System assessment in Australia: achievements, challenges and opportunities
This article is an abridged version of the paper System assessment in Australia: achievements, challenges and opportunities, the keynote address at the 11th Annual National Roundtable Conference, Perth 2006.
Before the 1990s tests and assessments were for the most part developed by teachers or were commercial standardised tests, and tended to provide comparative rather than descriptive data. Often the results did not reflect growth or development in a given learning area. Instead they provided a score or a position in a particular test group, sometimes norm-referenced against a population group.
In the 1990s the emphasis began to shift towards identifying what students had learnt. The shift brought with it the need for assessment data that is comparable at system, school and class levels. As part of this trend, full cohort testing became more common across jurisdictions and sectors.
Many teachers initially reacted to this development with some apprehension about how the data would be used by school leaders, the department or government. Since that time, however, we have seen a change in teachers’ attitudes.
There are a number of reasons for this turn-around. Much of it can be attributed to the efforts that jurisdictions and sectors have made in setting up programs to support schools and teachers in the use of assessment data. Through programs such as the Data Club in Western Australia, schools and teachers have witnessed for themselves the usefulness of data to support decision making and inform plans to improve learning outcomes.
IT applications encourage and support the exploration and interrogation of data. Over time, data provision has become increasingly user friendly. Web delivery has become an efficient way of providing assessment support materials in jurisdictions and sectors. The provision of diagnostic information, and of links to support materials such as the New South Wales School Measurement Assessment Reporting Tool (SMART), has enhanced the value of assessment data to teachers and schools.
Professional development has also been critical in developing the expertise required at the school level to interrogate and interpret data in order to understand a school’s needs. At the same time, jurisdictions included teachers in test item review processes and marking.
International, national and jurisdictional sample programs have played a key role changing teachers’ attitudes to assessment data. It is useful to reflect on two examples of testing programs that have helped to change teachers’ attitudes to data during this period. They are briefly outlined below and described in more detail in the Roundtable Conference paper from which this article is drawn.
The 1996 National English Literacy survey (NELS) was designed primarily to provide national data for monitoring purposes, but it also provided an opportunity to draw on and build teachers’ professional expertise by involving them directly in the assessment of their students. ACER conducted the survey under the direction of a management committee consisting of representatives from State and Territory governments, non-government school authorities, the Australian Government, teacher unions, professional teacher associations, parents and the business sector. This highly collaborative work was further supported by a reference group including literacy experts, representatives of multicultural education associations, teachers of English as a second language and Indigenous education groups.
The survey consisted of ‘common’ tasks, evidence collected under standardised assessment conditions but embedded in classroom practice over a six-week period, and ‘best work’ samples, or samples of regular classroom work. Teachers administered writing, reading, viewing, speaking and listening assessment tasks in a representative sample of schools across Australia. This included making performance judgements, for example in the speaking tasks.
To ensure comparability among teachers and across schools, external assessors trained and supported teachers in the assessments. One hundred external assessors were trained at a three-day national professional development workshop, using a ‘train the trainer’ model. The external assessors in turn trained the 900 teachers participating in the survey and supported them throughout the assessment period. Each of these 900 teachers assessed ten students.
Teachers were provided with detailed marking guides, rating scales and annotated work samples. Marker consistency was high – almost all teacher assessments were left unchanged. More than 90 per cent of Viewing and Listening and 98 per cent of Reading assessments were left unchanged. Writing was more subject to marker harshness or leniency. The National English Literacy Survey report provides more detail on this.
Many important findings were made about the literacy levels of Year 3 and Year 5 students in Australia. From the end-user perspective other important results were also obtained. Teachers gained a more holistic view of English literacy and were supported in making consistent judgements and ascertaining student development in various aspects of English literacy.
Teachers were provided with models of assessment in areas such as listening and speaking, where they previously tended to lack confidence in terms of assessment. The approach of judging aspects of speaking such as content and performance separately was valuable in supporting teachers to map student performance, clarifying what students could do and identifying areas where they required focused support.
Teachers were valued in the assessment process and also gained professionally from their involvement, building on their assessment repertoire.
Progress maps reflected growth and development in an area and were useful for teachers in assessing student progress. They also gave further meaning to reporting, and progress between years levels could be monitored.
The value of professional development in this survey was recognised and is now provided and promoted through the Literacy Assessment: Strategies from the National School English Literacy Survey website.
One important message highlighted by the survey was the unacceptably low attainment of Indigenous students.
The Monitoring Standards in Education (MSE) sample testing program, in Western Australia, has collected system level data on Years 3, 7 and 10 students’ performance across key learning areas since 1990. The program has been important for providing a model of assessment for areas of learning beyond literacy and numeracy.
The 2000 MSE Technology and Enterprise sample testing was one part of this program. Approximately 10,000 students across Years 3, 7 and 10 participated. Across the various assessment tasks students were required to apply knowledge, values, understandings and skills, which generally entailed responding to written questions and making a product.
Year 3 tasks focused on self and meeting personal needs. One Materials task required students to design and make a money holder. The task served to demonstrate expectations of the learning area and model best practice in teaching and assessment within standard classroom settings. By Year 10 students were expected to apply their thinking more broadly to meet the needs of communities or societal groups. For some Year 10 students an outback town provided the context for Design Technology, Information Technology and Home Economics design briefs and subsequent assessment activities.
One of the main challenges was to provide authentic, purposeful, practical and performance-oriented contexts for the assessments, all within a reasonable timeframe and in classrooms that often lacked specialised equipment or settings. Given the diversity of school settings and resourcing, decisions had to be made about what could reasonably be expected in terms of school facilities and the availability of specialised equipment and resource materials, as well as what could reasonably be provided to schools to ensure that students had sufficient opportunity to demonstrate their abilities.
At the time when it was first undertaken, system-wide testing of the Technology and Enterprise learning area was highly innovative. Similar programs overseas were rare, and certainly not of the scope undertaken by MSE 2000.
Feedback from participating teachers was very positive, with most finding the assessment process and results useful in confirming or challenging their estimates of student ability. Teachers also reported a high level of student engagement in the tasks and interest in their achievements.
Assessments of this kind demonstrated that it was possible to provide authentic and engaging assessment contexts and showed that assessment could model the depth and breadth of a learning area.
The new testing served to stimulate teaching itself in this subject area. At the time it was not uncommon for students to be limited to simple activities – for example, Year 7 students might still be restricted to activities such as making a pencil case in term 2 in woodwork. The new assessment suggested the use of more complex teaching activities.
Around Australia, system assessment was now increasingly being used beyond reading, writing and numeracy. The Australian Capital Territory included Viewing, Listening and Speaking in its Year 3–9 system assessments. Western Australia ran cyclic testing in Science, Technology, the Arts, Health and Physical Education and Studies of Society and the Environment at Years 3, 7 and 10. Victoria assessed Science and Studies of the Environment at Years 3, 5 and 7.
As a result of these developments, teachers’ earlier doubts about system-level assessment have been largely replaced by enthusiasm to learn about and take part in it. The thirst for assessment data is illustrated in a range of ways.
Assessment conferences in 2005 such as the Curriculum Corporation Conference Curriculum and Assessment – Closing the Gap and ACER’s Using data to support learning were more than fully subscribed long in advance. Teachers are increasingly keen to be involved in item development and item review processes, as evidenced by the number of expressions of interest received by jurisdictions.
In the Northern Territory teachers are involved in test construction and item development panels. Many are keen to attend these panels more than once because they value the professional development offered, which includes gaining a better understanding of the curriculum frameworks; having the opportunity to share their experiences and learn with other teachers; ensuring items will be appropriate for Northern Territory students; and sharing learning in their schools. Teachers who arrive with a more sceptical opinion of testing and assessment often leave at the end of the week as advocates of the program, having been made aware of the rigour involved in item development and test construction.
The high number of visits to assessment websites is a further indication of the developing thirst for assessment data. Between 1,000 and 3,000 unique visitors access the Assessment for Learning site each month. States, Territories and sectors have their own well-visited and rich assessment websites. Also indicative of the thirst for data, a number of Western Australian schools asked to be included in the National Assessment Program Science Literacy (NAP-SL) testing this year.
It is important that we nurture and grow the interest and thirst for data that has been developed, in large, by jurisdictions and sectors. Likewise, it is important to maintain and build the wealth of expertise and experience that has been developed across Australia.
Teaching and learning