Assessing scientific literacy nationally
Science literacy is important in enabling citizens to question the assertions of others, establish evidence, and come to their own decisions, in relation to scientific issues. The development of science literacy among Australian students is one of the aims of the National Goals of Schooling in the Twenty-First Century, endorsed in 1999 by the Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA).
In support of this goal, the science literacy of Australian children at Year 6 level is currently being evaluated through a test administered to a national sample of students. The evaluation is taking place through the National Assessment Program (NAP). The NAP assesses students’ ICT literacy and understanding of civics and citizenship, as well as science literacy, to monitor and report more widely on progress towards the National Goals of Schooling.
In general, the NAP assessments monitor students’ progress at Years 6 and 10. However, for science literacy only Year 6 students are evaluated by the NAP test. Currently, Year 10 students are monitored through participation in the OECD’s PISA assessment program.
The NAP assessments are conducted in a three-year cycle. The first stage of the evaluation was run in October 2003. The second phase now under way is being administered by Educational Assessment Australia and Curriculum Corporation, on behalf of MCEETYA’s Performance Measuring and Reporting Taskforce (PMRT) . The assessment materials currently being developed will be trialled later this year before being administered to a national sample of students in 2006.
A scientifically literate person is able to apply processes and concepts in relation to real-world contexts and problems. Assessing scientific literacy therefore means more than testing isolated knowledge and facts. For the purposes of the NAP science sample assessment, scientific literacy is defined as ‘the capacity to use scientific knowledge, to identify questions, to investigate and to draw evidence-based conclusions in order to understand and help make decisions about the natural world and the changes made to it though human activity’. This definition is based on the one used in the PISA program. However, the revised definition adds the term ‘investigate’, which enables the administration of performance tasks in the form of ‘hands-on’ practical activities. The rationale for the definition is explained in more detail in the PMRT’s background documentation.
A progress map has been developed based on this definition of scientific literacy. The map has three domains (or subdomains of scientific literacy) and describes progression in six levels, spanning the compulsory years of schooling:
In producing test materials linked to the progress map, test developers are also guided by the PMRT's table of major scientific concepts found most widely in the various State and Territory curriculum documents. The table provides broad statements of scientific understandings that Year 6 students can be expected to demonstrate. The concepts are drawn from the areas of Earth and beyond, Energy and change, Life and living and Natural and processed materials.
An assessment framework derived from the progress map will be developed to indicate the distribution of item types that are considered to be appropriate to a Year 6 assessment, covering each level in each of domains A, B and C. Item types might include multiple-choice questions, short constructed items that require only a one or two word response, or extended response items requiring students to provide up to a short paragraph if in text form (or a diagram or constructed data table of equivalent detail). It is expected that extended constructed response items will be included in order to assess the higher order skills demanded by the upper levels of the framework. Suitable contexts for Year 6 students will also be identified.
As was the case for the 2003 assessment program, the students will complete both a pencil and paper task and a practical task. Practical tasks will be piloted with classes prior to trial, to ensure that the activities proposed can be implemented with ease within the Year 6 classroom setting. In recognition that many primary schools will have limited access to sophisticated science equipment, and that students will have had limited exposure to such equipment, all materials required for the conduct of the practical tasks will be relatively simple in nature and provided to schools.
To guide development of the trial materials a detailed assessment framework will be presented to the PMRTs Science Literacy Review Committee for approval.
Key Learning AreasScience