Welcome to the Curriculum & Leadership Journal website.
To receive our fortnightly Email Alert,
please click on the blue menu item below.
Curriculum & Leadership Journal
An electronic journal for leaders in education
ISSN: 1448-0743
Follow us on twitter

Statistical literacy: a professional skill for today’s teachers

Helen Chick
Robyn Pierce
Robyn Pierce (r.pierce@unimelb.edu.au), Helen Chick (Helen.Chick@utas.edu.au)

Associate Professor Robyn Pierce works at the University of Melbourne, where she is academic leader of the mathematics education group of the Melbourne Graduate School of Education. Associate Professor Helen Chick works at the University of Tasmania, where she is a deputy head of school in the Faculty of Education. Both have interests in teacher education and statistical literacy. The work reported here was part of a project funded by the Australian Research Council (LP100100388), the Victorian Curriculum and Assessment Authority, and the Victorian Department of Education and Early Childhood Development. 

For an extended discussion of this topic see Pierce, R, Chick, HL, & Gordon, I 2013 (in press), 'Teachers' perceptions of the factors influencing their engagement with statistical reports on student achievement data', Australian Journal of Education; and Pierce, R & Chick, HL 2013, 'Workplace statistical literacy: Teachers interpreting box plots', Mathematics Education Research Journal, vol. 25, pp. 189205.


In today’s education environment, decision making based on data-driven evidence has become part of expected professional practice. In schools, student assessment data such as NAPLAN results, whole school reports, and Year 12 examination reports have the potential to inform changes in this practice. If education personnel such as teachers and principals can interpret the data to identify across-school and within-class areas of strength and concern, and even diagnose the needs of individual students, then it becomes possible to adapt teaching approaches and emphases in a highly targeted way, with the potential for significant positive outcomes.

However, analysing and interpreting quantitative data in the context of a school setting is not easy, and teachers now require a level of statistical competence not expected in the past. Teachers do not necessarily need the deep knowledge of statistical algorithms required of statisticians, but they certainly need appropriate levels of statistical literacy – which means sufficient understanding of numeracy, statistics, and data presentation to make valuable use of quantitative data and summary reports in a personal or professional setting. They also need a positive disposition towards the use of such data (Ben-Zvi & Garfield, 2004; Gal, 2002; Watson, 2006), if they are to apply data effectively to inform student learning.

We have been involved in a research project examining the statistical literacy that teachers require to make sense of the statistical reports now provided to schools by education systems. The project has also examined teachers’ attitudes towards the data, in particular the extent to which they see it as a means to improve student learning. Despite the high importance now ascribed to assessment data, it seemed plausible that teachers’ awareness of possible shortcomings in national testing processes might impact negatively on their attitudes.

Initial evidence was obtained through a trial survey involving approximately 150 teachers, followed by focus groups with selected participants. Responses informed the design of an online survey completed by 700 teachers from 63 randomly selected schools. The survey simultaneously examined teachers’ existing levels of statistical literacy, and their disposition towards the use of large-scale assessment data.

The project is intended to inform the creation of a set of professional learning videos for later distribution by the Victorian Curriculum and Assessment Authority. The focus of our work was Victoria, but many of the principles are likely to be generalisable to other states and to countries beyond Australia, where large-scale testing and educational data are part of the education process.  

The work is part of a project funded by the Australian Research Council, the Victorian Curriculum and Assessment Authority, and the Victorian Department of Education and Early Childhood Development.


Statistical literacy

We found it useful to construct a framework for teachers’ professional statistical literacy (for details see Pierce and Chick, 2011). This framework (see Figure 1) identifies levels of statistical focus – from the simple ‘Reading Values’ (eg being able to read a student’s numeracy score in a table), through ‘Comparing Values’ (eg for differences between students, or between a class median and the national median), to the more complex and holistic ‘Analysing the Data Set’ (eg where teachers are able to see the data set as a whole and identify trends over time or consider the extent of variation within a class or school). Each level of focus requires skills from the levels within it, as indicated by the nested circles.  

StatLit Chick Figure 1b

Figure 1. A framework for considering professional statistical literacy


Statistical literacy, however, is not merely a matter of technical statistical knowledge. To fully understand the data and put it to good use requires contextual knowledge. For the education workplace there are two contexts that matter, indicated by the two halves of the surrounding rectangle in Figure 1. First, there is knowledge of the professional context. In order to make sense of statistical education reports, teachers need to understand the origins and nature of the data, such as the structure of NAPLAN tests, how the ‘Bands’ of performance are determined, and the meaning of acronyms such as ATSI and LBOTE. This kind of knowledge is part of the professional knowledge of a teacher or principal. Second, teachers also require knowledge of their local context, since characteristics of the school or the class may impact on or explain the results. The specifics of this knowledge will vary from teacher to teacher as local context varies, but is needed in order to fully interpret the data.

The presentation of the data in educational reports varies among jurisdictions. For Victorian teachers one of the most common statistical graphs that they need to interpret is the box-and-whisker plot. The box-and-whisker plot is considered best practice in providing a range of information to the reader about the distribution of a data set. However these graphs, which teachers are unlikely to have encountered during their own schooling, are open to misinterpretation, particularly as they illustrate the location of set proportions of the distribution rather than frequency (see Pierce & Chick, 2013). Our research showed that although teachers could interpret some aspects of this representation, most did not understand the idea that the same proportions of their class might be depicted by boxes of different length.

We would expect that the more complicated representations used in other states may also create some significant demands on the statistical literacy of teachers.


Teacher attitudes and perceptions of data

Part of the questionnaire focused on teachers’ attitudes, subjective norms (what teachers perceive that others, whose opinions they value, regard as being the ‘right thing to do’), and perceived behavioural controls (factors that teachers regard as affecting the ease or difficulty of working with data). These arise from Ajzen’s Theory of Planned Behavior (Ajzen, 1991). More than two-thirds of teachers were positive about the value of attending to such data and were confident that they could adequately interpret these reports. In general they recognised the data’s potential usefulness for informing planning choices, although there were some concerns that the results often arrived too late in the year to make any significant difference for current students.

The results do however suggest that a significant minority of teachers remain ambivalent or negative about the value of large-scale assessment data as ways to inform teaching practice.


Professional learning

Grounded in the evidence collected, we developed a face-to-face workshop targeted at improving teachers’ statistical knowledge and interpretation skills. We aimed to provide an opportunity for teachers to quickly learn or revise the statistical knowledge required for the interpretation of reports. The workshop, presented on different occasions to various groups totalling about 60 participants, focused on the ‘big picture’ of the concept rather than the technical detail, while still acknowledging the complexity inherent in the data analysis.

The interactive nature of the workshop gave teachers a chance to apply their new understanding to local school data, and to make concrete the abstract statistical concepts by applying them to a class of students from a hypothetical classroom similar to their own. Teachers were able to adjust the test scores of particular students by physically moving images of the students on a box-and-whisker plot. In this way we aimed to give teachers a sharper sense of where individual students stood in relation to the whole class, and how changes to one student’s score affected the overall pattern of results for the class. One session of the workshop allowed the teachers to examine their own class data and gain a better understanding of how closely their class results matched state and national results.

Teachers provided feedback about the workshops, through both in-workshop comments and a subsequent online survey, indicating that the workshops produced positive outcomes for teachers’ attitudes and perceptions of barriers to their use of data, and improved their statistical literacy skills for understanding complex data representations. However, teachers also mentioned the need for reinforcement of what they had just learned.


Online tutorials

As a consequence of our experiences designing and delivering this professional learning session, a series of online tutorials have been produced that could be used by individuals or groups to revisit or learn key points emphasised in the face-to-face workshop. These tutorials, which will be publicly accessible later this year, will allow teachers to build their statistical literacy skills, especially for dealing with box-and-whisker plots, and  for understanding what this implies about their classes. These tutorials will allow teachers to revisit and consolidate topics on an ‘as-needs’ basis. This means that teachers can review key ideas at the time that data is released.

StatLit Chick Figure 2b

Figure 2. Screen shot from an online tutorial showing a box-and-whisker plot constructed for a class of students (from a tutorial to be released by the Victorian Curriculum and Assessment Authority)


Conclusion

The information provided in educational data such as NAPLAN reports has the potential to be very useful to teachers. We now have a better understanding of teachers’ perception of the usefulness of this data and of the statistical literacy required to interpret and use such reports. This has made it possible to deliver targeted professional learning designed to build appropriate statistical skills and lead to more effective application of educational data.


References

Ajzen, I 1991, ‘The theory of planned behavior’, Organizational Behavior and Human Decision Processes, vol. 50, pp. 197-211.

Ben-Zvi, D & Garfield, J (Eds.) 2004, The challenge of developing statistical literacy, reasoning and thinking, Kluwer Academic Publishers, Dordrecht, The Netherlands.

Gal, I (2002), ‘Adults’ statistical literacy: Meanings, components, responsibilities’, International Statistical Review, vol. 70, pp. 1–51.

Pierce, R, Chick, HL, & Gordon, I 2013 (in press), ‘Teachers’ perceptions of the factors influencing their engagement with statistical reports on student achievement data’, Australian Journal of Education.

Pierce, R & Chick, HL (2011). ‘Reacting to quantitative data: Teachers’ perceptions of student achievement reports’, In J. Clark, B. Kissane, J. Mousley, T. Spencer, & S. Thornton (Eds.), Mathematics: Traditions and (new) practices. Proceedings of the 23rd biennial conference of The Australian Association of Mathematics Teachers Inc. and the 34th annual conference of the Mathematics Education Research Group of Australasia Inc. (pp. 631-639). MERGA/AAMT, Adelaide. http://www.merga.net.au/publications/counter.php?pub=pub_conf&id=1714

Pierce, R & Chick, HL 2013, ‘Workplace statistical literacy: Teachers interpreting box plots’, Mathematics Education Research Journal, vol. 25, pp. 189-205. DOI 10.1007/s13394-012-0046-3

Watson, JM 2006, Statistical literacy at school, Lawrence Erlbaum Associates, Mahwah, NJ.

KLA

Subject Headings

Assessment
Educational evaluation
Teaching and learning
Professional development
Statistics
Numeracy