NAPLAN Test comparisons are meaningless for small schools

This is the text of a letter posted to all Korowal families in 2010 to explain our concerns about the information published on the My School website and in the Sydney Morning Herald.

"Dear Parents and Friends,

We are writing to all parents with children currently at Korowal and on our waiting list to express our concern about misleading and inaccurate information that has been published on the Federal Government’s My School website and in the Sydney Morning Herald.

The My School site has taken the results of every school’s NAPLAN tests and compared these to a national average, and to a list of “similar schools”. The Sydney Morning Herald took the information from this website and converted it into a simplistic rank for schools.

What is the NAPLAN test?

The NAPLAN (or “basic skills”) test is conducted in schools across Australia in grades 3, 5, 7 and 9.  It assesses a narrow range of literacy and numeracy skills in a particular way on a particular day. We do not teach to the test or prepare students for it. Our students are not familiar with testing procedures. It is compulsory for schools to offer the test.  Many of our parents allow their children to sit the test, but many opt out. Until now we had considered the test to be a relatively harmless way of providing parents and teachers with feedback about some aspects of literacy and numeracy. We believed that the test results would only be used to compare individual students to a national average.

Korowal’s performance in the NAPLAN test.

The My School website compares the performance of each school in these tests to a national average and to a list of supposedly “similar schools”. It reports that Korowal’s overall performance was generally below average in 2009 and above average in 2008. The only class ranking above average in 2009 was the largest group tested – our year 9 class.

How are “similar schools” determined?

Similar schools are based on measure of socio economic advantage by postcode, numbers of Aboriginal students, and remoteness of the school. There is no account taken of the size of a school, how many students sat for the test, how many students have special needs, or whether students speak English as a first language.

The Sydney Morning Herald rankings.

The Herald combined the misleading data from the My School site and ranked these rankings! They used only one year’s results. On this basis Korowal high school was ranked relatively well and our primary school ranked poorly. In doing this they have contravened s.18A of the NSW Education Act which prohibits newspapers from publishing school results in a way that compares schools, unless with the permission of the school principal.

What do these results mean?

Korowal's results are statistically meaningless.

The My School website explains that “comparisons are more accurate when more students are tested and when their results are consistent.” This means that Korowal’s results are very inaccurate, because:

• Korowal has small classes and some of our parents withdraw their children from the test. In year 3 last year only six students sat the test. The website states that in small groups such as ours the error rate is as high as 35 points.
• When there is a range of results in a group – some students performing highly and others who might have learning difficulties or special needs – this makes the results even more inaccurate.

Therefore, comparing results for small schools against any average is meaningless. A group of at least 25 students needs to be tested in order to achieve fairness and accuracy.

It is obvious that groups in a small school will vary greatly from year to year – especially in a school that encourages the integration of students with varying abilities

What is Korowal’s approach to ranking?

• We do not encourage ranking and grading but we provide continuous opportunities for meaningful and constructive assessment.
• Our approach is holistic – students learn literacy and numeracy skills in main lessons and focus studies and these are supplemented by specific skills classes in small groups. We do not rush students to achieve “basic skills” targets at an early age.  We integrate students of different age and skill levels within learning groups.
• We accept that our students’ School Certificate and Higher School certificate results will be publicly compared. We prepare our students for these examinations and are extremely proud of their achievements. However, we value far more than statistically high marks. Because our senior classes are small, our results vary when compared from year to year. Some year groups perform outstandingly. Some students attain very high marks. More importantly, most exceed their expectations and complete school with a positive attitude to learning.

Our objections to the NAPLAN test comparisons.

The comparative information published on the My School website and simplified even further in the Herald is:
• inaccurate and misleading.
• damaging to small schools.
• an extremely  narrow way of ranking schools.
• a breach of trust. Our teachers, students and parents agreed to these tests believing that they would provide information about individual students in relation to a national average. We had no idea that they would be used to rank schools.

What will we do about this?

The 2010 tests will be conducted in May. Our staff and School Council will review our approach to the NAPLAN tests in the light of this changed political context. It is a Federal Government funding  requirement that we offer the testing. However, parents are free to withdraw their children from the test.
Many schools will respond by “playing the game” – by teaching to the test, conducting practice tests, and by selecting the students for the test who are likely to achieve most highly and encouraging others to withdraw.
Along with other small schools in the Mountains we have begun lobbying to have test results from groups of less than 50 students excluded from any comparative tables.

We will continue to promote the idea that assessment, success and achievement in education cannot be measured by a narrow test of literacy and numeracy.

We understand that parents choose Korowal because they value our integrated, human-centred principles and understand that learning encompasses far more than literacy and numeracy. They understand that education is not a competition with winners and losers. Nevertheless, we understand that students must learn many skills, including literacy and numeracy skills  and we can assure you that a Korowal education will give them those skills. We can also assure you  that students who struggle in any aspect of education will be supported, and that Korowal is the ideal and proven learning environment for students who aspire to high HSC scores.

Mark Thomas
School Coordinator
2 February 2010"