Research by the NSW Secondary Principals’ Council (SPC) has revealed major flaws in the way that the Australian Curriculum, Assessment and Reporting Authority (ACARA) calculates the Index of Socio-Educational Advantage (ICSEA) for 2010.
The ICSEA is the basis for the lists of “statistically similar schools” which the Deputy Prime Minister urges parents to use to judge the performance of their child’s school. It is one of the major pillars on which the My School website is founded.
The SPC President, Jim McAlpine, said today: “A clerical oversight in setting up the My School website gave SPC researchers a window into the ‘before and after’ versions of the ICSEA for hundreds of schools”.
Instead of using each school’s actual student enrolment data, ACARA used average data from the 2006 Australian Census to calculate the degree of educational advantage or disadvantage for each family in each school. The Australian Bureau of Statistics (ABS) has cautioned against using its statistical data in this way.
Mr McAlpine said: “The lack of reliability involved in using census averages instead of actual student data is underlined by the fact that ACARA relied on school system authorities to help them corect mistakes before they go on the My School website. We believe that ACARA should be using actual student data to construct the ICSEA. Their present “short-cut” approach is piecemeal, statistically unreliable and reflects the indecent haste with which My School was brought online to meet the Deputy Prime Minister’s timetable”.
“Population data published in early editions of My School showed that the ICSEA for one western Sydney high school should have been just under 1000 using ACARA’s methods, but the published figure – as corrected by the NSW Department of Education and Training – was actually close to 700. That’s a difference of more than 250 ICSEA points”, Mr McAlpine said. “It’s a good thing for that school that the NSW DET knows its schools and students far better than ACARA apparently does.”
“While we can be confident that the NSW DET has robust and reliable data for its students, we cannot be similarly confident of other systems in NSW and systems in other states, all of which are being compared with NSW schools on the so-called ‘similar schools’ lists. We understand that the reason ACARA uses census data is precisely because all systems around Australia do not have the accurate, student-level data that is needed for a reliable result”
“The ICSEA is a good idea and the SPC supports its use in principle as a basis for resource allocation, among other things, but it must be calculated from real student enrolment data or not at all”, Mr McAlpine said.
“In a high-stakes situation where a few ICSEA points can mean a completely different list of comparison schools, we have evidence that more than 200 NSW public schools had to have their ICSEAs adjusted from the original calculation, several of them by 100 points or more. Shifts like that can completely change the numbers and the colour scheme that My School uses to indicate school performance to parents and the public.”
President, NSW Secondary Principals’ Council (SPC)
02 4868 2364 (office), 02 4869 3104 (fax)
0411106267 (mobile), 02 4869 3103 (home)
PO Box 1074, MOSS VALE 2577, NSW, AUSTRALIA
18 April 2010
Download full Media Release which includes table of ICSEA differences