SPC and the MySchool website

The SPC has played a significant role in trying to make a positive influence on the Federal Government’s decision to create the My School website. A Task Force of informed principals and former principals was formed for this purpose, and much correspondence has been entered into. Letters have been written to politicians at state and federal level, papers have been produced that have relied on solid evidence, meetings have been convened with significant personnel, and our members have been kept fully informed. In the interests of transparency, the SPC is now making this information available. Please read on to see what we have done.

Living with ICSEA – The Index of Socio-Educational Advantage

The story so far …

ICSEA first hit our consciousness when the My School website went live in 2010.  The number was calculated by ACARA for every school in the country as a basis for setting up “statistically similar” groups of schools from which My School users could draw conclusions about the relative merits of the schools’ work.

That last sentence raises a whole raft of issues that will be set aside here in order to focus on the ICSEA itself and what you need to know about it as a principal.

The SPC is in no way opposed to the identification and measurement of socio-educational advantage or disadvantage.  On the contrary, the concept of an ICSEA is most welcome if it can provide an accurate, objective means of deciding, say, the allocation of resources according to educational need as happens in New Zealand.  However, even for this relatively benign purpose, the accuracy of the measure is critically important if the misallocation of scarce taxpayer funds is to be avoided.

In human and educational terms however, the stakes are much higher if the ICSEA is to be turned into a public statement about a school’s community and even more so when it is used as a basis for judging the work of schools in the public domain.  It was in that context – and totally unprepared – that we first came face to face with ICSEA.

How ICSEA is derived
There is a reasonably well-defined collection of family attributes that correlate well with the educational success of their children.  Those attributes represent a foundation of “socio-educational advantage” (SEA) that each child brings to school.  The hypothesis is that whatever happens next in education is a “school effect”:  if groups with similar levels of SEA end up performing differently, then those differences reflect the work done by teachers and the school.

Calculating a measure of SEA for every school in the country is no small task, given that school populations are a moving target and the available data on students’ family circumstances – where they exist at all – vary greatly in their nature and quality.  ACARA decided to use a suite of measures derived from 2004 census data by the ABS, because they were at least consistent and available for most of the country.  The problem with the measures (apart from the 2004 date!) is that they are only available as averages and the smallest division for which they are released is the “census collection district” (CCD) – around 200 families.

Your school’s ICSEA
In essence, the calculation of your school’s ICSEA for the 2010 My School website started by assigning to each of your students the average indices of the CCD where they live.  A given school might draw from as few as one CCD or as many as a dozen or more.

The fact that this kind of use of their data is advised against by the ABS was clearly set aside by ACARA.  The (shakey) assumption on which it rests is that each of your students, or the aggregate of them, is representative of the CCDs they come from.  While this may be a fair approximation in many cases – but an approximation at best – it is clearly not for all.  Several lines of research show that where parental choice is available, on average the more advantaged students from any particular CCD will tend to go to a private school (producing an under-stated ICSEA for that school) and the less advantaged students from that CCD will attend a public school (producing an over-stated ICSEA for that school).

Aware of this and other influences, ACARA permitted school system operators to review the “draft” ICSEAs before publication and to recommend changes where they had reason to believe the figure did not represent the school’s reality.  While the process was not transparent, for a range of reasons, including the fact that they had superior, student-based data available, the NSW DET was able to recommend many changes for its schools’ ICSEAs, correcting some quite serious anomalies.

So, notwithstanding some remnant anomalies, NSW public school principals can have more confidence in their 2010 ICSEA than many of their interstate and inter-sector colleagues.  Unfortunately, while the rest of Australia’s ICSEAs are not similarly accurate, this doesn’t improve our confidence in the My School comparisons.

Why didn’t more schools object to their ICSEA?

Probably the most second most specious claim in the aftermath of the My School launch was the much-publicised defence  that “only 24 schools around the country complained” [about their assigned ICSEA].
Considering that:

1. only those schools on the wrong side of ICSEA-based comparisons would be motivated to complain in the first place.
2. The ICSEA value appeared “out of the blue” on the My School site, unannounced and completely without pre-consultation.
3. The mathematical basis of the ICSEA was only given in broad outline, with only limited methodology or process details.
4. Even if the mathematical process could be guessed at, the data necessary to calculate or verify an individual school’s ICSEA is privy only to ACARA.
5. The ABS data relating to particular CCDs is not available except by purchase and would be impossible to mesh with the other ICSEA data anyway.

… then it is surprising that as many as 24 schools felt confident enough to formally challenge the determination.

The best that most principals could manage was a “gut feeling” that something was wrong, which large numbers of them were quite ready to share and discuss with their staff and colleagues.  However it should be no surprise that the vast majority either lacked the mathematical confidence or judged that any attempt to take the matter further would be futile and they had more pressing issues to deal with.

Want to mount a challenge?
It is the view of the NSWSPC that those principals who wish to challenge their assigned ICSEA, or at the very least to seek a detailed justification of it, should be given every facility to do so.  We are hopeful that ACARA might take up this issue and provide some practical advice, but at the moment the esoteric mysteries surrounding ICSEA present an intimidating barrier to any thought of challenge.

The only foothold to analysis available to principals at the moment is the population quarters listed on the website which show ACARA’s view of where your student’s families sit on the socio-educational scale.  This may well be at variance with your understanding of your school’s community and if this is the case, you need to understand and more importantly, be able to substantiate these differences.

While you won’t be able to use these in any direct way, you could begin to form a perspective of your schools level of SEA based on data you have (or may be able to obtain readily and legitimately) about your students and their families.  You need to follow the ICSEA inputs as closely as you can, so you will need percentage breakdowns of things like:

• Aboriginality
• Single-parent families
• Unemployed families
• Families without an internet connection at home
• Families who are non-English speakers

– all of which are direct inputs to ICSEA.  Other key inputs include family income and parent education and occupation level, which you won’t have, but your school records can provide parent occupations which could be worked into something useful with effort.

There is no way you can process these data through a formula and come out with an ICSEA, but you may well find a case among them that you could put to ACARA to seek their response.

ICSEA and the Future
The SPC’s policy is to abandon ICSEA-based comparisons on the My School website.  While there is no real prospect of that happening at the moment, we are hopeful that current developments will lead to the calculations being based on actual student data, rather than CCD averages, which would be an improvement if it could be achieved universally.  Unfortunately, not all jurisdictions collect student-level data of the kind that would be necessary and some that do are reluctant to provide it to ACARA.  Again, it is SPC’s policy to abandon ICSEA-based comparisons, if not completely, then at least until they can be done with more reliability than at present.