|
The true impact of school diversity?
Stephen Gorard Department of Educational Studies University of York, UK
This paper presents a summary of the findings from a series of investigations conducted by the author into the relationship between school organisation and school outcomes in England and Wales. It shows that increased diversification of secondary schooling in the UK is generally associated, by commentators and advocates, with two inter-related claims. First: increased diversification does not lead to increased segregation of students between schools. Second: the new school types driving diversification are more effective in producing higher levels of student attainment with equivalent student intakes than their predecessors. Neither claim can be shown to be true. Then the paper considers a relatively new type of school - the Academy - that adds an apparent complication. It shows that Academies do, in fact, fit the same overall pattern of 'sleight-of-hand' school improvement as grammar, faith-based, specialist and Welsh-medium schools (among others). If all types of schools produce pretty much the same level of student attainment once their intakes are taken into account, why then does the segregated nature of schooling matter? If the drive to increase standards through diversification has merely produced more segregated schools, apparently nothing has been gained but nothing lost. The paper ends by arguing that segregation matters, not for school outcomes as narrowly envisaged, but for schools as mini-societies in which students develop a sense of what is just and what is not.
School diversity and segregation
In a major study of the composition of secondary schools in England and Wales from 1989 to 2001, I have demonstrated that increased diversity of school types is strongly associated with increased segregation of student intakes (Gorard et al. 2003). 'Segregation' here refers to the unevenness of student allocation to schools in terms of their background characteristics - most notably in terms of official designation as 'living in poverty'. It is a measure of the extent to which poor children are clustered in specific schools (Gorard and Taylor 2002). At the start of the period of study (1989) the school system in England and Wales was largely comprehensive in structure (with a minority of selective and faith-based schools), so that most state-funded schools were open to students of any ability or socio-economic background, and students tended to use their local designated school. This meant that the student body in most schools reflected the nature of local housing and its residents more than anything else. The historical long-term trend of both residential and school segregation was down.
By 1995 this long-term trend towards comprehensive school intakes had stalled, and from 1997 onwards began to reverse. The growth in segregated school intakes coincided with considerably increased diversity of school types both temporally and spatially. As the proportion of 1989-type local comprehensives declined so segregation increased, and was highest in those areas with a low proportion of comprehensives (Gorard and Taylor 2001). Once the nature of local housing patterns had been taken into account, the least segregated areas were those with no selection by schools, little or no diversification of school types, where choice prevailed over the rigid allocation of school place via catchment areas, and finally where schools were constrained to admit a proportion of students across all of the ability bands represented in the area ('banding').
Readers are recommended to pursue the references given throughout this paper in order to gain a more sophisticated picture of the problem. However, in summary it would be fair to say that, ceteris paribus, diversity of school types is clearly linked to increased segregation of school intakes. This applies whether the diversity is in terms of fee-paying:state-funded, faith-based:secular, local:central control, selective:all-ability, specialist:traditional, or Welsh:English medium (Gorard 2000a). Some conservative commentators, especially in the UK research community, objected to the findings of this analysis, presumably on ideological grounds given that their substantive objections were flimsy and sometimes even dishonest (Gorard and Fitz 2006). For advocates of increased diversification of schools it is important to maintain the fiction that introducing new types of schools does not drive up segregation, because their advocacy is generally based on the claim that new type of schools produce superior academic outcomes without changing their intakes. How true is this?
School diversity and improvement
In order to show that a school is differentially effective we have to establish that exactly equivalent students would achieve lower test scores after education at another school. In order to show that a school has improved we have to establish that there has been an improvement in test scores that cannot be explained by a change in the nature of the school intake. Otherwise, SESI is 'sleight-of-hand'.
It is not possible to compare directly the progress of the same student in two schools. Trying, instead, to match the characteristics of students, curricula, qualifications and schools, leads to multiple difficulties of comparability (Gorard 2001). This leads to an asymmetry in the burden of proof. A commentator claiming to have evidence that one type of school is differentially effective has, therefore, to explain how they have overcome the problems of comparability and of matching equivalent students. A commentator denying evidence differential effectiveness, on the other hand, need only point to any lack of prime facie evidence or to a contextual difference between types of schools that has been omitted from the analysis.
In any study of school effects, typically between 0 and 25% of the variation between school outcomes remains to be explained, and this residual includes a very important error component. This major finding of work in the school effectiveness genre has been quite consistent over time (Gorard and Smith 2004a). The larger the study, the more variables available for each student, the more reliable the measures are, and the better conducted the study, the stronger is this link between school intake and outcomes. Where sufficient background or prior attainment data is available, it is possible to explain apparent raw-score differences between types of schools simply in terms of differences in student intake (Gorard 2000b). Once their intake is taken into account, specialist, grammar, faith-based and Welsh-medium schools appear no more effective than 'bog standard' comprehensives. In fact, it becomes quite difficult to establish any school effect at all (Gorard and Smith 2004a). But school effectiveness and school improvement (SESI) forms an entire field of research and policy endeavour based on the dual premise that some schools are differentially effective with equivalent students, and that is possible to transfer good practice from the more successful schools to the less successful ones.
SESI advocates try to guard against being misled by not using raw-scores outcomes, and using value-added models instead. Such models are intended to take the prior attainment of each student into account, and so to produce scores that are 'a measure of the progress students make between different stages of education' (DfES 2005). Unfortunately, these models fail for a very simple but important reason - the value-added scores they produce are so little different from the raw-score figures they are derived from (Gorard 2006a).
The UK Department for Education and Skills (DfES) value-added school 'performance' figures have a correlation of +0.84 with the raw-score outcomes they are intended to replace, which themselves have a +0.87 correlation with the prior KS2 scores (Figure 1). This means that 71% of the variation in school value-added scores is explicable in terms of their raw-scores alone. There is a clear pattern of low attaining schools having low VA, and high attaining schools having low VA. Value-added scores are no more independent of raw-score levels of attainment than outcomes are independent of intakes. This means that all policies concerning schools, and judgements about the relative effectiveness of different schools and types of schools, will have been misled where they have been based on such a 'value-added' analysis. School improvers and school improvement researchers, relying on value-added analyses, will have been misled in their explanations and in making recommendations for practice. Once accepted, this re-consideration suggests that school improvement policies, at least in this narrow sense, are always likely to be ineffective. Therefore, all claims to differential effectiveness of one kind of school over another are suspect, and the risk of increased segregation through diversification is being taken for no reason at all.
Figure 1 - The link between GCSE A*-C benchmark and KS2 to GCSE value-added, all secondary schools in England, 2004 Note: figures provided by DfES. The figures exclude special schools, school with no GCSE entries, no GCSE 'passes' or no Key Stage 2 results, independent schools opting out of the value-added scheme, those whose results were suppressed by the DfES on grounds that individuals might be identifiable, and those with fewer than 30 cases in the cohort. Note: the value-added scores are those published by the DfES, with a mean of around 988. Note: the perceived 'width' of the scatter depends upon the scales used. The correlation coefficients quoted are a better guide.
The Academies programme
One recent model of school diversification in England appears, at first sight, to break this link between increased segregation as a price to pay for less than convincing academic improvement. This is because these new schools were intended to be selected as those with the highest levels of disadvantage, so that a change in their intake towards less deprived families would inevitably lead to less local segregation, rather than more. In 2004, the BBC reported the considerable success of the Academies (originally City Academies) programme, wherein troubled schools in deprived areas were rebadged, given extra funding, new management, and curricular freedom. The report stated: 'The government has released GCSE figures from three of its new flagship Academies in England. All the schools, which were set up in deprived areas, showed remarkable improvements in results' BBC (2004). The first three Academies were set up in 2002, so that by 2004 it was only these three about which one could tentatively draw before-and-after comparisons. One of these was the Business Academy, Bexley, where the Prime Minister stated that he had 'seen the future of education'. 'In its first year, the Business Academy, Bexley achieved an increase in pupils attaining 5 or more A*-C grades at GCSE from 7% in 2002 to 21% in 2003' (DfES 2004).
In fact, of course, Academies did not break the link between school composition and attainment (Gorard 2005). Many Academies, including one of the first three, were nowhere near the most deprived schools in their areas. The others have all reduced the proportion of deprived children in the school since becoming Academies. For example, Table 1 shows the proportion of students in the Bexley Academy eligible for free school meals, which is an indication of family poverty. For the two years after conversion to an Academy (2002-03), this school took its lowest proportion of FSM students since records began in 1989. The proportion of FSM students was, and remains, higher than the national average so this reduction could have been claimed as a success of the policy of turning schools around. However, this reduction in FSM had to be ignored because the school and the government wished to claim that a purported increase in examination success was achieved with no discernible change to the school intake.
Table 1 - Pattern of disadvantage over time in Bexley
But just how 'remarkable' were the improvements in GCSE scores acclaimed by the Prime Minister? Table 2 shows that, in recent years, the best year for GCSE results in the school that became an Academy in Bexley was 1998, long before the change to Academy status. When the DfES claimed success for the early Academies, because the GCSE benchmark for the Business Academy rose to 21% in 2003, they neglected to mention that the predecessor school had a benchmark of 24% in 1998 at a time when national benchmarks were lower on average anyway. The historical sequence of events here is quite wrong for a successful claim to a policy intervention.
Table 2 - GCSE results over time for the Business Academy
Note: the DfES Standards Site does not provide results for 2002, the year of changeover
In addition, the more recent GCSE 'success' remarked on by the government occurred when the school had a lower level of disadvantage. This makes the 1998 success more remarkable. Of course, the intervention is still at a very early stage, and perhaps we should not expect to see differences until an entire cohort has gone from entry at age 11 to GCSE. The point is that by 2004 success was already being claimed by some (see above), and that claim does not stand up to even superficial scrutiny. Because the strong link between school compositions and school outcomes remains, the apparent improvement of Academies is, just like other attempts at improvement through diversification, sleight-of-hand.
The true impact of school composition?
Diversification of schools tends to lead to increased student segregation, which is a concern for some commentators. But if, as appears to be the case, the precise mix of students in any school makes no substantive difference to their levels of attainment (Gorard 2006b), then increased diversification of schools appears somewhat pointless but also relatively harmless. Why does it matter who is educated with who? I have two tentative responses to that important policy question.
First, if there is no need to re-mix school intakes in order to improve attainment (because that has been shown to be ineffective) then we can feel free to use other criteria for deciding on the pattern of intakes to schools. These criteria might include efficiency or convenience, but we could also try equity as a guiding principle. This was the approach that led to comprehensive schools in the first place, and to area-level ability banding in particular.
Second, it is important to bear in mind that the outcome measure in all of the foregoing is based on examination outcomes. Schools are, however, about much more than these. The school mix appears to matter most because it provides the context for creating students' awareness of equity (Halstead and Taylor 2000, Meuret 2001). Close to the heart of developing a model of citizenship among students is the need to encourage children to develop their own concepts of fairness (DfES 2002). Probably the fundamental influence on pupils in developing their perceptions of what constitutes a fair and equitable society is their experience of school (Howard and Gill 2000, Davies and Evans 2002). Inclusive schools are generally more tolerant (Slee 2001), and exhibit that tolerance in racial, social and religious terms, and this is also associated with greater civic awareness (Schagen 2002). In fact, the simple act of segregating students, whether by race, class or ability, might be considered an affront (Massey and Denton 1993).
This matters because the level of ethnic, and other, segregation in schools can affect racial attitudes, subsequent social and economic outcomes, and patterns of residential segregation (Clotfelter 2001). The experience of Northern Ireland shows that, if true, this can be a force for even greater societal segregation, where the segregation in schools feeds back into society (Smith 2003), and where teachers in schools are, therefore, unwilling even to discuss issues of sectarianism with their (segregated) students (Mansell 2005). In divided societies, citizenship education can actually generate negative results, including the ghettoisation of minority communities, perhaps culminating in greater social unrest as it has in some central European countries (Print and Coleman 2003).
My own research shows that school students have a clear notion about what constitutes a fair and equitable national education system (Gorard and Smith 2004b). The system should be an egalitarian one which benefits all students equally. In general, most students in EU countries were of the opinion that a fair and equitable national education system would be one in which all students were treated in the same way, although there was also considerable support for the notion that the less able students should receive a disproportionate amount of the teacher's attention. However, the extent to which the students report that this was what they actually experienced in school varied across the EU. There are preliminary indications that the nature of their national school system is related to students' formation of views of justice (Smith and Gorard 2006). Perhaps, therefore, we should be more concerned with the experience of schooling as something in its own right, and not always for something at a later date (qualification, participation, employment and so on). Perhaps the school mix is the school effect, affecting pupils' views of school, and therefore of later educational opportunities, but also affecting their notions of social justice in the present. And perhaps it is this that is most at risk from some of the proposed changes to school organisation presented in the Schools White Paper for England before the UK parliament at time of writing.
References
BBC (2004) Academies getting results at GCSE, http://news.bbc.co.uk/go/pr/fr/-/1/hi/education/3602818.stm, (accessed 17/11/04) Clotfelter, C. (2001) Are whites still fleeing? Racial patterns and enrolment sifts in urban public schools, Journal of Policy Analysis and Management, 20, 2, 199-221 Davies, I. and Evans, M. (2002) Encouraging active citizenship, Educational Review, 54, 1, 69-78 DfES (2002) Citizenship: the National Curriculum for England, http://www.dfes.gov.uk/citizenship [Accessed August 2003]. DfES (2004c) Academies that are open, http://www.standards.dfes.gov.uk/academies/projects/openacademies/?version=1, (accessed 2/11/04) DfES (2005) Value-added Technical Information, http://www.dfes.gov.uk/performancetables/schools_04/sec3b.shtml, (accessed 25/2/05) Gorard, S. (2000a) Education and Social Justice, Cardiff: University of Wales Press Gorard, S. (2000b) 'Underachievement' is still an ugly word: reconsidering the relative effectiveness of schools in England and Wales, Journal of Education Policy, 15, 5, 559-573 Gorard, S. (2001) International comparisons of school effectiveness: a second component of the 'crisis account'?, Comparative Education , 37, 3, 279-296 Gorard, S. (2005) Academies as the 'future of schooling': is this an evidence-based policy?, Journal of Education Policy, 20, 3, 369-377 Gorard, S. (2006a) Value-added is of little value, Journal of Educational Policy, (forthcoming) Gorard, S. (2006b) Is there a school mix effect?, Educational Review, 58, 1, 87-94 Gorard, S. and Fitz, J. (2006) What counts as evidence in the school choice debate?, British Educational Research Journal (forthcoming) Gorard, S. and Smith, E. (2004a) What is 'underachievement' at school?, School Leadership and Management, 24, 2, 205-225 Gorard, S. and Smith, E. (2004b) An international comparison of equity in education systems?, Comparative Education, 40, 1, 16-28 Gorard, S. and Taylor, C. (2001) Specialist schools in England: track record and future prospect, School Leadership and Management, 21, 4, 365-381 Gorard, S. and Taylor, C. (2002) What is segregation? A comparison of measures in terms of strong and weak compositional invariance, Sociology, 36, 4, 875-895 Gorard, S., Taylor, C. and Fitz, J. (2003) Schools, Markets and Choice Policies, London: RoutledgeFalmer Halstead, J. and Taylor, M. (2000) Learning and Teaching about Values: a review of recent research, Cambridge Journal of Education, 30, 2, 169-202 Howard, S. and Gill, J. (2000) The pebble in the pond: children's constructions of power, politics and democratic citizenship, Cambridge Journal of Education, 30, 3, 357-378 Mansell, W. (2005) Don't mention the troubles, Times Educational Supplement, 18/2/05, p.16 Massey, D. and Denton, N. (1998) American Apartheid: Segregation and the making of the underclass, Harvard: Harvard University Press Meuret, D. (2001) School Equity as a Matter of Justice, in In Pursuit of Equity in Education, Hutmacher, W., Cochrane D., Bottani N. (eds.), Kluwer Academic Publishers, Dordrecht. Print, M. and Coleman, D. (2003) Towards understanding of social capital and citizenship education, Cambridge Journal of Education, 33, 1, 123-149 Schagen, I. (2002) Attitudes to citizenship in England: multilevel statistical analysis of the IEA civics data, Research Papers in Education, 17, 3, 229-259 Slee, R. (2001) Driven to the margins: disabled students, inclusive schooling and the politics of possibility, Cambridge Journal of Education, 31, 3, 385-397 Smith, A. (2003) Citizenship education in Northern Ireland: beyond national identity, Cambridge Journal of Education, 33, 1, 15-31 Smith, E. and Gorard, S. (2006) Pupils' views of equity in education, Compare, 36, 1, 41-56
|