THE SCHOOL CALCULATION
Simply put, a pupil's CVA score is effectively
the difference between a set of results predicted according to
the CVA model and the results actually achieved. A school adding
value will enable the pupil to outperform the results expected
of him or her according to the CVA model. In order to calculate
the CVA score for a school as a whole, the average CVA
scores of its pupils is taken, then an adjustment is made for
the number of pupils in a school's cohort (the shrinkage factor).
92. This provides a measure of school effectiveness.
The Government said that:
CVA is a powerful tool for analysing school performance,
but it is a relative measure, only effective when seen in combination
with other factors, including raw scores, value added based on
prior attainment, school self-evaluation, inspectors' judgements
and the content of the school profile. It shows a school's past
performance in relation to other similar schools, but cannot project
performance into the future. Hence it cannot be used to set future
targets.[140]
93. The CVA measure is only an estimate, even within
its own, limited terms. The score is based on a prediction which
is, in turn, based on the actual attainment of a pupil in a given
exam on a given day. On another day with the same pupils, a school
may well have achieved somewhat different results. This degree
of uncertainty is reflected in the confidence interval, which
is provided in the performance tables alongside the school's CVA
score. The confidence interval is, essentially, the range of scores
within which one can be statistically confident that the "true"
school effectiveness (according to the model, at least) will lie.
It gives a measure of the uncertainty inherent in a school's CVA
score and the size of the confidence interval will be determined
by the number of pupils in the calculation.
CVA information for each school is presented on the
DCSF website, illustrated by the following example:

Source: DCSF website; Achievement and Attainment
Tables.
94. The major issue which arises out of CVA is, as
with performance tables generally, that it is not readily understandable
to the layman, and parents in particular. It is not clear from
the table above what the practical difference is between a school
with a Key Stage 1-2 score of 99.5 and a school with a score of
99.9. In fact, other tables available on the DCSF website show
that the absolute results for School A are low, whereas the absolute
results for School B are high. The implication of the CVA scores
is that both schools are similarly effective, albeit with very
different intakes (School A has a high number of SEN pupils, School
B a relatively low number, according to yet another table). However,
this interpretation is not obvious unless one undertakes a thorough
analysis and comparison of several tables together. It follows
that the intended use of CVA scores, to place the absolute results
in context, is diluted because many do not know how to interpret
them.
95. We put this concern to the Minister. He told
us that:
I do not think that it is that difficult to understand
that in CVA terms, 1,000 is the norm. If you are above 1,000,
you are adding value better than the norm. If you are below 1,000,
you are adding value lower than the norm. If that is all people
understand, then it is pretty straightforward.[141]
Mr Tabberer added that, in publishing performance
tables, including CVA scores, the Department is "following
the principle of being transparent about all of the analyses so
that parents can access the information that they understand or
the information that they want". He said that the Department
tries to ensure that the public can see comparators and benchmarks
and that CVA scores were considered a fair means of comparison.[142]
We do not take issue with this, although we have already noted
the limitations of any kind of evidence based on test scores alone.
We cannot agree, however, that the meaning of CVA scores, as they
are presented in the Department's own performance tables, is by
any means obvious.
96. We consider that CVA scores are important because
there is a strong correlation between the characteristics of a
school's intake population and its aggregated test results and
CVA attempts to make some compensation for this. Schools with
an intake of lower-performing pupils will do less well in the
performance tables of raw scores than schools with an intake of
higher-performing pupils.[143]
Mick Brookes put it bluntly, claiming that performance tables
simply indicate "where rich people live and, sadly, where
poor people live as well".[144]
Dr Mary Bousted, General Secretary of the ATL, told us:
Variation between schools is much less than variation
within schools. We know a school can make about 14% of the difference;
the rest of the determining factors on a child's achievement come
from their background, actually. And 14% is a lot; I am not undermining
what a school can do. However, that means that schools in the
most challenging areas have to work extremely hard to get the
results that they do.[145]
97. CVA scores, then, go some way towards levelling
out these inequalities. However, they are not a transparent measure,
and it is not easy to judge the validity of the variables used.[146]
CVA is still a relatively blunt instrument for making comparisons,
based as it is on the limited dataset of test results and on a
series of assumptions and generalisations about a pupil's background.
Nigel Utton, Chair of Heading for Inclusion, illustrated this
point:
In the case of my own school, by removing two
children with statements for educational needs from the statistics
we move from being significantly below the national average to
being within normal boundaries. If the contextualised value added
measures were sufficiently sophisticated, that would not be possible,
as it would weight children with SEN to factor out such a discrepancy.[147]
98. Whilst we consider that Contextualised Value
Added scores are potentially a valuable addition to the range
of information available to parents and the public at large when
making judgments about particular schools, we recommend that the
information be presented in a more accessible form, for example
graphically, so that it can more easily be interpreted.
99. We are concerned about the underlying assumptions
on which Contextualised Value Added scores are based. Whilst it
may be true that the sub-groups adjusted for in the Contextualised
Value Added measure may statistically perform less well than other
sub-groups, we do not consider that it should accepted that they
will always perform less well than others.
100. In addition to these specific recommendations
about Contextual Value Added scores, we recommend that the Government
rethinks the way it publishes the information presented in the
Achievement and Attainment Tables generally. We believe that this
information should be presented in a more accessible manner so
that parents and others can make a holistic evaluation of a school
more easily. In addition, there should be a statement with the
Achievement and Attainment Tables that they should not be read
in isolation, but in conjunction with the relevant Ofsted report
in order to get a more rounded view of a school's performance
and a link to the Ofsted site should be provided.
101. We have received some evidence that Ofsted places
considerable weight on test scores when making judgments about
schools under the new, lighter touch, inspection regime. The NUT
said that Ofsted relies on CVA as the baseline measure for school
evaluation.[148] Heading
for Inclusion similarly stated that Ofsted inspections:
[
] focus almost entirely on a school's
ability to produce high results in tests at various stages, whether
they be Key Stage SAT results or GCSEs. This has led to schools
devoting much of their time to 'playing the game' and teaching
the children to pass the tests.[149]
Other witnesses also expressed concern that Ofsted
uses information in the performance tables as key inspection evidence.[150]
Cambridge Assessment made the point that the new Ofsted inspection
regime is far more dependent on national assessment data than
previously. Although Cambridge Assessment stated that the new
regime has been broadly welcomed by schools, it argued that the
regime fails to take into account the essential weaknesses in
these data.[151] The
IPPR gives a measured assessment of the situation:
[
] the results of national tests are a
critical input into Ofsted inspections, and a bad inspection may
result in a school being issued a notice to improve, or risk being
placed in special measures. Entering special measures means that
a school loses its autonomy and represents a severe criticism
of the leadership of the school. [
]
It is quite right that there should be a robust
inspection mechanism to provide schools with powerful incentives
to improve, and especially to ensure that no school falls below
a minimum acceptable standard. However, if test results are to
play an important role in such a powerful incentive mechanism,
it is all the more important that they are robust, valid, and
do not negatively impact on other desirable aspects of the learning
environment.[152]
The IPPR added, however, that it is important not
to overstate these arguments and that Ofsted does take into account
a wide range of other factors in its inspections.[153]
102. The scope of this inquiry does not extend
to a thorough examination of the way Ofsted uses data from the
performance tables under the new, lighter touch, inspection regime.
However, we would be concerned if Ofsted were, in fact, using
test result data as primary inspection evidence in a disproportionate
manner because of our view that national test data are evidence
only of a very limited amount of the important and wide-ranging
work that schools do.
103. So far, we have considered objections to performance
tables, including CVA measures, based on arguments that they are
not a valid measure of the performance of schools judged across
the full range of their activities; and they are not readily understandable
by those who may wish to use them, especially parents. However,
the most serious and widespread objection to performance tables
is, as with performance targets, the distorting effect that they
have on the education which takes place in schools.
104. The Government states that the performance tables
are "an important source of public accountability for schools
and colleges".[154]
The use of performance tables for school accountability means
that a school's standing in the performance tables is a matter
of significant importance to that school, directly or indirectly
affecting the morale of pupils and teachers; the attitudes of
parents; the school's standing in the local community and within
the wider local authority; the resources allocated to it; and
perhaps even the school's very survival. The stakes, as many witnesses
have pointed out, are high.[155]
105. The evidence we have received overwhelmingly
suggests that these high-stakes lead to serious distortion of
the education experience of pupils (and see further Chapter 4):
teaching to the test, narrowing of the taught curriculum and disproportionate
focus on borderline students.[156]
Witnesses have commented that the use of performance tables as
accountability measures has had the effect of "undermining
good practice in many classrooms"[157]
and has encouraged a "risk-averse culture"[158].
Performance tables "depress and demotivate teachers who struggle
to make children achieve grades they are not quite ready for".[159]
The NASUWT told us that the practical effect of performance tables:
[
] is to contribute to a skewing of the
curriculum, generate unacceptable levels of pressure and workload
at school level and entrench a competitive rather than collaborative
culture between schools. They are also responsible for many of
the pressures that inhibit the ability of teachers to exercise
an appropriate level of professional discretion and autonomy.[160]
Professor Tymms argued that:
We are forcing teachers to be unprofessional.
League tables are an enemy of improvement in our educational system,
but good data is not.[161]
106. We consider that schools are being held accountable
for only a very narrow part of their essential activities and
we recommend that the Government reforms the performance tables
to include a wider range of measures, including those from the
recent Ofsted report.
107. We have considered in this Chapter some of the
issues which arise from the use of national test results for the
purposes of accountability and monitoring of schools through performance
targets and tables. In the next Chapter, we shall consider in
more detail some of the unintended consequences of this regime
and suggestions for radical reform of the accountability system.
93