APPENDIX 2
Memorandum from Professor Colin Richards
(OAR 02)
Though welcome with its upbeat messages and positive
tone about developments in primary education I would like to draw
the Education Sub-committee's attention to the following weaknesses
which invalidate, or at least render questionable, some of the
claims to improvement in primary education made in the report
and which represent fundamental criticisms of the OFSTED inspection
process:
— the report construes "educational
standards" narrowly—I and many others would say too
narrowly;
— it makes much reference to pupils' "progress",
though the ability of inspectors to make judgements of progress
during inspections is problematic;
— it includes lists of schools which,
it is claimed "have made excellent improvement since their
first inspection", yet the ability of inspectors to report
on such improvement rests on uncertain, fallible evidence;
— it claims that "the quality
of teaching in primary schools continues to improve" but
much of that "improvement" may be spurious—with
teachers increasingly engaged in approved rather than necessarily
improved teaching methods.
1. In reports issued by HM Inspectorate
prior to its demise in 1992 and in reports issued by OFSTED for
some years after standards in the primary curriculum were
broadly conceived and related to children's attainments across
the full range of subjects of the national curriculum, religious
education and any other curricular areas taught. Standards were
judged drawing on a wide range of evidence, in particular evidence
gathered as a result of first-hand observation.
2. In this year's Annual Report as in last
year's standards in English, mathematics and science in
primary schools are construed much more narrowly—solely
in terms of pupils' test results at the end of key stages. Inspectors'
own judgements of standards in these and other subjects are not
reported at all. Their judgements count for nothing in the Annual
Report. Does OFSTED not trust their judgements? Why then should
OFSTED (or the rest of us) trust their judgement in other areas
inspected?
3. In its report of foundation subjects
in the primary curriculum OFSTED makes much reference to the progress
pupils make as a criterion for judging improvement. In judging
progress inspectors are required to assess "the rate, breadth
and depth of learning in each year based on the gains in knowledge,
skills and understanding pupils make in lessons and over a period
of time". Such assessments are made in the basis of observation
and scrutiny of pupils' work.
4. Assessing progress in individual
lessons involves at least two sets of judgements: (a) judgements
of pupils' knowledge, understanding, skills or attitudes at the
beginning of a lesson; and (b) judgements of these at the conclusion
of a lesson. Usually inspectors cannot gain detailed, informed
knowledge of either (a) or (b) because of the constraints of time
and therefore cannot gauge the degree of change in pupils' knowledge,
understanding etc. Of course we hope and believe that most pupils
do make progress in lessons but this is not normally detectable
to any significant extent through observation by inspectors. It
might be assessable if inspectors had chance to question pupils
closely before and after a lesson but such conditions rarely obtain
in the frantic conditions of inspection. The judgements about
progress which Ofsted inspectors are required to make and
on the basis of which Ofsted creates so much data are inferences
only, based mainly on judgements of the quality of teaching and
the quality of pupils' observable response. Has OFSTED ever tried
to attempt to establish the reliability and validity of such inferences
before including judgements of progress in its Annual Reports?
I fear not.
5. Equally problematic are judgements of
progress made over time on the basis of a scrutiny of children's
work from different year groups. When required to make judgements
of the progress children make over the course of their primary
education in a particular school inspectors have to resort to
samples (only) of the work of different year cohorts of pupils
at any one time, not extensive samples of work of the same cohort
of children over time. In such scrutiny like is not being compared
with like; the populations are likely to vary in a variety of
ways. Scrutiny of pupils' work, usually conducted hurriedly and
superficially after school one evening, is a totally inadequate
basis for judging the progress children make through school. This
throws great doubt on the weight OFSTED places on its overall
judgements of progress, as evidenced by copious references in
the 1998-99 Annual Report.
6. The publication of a long list of "improved"
schools in the Annual report testifies to OFSTED's belief in the
ability of inspectors to report validly and reliably on improvement
from one inspection to the next. To validly judge the extent of
such improvement requires at least the following: (a) in-depth
knowledge of the school at the time of its first inspection; (b)
in-depth knowledge of the school at the time of its second; (c)
the same criteria employed on both occasions; and (d) the criteria
employed in the same way on both occasions. In reality published
inspection reports, even when complemented by statistical information
cannot provide the detail required for (a) and (b). There have
been very significant changes to the inspection criteria over
time, and changes in the instructions given inspectors on how
to apply them, which render inter-inspection judgements highly
questionable. Finally OFSTED has no way of securing or guaranteeing
(d). These objections plus the shortcomings in at least a proportion
of inspections during the first cycle render very problematic
OFSTED's claim to be able to report improvement from one inspection
to the next.
7. The Annual Report makes much of the improvement
of teaching in primary schools. If that improvement is real, then
it is right to do so. It could well be that as a result of adoption
and adaptation of both the literacy and numeracy strategies teaching
has actually improved. My hunch is that it has, but by how much?
Part of the reported improvement has resulted from a clear national
steer as to what constitutes approved teaching methods; from inspectors'
preoccupation with looking for, and finding, the features of those
approved methods; and from teachers' willingness to go along with
those approved methods partly at least to give the inspectors
what the teachers know they want to see and thus secure more favourable
grades and a better published report. Because of the "inspection
effect" it is impossible to know whether the reported improvement
is genuine, spurious or, more likely, a mixture of both.
Professor Colin Richards
February 2000
| 
