Memorandum from Professor Carol Fitz-Gibbon
Below are three questions that should have been
asked before Ofsted was allowed to operate. Alas, the standards
needed for research are not seen as relevant to Ofsted. Morally
and economically, however, even more stringent standards should
apply to Ofsted because of the dire effects on the teaching profession
and the huge financial cost.
1. Sampling: What studies have been
undertaken to check that the size and representativeness of the
sample is adequate? For example, how many lessons need to be observed
before the ratings awarded settle down to a stable level? For
how many days/hours/months does a team need to be in a school?
What evidence has been sought to illustrate the effect of the
Ofsted visit being pre-announced? (Would Health and Safety ever
find cockroaches if they announced their visits? In education
the staff would probably be less stressed if visits were not pre-announced.
. . But we need evidence. . . )
2. Reliability: To what extent do
different inspectors come to the same conclusions if working independently
on the same topic? Ditto for teams of inspectors. (Schools certainly
believe it matters who you get. Many inspectors have been fired.
. . did the schools receive apologies for having been sent dud
inspectors? What proportion of all inspectors have been fired?)
3. Validity (the sine qua non): How
does Ofsted know that its conclusions are correct? How does it
explain the fact that it defines a good school as one in which
pupils make better than average progress and yet has called several
such schools "failing"? How does OFSTED know that its
assumptions about what makes a good lesson are applicable, relevant,
accurate, valid? How valid? Predictive validity = 0.9 or 0.5 or
0.2 or zero?
4. Impact (not a research criterion but
socially very important): What evidence has OFSTED for the
effect on education of the entire system of inspection. . . How
can they prove it has not made schools worse? And the crisis in
teaching greater? (They can't. . . it's not entirely their fault
but they should acknowledge the severe limitations on our knowledge
of how to evaluate a school.)
Professor Carol Fitz-Gibbon