EXAMINATION
OF WITNESSES
(QUESTION NUMBERS
40-59)
KEITH BARTLEY,
MICK BROOKES,
DR JOHN
DUNFORD, MARTIN
JOHNSON, CHRISTINE
BLOWER AND
JOHN BANGS
16 MARCH 2009
Q40 CHAIRMAN:
I recognise that it is well used in commercial organisations as
a management development tool, but do you know if it is replicated
in parallel sectors in the education sector?
DR
DUNFORD: Elsewhere
in the education sector, colleges certainly have a very strong
self-review process.
MARTIN
JOHNSON: I just
want to observe that the whole top-down ethos of the public sector
in recent years militates against that kind of approach.
CHAIRMAN: But
we are using it here.
MARTIN
JOHNSON: That
is because schools continue to resist top-down impositions.
Q41 CHAIRMAN:
Self-evaluation was not the schools' idea, it was Ofsted's, was
it not?
DR
DUNFORD: No,
it comes from him.
JOHN
BANGS: Yes,
it was John and I—it is our fault.
Q42 CHAIRMAN:
I can see the "Wanted" posters. You are saying that
this was not a plot to save money, but that you successfully lobbied
to have a measure of self-evaluation—
JOHN
BANGS: May I
expand on that and give a bit of history. The reaction to Ofsted
in 1992 was so strong—at Crook Primary School, which had
the first Ofsted inspection, the press were on girders looking
into the school with their television cameras—that we commissioned
Professor John MacBeath, who was then at Strathclyde University,
and his team to see whether the Scottish model of school development
and self-evaluation could be used in England. That was the purpose
of the research. His findings were that, yes, it could. That captured
the imagination of the then Chief Education Officer for Newcastle,
who then, I believe, carried that. Separately and independently,
you were coming to the same conclusions, John—but John can
tell his own story.
DR
DUNFORD: That
is right—as part of looking at the new inspection process
and thinking about quality assurance. The way that Ofsted came
in at the beginning, it was about quality control. Industry had
moved way beyond quality control, and was very much into quality
assurance. We felt that in education it was the bringing together
of self-evaluation and external inspection that gave quality assurance.
JOHN
BANGS: Your
question, Chair, is absolutely spot on. There are models in the
private sector, particularly promoted by Deming and a range of
management consultants, that are about owning the product that
you are producing. That is absolutely behind self-evaluation.
KEITH
BARTLEY: It
is important to differentiate between the SEF and school self-evaluation.
The SEF is a very restricted form of school self-evaluation. Those
schools that have the cultures, practices and processes in-built
and well established around self-evaluation are probably those
schools that will have the greatest capacity to respond to the
outcomes of an inspection or, indeed, to the evidence that they
present in a SEF. I think that we need to see it in its place,
rather than assume that it is the process.
CHAIRMAN: Thanks
for that. Let us move on. Sorry to hold you up.
Q43 MR CHAYTOR:
Picking up Keith's point, in terms of the processes other than
ticking the boxes on the form, what is best practice in the process
of self-evaluation?
KEITH
BARTLEY: Do
you mean in preparation for inspection, or more generally in terms
of school improvement?
MR CHAYTOR:
Both really. You are making the point that SEF alone is not enough.
In the case of a successful school, there is likely to be a sound
and solid process. My question is how do we know? How do we evidence
the process? What kind of processes are generally considered to
be good practice?
KEITH
BARTLEY: For
a start, the features of good practice are about schools in which
all of the staff are encouraged to be part of that reflective
process. In other words, the school's model of organisational
development and, indeed, the store by which it sets teachers'
professional learning and continuing professional development
are very much focused on an examination of practice, a reflection
on why practice may be as it is or how it could be changed or
improved, and then some consequent planning on that. If those
features are evident within a school's self-evaluation processes,
they should manifest themselves in improved outcomes for children,
which is vitally important, but they are discernible and inspectable
features of a school as well.
Q44 MR CHAYTOR:
And the form itself—is the form fit for purpose? Or does
it need further refinement?
KEITH
BARTLEY: I want
to defer to my colleagues, who will be closer to that, because
I have neither inspected nor completed one.
CHAIRMAN: Mick,
you have been quiet for a moment.
MICK
BROOKES: It
is a reasonable framework, which is why we are saying that schools
need to take it and shake it down, so that it fits their context,
rather than it being a one-size-fits-all document. Schools that
do that and own it in that way have it as a working tool in a
school, rather than as a document that gets done and put on a
shelf.
Q45 MR CHAYTOR:
Do you have an input? Do the teachers associations or the head
teachers have input into amending the form year on year, or is
it a given and that is it?
DR
DUNFORD: I think
you should ask the Chief Inspector about how the form is constructed.
The one really good thing that I would say is that the Chief Inspector
acts as a gatekeeper against the Department for Education adding
bits and pieces every two minutes to the SEF. It is only changed
once a year, thank heavens. The schools for which I have the greatest
admiration are the ones that have the courage not to complete
a SEF. There are schools that are "outstanding", but
do not complete a SEF and have very rigorous self-evaluation processes.
Q46 MR CHAYTOR:
How do we know?
DR
DUNFORD: Because
in that situation, inspectors have to look at the self-evaluation.
They do not just look at the SEF or the box-ticking exercise,
they have to look at the self-evaluation. The SEF is not self-evaluation,
it is simply a summary, in a sort of tick-box way, of the real
self-evaluation that has taken place. That comes back to the point
that Keith and I made earlier, which John alluded to, that the
best kind of self-evaluation involves all the staff. The inspectors
coming in can recognise that. When they are talking to an individual
teacher of mathematics—not even the head—they will
recognise that self-evaluation culture in the school.
Q47 MR CHAYTOR:
There could be a particularly skilful teacher of mathematics who
is good at talking self-evaluation language. I spoke at a conference
not too far from here, which was set up by an organisation specifically
to train head teachers how to fill in their SEF correctly and
get a good score with Ofsted. These things are not difficult to
do with a bit of training. I am interested in all this stuff about
process. Where is this document? How do we know whether over the
last year—or the last three or five years—the school
has been actively implementing a self-evaluation process?
DR
DUNFORD: Because
the best inspectors go behind the SEF to look at the processes
of self-evaluation that have led to it. In secondary schools,
for example, they talk to the head teacher about the evaluation
discussions that take place every year with heads of department.
They then go and talk to the heads of departments about that,
so they see both sides.
MARTIN
JOHNSON: I do
not have a lot to add, except to David's last remarks. This is
the whole point about the accountability problem. Where there
are high stakes mechanisms, you get negative kinds of reactions.
John is talking about the exceptions that prove the rule. For
too many schools, it is an exercise in form filling and compliance.
For the last quarter of an hour we have been arguing for embedding
a culture of improvement, partly through self-evaluation but not
only that. You will not get that in a high-stakes inspection regime.
MICK
BROOKES: It
is not only the high stakes, it is also the mechanical nature
of it. If the mechanical nature of Ofsted means that someone can
tick the boxes and get the right answers, we can play the game
as well as anybody. Until we have moved from that to a values-based
inspection system where the context of the school is what matters,
there will be people playing the game rather than owning their
own material to move the school forward.
JOHN
BANGS: I think
David has asked a really good question. The fact of the matter
is that if you have an "outstanding", or even a "good",
from Ofsted, you have permission to do anything. You can try out
a set of individual instruments constructed within your school
community—which is what true self-evaluation is about—that
are fit for purpose and stand up to external interrogation about
their validity. Those can be tried out because self-evaluation
is essentially a creative activity. You are finding out information
that you can use, so that you can improve on what you are doing
internally within the school, using the instruments that are fit
for purpose. I have seen some fantastic self-evaluation on a European
basis; for instance, not written self-evaluation, but youngsters
taking photographs of the things that they like and do not like.
It could be films, or small video streaming, or whatever, actually
saying what we like, what we don't and what we think we can do
to improve. The fact is that there are a small number of schools
with that confidence. At the other end, there is the picture of
the head teacher with the moon in the sky, up against their computer
late at night, buffing up their self-evaluation form because the
end of the 3-year cycle is coming up and they know that they have
to do it. There could not be two more stark extremes. Martin put
it very well: it is about getting to the guts of how you embed
a culture with a rigorous sense of how you can improve, knowing
that you own it, knowing that you can improve it, and feeling
professionally empowered to do so, but without the kind of high-stakes
culture that says that someone else who does not know the process
that you have go through is not going to come in and hammer you,
using a delivery system that is entirely conformist in approach
rather than encouraging innovation at school level.
CHAIRMAN: A
quick one from you, Keith. Then we do need to move on.
KEITH
BARTLEY: I want
to make two quick points. First, the study conducted by York Consulting
showed real evidence of the high correlation between the best
SEFs and the best practice in schools and supporting self-evaluation.
It is proven that there is a correlation. The other point that
I would like to make goes back almost to where we started. If
we have an accountability framework that is focused on the impact
of schools' work on children and young people, then schools' self-evaluation
provides an opportunity to reflect upon that broader sense of
the outcomes—the differences—that the school is capable
of making for each child and young person. You cannot do that
in a restricted format.
Q48 MR CHAYTOR:
Does the role of self-evaluation and the SEF have a particular
weighting in the total inspection process? How does it fit in?
How is it integrated into the rest of the inspection?
DR
DUNFORD: Again,
I think that you come back to the proportionality inspections,
and the point that John made. It is different looking at a SEF
in a school that is good or outstanding, where all the trends
are going in the right direction, to a school where things are
badly. I think that is the only way I can answer that question.
Q49 MR CHAYTOR:
Before we leave the SEF and Ofsted, what about the cost? Martin
touched earlier upon that question. Do we spend too much on inspections?
DR
DUNFORD: Perhaps
we would ask the question—
MR CHAYTOR:
We are asking the questions.
DR
DUNFORD: Perhaps
we would answer that question by raising the question of what
is the cost of all of this put together, when you include the
opportunity cost of the time taken when the moon is in the sky,
and filling in the SEF, when you are filling in the school profile—which
we have not mentioned; as part of the accountability system, it
is completely useless—and when you are involved in numerous
discussions, with people coming in asking you about your targets
and so on. I think it is not just about the cost of Ofsted; it
is about the cost of all of those things.
Q50 MR CHAYTOR:
Five minutes ago, you were putting the case for good self-evaluation
processes, saying this is integral to the culture and management
of the school, it takes time, and it involves the head teachers
talking to the classroom teachers. You cannot suddenly say, "Hang
on, there's a cost to that."
DR
DUNFORD: No;
we would probably all say that it is money well spent. Particularly
if it is done well, that is money well spent. If that money is
well spent, perhaps we should put more resources into that, in
order to spend less on the extended inspection coming along and
validating it.
Q51 MR CHAYTOR:
The specific question that follows is, do we spend too much on
Ofsted?
DR
DUNFORD: We
probably spend too much on Ofsted investigating individual schools
and not enough on Ofsted investigating how the system is going
as a whole, which was a point that we made earlier.
Q52 CHAIRMAN:
Why should Ofsted be responsible for all schools? Why should it
not take a few schools?
DR
DUNFORD: How
do you mean?
CHAIRMAN: Why
don't we have a much trimmed down Ofsted that has only a few schools—many
fewer schools?
DR
DUNFORD: You
could have a system that did that if, for example, you relied
more upon school improvement partners, who are having a regular
support and challenge conversation with the head teacher. You
could also use Ofsted less if there was a much better relationship
between inspection and school support, which we talked about earlier,
and you focused more resources on the support aspect rather than
on the inspection aspect and all that goes with it.
Q53 CHAIRMAN:
I asked that because, as I listened to the Laming inquiry discussions
last week, the one question that was left unasked was the fact
that what Laming recommends is enormously expensive in resource
implications. If that is true, somewhere there is going to be
a shift of spend from schools across to other children's services.
I am just wondering whether inspection might be an alternative.
DR
DUNFORD: We
would magnanimously give up some—
CHAIRMAN: I
thought you might say that. Sorry, I cut across and someone was
very frustrated about not getting in there—Martin.
MARTIN
JOHNSON: I just
want to emphasise what I think John was getting round to. I would
be very surprised if this Committee did not look at the costs
of all the agencies involved in both inspection and improvement
work. If you look at Ofsted, the National Strategies, the Specialist
Schools Trust, the SIPs and local authority improvement teams,
they are all doing overlapping work. Maybe it is for you to recommend
how that is rationalised—I have given my take on that—but
it certainly needs rationalisation and savings would accrue.
Q54 MR CHAYTOR:
Each of your organisations has submitted a written statement to
the Committee, and one of them has called for big cuts in budgets
either to Ofsted or any of the other agencies that were referred
to, Martin.
MICK
BROOKES: Without
a shadow of doubt—I have said this to whoever would listen—when
it comes to a choice between front-line services and everybody
who purports to support schools, we would be voting for front-line
services. If that meant putting a greater emphasis on trusts in
schools properly to self-evaluate, with light-touch approval accreditation
of that, we would vote for it.
MARTIN
JOHNSON: We
specifically said in our evidence that we believe that Ofsted
should no longer carry out section 5 school inspections. There
is a saving there.
Q55 MR CHAYTOR:
We have touched on the question of SIPs. No one seems to be dissenting
that SIPs have been a useful innovation. There may be a difference
of view in the approach. Do you think they will be there for ever
or is it temporary?
CHAIRMAN: Are
you all right?
JOHN
BANGS: I was
trying—
DR
DUNFORD: To
get the first word in.
JOHN
BANGS: The idea
that an individual school improvement partner can be this Olympian
character through which advice can go two ways, data can pass
two ways—that they can be the person who provides the judgment
about the individual school to the local authority—I find
extraordinary. I think that SIPs betray, to a certain extent,
a lack of trust in the school. Conceptually, school improvement
partners are wrong. Actually, if you talk to head teachers, they
often talk with fond memory of the original external advisers
who used to come in, because those external advisers, albeit they
were employed by CEA, actually were there to provide the—to
use the parlance—challenge and support to the individual
head teacher alone. Often, those head teachers who have been a
long time in the business will remember that fondly. Probably
one of the best aspects of the old school—the pre-appraisal
scheme in the mid-'90s—was the fact that head teachers and
chairs of governors used to be involved in school appraisal, and
that actually seemed to work relatively well too. It comes back
to the issue about what we spend our money on. The balance between
external evaluation and money for school improvement through professional
development and the identification of individual professional
development is entirely skewed. There is far too little spent
on the outcomes of an appraisal or a performance management evaluation
compared with the enormous weight of external inspection, whether
at local authority or national level. I know we are coming to
the end. We have argued in our submission, and have consistently
argued for the past 10 to 15 years, since Ofsted came in, that
there ought to be an independent review of the accountability
system as it actually bears down on schools. We ought to have
had a national debate about the nature of the accountability system.
We did not have one. It was simply imposed, if colleagues remember,
back in '92. It was a deal done between the Conservative party,
which thought it was going to lose the election, and Labour about
getting a Bill through Parliament. There was no debate at the
time. Essentially, we have a rushed and truncated model of a top-down
inspection system that has gone through various iterations since,
but nowhere have we sorted out how you actually evaluate the institution
as a whole. That seems to me the key issue.
Q56 MR CHAYTOR:
Just one more, before we move on—the question of other initiatives,
such as National Challenge. There was a little furore when the
failing schools were originally identified. Has that settled down
and does anyone now deny that the National Challenge programme
is targeting resources where they are most needed?
MARTIN
JOHNSON: We've
got a thing about the National Challenge. It's lucky Mary Bousted
is not here, otherwise you would have a 10-minute barrage. The
fact is that the challenge in particular, but some of the other
agencies as well, has a not dissimilar effect on many schools—not
the very self-confident ones that have the "outstanding"
badge—as an Ofsted inspection. They create the impression,
perhaps inadvertently—if you talk to the national strategies
people, they declare that of course it is not their intention
and not what they do—that, as perceived in too many schools,
National Challenge is another example of the imposed conformity.
They say they give advice, which they do, but it is perceived
as a demand for a rigid answer on why a subject might be taught.
There is a situation where the QCA has been trying to free up
the curriculum to quite a lot of support—we await what is
suggested for Key Stage 2 to see whether it mirrors Key Stage
3—on the one hand, but the assessment regime is under a
lot of pressure as well. You still have these agencies saying
that is the way to do it—as perceived by schools.
CHAIRMAN: A
quick one from you, John, because we have to get to this last
section on schools.
DR
DUNFORD: National
Challenge is a huge £400 million project that was introduced
with no project planning. You have local authorities told to create
improvement plans over the summer holidays. We had National Challenge
advisers not appointed until November—and they are the key
people in this. We have had the funding only in the last week
or two getting into some of the schools. There is also some very
questionable targeting of the resources in that £400 million—into
some school reorganisations, but also into simply improving the
results of Year 11 in the next two years, not on deep school improvement.
I have huge questions around National Challenge.
CHAIRMAN: A
quick bite from Mick and then we move on.
MICK
BROOKES: It
is an example of how accountability is being misused, sitting
in the Department saying, "Oh woe! All these schools have
been described as failing schools. That is not what we intended."
But it was clearly going to happen. When you have accountability
based on a very narrow spectrum of results, you will get those
things. The concept that you can have a good school working in
a very tough environment moving forward—maybe not as fast
as other people would like—has not been understood by politicians.
CHAIRMAN: School
improvements. Andrew, would you open on that?
Q57 MR PELLING:
I really wanted to deal with the school report card. I apologise
for arriving late; I stayed for the statement in the Chamber.
I also apologise for the fact that I have come to the conclusion
that this debate about school accountability has become so confused
over the years that I would like to return to the idea of some
accountability to the local community and through local education
authorities. Having declared my prejudice, do you think the school
report card has a legitimate and useful place in terms of accountability
for schools? I know that there was consternation about some of
the proposals. How could the proposals be adjusted to make them
better suited?
DR
DUNFORD: We
had a debate about this at the weekend, which you may have read
about in the press. We are not keen on the whole school performance
coming down to a single grade. However, in principle, a school
report card could represent more intelligent accountability. That
depends on the detail; the devil really is in the detail. How
do you measure improvement and progress? What wider achievements
of the school will be brought in? We must have a discussion over
the next couple of years about what those elements will be, how
they will be combined and how they will be graded. In principle,
replacing league tables with a more sophisticated report card
that has been well thought through—with the input of the
profession, parents and other stakeholders—could be useful.
We have set out 10 principles, which I could send to you, about
what a report card should look like. I will mention just one of
them. A good school serving a challenging area should have the
same chance of getting a good grade as a good school serving a
more favoured area. We will judge all the proposals on the report
card against that principle and the others that we have set out.
JOHN
BANGS: To follow
on from John, there is the germ of a good idea in the school report
card. It could be a rich definition of the school's evaluation
checked by an external evaluator. Currently, the school report
card is another damn thing. It comes on top of the Ofsted inspection
result. The consultation document does not resolve the question
it poses itself: what do you do if you have decided, given all
the data, that you have an "outstanding" and Ofsted
comes up with another judgment? The Government refuse to pose
report cards as a substitute for or alternative to school performance
tables. This proposal suffers from the greatest sin of all, which
is cherry-picking a system from one country and dropping it in
elsewhere. It genuinely is cherry-picking because the New York
report card is used by individual schools as a way of arguing
for better funding. That is not part of the consultation. The
United Federation of Teachers agrees with the school report card
in New York because it is a vehicle for negotiations with New
York City about extra funding for schools. I do not see that here.
The idea of a model or framework for describing in a sensitive
way the strengths and weaknesses of a school so that it is understandable
for the whole community is good. However, you must get rid of
the baggage, such as the overlapping accountability systems that
Martin mentioned.
MICK
BROOKES: If
the report card leads to a wiser way of describing and narrating
a school's progress and what it is doing for the community, we
would support it. However, there is a reductionist theory about
trying to get something very simple for parents. As John said
at his conference, there are answers to every complex question
that are simple and wrong.
MARTIN
JOHNSON: I want
to chuck in something very unpopular. The notion that we can divide
our schools into four categories is absolutely bizarre. We are
being consulted on whether we would prefer to use A to D or 1
to 4. It is unfortunate from the point of view of policy making
that the larger the study, the more it becomes seen as the fact
that schools differ in their effectiveness hardly at all. That
is the opposite of an assumption that is made throughout policy
making, but it is the case. I did not manage to get this into
our CVA discussion, but CVA figures only confirm that. Although
there are some outriders, the vast bulk of schools' scores vary
little, when allowing for statistical issues. The idea of the
score card and actually dividing schools into sheep and goats
is fundamentally flawed. I know that my words will go out there
into the ether and be disregarded. In a way, it is counter-intuitive—it
just happens to be the case.
Q58 CHAIRMAN:
Do you also mean that good teaching doesn't do any good?
MARTIN
JOHNSON: There
is quite a significant classroom effect—a teacher effect—but
there is very little school effect. That is a very significant
difference—or should be—for education policy. The
importance of the teacher in the classroom is becoming more understood
but it is still submerged in terms of policy making. I am not
saying anything of which you are not fully aware but, to recap,
a school is a very complex organisation. The idea that you can
summarise, even in a few pages, as John aspires to, what it is
like and likely to be for a range of learners, is just a myth.
As for reducing it to a single digit or letter—that is a
joke.
Q59 MR PELLING:
So Martin is saying that past education policy has been too obsessed
with the idea of school organisation or effectiveness, and that
politicians should concentrate on a classroom, pedagogical level.
MARTIN
JOHNSON: Absolutely.
| 
