Examination of Witnesses (Questions 311-319)
22 APRIL 2009
DR KEN
BOSTON
Q311 Chairman: Can I welcome Dr
Ken Boston to our proceedings. Ken, we have been waiting a long
time to have evidence from you. Your interpretation of your contract
of employment was that when you were under suspension you could
not appear before the Committee. One member of the Committee may
ask you a question about that. You will know that we broke all
previous records by holding a meeting of this Committee in the
summer recess last year, such was the urgency and importance,
we thought, of the tests situation at that time. So you know it
is of concern to this Committee. You know that we have in a sense
wrapped the issue up with the problems that we faced with education
maintenance allowances, which are totally separate and not part
of your remit at all. But there were two systems, both of which
seemed to fail. Independent companies that were hired to do a
job did not seem to be able to deliver satisfactorily. We hope
that this evidence session will allow us to come to some conclusions
and to write up our report, so there is a lot hanging on this
meeting because we have been waiting for such a long time. This
is the only Committee that actually got evidence from the American
company that failed to deliver. That company did not give evidence
to the Sutherland inquiry, but it did give evidence to us, so
we have a particular responsibility in this matter. Ken, we have
known you for a long time in your role. We have had a valued relationship
with you, so we start on that basis, but our job today is to find
out what went wrong and why it went wrong. As with all witnesses,
I am going to ask you if you want to make an opening statement.
Dr Boston: Thanks, Barry. I would
like to make an opening statement, which touches on three things.
I think it might be a useful start. One is the letter that I sent
to the Select Committee.[1]
The second is the terms of reference for Sutherland, because I
think there is a lot to learn from that. Thirdly, I have a couple
of points about the role of observers on the QCA, which I think
has some lessons for Ofqual. Let me begin by saying, in relation
to the letter, that I have resigned and I am not seeking to mitigate
that in any way. Even if Lord Sutherland had been able to get
to all the causes of the failure, and he was not able to get to
all the causes because of his terms of reference, I would still
have resigned because of my part in it. So let there be no question
that what I have to say is attempting to go back over that ground.
The letter I sent to you sets out what I think are two major flaws
in the evidence—quite incorrect and unsound evidence—which
has been used against me. At 2 o'clock this morning on the radio
I heard that the Department had in fact put out a statement saying
that Jim Knight had corrected this evidence some time ago. I was
not aware of that, but I think any correction raises more questions
than it answers. The whole point of my letter is the last sentence,
which seeks an explanation as to why paragraphs 4.92, 4.93 and
4.137 in the Sutherland report are without foundation. There is
no fact to them, and yet they were used against me, to my disadvantage,
in the House of Commons and in this Committee. I want an explanation
for that, and the explanation cannot be simply that Jim Knight
was mistaken about the date on which he thought he met with David
Gee and me, because, as my letter sets out, there was no other
meeting with which it could possibly have been confused. Secondly,
there were no occasions—other than the occasion on 2 June—when
I was pressed by Ministers for answers. You will recall that in
the House of Commons there was some banter about what "usually
pressed" means, and the Secretary of State clearly used it
to mean "pressed on several occasions". There was one.
I want to know why the document was so flawed; why the flaw was
not discovered sooner; why the DCSF official, who took the notes
of the meeting, did not draw the mistake immediately to the Minister's
attention when the transcript of the evidence came in a few days
later from Sutherland, when the error must have been clearly there;
and why evidence given on 14 October last year was not corrected,
to my knowledge, until 2 am this morning. I think that reasons
are required, and an explanation is needed as to what that is
all about. And what does Lord Sutherland think? He was given,
in good faith, false evidence, which he had published. If I were
him, I would be furious, and I would be seeking an explanation.
On the second point, on the issue of the terms of reference of
the Sutherland report, I believe that one of the lessons that
we have learned—because we need to look ahead—is that
Lord Sutherland should have been given a remit to examine the
role of all three partners involved in the development and the
delivery, which are the Government, the QCA, and the contractor,
ETS. His inquiry, as we know, was restricted to the QCA and ETS
alone. Yet the Government, through the DCSF, determine the nature
of the tests to be procured, the pupils who are to be tested,
which subjects they are to be tested in, how and when they are
to be tested and how much money has to be spent on testing. The
interval between the date of the test and the date for results—the
eight weeks' interval—is set by the Government. They also
determine whether the tests are marked manually or onscreen; which
results data are collected and how they are collected; the form
in which the data are required; and how the results are returned
to schools. In those core elements of test development and delivery,
neither the QCA nor the supplier has any discretion—decisions
are made by the Government alone. The development and the delivery
of these tests are not at arm's length from the Government; the
Ministers and the officials are at arm's length only from the
detail of the test questions, and from the marking and level setting.
I believe that the failure in the delivery of the Key Stage tests
in 2008 presented Ofqual with its first opportunity to establish
itself as a truly independent regulatory authority. If the Government
had genuinely wanted Ofqual to assume such a role, and to get
to all the root causes of the failure, it would have asked Ofqual
to establish terms of reference for Lord Sutherland which encompass
the role of all three partners in test development and delivery.
Consistent with Government rhetoric, it should have been beyond
question that Ofqual, with Sutherland as its agent, would account
to Parliament through the Select Committee, not to the Secretary
of State, and report on the roles of all three partners in the
delivery failure. I think that in giving Lord Sutherland specific
terms of reference to look at the QCA management of the ETS contract,
which inevitably would have been the critical part of any Ofqual
inquiry, and in asking Lord Sutherland to report to him—not
Ofqual—on that matter, the Secretary of State narrowed,
not broadened, the scope of the inquiry. His intervention put
a protective fence around the DCSF and Ministers, focused the
spotlight on the QCA and ETS, prevented some major causes of the
failure from being identified and compromised the independence
of Ofqual and its capacity to determine its own affairs. No doubt
the Government wanted to avoid any risk of a public inquiry into
the purpose and use of the Key Stage tests. But I think that Ofqual
should have been trusted, on that point, as a regulatory authority
rather than a body for policy review. It is beyond question that
it could have been relied upon to give Lord Sutherland terms of
reference which focused broadly on the programme. I have no doubt
that decisions made by Ministers and officials in 2006 had a marked
impact on the failure in 2008, most notably staying with manual
marking rather than moving to the phased introduction of onscreen
marking, as with GCSEs and GCEs. Four years after it was recommended
to Ministers by me and others in the QCA, onscreen marking has
become one of Lord Sutherland's principal recommendations. The
further contributory factor, of course, was the question level
data being captured in the marking of the 2008 tests—Lord
Sutherland's references to several thousand clicks being required
for each bundle of papers, you know. My final point, briefly,
is that I think the appointment of DCSF observers to the QCA board,
and other QCA bodies, has undercut the authority of the QCA and
will undercut the authority of Ofqual if we are not careful. By
reducing the formality of the relationship between the two organisations—the
Government and the QCA—it leads to negotiated compromises
from time to time, which erode public accountability. The role
of observers on boards and committees has been to take part in
discussions, to provide advice to Ministers separately from the
QCA and to relay ministerial feedback. It is in that third area
where the problem arises. Observers typically advise in committees
and boards that Ministers would be "minded to" or "not
minded to" agree with this or that proposal or "content
to" or "not content to" agree to a particular recommendation,
or even that, "If I put that idea to the Minister it would
be laughed out of court." Now certainly, prior consultation
with the DCSF is important to inform subsequent decisions by the
QCA on what advice it should offer Ministers, but too often boards
such as that of the QCA are put into a position where it is expected
that they will seek to negotiate that advice in advance. I think
that is a pernicious process that compromises integrity and independence,
and if we are not careful, in relation to Ofqual, it will cause
real difficulties there. I do not think that for Ofqual the runes
are propitious at all. The reality of true independence for Ofqual
will need to be asserted, defended and won. QCA, from time to
time, has been seen as far too independent, and that is one reason
why its wings are being clipped as the QCDA. The greatest risk
for Ofqual at the moment is of being cynically outmanoeuvred by
Government. I think we have to be very careful about that.
Q312 Chairman: That is a very
powerful survey of how you see the situation, and we will now
start some questioning, but can we just clear one thing up. You
talked about Jim Knight's response. We have given you a copy of
a letter received by the Committee—I am not sure whether
you saw it; from what you said, you did not.[2]
Jim Knight admitted that he had made an error in evidence given
to Lord Sutherland; his letter is dated 9 February. Then you have
Lord Sutherland's reply, saying that he did not think that that
made much difference to his report. I just thought that we should
clear that up before we go on to the substance.
Dr Boston: This is the first time,
Mr Chairman, that I have seen this letter. I think that it is
disgraceful that, if I was misrepresented by a Minister in evidence
in a public report, I have not been contacted by the Minister.
Q313 Chairman: So you have never
seen this letter?
Dr Boston: No, I have not. I have
not been given any information on this at all. I think that is
outrageous. If as, just glancing through, I see in the last paragraph
on the first page, the Minister is confusing it with a meeting
that took place on 2 July, and if that meeting is the subject
of paragraph 4.92, that is also wrong. The Minister knew on 26
June that there was a failure. The meeting on 2 July was put together
at his request to talk about where we would go from there, with
regard to releasing the results. It was the first meeting I had
with the Minister along with David Gee, and others were present.
Q314 Chairman: That was with Jim
Knight?
Dr Boston: Yes, with Jim Knight.
We had before us a set of data from ETS that was absolute nonsense.
It showed very clearly that it was going to fail, but even from
ETS we were getting some confusing signals. The president of ETS
in America sent an e-mail the next day, 3 July, as you might recall
from the evidence in Sutherland, saying that he thought that they
were still going to make it. All the evidence we were getting
from the ETS people locally was that they were not. Jim Knight
knew on the previous Friday that this was not going to succeed.
The meeting on 2 July was to sort out where to go from there.
Therefore, that was in my view beyond the terms of reference of
Sutherland. Sutherland was looking at the reasons why failure
occurred and the whole run-up to the failure. At that point, on
2 July, failure was evident and known. The key argument we had
at that meeting, which was very intense, was whether to release
the results. Jim Knight asked me for formal advice on delaying
the results day by a week, and I refused to give that advice.
My advice was that the results available on 8 July should be published,
and as events turned out, 93.5% of Key Stage 2 results and 85%
of Key Stage 3 results were available on the results day of 8
July. A large number of schools, a minority, would still not have
got results for all three subjects at the one time, but my argument
was that letting the results go out would at least allow the majority
of schools to have results, rather than penalising the whole lot
so that they would not be able to complete their evidence. The
Minister stuck with the decision that the results should be delayed,
and I agreed to provide him with advice on the length of that
delay. That is in fact another element of the Sutherland report
that is wrong. It reports—I do not recall the exact words—that
I advised the Minister to extend the date, but I did not. My advice
to the Minister was to stick with the date, and when he asked
how long it should be extended if he insisted on doing so, I gave
him the figure of a week. I remember the 2 July meeting very well.
It is beyond the terms of reference of Sutherland, and the description
of it—it was quite an intense meeting—has no relationship
to the characterisation of my performance or behaviour in the
meeting which is set out in paragraph 4.92.
Q315 Chairman: Can I take you
back. On the one hand, the Committee wants to find out what went
wrong. You, as the QCA, hired this American company, and all of
us agree that it failed to deliver in the way we anticipated it
should, in terms of our hopes for the students taking the exams.
Before I bring you back to that, let us be clear that you are
very angry about how your role in this has been characterised
by the Government.
Dr Boston: Yes.
Q316 Chairman: That is very clear,
and I understand that and you have had three or four months to
think about it. On the other hand, would not the Ministers say
to us that they had asked the QCA to hire an independent company
to provide value for money and deliver the test results but that
did not happen? Whatever we say about which Minister met you and
when, the fact is that the QCA hired these people and they failed
to deliver. What do you say to that?
Dr Boston: I say that is exactly
true. That is why I resigned. I was managing an organisation that
had a difficult task ahead of it—we can talk about aspects
of that—but it failed and I resigned. What I resent is evidence
against me being sexed up in a report by Lord Sutherland, on the
basis of false evidence given by Ministers, to characterise me
as something I am not. I failed to deliver and I resigned. That
is it.
Q317 Chairman: In terms of the
broader brush, though, in your introductory remarks you suggested
that there was something much deeper at the heart of the problem,
and that is that the QCA has been abolished—we now have
two organisations—and the Ofqual that is emerging is weaker
than it should be.
Dr Boston: We were first advised
of the changes being made to the QCA by Ed Balls on 19 September
2007—that he wanted us to split. I support absolutely the
notion of a regulatory authority reporting to Parliament rather
than to Ministers. That has always been a difficulty with the
QCA: it is responsible for maintaining the assessment standard,
the height of the hurdle, yet reports at the same time to the
Ministers who want to drive up the performance standard—the
number of people who leap the hurdle. There is a real inconsistency
there. In my view, and this was my view way back in 2003 and 2004,
it would have been much better either to have hived off the operating
arm, the National Assessment Agency arm, to an awarding body,
or to have set it up as a separate authority, or even to have
abolished it and given an awarding body that function and the
contract, and put the whole of the rest of the QCA, including
its regulatory side and its curriculum side, in a statutory authority
that reported to Parliament and not to Ministers. That would have
been a better solution. Regarding the notion of now taking the
statutory authority role of the QCA away, there has been agitation
there, and that became very clear in the meeting we had with Ed
Balls and Jim Knight about the future of the QCA on 18 March.
One of the great sources of agitation has been the way in which
I have played my role and allowed other senior people to play
their roles, including particularly perhaps Mick Waters, the Director
of Curriculum. The QCA was set up as a statutory authority under
the 1997 Act not to be a critic of government but to be a constructive
participant in educational discussion nationally, and I have seen
it lead part of that discussion. I have seen how it has not been
bound by a set of speeches that were basically constructed around
the Government press releases of the day, and how it has never
publicly criticised Government but has been prepared to shoot
the breeze about other ways of doing things. Notably I, and Mick
in particular, have done that on curriculum and indeed on changes
to assessment and modernisation. Now, the QCDA—the big debate
has been about the right role of the word "authority"
rather than "agency"—will not have that capacity.
It will be virtually an arm of the Department, and my personal
view is, why not incorporate it within the Department? What use
is it going to be as a body that is at arm's length in one sense
but not able to do anything in terms of having a public role?
Q318 Chairman: This is the last
question from me. In your view, the terms of reference of the
Sutherland inquiry were too narrow and excluded looking at the
Department's role in all this.
Dr Boston: Yes.
Chairman: But what about the overall
independent scrutiny of Sutherland? Sutherland, as you know, is,
like you, a highly respected person in the educational world.
Do you not believe that he made a fair appraisal and did a fair
job, given his limited remit?
Dr Boston: I think he did a very
fair job, and I have said that on the public record several times.
I think that it was a very good report, but he was able to look
at the role of only two of the participants, not all three, and
he was fed some information that was wrong. If I were him, I would
be furious that it was wrong, given that he had given two opportunities
through the process for it to be corrected. It now stands on the
record erroneously. If I had been the inquirer, I would be very
angry about that.
Q319 Chairman: You had a good
and quite harmonious relationship with the ministerial team for
five or six years.
Dr Boston: Yes, indeed.
Chairman: So the relationship between
the Department and the QCA is something that you have thought
about and reflected on since. I am trying to tease out when the
unhappiness that you had about that relationship and the changes
that were mooted in the QCA emerged. I am trying to ask you whether
these are the words of a bitter man who feels that he has been
wronged and is picking a fight with the Government, or have you
had these views for some time?
Dr Boston: I have had them since
the decision to set up Ofqual, which I strongly support, but I
have had real concerns about the related decision to dismantle
the QCA. The meeting on 18 March was very tense. I was not the
only one involved; the chairman and another board member from
the QCA were also present. It was very clear that the Secretary
of State and the Minister thought that we were handling our job
in the wrong way. It has been difficult from then on. In fact,
in the whole run up to the test delivery series to the end of
June, I had only two meetings with Ministers—in the entire
year—as my letter says, one of which did not even cover
anything about the national curriculum tests. When I go back through
my diaries for previous years, that is very much at odds with
the previous pattern; during the crucial last eight or 10 weeks
I would have met with Ministers four or five times in previous
years.
Chairman: It is certainly true that when
you gave evidence to our inquiry on testing and assessment you
shared some of your concerns at that time. Can we hold that for
a moment David.
1 See Ev 48 Back
2
See Ev 77 Back
|
