Examination of Witnesses (Questions 320-339)
22 APRIL 2009
DR KEN
BOSTON
Q320 Mr Chaytor: There are two
separate issues here. One is the relationship between yourself
and the QCA and Ministers in the context of the appropriate degree
of independence for the QCA and its successor bodies. The other
is the responsibility for the events that led to the failure of
ETS to deliver the tests on time by 8 July. You are trying to
conflate the two and to suggest that there was a hidden agenda
that led the Sutherland inquiry to focus only on your responsibility
in not delivering the contract to time. In retrospect, do you
not accept that there was a responsibility on the QCA and the
NAA, as the agency directly responsible for supervising the contract,
to inform the Government at an earlier stage of the scale of the
risk that you had identified? You said in your evidence last year,
and you say again in your letter,[3]
that there had been weekly meetings from September with ETS and
daily meetings from the beginning of May. Surely that is exceptional—if
there was no problem or if the problems were being dealt with,
why was there a need to have daily meetings from 1 May?
Dr Boston: No, that was not exceptional.
That was in fact the plan, and it was very good management and
was similar to previous years. The daily meetings in the final
few weeks are the norm. It was not because there was any particular
crisis. As I have said in evidence before, right up until late
May and the start of June there were some problems with getting
papers out to markers. There were problems with distribution and
return. They were fixed, and by 6 June 50% of the marking had
been done. We were on target, just as we had been in previous
years. The signs then were very good. It was not until, in my
case, 25 June that it became clear that we were not going to make
this.
Q321 Mr Chaytor: Given the scale
of the problems described by yourself and by Sutherland and acknowledged
by ETS in its evidence to the Committee, it seems incredible that
as late as 24 June there could have been a general acceptance
that the target date of 8 July was going to be met. You refer,
for example—I have the transcript of your evidence last
year—to this backlog of 10,000 e-mails that ETS had not
responded to. This led to an identification of the support provided
by the NAA. This is exceptional, is it not? Which organisation,
operating normally, has a backlog of 10,000 e-mails?
Dr Boston: Yes. The Department
was fully aware that that was the backlog, fully aware that we
were coping with it, fully aware that we had put in resources
to deal with it. It was a problem, and we were working to fix
it. In fact, the backlog was removed. It is not a matter of the
Department not being aware of issues from day to day. The contact
we had through the NAA and elsewhere was very close. Department
observers attended the meetings we were having with ETS and others.
The broader question is that every year this has been a high-wire
act. It is to do with the extraordinarily large scope of the job,
done manually, and packed into eight weeks. We failed in 2004
when Key Stage 3 was late. We were within hours of failing in
2005, even though we had extended the Key Stage 3 results date.
In 2006 and 2007 we were fine. In 2008 the whole thing collapsed
again. The enterprise, the whole archaic nature of this thing,
is incredible, and we are persisting with it for 2009. Lord Sutherland
has made recommendations about end-to-end testing which simply
cannot be met for 2009. Key Stage 3 is not being run this year
but that does not halve the risk. It simply means that there will
potentially be more markers who will be able to do Key Stage 2.
In the general qualifications where the major awarding bodies
have applied technology, are doing onscreen marking, producing
valid and reliable results and identifying aberrant marking quickly
and delivering a top-quality product securely, an enormous achievement
has been made. Key Stage 2 and Key Stage 3 is still the cottage
industry it was in 2002. The only change is that papers are now
no longer left on doorsteps and post office counters—they
are barcoded and tracked. That is the only change that has occurred
in this period.
Q322 Mr Chaytor: You said earlier
to the Chairman that you had recommended four years ago that onscreen
marking should become the norm for Key Stage 2 and Key Stage 3.
What was the Minister's response to that recommendation?
Dr Boston: There was extensive
discussion with Ministers and officials particularly over a long
period of time about introducing onscreen marking. We wanted to
introduce onscreen marking for security, better quality and greater
validity of results. The Department was pushing continually for
a trial that would have had some papers marked manually and some
marked onscreen and, provided that the results were the same,
onscreen marking would be gradually phased in. The whole point
is that the results would not be the same because the onscreen
results would be better. I do not know whether that would make
the league tables look better or worse, but the one thing you
can absolutely bank on is that you will get a different, better
and more valid result with a process that identifies aberrant
marking and other errors early, does item level data collection
automatically and eliminates all that manual stuff. You will get
much better results, but they will be different. The compelling
argument, finally, behind closed doors where all of this was thrashed
out was, "Well, it's too risky—the results might look
different."
Q323 Mr Chaytor: Can I ask also
about the role of the DCSF observers, because your argument is
that they were involved with the NAA and the delivery of the contract
at every stage? They were present at the weekly meetings from
September. They were present at the daily meetings from 1 May.
Why did they not report to Ministers earlier the scale of the
risk and the likelihood of failure?
Dr Boston: I cannot answer that.
I don't know. But it is true that they were present at every level,
including board level. They were aware fully of decisions being
made and in fact were participating and contributing to that.
The ETS contract was not even signed by the QCA chairman until
we had the then Secretary of State's approval for it to be signed.
She was being briefed by officials on, presumably, where this
whole thing was heading throughout. I am not objecting to that,
nor am I absolving the QCA from responsibility. It was the management
agent. I am not saying that we should not have taken the rap on
this. It happened on our watch, but it cannot be said that it
came out of the blue or that the warning signs of continued briefings
were not making it apparent when difficulties arose.
Q324 Paul Holmes: I have two questions.
First, in the letter that you sent to the Committee on 15 April[4]
you say, "I was not asked to meet directly with the Schools
Minister in the months leading up to the delivery failure."
We have already talked a bit about that. Why were you not asking
to meet the Minister? Why had you not spotted that problems were
emerging? You have just said that it did not become apparent until
26 June. How come you did not spot that at an earlier stage?
Dr Boston: We were certainly spotting
the problems and dealing with them. As I said in my evidence to
Lord Sutherland, it is a fraught process. There are difficulties
the whole time. Our approach was to roll up our sleeves and get
on with solving it, rather than running round telling people that
there was a problem. If I had had a meeting with Jim Knight every
week for the last eight weeks in the run-up to the closure, it
would not have made him accountable for the tests. It was still
my accountability. It would not have helped simply to put problems
on his desk. Our objective was to keep him and his people briefed,
but solving the problems and addressing them and bringing the
tests in. We have done that year on year on year, except in 2004
and 2008. It has worked before. Our anticipation was that it would
work again, but the risks in this are so high. Sutherland is critical
of the QCA board having the tests at an amber risk on its risk
register for 14 months. Amber means that likelihood is possible
and the impact would be very high. I cannot see these tests ever
having a risk rating less than amber. They could always fail,
given the eight weeks in which the delivery occurs when they are
done manually. And the impact when they fail is—well, we
saw it in 2008.
Q325 Paul Holmes: We will come
back to the lines of accountability in a minute. Given that you
are saying that it is always on a knife edge because of the sheer
volume of tests and the short time scales, why were you not banging
alarm drums much earlier, both last year and the year before,
and the year before for that matter? Both you and I were at an
educational seminar—this was in the early stages of the
development of diplomas. When we were chatting, you said you were
worried about what was happening with the development of diplomas,
and that you thought the Government were being too complacent
and that there was not enough feed through from those people developing
diplomas to the schools and employers. You asked me to urge the
Committee to look at that. I think you also talked to the Chairman
separately and probably to other people. The Committee looked
at the matter at that stage, and it did gee the Government up
to get their act together a bit more. In that case you were really
on the ball. Because you were not getting the response that you
wanted from the Government, you used the Committee to try and
push things along. If you were very much on the ball in the early
stages regarding concerns about diplomas, why if year after year
the marking was such a problem were you not banging the drum year
after year? If you were not getting anywhere with the Ministers,
why did you not raise the matter more widely?
Dr Boston: That is a good question,
and it is one that I have asked myself. I have frequently spoken
publicly about different approaches to testing and the problems
with this particular approach and how it was administered. I have
also pressed privately for reform with Ministers and officials—there
is no question about that. Over the past few months I have asked
myself why I did not resign in 2006. At that point it was clear
to me that I was not going to get Government buy-in to necessary
reform. I continued through 2007 and 2008 to run an organisation
that was faced with close to an impossible task, which I had seen
go belly up in 2004 and had the prospect of doing so again. Probably
my greatest regret in all of this is that I was not more honest
with myself at that time and did not simply say, "It can't
be done."
Q326 Paul Holmes: Is there a wider
institutionalised problem about lines of accountability in general
between the Government and different arms of business that are
being hived off? For example, in this case a private company,
ETS, was brought in, and it was a disaster. We had a similar thing
some years ago with the delivery of individual learning accounts
and Capita and so on. We get private companies, and the Ministers
say, "It's up to them now—don't ask us about it, ask
them." The companies say that it is commercial confidentiality
and it all collapses and is a disaster. That seems to happen over
and over again. My second point concerns quangos, whether it be
the QCA or NAA or whatever. We have seen the Learning and Skills
Council make a total disaster of sixth-form funding allocations
and of funding for college buildings. However, the Minister keeps
saying, "It's nothing to do with us, you'll have to ask them."
Have we got an institutional problem, which has developed over
the last 10, 15 or 20 years, about what the lines of accountability
are between the Government and the private sector and quangos,
who are delivering things further down the system?
Dr Boston: Yes, I think we do.
From my experience, I certainly think that there is a problem.
Looking forward, that is a critical reason why Ofqual must be
completely separate from Government and report to Parliament through
the Select Committee. The QCA's problem has been that it is not
at arm's length—to use the Government's term—yet it
can end up carrying the blame for a whole set of decisions over
which it had no control. In my statement I pointed out all the
things to do with national testing that the Government determine,
and it is virtually everything except what the standard is and
responsibility for the marking. If one goes further, and looks
at awarding bodies, one sees that the delivery relationship works
effectively because awarding bodies are quite remote. They do
not work with Government even on a contractual basis. They deliver
a public service and are funded through schools to deliver that
service, and it works. Distance, with Ofqual, is going to be critical.
If it does not get that distance, there will be a problem. As
I said, I believe that Ofqual was made to stumble at its first
hurdle. That is not an auspicious sign for the future of independent
regulation.
Q327 Mr Stuart: As you have described
it, the system effectively has been specified by Government—they
are the architects of the system in almost all its components,
except for the content of the exams and the style of the marking.
We have a constitutional system, where Ministers are always responsible.
You said that if you had met every week with the Schools Minister,
he would not have been accountable—you would have still
felt accountable. To follow up the last question, do you feel
that the constitutional convention of Ministers being ultimately
responsible for delivery, particularly when the system is not
genuinely independent, has failed to be honoured in this instance?
Dr Boston: Yes, I do. There comes
a point where you have to accept your accountability. From my
own point of view, I accept the accountability and I accept the
responsibility. On matters of detail, I was not at fault, because
I was not doing those detailed things, but I was responsible for
the management of the people who were delivering them and for
the overall result. It would seem to me that a time does come
with all Ministers—over my career, I have worked with more
than 40 Ministers—when a Minister has to accept accountability
and go.
Q328 Mr Stuart: The evidence that
Sutherland quotes, particularly paragraph 4.137, says: "In
practice in 2008 what happened was that DCSF observers escalated
their own assessment of risks to the DCSF ministers on a number
of occasions. On this basis, ministers usually pressed the QCA
Chief Executive for answers." Is that an accurate account
of what happened?
Dr Boston: No, it is totally false.
I was pressed, if asking a question is being pressed, by Ministers—Ed
Balls and Jim Knight—on 2 June, at the start of a meeting,
as the first agenda item in a meeting that covered a series of
agenda items. I am aware that in the House of Commons two Members
sought clarification of what "usually pressed" was,
and the Secretary of State referred to that as meaning "frequent
pressing"—those were not his exact words, but that
was the clear implication.
Q329 Mr Stuart: So Ministers'
role in the system was to ensure, the Schools Minister told Sutherland,
that "the reality clearly is that some of these tasks that
they are doing are so mission-critical that we need alongside
that arrangement to be satisfied for ourselves that things are
going well." That was the responsibility on Ministers. Is
it your evidence to us that they falsely maintained to the House
of Commons that they had carried out that duty when in fact they
had not done so?
Dr Boston: I am not certain about
that, because I am not privy to the amount and detail of briefing
that Ministers were given by DCSF observers at our own meetings.
I cannot really comment on that. Presumably, the briefings were
sound. That is why I find it so curious, for example, that the
senior DCSF official who took the note of the meeting of David
Gee and Knight on 17 June did not draw the Minister's attention
to this mistake a few days later when the transcripts came back
from Sutherland. The relationships between Ministers and DCSF
officials over this I cannot fathom.
Q330 Mr Stuart: I know that it
was a complex set of circumstances that led to the failure of
delivery. However, do you believe that if the Government had accepted
your advice and brought in phased onscreen marking rather than
manual marking the deadlines would have been met?
Dr Boston: I am certain of it.
The evidence for that is what we see with the awarding bodies
and the general qualifications. There is no question that these
tests could be delivered effectively and well within eight weeks
using technology, giving us much more valid and reliable results
than we get at the moment.
Q331 Mr Stuart: So the chief author
of this failure, in your opinion, was the Department, because
of the specification that it set for the way that these tests
were to be assessed. And, in the evidence to the Sutherland inquiry,
the Department failed to accept that responsibility.
Dr Boston: I certainly believe
that, if we had been successful in introducing onscreen marking
several years ago, 2008 would not have happened.
Q332 Chairman: I have just listened
to your replies to Paul Holmes and Graham Stuart. Are you not
trying to have it both ways? On the one hand, you are complaining
that the record is not straight and that you hardly ever saw Jim
Knight or Ed Balls. In other words, you were left alone to get
on with the job. On the other hand, you are saying that there
should have been greater ministerial responsibility, because when
everything went wrong Ministers should have said, "Hands
up, it's all our fault, because of decisions that we made, or
failed to make, some years ago." So let us get this straight.
You are complaining: you said that Jim Knight made a mistake,
saying that you were at a meeting that you were not at. It seems
to me that someone could interpret what happened as the Government
saying, "Get on with the job, Ken. Hire someone to do this
job. You are responsible. You're a good guy and you've got a good
reputation, so get on with it." Actually, Ministers do not
really get involved until everything seems to be going wrong.
That is true, is it not?
Dr Boston: Yes, that is certainly
true. Ministers were not getting involved in the detail of it
at all. Look, I don't think I am trying to have it both ways.
I have very clearly fallen on my sword over this. I have taken
responsibility, taken accountability, and that is the end of it.
Q333 Chairman: But I am talking
about the ministerial responsibility, Ken. On the one hand, you
are saying that the Ministers were never there, because they left
you to get on with the job, and now you are saying, "But
we need clearer ... ", and we need Ministers to stand up
and say that they got it all wrong.
Dr Boston: If I had met weekly
with Jim Knight, I think that this problem would still have emerged.
I am not saying that his failure to meet regularly was the cause
of what happened. I take your point. However, I certainly do not
think that I am, in any way, trying to diminish, reduce or explain
away my own responsibility. I was foolish enough to continue running
an organisation doing this high-wire act, when I knew fundamentally
that it was deeply flawed and that the only way that I could have
made it secure was to get ministerial agreement to do so several
years ago.
Q334 Fiona Mactaggart: When did
you first learn of the public account by Jim Knight that was wrong,
and what attempt did you make to correct it? Was it the letter
that you wrote a few days ago to our Chairman?[5]
Dr Boston: Yes.
Q335 Fiona Mactaggart: Why?
Dr Boston: I believed that I was
unable to do anything prior to that. Let me say that I first became
aware of this error—the flawed evidence—on 11 December,
when I had a pre-release copy of the Sutherland report to read
in the Department, and these three paragraphs just stood out as
absolutely flawed. I resigned the following day. I would have
resigned even if those three paragraphs had not been there. On
the following Monday, 15 December, the day before the Sutherland
report was released, I was then suspended by the QCA board at
the proposal of the DCSF permanent secretary. I have contested
that suspension, or reserved a right to contest it, because I
believe that I simply should have been put on gardening leave,
rather than receiving some sort of ill-defined suspension. I certainly
was astonished to be suspended before the report had even come
out and before anyone had even read it. Once that suspension had
occurred, I received a letter setting out what a suspension meant.
The letter did not explicitly state, "You can't speak to
the Select Committee or write to newspapers," but that would
clearly have been inconsistent with the spirit of the letter,
which set out that I was not to contact the QCA or do a host of
other things. I simply felt that the proper thing was not to make
a media issue or anything else out of it, but to come back to
the Select Committee in due course when the suspension was lifted
and deal with the issue in the forum in which the faulty evidence
was used against me. That was my decision.
Q336 Fiona Mactaggart: So the
correction that Jim Knight made in February to Lord Sutherland
and to this Committee was not because you raised any query; it
was just done by him.[6]
Dr Boston: I am uncertain about
that. The QCA board had to set up a panel to conduct a review
in light of the Sutherland report, and that panel met on 14 January.
On that date, I gave the panel, in a confidential meeting, an
account of my performance, as well as an analysis of it in relation
to the Sutherland report. The QCA panel, and subsequently the
QCA board, then became aware that I had not been at that meeting.[7]
The panel then contacted the Department to obtain a copy of the
note of the meeting, which confirmed that I was not in fact present
and had not made a contribution. It seems clear to me that that
action, and the QCA board getting on to the case with the DCSF,
must have led to this letter being written.[8]
As I have said, I have been totally unaware of this letter until
today, and of the fact that there had been a statement until 2
o'clock this morning. I think, in all fairness and decency, that
I should have been given at least this letter and an explanation
as to why I was so misrepresented.
Q337 Fiona Mactaggart: That is
a reasonable request, but one of the things that you have alleged
in your evidence to the Committee today is that Ministers—in
the phrase you used—"sexed up" the issue. I am
slightly tempted to use the same phrase to describe the way that
you have dealt with it, because you have made much of the "fiction"
of Ministers. But as you now know—I am quite prepared to
accept that you did not know this when you started—Ministers
had made a correction to the Committee and to Lord Sutherland.
I am concerned that, although you have raised some serious issues
before the Committee, they might be designed to, if you like,
divert the attention from the very substantial criticisms of your
role in the Sutherland report.
Dr Boston: Let me say this: if
there has been an edge to my remarks, there is a reason for it.
I gave evidence to Sutherland on 18 September 2008. On 24 September,
which was six days after I gave evidence, three weeks before Balls
and Knight gave evidence and nearly three months before the Sutherland
report was published, which was certainly before Sutherland had
even put pen to paper, the permanent secretary sought to negotiate
my early resignation on the grounds that the Sutherland report
would be bad for me. The Ministers had not even given their evidence.
The permanent secretary said to me on that date that he wanted
me out of the job before March 2009. I engaged an employment lawyer
who wrote to the permanent secretary. I made it clear that my
intention was to remain in post until the end of the contract
and that I would be willing to consider departure only on the
basis of the contract being paid out. That was, in fact, the end
of the correspondence—once the permanent secretary had replied
to Lewis Silkin. The Ministers' evidence came after that—Knight
on the 14 October and Balls on the 15 October. On the 11 December,
as I said, for the first time I became aware that there was a
problem.[9]
I make no allegations or assertions, but it was very clear that,
by the time the two Ministers gave their evidence, the skids were
under me—or it was proposed to put the skids under me—at
Department level. That goes back to the meeting on 18 March and
the whole issue of the future of the QCA.
Q338 Fiona Mactaggart: So your
contention is that this is—I am going to use the word "plot",
but I am not saying that you are saying it is a kind of secret
plot—a plot devised well before the failure of the curriculum
tests to get rid of you from your role, with the failure of the
curriculum tests being used as an excuse.
Dr Boston: I think that I was
seen as a troublesome priest, and the failure of the tests was
perhaps a catalyst seen to offer an opportunity to allow those
events to move forward.
Q339 Chairman: When would your
contract have run out anyway?
Dr Boston: On 30 September 2009.
3 See Ev 48 Back
4
See Ev 48 Back
5
See Ev 48 Back
6
See Ev 77 Back
7
Note by Witness: The meeting which took place on 17 June
2008 between David Gee and the Rt Hon Jim Knight MP. Back
8
See Ev 77 Back
9
Note by Witness: On the 11 December I read the final pre-release
Sutherland Report, and for the first time I became aware that
there was a problem. Back
|
