Examination of Witnesses (Questions 172-179)
ISABEL NISBET
AND KATHLEEN
TATTERSALL OBE
10 SEPTEMBER 2008
Q172 Chairman: May I welcome Kathleen
Tattersall and Isabel Nisbet of Ofqual to our proceedings? You
are not as unfamiliar to the Committee system as the previous
witnesses, so we do not have to ride you through the process.
However, Kathleen, I will give you a chance to say a few words
to open the session. You have been sitting here, listening to
the proceedings, and, knowing your background and that of Isabel,
I know that you will have looked at every bit of evidence that
we have had in this inquiry already, so you will be well primed.
Kathleen, may I ask you to say a few words to open the session?
Kathleen Tattersall: Thank you
very much, Chair, for inviting Ofqual to give evidence today.
As you know, we are the new regulator for examinations and tests
in England. In our interim form—that is, before the legislation
governing us is passed—we are part of QCA. We began our
operations on 8 April 2008, with myself as the chair and Isabel
as the acting chief executive. Before Ofqual came into being,
regulation was undertaken by the regulation and standards division
of QCA and Isabel was the director of that division. This summer,
we have monitored both the national curriculum tests and the A-level
and GCSE examinations. We have had a very successful summer with
A-levels and GCSEs. Sadly, that was not the case with the national
curriculum tests, where there have been some very well documented
problems. We share this Committee's disappointment that the delivery
problems were such that many students are still waiting for results,
as was said earlier. It is unacceptable that pupils and schools
did not receive those results in the time scale that was laid
down. However, I must say, as an evidence-based regulator, that
there were many processes in the national curriculum tests this
year that were successfully delivered. You heard about some of
them earlier. They included the delivery of the test papers themselves,
the printing and delivery of 10 million of those papers to schools
and the administration of the tests by the schools themselves.
So the pupils at that point had had a good experience. I think
that it is also important to note that the plans that were in
place for controlling the quality of marking were better than
in previous years. However, the marking process is the heart of
testing and examinations. Communications with schools and markers
and the data entry processes all went badly wrong. We have set
up an independent inquiry, as you know, led by Lord Sutherland.
He will be looking at the whole of the processes, including our
actions as the regulator. The DCSF has also asked Lord Sutherland
to report to it on areas outside our remit. We are continuing
to monitor the national curriculum tests and NAA's management
of the review process. However, from the processes that we have
observed so far, we believe that the quality of marking is as
good as in previous years. We will, of course, be looking carefully
at the reviews. Going forward, the priority is to regain the confidence
of teachers, pupils and the public in national curriculum tests.
To do that, we will be looking carefully at the 2009 series and
monitoring it carefully to ensure that pupils get what they deserve.
We want to ensure, as far as we can, that the problems that have
been experienced this year do not recur in 2009. Thank you, Chair.
Q173 Chairman: Thank you, Kathleen,
but you have not said anything about what went wrong. You heard
the evidence that was given to the Committee just now and you
know about the evidence from the QCA. Who do you think made things
go wrong? Was it an inefficient contractor or too much interference
from the NAA and the QCA?
Kathleen Tattersall: The evidence
that we have all heard today is the first time that I have heard
ETS give its side of the story, so there is clearly something
there to be absorbed. But from my point of view not only as chairman
of Ofqual, but previously as the chief executive of a very large
awarding body—the AQA—I know that at the heart of
any testing and examination process are the communications with
markers, the management of those markers and the management of
the scripts. In terms of the problems that we experienced, the
way in which markers were managed and dealt with throughout the
marking process was very much at the forefront. It is difficult
to recruit markers, and you always have to be acutely aware that
these are professional people who want to do a good, professional
job, but that they are doing that job over and above the job that
they hold as teachers, so you have to treat them in a very professional
manner. Some of the stories that we have heard would suggest that
markers were very upset by the way in which they were treated,
so that is one of the aspects. There are also the basic logistics—the
movement of scripts from schools to the markers concerned. Again,
there was some evidence that the scripts in particular subjects
did not make it to the markers of that subject and had to be redirected.
There was also evidence that some scripts did not make it at all
to markers. I would have thought that one of the basic requirements
of any system of assessment and examination is that there is a
checking process to ensure that everything is out with the markers.
So the logistics around script movement—getting the scripts
back to the warehouse and checked for marking—and the handling
of markers are at the heart of the problems. But I would emphasise
what I said in my statement, which was that as far as the training
of markers was concerned—regardless of what was said about
whether it should be online or face to face—what we observed
was very good, and that has led us to the view that the quality
of the marking should be okay.
Q174 Chairman: But Kathleen, with
great respect, you were the chief executive of the AQA and you
heard representatives of a similar organisation giving evidence
to the Committee just now, saying that they had signed a contract
and that that contract was quite dramatically altered, which made
it difficult for them to deliver on the basis that they thought
they were going to deliver, particularly in terms of the training
of the markers. Was it your experience as the chief executive
of a major examination board that that is normal? Would the QCA
or the NAA dramatically change the nature of a contract after
you had signed it, when you were actually running the operation?
Did that happen to AQA?
Kathleen Tattersall: The examination
system is slightly different. Certainly, the awarding bodies do
not operate under the same contractual arrangement with a local
education authority as ETS did. In terms of the contract, the
events that were described happened before I joined Ofqual, but
my colleague Isabel might be able to throw a bit more light on
the issue.
Q175 Chairman: If that contract
had been changed dramatically, would you not have been shocked?
Kathleen Tattersall: I would not
expect a major part of the requirements to be changed unless it
was clear that the process was not working as it should do, in
which case it would be very foolish to continue with whatever
was there in the contract. There would need to be in a place a
fallback system to enable all the processes to go forward. If
online training caused a problem—with your permission, Mr
Sheerman, I shall ask Isabel to talk about that—there would
have been no point in persisting with it because we might well
lose the very people whom we wanted to retain in the marking system.
Throughout an examination cycle, including the awarding bodies
as well as the national curriculum tests, things do go wrong and
one has to be fairly nimble footed to change tack to accommodate
that. Things do not always go as smoothly as we plan. As for online
testing, I would rather Isabel commented on that.
Isabel Nisbet: In February 2008,
Ofqual, the regulatory part of QCA, attended a meeting with the
NAA and ETS when they reported back on a pilot that they had been
doing on different aspects of quality assurance.
Q176 Chairman: What date was that?
Isabel Nisbet: It was 21 February
2008. On that occasion, one of the things reported back on was
the online trials of the online training of markers. The response
of the markers to that was poor. The conclusion at the meeting
was that it would be better to give them the option of face-to-face
training. The regulatory part of QCA and I agreed with that judgment.
The reaction to the online training was not good. In addition,
the regulator was not responsible for the detailed scrutiny of
the contract, but we were monitoring how everything was going
overall. A lot of things that went wrong later were not about
training, but about parcels not being checked before they were
sent out, for example. I should say that, as regulator, we had
great difficulty getting good data on where all the scripts were.
Q177 Chairman: Who was responsible
for the details of the contract, if you were not?
Isabel Nisbet: The contract was
between the NAA and ETS. We were not supervising the contractual
aspects. Our duty, as set out in our regulatory framework, was
to look at the quality and to make sure that the tests were delivered
in a way that would command confidence.
Chairman: Thank you for those opening
responses.
Q178 Mr Pelling: I shall ask some
basic parameter questions. What was the role of the regulator
in the actual procurement of the contract with ETS?
Kathleen Tattersall: The procurement
process was long before Ofqual came into existence. Again, I ask
Isabel to comment on that.
Isabel Nisbet: As regulator, my
part of QCA did not have any formal part in the contractual process.
The proprieties of the contracting were overseen by the Office
of Government Commerce, which reported on it several times. Our
responsibility was to make sure that our regulatory requirements
were met. They are that the levels are properly set; the standards
are properly made; the quality of the marking is good; and the
tests are delivered in a way that meets the standards of previous
years. Clearly, a lot of the things in the contract were to do
with that, but the actual contracting was not overseen by us.
Q179 Mr Pelling: I presume that,
if the processes of the procurement were so limited, there would
have been no oversight role for the regulator.
Isabel Nisbet: It is fair to say
we oversaw the whole delivery of the tests, but we were not monitoring
the contract. We were not a contract checker. Our requirements
are set out in our regulatory publications, and those are the
issues that we were checking against, not the terms of a contract.
Kathleen Tattersall: It should
be said that our regulatory role was in respect of NAA. That is
where our regulation falls. NAA managed the contract with ETS.
That is the line of accountability.
|
