Examination of Witnesses (Questions 60
- 79)
MONDAY 17 DECEMBER 2007
DR KEN
BOSTON
Q60 Chairman: In broad terms, do
you think that this morning's proposals are to be welcomed?
Dr Boston: Yes.
Q61 Chairman: In their entirety—there
is no hesitation, qualification? I won't say the Australian equivalent
of welcome, but you know what I mean.
Dr Boston: With a modest, restrained
British approach to things, Mr Chairman, yes, these proposals
are to be welcomed.
Q62 Chairman: Let us drill down a
little. In this Committee, and the previous one, we did not see
great public demand for these changes. Do you believe that the
public were knocking on people's doors—they were certainly
not knocking on my door—saying that they wanted a more independent
relationship? Or is it that they were worried about standards?
There was always a fuss in August when the results came out—the
Daily Mail would always tell us that standards were going
down and that there was grade inflation and much else. Is that
what people are responding to? Is that what the Government have
responded to—the furore that goes on in August?
Dr Boston: Certainly, the Government
have listened to and heard our concerns about the ambiguity present
where there is a body that, among other things, is responsible
for regulation and reports on the maintenance of assessment standards
to a Government who are committed to driving up standards to meet
particular targets. As I said, in reality, we have not been troubled
by this. I do not think that anyone could point to an occasion
when pressure has been put on the organisation by the Government
or civil servants with regard to standards— certainly, I
am totally unaware of it, and I am certain that it has never happened.
However, if we consider one of the causes of the August debate
to be that the separation of the regulator from Government is
not perfectly clear, then that August debate might be diminished
if the separation were made more apparent. Of course, there may
be other issues in the August debate that are not resolved by
that situation.
Q63 Chairman: As you know, August
is a slow news time. They always bring the education correspondents
back for August, so if they have to write about something, I am
sure that they will do so. What is your view of the balance between
the agency and the other body? How will it be handled, and how
will the two organisations develop?
Dr Boston: The
Secretary of State has asked us to set up an interim regulatory
authority. That should be done virtually immediately, and there
should be as much distance between the regulatory body and the
parent body—the QCA—as is possible by the summer examinations.
Of course, the legislation will not be passed and take effect
until 2009. The way we are looking at setting up the interim
arrangements is for the QCA board, which cannot be discharged
of its regulatory responsibilities without a change in the Act,
nevertheless carrying out those responsibilities, not through
me as Chief Executive, but through Isabel Nisbet, the Head of
Regulation and Standards, who, it has been announced today, will
be the Acting Chief Executive of the new regulatory authority—Ofqual,
or whatever shorthand we might finally use to describe it. That
organisation will be operating in shadow form from April. I will
not be dealing personally with the awarding body chiefs on matters
of standards and I will not be setting levels in relation to national
curriculum tests, as I do at the moment. That will be done by
David Gee as head of the NAA. I will be responsible for managing
the affairs of the board. I will remain the accounting officer
for the entire organisation, but the shadow regulator's funds
will be ring-fenced. An interim board with an interim chairman
will be established for the shadow regulator, and the proposal
is that, to all intents and purposes, it should function as a
separate body from about April. Not only will it function separately,
but it will do so from Coventry, because many of them would otherwise
be moving to our temporary premises in the old Adult Learning
Inspectorate.
Q64 Chairman: We must get on to the
last thing. We have dipped our toe into the area of testing and
assessment. We have already had a lot of written evidence and
we have had a seminar. People mostly wanted to talk about, not
the constitutional role of the two organisations, or the split
between the roles of the organisations—that was hardly mentioned—but
too much testing, grade inflation, and a range of things that
concern parents, students and commentators. It seems that this
is to take our eye off the ball, so that we can say, "Look,
this is all alright. We are making some big, grand, but complex
changes out there," whereas most parents and students are
worried about other things entirely, such as too much testing.
Everywhere in the world they say that there are too many tests.
Academics come before us and tell us that we test the wrong things
or too many things. Those are the real issues, are they not?
Dr Boston: Yes, they are. Certainly,
during the interim period, we will not be taking our eyes off
those balls.
Chairman: Let us get drilling now with
David.
Q65 Mr Chaytor: To pursue today's
announcement a little further. What will it cost?
Dr Boston: I do not have an answer
to that, but we will be meeting to establish the shadow regulatory
authority for which we will need completely new front-of-house
facilities. From April, if you ring the regulatory authority,
you will not want someone from the QCA answering the phone. The
media will need to be different, as will the presentation and
delivery. We are looking at that, with a view to presenting a
budget bid to the DCSF for putting it in place.
Q66 Mr Chaytor: Do you know at what
stage your budget bid will be presented?
Dr Boston: It will be presented
within the next few weeks; by early January.
Q67 Mr Chaytor: In your opening presentation,
you put a lot of emphasis on the distinction between assessment
standards and performance standards. In 1996, the QCA's predecessor
and Ofsted published a report on assessment standards, saying
that there had been no weakening in the previous 20 years. In
2007, can the QCA say that there has been no weakening in assessment
standards in the previous 11 years?
Dr Boston: Yes. I would also have
to say that being able to say that is the product of vigilance
and monitoring. Of course, when looking at standards, which are
made by humans, and evidence produced by full-cohort papers—a
new, different paper each year—judgments have to be made
about the way in which one paper and performance equates with
previous papers and performance, and so on. Much of our work
on maintenance of standards is looking back over a period of time.
The reviews that we undertake of groups of subjects over a period
of time indicate, from time to time, that in one area there might
have been a drift, and that needs to be corrected. In a report
earlier this year we looked at music, including elements of the
music curriculum and music performance, and there appeared to
have been a drift there over five years. That then needs to be
corrected by altering criteria with awarding bodies. It is
a process of monitoring, review and adjustment, but taken in balance
as a whole—as an overview of the situation—my answer
clearly and unambiguously is yes.
Q68 Mr Chaytor: But will today's
announcement about the split of the QCA's functions in any way
reduce the likelihood of drift in assessment standards over the
next 10 or 20 years? Your argument seems to be that there has
been some drift here and there, which is largely the inevitable
result of human error and weakness of human judgment that has
been corrected. But is there anything in the new structure that
will stop that happening?
Dr Boston: No. The new body—the
regulatory authority—will use codes of practice similar
to those we have used in the past. It will use monitoring processes
with awarding bodies. It may choose to extend its work beyond
the work we fundamentally do, which is at the front end of the
qualification, developing the criteria and then accrediting the
qualification submitted to meet those criteria, and at the end
of the process, after the examination is running, looking at whether
the code of practice has been applied in the awarding process.
As we move forward with regulation—since Isabel Nisbet[4]
has been with the organisation, she has driven this very hard—we
need to be regulating more on the basis of the assessment of risk
and going into particular points through the process, rather than
focusing initially at the start and, finally, at the end.
Q69 Mr Chaytor: But none of those
issues could not be grasped by the QCA in its present format.
Is not that the case?
Dr Boston: That is true.
Q70 Mr Chaytor: There is nothing
about the new form of regulator that will give an enhanced guarantee
of no reduction in assessment standards.
Dr Boston: It is precisely the
same style.
Q71 Mr Chaytor: What I am trying
to get at is this: is the conclusion, therefore, that the only
argument for change is to somehow deal with the annual two-weeks-in-August
hysteria in the tabloid press?
Dr Boston: Well, I would not describe
it as dealing with the two weeks of hysteria, because while the
basis for that might be diminished I am not sure that it is going
to go away. The basis of the separation that is occurring is,
as I see it, the logical one: a regulatory authority should not
be reporting to the political party that is currently trying to
drive up standards.
Q72 Mr Chaytor: In terms of structural
change within the QCA, will the existing structure of the organisation
adapt itself neatly to a division into the two new functions or
will this require a major overhaul?
Dr Boston: No. This will require
some major separation of the organisation. The regulation and
standards division is clearly at the core of regulation, although
not all that it does will go to the new regulatory authority.
There are other elements in our curriculum division and in the
qualifications and skills division, where regulatory work is done.
The re-accreditation of A-levels, for example, which is essentially
regulatory, is done through the qualifications division as a 14-19
qualification. We have to unpick those functions and make provision
for that work to transfer to the regulator.
Q73 Mr Chaytor: Within the QCA as
it stands, there are three main divisions. The structure of the
organisation is based on three main areas.
Dr Boston: There are four: regulation,
qualifications and skills, curriculum and the NAA, which is the
operational arm that delivers the national curriculum tests and
the modernisation agenda.
Q74 Mr Chaytor: In terms of assessment
standards and performance, this is a blurring of these two functions
across the four divisions.
Dr Boston: Yes, organisationally
there is a bit of a blurring. This is meant to clarify it. Regulations
and standards or Ofqual—or whatever we end up calling it
in shorthand—sitting at Coventry, will be purely to do with
assessment standards and nothing else.
Q75 Chairman: We have a QCA. You
are the experts on the curriculum. The Government have just announced
yet another inquiry into curriculum, not by you, but by Jim Rose.
What is he doing being pulled into that? You are the competent
body. You know more about this than Jim Rose. Why are you not
doing it? I would be sulking if I were you.
Dr Boston: The intention announced
by Government is that the inquiry will be led by Jim Rose, but
that we will work with him as the chief source of advice on evidence
and as the body organising and managing a consultation, which
presumably will be very widespread. We need to take this out and
get genuine consultation with the professionals.
Q76 Chairman: Have they appointed
Jim Rose because he is more of a political fixer than you?
Dr Boston: I have no comment on
that, Mr Chairman.
Q77 Chairman: Some of us on the predecessor
Committee were not too keen on the Rose report. He went totally
overboard on synthetic phonics, but we hope that he will do a
better job with you on the curriculum.
Dr Boston: He is certainly a very
valued member of our board, and I believe that we will be able
to work together very effectively to achieve this. Finally, of
course, it will be his advice that goes to the Government. There
is no question about that, but we will provide the horsepower
in shaping that advice and carrying out the consultation.
Q78 Ms Butler: We are all aiming
for the same goal: to ensure that our children are very well educated.
We also want to ensure that schools are properly evaluated. In
your opinion, are there any other ways in which the effects of
national policy on the state schooling system could be effectively
evaluated? Do you have any ideas or opinions on how it could be
further improved?
Dr Boston: I am not quite sure
that I get the question. Do you mean methods other than the current
assessment system?
Q79 Ms Butler: Other than the current
system and how it works.
Dr Boston: That question takes
us fundamentally to the issue of the fitness for purpose of assessments.
What are we assessing and why? That is the area in which the paper
that Paul Newton from the QCA prepared for the Select Committee
is very helpful. The current Key Stage tests are absolutely fit
for the purpose for which they were designed. That is full cohort
testing in reading, writing, maths and science for our children
at two points in their careers and for reporting on the levels
of achievement. They are assessments that are developed over two
and a quarter years, and are pre-tested. They are run through
teacher panels, pre-tested again, and run through teacher panels
again. The marks scheme is developed over a period of time. In
terms of the way in which they are put together, if your purpose
is full cohort testing, in these dimensions, these are the Rolls-Royce.
You are not going to get better; they are fit for purpose. The
issue arises with any assessment when, having achieved an assessment
that is fit for one purpose, you strap other purposes on to it.
As Paul's paper shows, there are 22 purposes currently being served
by current assessments, and 14 of those are in some way being
served by Key Stage test assessments. Some of those purposes are
very close to what is the design purpose, the essential function—the
design inference, as Paul calls it. Some of the user inferences—the
purposes to which they are put—are much more distant. One
of the things that attracts me to the single level tests is that
the Government are now looking at a new suite of tests that will
have, not only the summative role—potentially when you add
up what children have achieved at the end of the Key Stage, to
get similar data to the summative data that you get now—but
potentially a formative and development role because they are
taken during the Key Stage test, and will potentially have less
impact on preparation for the test because you are not preparing
everyone to take the test at a particular time. You are building
children up to take the test when they are ready. My judgment
is that, given that there are so many legitimate purposes of testing,
and Paul Newton lists 22, it would be absurd to have 22 different
sorts of tests in our schools. However, one serving 14 purposes
is stretching it too far. Three or four serving three or four
purposes each might get the tests closer to what they were designed
to do. To take a very simple analogy, Barry, if you want to cut
paper or cloth, you have scissors; if you want to slice an apple
up, you have a knife; if you want to turn a screw, you have a
screwdriver; if you want to open a bottle, you have a corkscrew.
To some extent, we are not building tests, we are building Swiss
army knives here, and when you put all of these functions on one
test, there is the risk that you do not perform any of those functions
as perfectly as you might. What we need to do is not to batten
on a whole lot of functions to a test, but restrict it to three
or four prime functions that we believe are capable of delivering
well.
4 Isabel Nisbet, Acting Chief Executive of the new
interim regulatory body which will begin operations next year.
Isabel is currently the Director of Regulation and Standards at
QCA. Back
| 
