Examination of Witnesses (Question Numbers
40-59)
MR PETER
WILLIAMS, MR
DOUGLAS BLACKSTOCK,
DR NICK
HARRIS AND
DR STEPHEN
JACKSON
17 JULY 2008
Q40 Dr Evan Harris: What is going
on out there that improves peer review?
Mr Williams: Our training, our
feedback, our evaluation and our annual updating of reviewers.
Q41 Mr Boswell: For clarity, we are
taking about your peer review process as part of external audit.
Mr Williams: Yes.
Q42 Mr Boswell: We are not talking
about your views on peer review.
Mr Williams: No.
Q43 Mr Boswell: That is conducted
at the course level.
Mr Williams: This is what we do.
Q44 Mr Boswell: Your own systems.
Mr Williams: Yes, our own internal
quality assurance.
Q45 Chairman: On 23 June you produced
the outcomes from institutional audit, the three reports. You
noted in your reports and, indeed, in your press release that
accompanied them that there had been "much solid achievement".
On 23 June, Peter, you gave an interview to the BBC in which you
said that the system was robust, but then you made a number of
criticisms in language which might be characterised as headline-grabbing.
You said, for example, that the degree classification was "arbitrary
and unreliable" and you went on to say that it was "rotten".
Do you believe that key parts of the UK's higher education system
are defective? When somebody from QAA says the system is "arbitrary
and unreliable" and "rotten" you can hardly blame
the media for then picking that up and saying that is what you
said.
Mr Williams: Those particular
quotes were taken, as you will appreciate, from a much longer
discussion with a journalist, and of course he picked out the
plums—perfectly reasonably.
Q46 Chairman: That is what they do.
Mr Williams: Indeed, they do.
But it was part of a longer conversation. What surprises me is
not that they picked it up in June but that they did not pick
it up in April 2007 when I said exactly the same thing. I said
rather more robust things, reported on BBC Online, and nobody
at all commented. This is the specific question of how a student's
achievement is recorded. It is not about the general higher education
system in this country, which I believe is robust, is solid, and
is good.
Q47 Chairman: This is degree classification.
Mr Williams: It is degree classification.
That is the only thing I was talking about when I said that the
system was rotten—and I agree that rotten is a colourful
adjective to use. I have strong feelings about the degree classification
system which I have made clear on a number of occasions.
Chairman: That goes really to the heart
of confidence in our higher education system, does it not? Degree
classification employers think is important. People will get jobs
or their futures depend on that degree classification.
Mr Boswell: And the grading of universities.
Q48 Chairman: And the grading of
universities is that. When people are accused of inflating the
number of firsts—and we seem to have had a miraculous improvement
in student achievement in some of our universities—that
does not give confidence, does it? Is that what you were talking
about?
Mr Williams: All I was really
doing was echoing the discussions in the two Burgess committees
and now the implementation group, which came to the same conclusion:
the system is not fit for purpose. It is as if that system was
designed for other times, for a smaller higher education world.
We have now reached the end of the "display by" date
and we have now pretty well got to the "use by" date.
Q49 Chairman: Why did you not say
in your reports that in future you will not in fact inspect or
audit universities until they reform their degree classification
to give confidence in the system? Surely you can drive this.
Mr Williams: We are driving it,
in a way, in the influence we have had on the Burgess group and
the implementation process which is ongoing. Our Quality Matters
paper which we brought out in April 2007 left nobody in any doubt
about why we thought the system was time-expired. The decision
on reclassification is one for the sector; it is not one for individual
institutions. It would be extremely difficult for a single institution
to move unilaterally on that. In the early 1990s the University
of London tried to do it. It gave itself five years, I think it
was, for a transition period to get rid of degree classifications
in favour of a common transcript. It did not work. It did not
happen because people were not prepared to jettison that with
which they were comfortable.
Q50 Mr Boswell: Perhaps I could pick
up on that, not historically but conceptually. Clearly there is
a strong attraction for employers, and probably for students as
well, in having some assurance of some coherence about what they
have. I am no expert on the alternative system and I am not wishing
to advocate this, but, given that you can see the present system
is strained to the limits or gone beyond the limits—partly
because of the greater number of higher education institutions
and the diversity which I welcome—is there any common core
that ought to be kept out of that, so that, if we take the institutions
which are geographically equidistant from my home, the University
of Oxford and the University of Northampton, both of which I know
very well, there is some sense in which an attainment which is
excellent in one is reflected or could be said to be comparable
with comparable excellence at another. Is there something you
want to salvage out of the degree classification to provide some
common currency?
Mr Williams: This is contested
territory. A lot of people have different views on it and mine
is just one view. I think we have to agree the basic level at
which a British degree is valid.
Q51 Mr Boswell: That is the threshold.
Mr Williams: That is right, the
threshold. Once you get above the threshold, then it is much more
important that you are clear about what it is you have. Of course,
diversity means you get different things—not that one is
better than the other but they are different from each other—and
the important thing is that it is clear what it is you have. I
think that information about a student's achievement is more important
than trying to give them a brand, a five-point scale which is
claimed to be the same across the whole country. It cannot happen.
It is logistically not possible.
Q52 Dr Evan Harris: Forgive me, Tim,
if this was not your question, but is the question whether an
employer who has two candidates, one with a first class degree
from Cambridge and another with a first class degree from another
university which is much more recent—
Mr Williams: Uttoxeter is an example.
Q53 Dr Evan Harris: Uttoxeter, yes.
I am not sure I have ever been. Would an employer be correct in
thinking that those students had the same value of qualification?
Mr Williams: I think it would
be rather foolish to assume that. There is no common definition
of what a first is.
Q54 Dr Evan Harris: Let us say there
are allegations of parochialism, so that a major employer had
to have observed the awarding institution, would you say that
they would notice the difference, if they were not able to distinguish
in appointments between a first class degree from Cambridge and
one from Uttoxeter, as you have put it?
Mr Williams: It is impossible
to say.
Q55 Dr Evan Harris: Is it really
impossible to say?
Mr Williams: I think it is. It
is very, very difficult. The evidence suggests that there is no
consistency between subjects in institutions or between institutions.
Q56 Dr Evan Harris: I do not understand.
If you need to get, say, the equivalent of three As at A-level
to get into one institution and you can get into another institution
with something like three Ds at A-level, and you both end up with
a first class degree, is it reasonable for employers to take cognisance
of that? If not, what is the point of having that classification
or, indeed, any classification, unless it does distinguish between
ability? Exams, surely, should be the same standard.
Mr Williams: In this country we
have 118, I think it is, awarding bodies. That is 118 individual
institutions with the powers to award their own degrees and to
set their own standards. They do that as autonomous bodies. There
is not, in a sense, a national curriculum and a national examination.
Q57 Dr Evan Harris: I thought your
job was to check that people were not being awarded first class
degrees undeserved in one institution. In a consumerised market,
people can say "I've got a greater chance of getting a first
class degree there, given my entry qualifications."
Mr Williams: No, that is not part
of our job. Our job in that is to say to the institution, "Do
you have some rules and regulations about what a first is? Can
you tell us? Can you show us what you mean by a first? Do your
assessment systems guarantee that only the people who meet those
are given a first? Is that process operating satisfactorily?"
Q58 Dr Evan Harris: If some research
was done with new graduates that showed objectively there was
a massive discrepancy in the standard of knowledge attained in
that subject between two people holding first class honours degrees,
would that suggest that your system was not sufficient to identify
that? Would you be concerned by it? Would you commission your
own research?
Mr Williams: We have done that.
The April 2007 paper essentially is saying that.
Q59 Dr Evan Harris: You have taken
a graduate with a first from one place and a graduate from another—lots
of them to make it statistically significant—tested them
and looked at what the results say.
Mr Williams: No, we are not a
research organisation of that sort.
| 
