Examination of Witnesses (Questions 340
- 359)
MONDAY 9 MARCH 2009
PROFESSOR PAUL
RAMSDEN, MR
PETER WILLIAMS
AND MR
ANTHONY MCCLARAN
Q340 Mr Marsden: A very quick final
point. I think you said 90 per cent, or just about 90 per cent
of students. Are you going to wait until you get 100 per cent
compliance on this, or are you prepared to move when you have
got 90 or 95 per cent?
Mr McClaran: No, I think we are
moving actively and our website already has those statements on
it. We have a special advance diploma section of the website and
we moved very rapidly in terms of making sure that it was brought
within the tariff framework, so we have been extremely active
in terms of ensuring that member institutions had a good understanding
of that qualification and what it might mean in terms of making
offers to potential students.
Q341 Chairman: Thank you very much
indeed. Mr Williams, you are soon leaving the QAA, so you are
totally free this afternoon to reveal all! We have heard a significant
amount of evidence, and indeed you heard some from Professor Alderman
earlier, that the QAA, which is supposed to be the Quality Assurance
Agency with the word "Quality" within it, is not doing
its job about ensuring quality at all, that you have mainly presided
over a process-led organisation and that provided the process
is carried out accurately then you give a tick of good health.
You have no teeth and you do not look at standards. Is it time
really that not only do you move on to a new job but that in fact
we have a new agency altogether?
Mr Williams: Thank you for asking
that question! It gives me an opportunity not only to rebut, if
not refute, what Professor Alderman has just said but to in fact
try and establish in your minds the link between process and outcome,
that is to say that any system which does not look at the way
in which the outcomes are achieved, that is the process, is going
to lay itself open to the charge of a capricious and unsystematic
attempt to ensure standards. The relationship between process
and outcome is very important, especially in complex organisations
like universities. Someone once said that quality is not an accident,
it is always the result of intelligent effort, and I fear that
Professor Alderman's approach would lead us to a series of accidents,
accidents where the absence of process, the absence of proper
procedure, of a systematic, careful and conscious attempt to provide
the kind of education that higher education, universities, should
be providing will actually lead to—I have lost the thread
of my sentence now.
Q342 Chairman: You were getting excited!
Mr Williams: I was. I feel passionate
about this. Process and outcomes are very strongly linked. It
is not an accident. It is because things are done that other things
happen. Because teachers plan their teaching, then students will
learn. Because students are guided in their learning, they will
learn. It is that careful, systematic approach which is important
and it is even more important given the size of the system there.
If you are talking about one to one tutorials on a weekly basis,
you can get away with a lot, but when you are talking about hundreds
of students, all expecting, quite rightly, to be given a good
learning experience then that does not happen by chance. But not
happening by chance means good process, so we do expect to see
good process. We are also interested in seeing the relationship
which is established between that good process and a satisfactory
outcome.
Q343 Chairman: Are not the universities
simply giving you a run-around in reality? You are saying to us—and
indeed you have written to me on the record so I presume I can
refer to it, because you did write to me as the Chairman of the
Committee—that following last summer's concerns in the media
about standards in our universities, about plagiarism, about,
quite frankly, cheating by academics and by chancellors, you set
up a causes for concern organisation, that you had investigated
these elements effectively and basically there was nothing in
them, and yet we continue to get literally a significant amount
of evidence, some of which we cannot publish because it might
be libellous but basically saying that this is still happening.
So there is a divide, that is the point I am making. I am not
suggesting that you do not investigate, but somehow people are
prepared to say things to this Committee which they claim they
have said to their universities, which you claim you have investigated
and they do not meet up.
Mr Williams: We have looked at
everything that has come in and we have done a fairly extensive
analysis of the media stories of last year and I am coming to
the conclusion that there are some areas where there is probably
something which requires more systematic investigation than we
have been able to give it so far. There are two or three areas
which I think we ought to be looking at, probably, but we will
wait to see the full outcome of our review. So far as the causes
for concern are concerned, what we have found there is that the
vast majority of them are in the first instance either personal
complaints or grievances or, in the case of staff, post-dismissal
or cases where they have been to an employment tribunal; in other
words they are again personal cases. Now, some of those are still
sub judice and have not gone through the standard procedures,
so when they have worked through those we hope to see them back
to see whether or not there is anything in them. I think it is
also fair to say that it is sometimes quite difficult to discover
whether the personal case is masking a systemic problem or is
just a one-off administrative failure, and that is where we are
needing to do more work on individual cases, some of which remain
open because we are not satisfied that this thing is simply a
personal grievance and we want to come back and look at them,
but we cannot do that while the cases are open. We have not had
the great avalanche which I feared last summer. We have made considerable
efforts to publicise our process and to explain what the process
is and what we can and cannot do, but we have not had the avalanche.
When we looked at the media stories, what we found was that there
are two or three stories which actually get snowballed and the
same story gets repeated time and time and time again, sometimes
with accretions of anecdotes and sometimes without, but the whole
thing adds up and it looks as if it is adding up to something
which is much bigger than it is. If you have got evidence you
can let us have, please let us have it.
Q344 Chairman: Okay. Let us see if
you will agree with this first: the universities, not the QAA,
are responsible for setting standards. Do you agree?
Mr Williams: Yes.
Q345 Chairman: How can you have a
situation whereby an organisation itself sets the standards and
judges those standards? Of course it will always come out right.
Mr Williams: That is the nature
of this beast.
Q346 Chairman: It is like Manchester
United saying, "We are going to win the Football League,"
so they only play Accrington Stanley!
Mr Williams: Yes, but that is
the nature of higher education around the world. It is how it
works around the world. Harvard does not have external examiners.
Yale does not have external examiners. Princeton does not have
external examiners.
Q347 Dr Harris: No, but there are
processes, are there not, where they are have to be reappraised
every ten years?
Mr Williams: There will be an
accreditation process, which is a rather different thing than
an evaluation of their standards. It is a quite separate process.
The only area in America, to the best of my understanding, where
the standards themselves are looked at are in the area of specialist
accreditation, that is to say vocational courses, where we have
in this country a cognate process, that is to say the accreditation
by professional statutory and regulatory bodies.
Q348 Chairman: Can I ask you another
question, and then I will pass you on to Dr Harris. Is there any
university at all that you have looked at where you have said
that their process is clearly inadequate, in other words that
the standards they set, which they then meet, you think requires
some organisation (whether yours or somebody else's) to actually
intervene and say, "This is not acceptable"?
Mr Williams: Sorry, the process
is good and the standards are bad, or the standards are good and
the process is bad? You can have either.
Q349 Chairman: In either case, where
would you intervene, because you do not have any powers to intervene
in that process of them setting the standards and judging the
standards?
Mr Williams: We do it by proxy.
Q350 Chairman: It is a bit like Mystic
Meg this!
Mr Williams: No, it is not quite
as exciting as that! There are two structures that deal with this.
Given that there is no national examination—there is one
country that does run a national examination and that is Brazil,
where they have everybody come and they all sit down and write
the same paper so that they can check the national standard, which
may be of interest to you. I do not know how much that costs or
indeed whether it is any use or not, but in this country we have
two structures. The first is the way in which standards are established
individually by institutions but by reference to national expectations.
In other words, we are squaring the circle and this is through
our circle of academic infrastructure, the subject benchmarks,
the qualifications framework. I should just report to you that
the qualifications framework for England, Wales and Northern Ireland
has just recently successfully been certified as being compliant
with the European Bologna framework as well, so we are now Euro-compliant.
Q351 Chairman: Who did that compliance,
by the way?
Mr Williams: That is run through
a process called Self Certification, which has been -
Q352 Chairman: This is fantastic!
Mr Williams: This is a European
thing and the Self Certification involves a large number of external
reviewers from all over Europe. It is not just us doing it to
ourselves, and indeed that report is on our website and it tells
you how it is done. It is a perfectly reasonable process and quite
an extended one. So we have the framework, which actually talks
about the level of requirement of different degrees, what the
different degrees mean, Bachelors, Masters and Doctorates. We
then have the benchmark statements, which give the subject community's
view on what are the expectations of an Honours degree in a particular
subject area and that allows for the diversity of different types
of universities, different types of programmes for different types
of student, and those together provide an external reference point
which the universities themselves use in order to establish the
standard, to compare what they are offering against the standard,
the national expectation.
Q353 Chairman: The short answer is
that you have never had to intervene? You have never felt the
need to intervene?
Mr Williams: Not in higher education
institutions. We have in further education institutions offering
higher education.
Q354 Dr Harris: We have established,
as you have accepted, that the QAA does not control or directly
monitor the standard of individual degrees?
Mr Williams: Yes.
Q355 Dr Harris: Let us say you shortly
retired and the Secretary of State or potential Secretary of State
came along to you and said, let us say, "We are going to
have a radical look at this to bring even more confidence to the
system," would you say that there was an argument for giving
you, as the QAA, a role in checking more directly the standards,
or do you feel that peer review, the external examiner system,
would give you more assurance on that?
Mr Williams: It is interesting
you should ask this, because we have actually been thinking about
this ourselves within QAA, exactly what we should do. Quality
assurance is an evolutionary process. You do not do the same thing
again and again, you must move on.
Q356 Dr Harris: You are limiting
it to the current framework but I want to look outside your current
legal powers. If you were improving the system, looking at it
from the outside, what would you suggest?
Mr Williams: What I would suggest,
and what we are thinking through, is first of all to retain the
peer review process because that, I think, is a strength, to look
more at what I would call primary evidence instead of secondary
evidence. At the moment we look at secondary evidence. The third
is to negotiate with the better regulation setters the right to
be more intensive in our work. At the moment we are very constrained
in what we can do through various constraints placed upon us.
We would look at primary evidence. We would look at external examiners
to a greater extent. In other words, we would go to the institutions
and say, "We want to see the same things that you see when
you are setting and monitoring your standards."
Q357 Dr Harris: Could you send us
a note about primary evidence and secondary evidence, because
I do not think we have time to explore it now?
Mr Williams: Yes.
Q358 Dr Harris: On peer review and
external examiners, do you think the system is good, do you think
it is good but maybe it needs to be looked at, do you think it
could be improved, or do you think it is flawed? What is your
view on it?
Mr Williams: I think it is a good
system. It is creaking. There needs to be better understanding
of what it can and cannot do. I think the claims for it to provide
the kind of nationwide or whole cross-sector guarantee of consistency
of standards cannot be sustained, and we have said that in a number
of places. But to have the external check, however that is formulated,
is an extremely valuable element within the whole process.
Q359 Dr Harris: Do you have a view
on peer review and external examiners, because you heard in the
previous session some concern raised about whether they could
do the job, even if it was not, "You scratch my back, I'll
scratch yours"?
Professor Ramsden: My view concurs
very largely with Peter Williams's. I think it is a good system
at heart, peer review, because it goes along with fundamental
ideas about academic culture and it tends to look effectively.
As you will be aware, the RAE is done right throughout that too.
I think it does need work on it to make it better. The suggestions
I made in my report to the Secretary of State include the possibility
of more training, better development for external examiners, the
possibility of a register and the development of colleges of peers
to go beyond the external examining system and complement it.
So I think it needs work on it to make it better.
| 
