Examination of Witnesses (Questions 1
WEDNESDAY 23 JANUARY 2002
1. Welcome to our hearing on the Research Assessment
Exercise; you are very welcome. I do not think any of you have
been here before, and we look forward to your evidence; but perhaps
we could start, since we have kept you waiting; there are lots
of issues to discuss around education, science, technology, and
so on. If you could introduce yourselves, please, I am sure we
will get that written into the minutes?
(Mr Bekhradnia) Certainly, Chairman. Thank you for
inviting us, it is always a pleasure to come before a Select Committee.
It is particularly apt today, because you know, I think, that
the HEFCE Board was considering some of the very issues that are
exercising your Committee today.
2. We time things immaculately.
(Mr Bekhradnia) Clearly, it is perfectly
timed. I am Bahram Bekhradnia. I am Director of Policy at the
Higher Education Funding Council for England. John Rushworth,
on my left, is the Head of Research Policy at HEFCE; and John
Rogers, on my right, alas, shortly to leave us, was Manager of
the Research Assessment Exercise, and so knows the Exercise in
detail, intimately, if we should get into that sort of discussion.
3. Thank you very much. Can I start off the
questioning then by saying that your Chief Executive, Howard Newby,
wrote, indeed, as long ago as last November, that "As the
results are compiled, it is becoming apparent that there will
be a considerable improvement in performance. No-one should be
surprised by this." he said. Now were you surprised by the
size of the improvement, and when did this become apparent to
(Mr Bekhradnia) I think, gratified, as much as surprised,
would be true. I would have been very surprised and very disappointed
if there had been no improvement in the results of the RAE, bearing
in mind that the basis for our allocation of research funds and
the whole purpose of the RAE is that we allocate money selectively.
The RAE is a tool, no more than that. It is a very influential
tool, and I do not belittle it by saying that, but it is a tool
that enables us to allocate our money selectively. So, with that
in mind, back in 1996/97, when we allocated money following the
last RAE, there was a substantial injection of additional resource
into the top-rated departments. And so, in part, the improvement,
in the sense that there was a substantial increase in the proportion
of staff in the top-rated departments, was expected; if it had
not occurred then something would have been wrong with our funding
of research and the whole basis for our funding of research. Beyond
that, there is some research which we had seen that indicated
that, in terms of citations, for example, UK research in general,
but within that in England, was punching very much above its weight;
and that differential had increased, we knew that. So that also
indicated that there really ought to have been an improvement
in the RAE results. So, no, we were not surprised, we were gratified.
I think the extent of the improvement was greater than we might
have expected, but that there should have been improvement, no,
we were not surprised.
4. Clearly, we will come on to the details of
how those improvements came about, and so on, and the quantification
of them. The improvements in the results have been said by some,
I quote, to be put down to "fiddling, finagling and horse
trading." Do you accept that universities' playing of the
system has played a part in the improvement in results? It has
been equated to the Premier Division; well, perhaps not, perhaps
nationwide Division One, in terms of how they have managed to
manoeuvre people around to improve their ratings. Do you accept
there has been this kind of horse trading going on?
(Mr Bekhradnia) I think that depends, Chairman, on
what you would mean by, I am sorry, I forget, the finagling, fiddling
5. Transfer deals, if you like?
(Mr Bekhradnia) We are used to it. It is a sad fact,
in this country, perhaps not so much in other parts of the United
Kingdom, that when there is improvement we look at ways of explaining
it away as being unreal. I think there is good evidence from outside
to show why we should have expected an improvement; we also know
that research is a very much-better-managed activity in universities
now than it has been. One of the things that we did, following
the results from the Research Assessment Exercise, was to survey
a number of the most research-intensive universities, just to
see what management actions they had taken since the last RAE,
and there had been, in all cases, substantial programmes of management
action, in terms of merging departments, closing departments that
were not performing, bringing in new blood; that is management
activity, I do not regard that as negative.
6. But how big was the transfer market, in your
opinion, in universities, in the few years leading up to the Research
(Mr Bekhradnia) In terms of numbersand we can
send you the information about this, as it is published; we carried
out a review of research 18 months, two years ago, and this was
one of the questions we looked atin terms of numbers, not
great. There was no evidence that numbers of staff moving university
was affected by the Research Assessment Exercise. It may have
had other impacts; it may have led to salary movements that were
Research Assessment related, perhaps people being offered incentives
not to move, for example; but, in terms of strictly the question
that you have asked, which is staff moving between universities,
the evidence is that that was not very great. There are some that
say that movement of academics in this country is not great enough;
in the United States, for example, where they have no Research
Assessment Exercise, or anything like it, there is a far higher
degree of staff movement between universities, precisely in order
7. That is because they pay them twice as much;
they actually bribe them, in the United States, do they not?
(Mr Bekhradnia) Yes; whatever the mechanism for doing
it, they do it. And one of the things that is said about research
in this country is actually that there is not enough transfer
between institutions, whether due to the RAE or for other reasons.
8. So you do not think the improvement at all
correlates with any kind of transfer market that has been going
on; the improvement does not correlate in any way with that?
(Mr Bekhradnia) Not overall, Chairman; because, if
you think about it, if one university is gaining, another must
be losing. So, in terms of the overall increase in improvement,
I think, certainly not, no.
9. And you do not think universities have just
got slicker and smarter at playing the system?
(Mr Bekhradnia) Again, I say, it depends on what you
mean by playing the system. I think that research is a much better-managed
activity now than it has been, I think that it has got a focus,
I think that there is more strategy in research activity in universities.
And, yes, we do know that after the last Research Assessment Exercise
universities did take actionif you look, for example, there
are very few 1-rated or 2-rated departments now submitted to the
RAE; this is not because of games-playing, of any sort, I do not
think, it is because universities have taken management action
to merge, to close, to improve areas where they were not so good.
10. I should declare an interest. I did manage
a department, as you would put it, in the last Exercise, and I
knew that there was a phenomenon of keeping certain names off
the list, so that you put in only a certain percentage of your
people as research-active staff. Now that was not putting in the
whole staff, that was playing a game, in a sense; I dropped off
names of people because they had not published poor quality papers,
for example. Now did that go on to a greater extent this time
than it did, say, in 1996, it would be?
(Mr Bekhradnia) John can tell us whether there was
a significant change in the percentage of staff submitted; there
was a change, but I think certainly it could not have accounted
for anything like a significant part of the improvement that was
11. There is a submission here, in the pile
that I got, which refers to materials science; 25 per cent fewer
submissions than five years ago, that is in the evidence we have
received. Now that is significant?
(Mr Bekhradnia) Is that fewer staff submitted, or
12. Fewer submissions.
(Mr Bekhradnia) Yes, but then people may be moving
out of materials science. They may be merging materials science
departments with other departments, and submitting them in a different
way. I think that is not strictly comparable. But if you look
at the system as a whole, yes, there was a reduction, but I think
it was not one that would explain the improvement, in the way
that you have described it.
(Mr Rogers) In terms of numbers of submissions, in
1996 there were 2,894 submissions; in 2001, 2,598. So there was
a reduction in the number of submissions. But the number of research
academic staff returned was virtually identical in 1996 and 2001.
So we have a number of things happening. The weaker submissions,
those that were obtaining 1s and 2s, have largely disappeared,
because their universities have withdrawn from research in those
areas, or through some other management action, such as Bahram
has described. So we have seen a reduction in submissions there.
We have, in other areas, a number of institutional mergers, which
have had an impact on the capability of institutions to submit
the same numbers, but we have also seen, in the top-rated submissions,
significant growth within departments, which explains a large
part of the increase in volume of staff returned in the top grades.
So there is a variety of things happening.
(Mr Bekhradnia) If I may add, that last is precisely
the expected improvement; that is selectivity at work. We provided
more funds to those departments in 1996, and we would expect universities
to use those funds to recruit more staff.
Chairman: We have a division. We will have a
break for ten minutes while we go and cast our democratic vote,
on identity cards.
The Committee suspended from 4.39 pm to 4.47
pm for a division in the House.
13. We apologise for that. We will return to
the subject we were discussing before the division bell. I wanted
to ask about the selection of panels, how you select the panels,
if there is an indent bias in it in any way; would you like to
comment on that first?
(Mr Bekhradnia) Certainly. Would you like me just
to elaborate on one point from your previous line of questioning,
Chairman, if I may?
14. Yes, if you like; you have had time to reconsider,
(Mr Bekhradnia) No, no; the advantage of your absence
was that I was able to confer with my colleagues. And I can tell
you that there was virtually no change in the proportion of the
total staff that were submitted, over the whole sector, between
1996 and 2001. Now I cannot comment on materials science, but
will gladly provide you, subsequently, if you have got any specific
subjects that you are interested in, with specific information,
but across the sector as a whole I can tell you that there was
a virtually unchanged proportion of staff submitted between the
two Exercises. So, in answer to your general interest, I think
it is highly implausible that that would have accounted for any
part of the improvement between the two Exercises.
15. Right; well, we will come back to that subject
area at a later point.
(Mr Bekhradnia) On the question of the panel selection,
Chairman, again, I will ask John Rogers to comment in more detail,
but, in general, let me tell you how we go about this. First of
all, I think, we advertised widely, not just in the educational
press but more widely.
16. You advertise in The Guardian, do you?
(Mr Bekhradnia) And, I think, the Financial Times,
and the Times Higher Educational Supplement. It is invidious to
mention specific newspapers. But we advertise fairly widely, asking
for people to tell us about bodies which might have an interest
in research and research assessment, and therefore be interested
in nominating people. And we have an extensive list; we had a
list of over a thousand, I think, 1,300 bodies that we knew about,
that might have an interest in nominating members of panels. We
then contacted, we wrote to, all 1,300 bodies, asking them if
they would like to make nominations. I think we set out the areas
of research of the 69 different panels that we had and the areas
that they covered, and asked them, in making nominations, to bear
this in mind.
17. You do not think this is a `clubby' atmosphere,
(Mr Bekhradnia) We try to avoid that, Chairman. The
reason for advertising and for opening up as widely as possible
is to avoid that. That would be a danger, and that is the reason
why we do it, to avoid that.
18. Where do you draw the line? If somebody
came forward and said, "I did this subject ten years ago,"
for example, "I remember Mendel's laws but I can't do genetic
recombination," are they out?
(Mr Bekhradnia) I think they ought to be out, probably,
if they were not credible people. Because, I think, one of the
great things that I will claim for the Research Assessment Exercise
is that it has a considerable degree of credibility; and, I think,
there, I draw a distinction between that and some of the other
processes that are around, and we try to maintain that; one of
the important things in that is that the people that are doing
the Exercise should be credible. So the nominations come from
there, but we have to ensure that the subject is covered in all
its dimensions. So the chair of the panel, who is elected by the
19. Elected; is it contested ever, has it ever
(Mr Rogers) It was done by a single transferrable
vote system within the panel.
(Mr Bekhradnia) We should not talk about that, in
this sort of place, but it is not a `first past the post' arrangement.
But the serious point, Chairman, is, yes, we do recognise the
danger that you are describing, we do try to open it up as far
as we can, and there is a balance between continuity from the
last Exercise and the injection of new blood and also a need to
cover the full range of subjects that the panel will cover.