Examination of Witnesses (Questions 60-79)
SIR ALAN
WILSON AND
PROFESSOR DAVID
EASTWOOD
10 JULY 2006
Q60 Chairman: It is a big shift away
from decision-making from academics to bureaucrats and quangos
like yourself. That is the shift of power, is it not, you are
taking away from academics, you are giving them much more time
to play with their research, but the decisions will be made by
people like you who really will jump to what the Government wants?
Professor Eastwood: I think that
is a very stark characterisation and not one I would necessarily
agree with. I think the first thing to say is—
Q61 Chairman: Professor Eastwood,
I have been in this chair for quite some time and I am used to
dealing with people in your job and I have to tell you that it
is very different, if HEFCE does something, a whole bunch of academics
who are independent . . . Once you become a paid employee of HEFCE,
you do become a very different kind of person, do you not, you
must realise that?
Professor Eastwood: Perhaps about
next year I will come and comment further on that.
Q62 Chairman: It is interesting,
is it not? On a serious point, Professor Eastwood, the sort of
thing you have been telling this Committee today is not what Howard
Newby was telling the Royal Society only 10 days or two weeks
ago when I chaired the meeting over there and it is certainly
not what Gareth Roberts said either, so the view there is some
kind of unanimity, your predecessor at HEFCE is not singing from
your hymn sheet today and Gareth Roberts is not singing from yours,
Sir Alan, that is the truth of it, is it not? You have come here,
all agreed, everyone out there, all reasonable men and women agree
with us today; you know that is not true?
Sir Alan Wilson: Chairman, as
I said to Mr Wilson, it is a highly controversial business and
I think we can all agree about that, but that goes both ways round,
it is not the successive proposals against traditional RAE, it
is very complicated, but I think what I would argue, Chairman,
and it goes back to your comment again about taking academics
out of the loop, I mean I think what in a sense we are trying
to do is to put the academics back into another and more productive
loop in terms of research. There will still be bidding for funding
from research councils, there will still be peer review there
and indeed some of them, Chairman, will still be on Professor
Eastwood's council so it is not simply, as it were, officials
deciding what happens to the funding. I think what is important,
and I go back to your earlier point on this, is that whatever
we do and whatever the system produces in the end, is transparent
and understandable and I do not think this has to go into obscure
mathematics and in fact if you actually take the RAE in its present
form, I think Professor Eastwood has been hinting at this, there
is some subjective academic judgment in actually producing research
ratings, getting from those research ratings and then starting
to use metrics, getting from those research ratings to a funding
formula involves the same kind of complications, whether it is
in the RAE framework or whether it is in the metrics framework
and I think we do have a responsibility between us to explain.
Q63 Chairman: Sir Alan, you know
I am a great admirer of yours and of Professor Eastwood, but it
is my job as Chairman to this Committee to tease out some of these
commonsense things that people will ask and they do ask the thing,
"Is this movement", like the Learning and Skills Council
in further education, "becoming the all powerful body?"
Here in higher education are we seeing a transfer from a whole
group of committees and academics making the decisions to a bureaucracy,
wherever nicely described, the civil service, in your case Sir
Alan, and a quango in your case Professor Eastwood, is that a
shift of power and should people be worried about it? Perhaps
they should not be worried about it.
Professor Eastwood: I think in
terms of the process that we have set out, the first serious commentary
on the alternative will come from the shadow metrics exercise
in 2008, it will come from our panels, so the first group to offer
substantial commentary will be academics. The consultation suggests
that there might be a different approach in STEM subjects from
the arts and humanities and some of the non-quantitative social
sciences and in the areas where we think metrics might be more
robust and that will be tested in 2008, there will be panels to
advise, so I think it is a process which might dial out some of
the complexity and we hope might dial in some transparency, but
I do not think it is a process which can be described, as it were,
simply leaving the funding to apparatchiks. On the contrary, the
judgments which underpin the metrics are all peer review judgments,
whether it is the distribution of funding on the one hand or the
decision to cite an article or research output on the other, so
academics are absolutely at the heart of this and what we are
trying to do is to find the most efficient way of deriving quality
judgments from those data, ensuring that they are valid and validated
and then the Funding Council, in a way which is appropriate, will
translate that into funding outcomes as it has done since the
1980s.
Q64 Mr Marsden: What I would like
to do now, if I may Chairman, is to move on to some of the specific
impacts of these changes both in the so-called STEM science technology,
engineering and maths subjects and also in arts and humanities.
Just to make the observation in passing to you, Professor Eastwood,
and you talked about disputed excellence, but there is also of
course the law of unintended consequence and I do not think anyone
on this Committee doubts your bona fides in that respect, but
we are trying to examine what some of those unintended consequences
might be. The first question I have got is for you and it is very
much focussed on the issue of arts and humanities. How can a metric
system in any shape or form, however modified, replace the RAE
when a metrics based system cannot deal with arts and humanities
subjects which account for about half of the existing research
activity?
Professor Eastwood: The first
thing to say in response to that is that HEFCE and the AHRC—Arts
and Humanities Research Council—have set up a group to advise
precisely on that and the view interestingly of the AHRC is that
it is possible with or without some elements of traditional peer
review—
Q65 Mr Marsden: It is possible?
Professor Eastwood: With or without
some, and this will be determined, some elements of—
Q66 Mr Marsden: They are not ruling
out right from the beginning metrics as an ingredient?
Professor Eastwood: No, indeed
they are not, and indeed the humanities' panels in 2008 will be
using some metrics because all panels in 2008 use at least 15%,
or at least 15% of their judgment is metrics based, so metrics
are there and what this group chaired by Professor Michael Wharton
of UCL will do is evaluate the way in which metrics, particularly
bibliometrics, have developed in the humanities and feed in to
the consultation in the autumn its judgments.
Q67 Mr Marsden: Can I just ask you,
I mean in terms of we toss this phrase "bibliometrics"
around, but I know from a previous existence that journal articles
carry different weights depending on the journals in which they
appear; a very specialist journal article might attract a particular
weighting, a more generalist journal article would attract a broader
rating. Is that going to be spelt out, made more transparent under
a metric system, a bibliometric system, any more than it is under
the RAE system?
Professor Eastwood: You mean the
weightiness that particular journals might carry?
Q68 Mr Marsden: Yes.
Professor Eastwood: I think we
can already see it in some areas of the social sciences and actually
in some areas of the humanities that there are particular journals
which carry particular weight, particular journals that academics
are desirous of publishing in.
Q69 Mr Marsden: And is that, to come
back to the original point that I raised, is that transparent
in the same way as we have league tables for schools, are you
envisaging a situation where anyone could go along and say, "Professor
X has contributed to that department because he got three points
from", I do not know, "his New Scientist article
as opposed to two points from a more specialist article on biochemistry",
for example?
Professor Eastwood: I do not think
that one precludes the other because I think it is in the end
about quality. I think the issue for the arts and humanities in
particular is how far they are down that particular road and the
kind of bibliometrics that are now increasingly available. For
example, there are metrics around monographs becoming available
for the first time which is obviously of a very considerable importance
to the humanities. Having set up this group to advise, I do not
want to pre-judge the outcomes.
Q70 Mr Marsden: I am not asking you
to pre-judge it, I am asking you to try and elucidate what is
quite an opaque subject, even the term "metrics" is
an opaque one and to try and translate what it would mean for
your average academic or your average university department in,
for instance, arts and humanities under a new system. I would
like to ask you another question in that context and that is to
say, supposing I am a young academic in my mid to late thirties
and I am trying to decide, I want to do a big book, it might be
a big history book or it might be a big book on physics or whatever,
you know as well as I do that one of the criticisms of the present
system is that the RAE forces that young academic to do chopped-up
little articles in rather narrow journals rather than having the
time to pursue bigger issues in a bigger book. Is a metric based
system likely to, or a system that takes in metrics, likely to
make that easier or more difficult for someone in the arts and
humanities?
Professor Eastwood: If the conclusion
was that metrics can measure the impact and quality of monographs,
which is a claim which is now being made, then if there was a
problem there it will start to address that problem. My own view
is that some of the effects of the RAE in this regard have been
exaggerated, particularly in a seven year cycle it is perfectly
possible to produce a monograph to produce major articles and
even to produce articles for noble publications such as History
Today.
Q71 Mr Marsden: This is not what
many of your historian colleagues say, people in the Royal Historical
Society, the Institute of Historical Research and in other organisations
who are concerned that the current system in RAE is stopping some
of those big books being produced by younger academics, let alone
a more metrics based one?
Professor Eastwood: I think I
would say with the greatest of respect to my colleagues that—
Q72 Mr Marsden: You acknowledge people
are saying this?
Professor Eastwood: I acknowledge
people are saying this, but I think it is a misdiagnosis, I think
there are a number of wider trends which have little to do with
the RAE and that are attributed to the RAE and what we are seeing,
for example, in monograph publications, something which is happening
in publishing, and the difficulties of getting monographs published,
is much less an artefact of the RAE than an artefact of the publishing
industry and the information revolution.
Q73 Mr Marsden: Let me turn to the
STEM subjects then. Do you accept, Sir Alan, that it is going
to be a lot easier to produce a metrics system based on STEM subjects
than arts and humanities?
Sir Alan Wilson: It is certainly
true at the present time, partly so because of the volume of research
council and other research funding, so you potentially get more
sensitive indicators and hence the Annex A to our consultation
paper where we explore a wider range of indicators and input on
arts and humanities.
Q74 Mr Marsden: You said yourself
earlier on that you had been influenced by the concerns that had
been expressed about the current RAE system perhaps inhibiting
collaborative and disciplinary research costs and STEM subjects
on a number of different departments. Are you convinced that that
particular issue, which is a real issue, is one that would be
addressed by a predominantly metrics approach in those subjects?
Sir Alan Wilson: I think it makes
it a lot easier. I think research councils and other research
funders are putting an increasing proportion of their funding
into interdisciplinary projects and that would be represented
in a funding formula.
Q75 Mr Marsden: What would you say
to those people who say that these proposals have essentially
been drafted, or this consultation has essentially been drafted,
with a focus on STEM subjects and that the arts and humanities
have really been a tagged-on after-thought?
Sir Alan Wilson: Well it is certainly
not an after-thought, Chairman, and again hence the annex. I mean
I think we wanted to produce a comprehensive response. I think
perhaps I could make one related comment that builds on the big
book question or the individual academic question. I mean one
of the things that we have said is I think there is a danger in
thinking that we are producing a system that focuses on individuals.
It will actually operate at a much more aggregate level and what
we have said in the consultation paper is that we believe that
universities are now increasingly in a position where, because
of past RAEs, they now have to manage their research internally
and I think the young historian who wants to produce a big book,
in that case I would be disappointed if the university management
in either the department or the university was not, in effect,
supporting that and I think that is one of the challenges.
Q76 Mr Marsden: I would love to believe
that and anecdotal evidence suggests in a sense that it does not
always happen. Can I ask you, Professor Eastwood, a final question?
When you were being pressed earlier about the possibilities of
measurement you referred to "input" and "output",
you said, "We want to be able to measure input, we also want
to measure output". I mean output surely in terms of research
also involves the impact of that research; I mean obviously it
may involve something very directly like a scientific application,
it also is an impact on various groups of people. Does not the
impact of research also involve the impact on teaching and dissemination?
Professor Eastwood: Yes, I think
it does.
Q77 Mr Marsden: In which case why
are you not, or maybe you are going to, why are you not going
to use the opportunity of this consultation to look at the impact
of research on teaching and how you would measure that either
within the existing RAE context or the metrics context?
Professor Eastwood: I think I
would make two comments there: one is that through a different
initiative there is now a funding stream available to ensure that
those institutions which have less QR, nevertheless—
Q78 Mr Marsden: But a much smaller
amount of money?
Professor Eastwood: It is a smaller
scheme, but nevertheless it goes in the direction of the point
that you are making which is that in higher education, the teaching
should come out of the research an informed environment, so that
is the first point I would make. The second point I would make
is in the current RAE, particularly in the humanities where what
one has are, if you like, publications which represent a synthesis
and the creativity is at the level of that synthesis, then they
are publications which, quite appropriately, are assessed within
the RAE. I think there is a deeper issue which we may return to
on another occasion no doubt which is given that we might shift
the focus to teaching, what is the appropriate funding for teaching
which will facilitate that kind of research and teach, but that
is a related, but I think a distinct question.
Jeff Ennis: My first question, Professor
Eastwood, is really a follow-on to the line of questioning that
both the Chairman and Gordon has just been pursuing recently in
terms of the utterances from Sir Howard Newby in terms of the
way we are approaching this issue, because he said on RAE at the
HEPI conference on 21 June that the order of any discussion must
be to clarify policy goals first, then consider the RAE and then
consider metrics—not to begin with a consultation on metrics
which will impact on the RAE which will impact on research policy
and we have obviously had organisations such as Universities UK,
to name but one, who have been calling for a full debate on this
issue. How would you answer those particular points?
Q79 Chairman: When do you take over
Professor Eastwood?
Professor Eastwood: I take over
in September.
| 
