APPENDIX 1
Memorandum submitted by Professor Richard
Green, Department of Economics, University of Hull
1. As currently constituted, the Research
Assessment Exercise forces universities to gamble for public money.
The Committee may wish to consider alternative methods of distributing
taxpayers' funds. I also comment on an example of the law of unintended
consequences.
2. Each unit of assessment chooses which
of its academic staff to submit for the RAE. Those submitted are
known as "research active"; those not submitted are
designated "research inactive", but the designation
is frequently misleading. It would be more accurate to call them
"research less successful", as many researchers with
respectable publication records will have been left out of the
exercise.
3. The gamble in the RAE comes from the
"cliff-edge" nature of the funding and grading decisions.
A unit graded at 5 currently receives 3.375 "units"
of funding per researcher submitted—the amount depends upon
the subject, but it is roughly £20 thousand per researcher
in the Arts and Social Sciences. If the unit was able to get a
5* by leaving out a small proportion of its staff, it would receive
20 per cent more per head. In Economics, I would guess that both
Oxford and Cambridge tried this gamble, submitting about 90 per
cent of their staff. If it had come off, they would have got 10
per cent more money than if they had submitted everyone and got
a 5. In practice, neither department left out enough people to
score a 5*, and so both got 10 per cent less money than if they
had submitted everyone. I expect most academics could guess at
similar stories in their own subjects. Lower down the funding
scale, the cliff edges get larger—submitting one person
too many and moving from a 5 to a 4 means that you will lose almost
a third of your research funding. The funding councils are likely
to change the funding scale in the near future, but the basic
flaw remains.
4. I have heard that the original reason
for allowing universities to choose whom to submit was to allow
small research units to remain visible within larger "mainly
teaching" departments. This was a sensible aim, but the consequences
have been severe. Universities have discovered the gamble at the
heart of the system, and waste hundreds of hours of senior staff
time in deciding on their submission strategies. Where the gamble
goes wrong, the university loses a significant amount of income.
The personal and professional consequences for researchers who
are deemed unworthy of submission can be severe.
5. Most importantly, the results are starting
to lack credibility. Many departments have learned that if they
submit two-thirds of their staff, they can hope to get a "4"—all
of the research is of national excellence, with some of international
standing. An identical department might submit everyone and qualify
for a "3a"—two-thirds of its staff of national
excellence. The two results might seem equivalent, and (at present)
bring the same amount of funding, but the "4" department
will be wrongly identified as the stronger.
6. I wish to propose a system that gets
rid of the "cliff edges" in funding, and gives universities
no reason not to submit all of their academic staff. RAE panels
would identify "national" and "international"
researchers, as they do at present. They would publish the number
(but not the names) in each category in each unit of assessment.
The funding councils would then give, say, £10,000 per year
for every "national" researcher (the amount varying
by subject) and two and a half times the first figure for every
"international" researcher. The amounts distributed
would be remarkably close to the current funding scale. If we
need to skew funding towards "top" departments, then
the amount for "national" researchers could be reduced,
and that for "international" raised.
7. The amounts allocated might be similar
to the existing system, but the incentives would be different.
There would be no need to gamble. Universities could no longer
lose money by submitting "marginal" researchers. Most
of the time spent in agonising over the submission decisions could
be used for more productive activities. The numbers of researchers
assessed as national or international could be used to assemble
league tables, but people who do not meet those criteria would
not be publicly identified in the way that "research inactive"
staff currently are.
8. My example of the law of unintended consequences
concerns a second flaw in the system. By the 1996 RAE, it had
become noticed that some universities were poaching good researchers
shortly before the census date, and getting funding for them,
while the institutions that had developed them received nothing.
This was felt to be inappropriate, and so the A* system was developed.
People who move between UK universities in the year before the
RAE census can submit two publications for assessment on behalf
of each institution, and count fully when that institution's quality
is assessed. The institution employing them at the census date
receives all the funding.
9. It is ironic that a system intended to
discourage poaching seems almost designed to make it worse. The
A* system gives less successful researchers the opportunity to
be poached. Almost anyone will look better if they are assessed
on their best two publications rather than on their best four.
A department that has just hired a number of new staff will therefore
tend to be given a higher quality rating than if it had not recently
recruited, and will receive funding for all of its new members.
10. If the RAE really wanted to be neutral
to staff movements, then all researchers should be judged on the
same basis—presumably four publications. Staff who have
left an institution within the past year should be included, whatever
their destination, but counted at half their previous FTE when
the department's overall quality and funding are determined. People
who have joined an institution within the last year should also
be counted at half their actual FTE.
11. If a unit loses a researcher and replaces
them with someone of the same quality, this system would be neutral—they
get half the funding for the departed researcher, and half for
the new one. If someone leaves unexpectedly and has not been replaced
by the census date, the department only loses half of the research
funding for that person, reducing the impact of chance events.
Departments will be discouraged from trying to expand rapidly
just before the census date, because the proportion of the extra
costs funded by the RAE will be lower. I have seen press speculation
that some of the universities that spent heavily in the run-up
to the RAE are now facing financial troubles, and hope that a
more rational system could help prevent such problems in the future.
2 January 2002
| 
