APPENDIX 19
Memorandum submitted by Scientists for
Labour
SUMMARY
Scientists for Labour is an organisation open
to members or supporters of the Labour Party who are interested
or involved in UK science and technology. Since its establishment
in 1994, it has become a strong political voice for science. In
preparation of this response we provided our membership with a
series of questions related to the major issues raised by the
RAE. The outcome of this consultation exercise may be summarised
in the following key points and recommendations:
— SfL believes that some mechanism
is necessary to enable the higher education funding bodies to
distribute public funds for research selectively on the basis
of quality.
— The RAE provides an appropriate mechanism
for the distribution of funding but significant changes should
be implemented prior to the next exercise.
— SfL believe that it is timely for
an objective assessment of the effectiveness of the RAE to be
undertaken.
— SfL recommends that the whole process
is simplified and streamlined.
— SfL recommend that the interval between
exercises should be increased, perhaps to seven years, to concur
with the general time-scale for strategic research developments
to come to fruition within departments.
— SfL believe that the current structure
of the RAE discourages the establishment of high-quality interdisciplinary
research programmes. The mechanisms within the RAE for dealing
with interdisciplinary research should be re-assessed so that
they act as an incentive for Universities to establish such programmes.
— SfL believes that the overall level
of Government funding for University research in the UK (in terms
of GDP) must increase to bring it in line with competitor countries.
— Where departments increase their
grading through the RAE exercise the funding mechanisms should
be in place to ensure financial benefit in terms of increased
funding. To do otherwise will lead to significant de-motivation
across the sector. If the Government is unwilling to increase
the funding for research, even if it is improving, then there
is little point in having a RAE.
1. THE RESPONSE
In preparation of this response we provided
our membership with a series of questions related to the major
issues raised by the RAE, which are listed below. We have set
out our response according to this format. Responses were received
from a wide variety of interested individuals, at all levels and
across disciplines ranging from Engineering and Mathematical science
to Biological sciences and Earth sciences.
(1) Has the RAE been successful in its goal
of improving the quality of research in UK University departments
over the period 1992-2001?
(2) Does the RAE encourage long term forward
planning within departments?
(3) Does the RAE represent a good investment
of time for academics involved in preparing submissions?
(4) Should departments be able to "play
the game" by submitting only those staff they choose or should
all staff have to be submitted?
(5) Does the current RAE structure lead
to unfair discrimination against academic output which may not
be in the form of peer reviewed papers (eg contract work, soft-ware
or hard-ware development)? In addition does this provide an undue
bias against the New Universities who may concentrate on this
type of activity?
(6) Does the RAE structure lead to unfair
discrimination against groups involved in interdisciplinary research,
which may not fit comfortably within a specific Unit of Assessment,
thereby favouring traditional disciplines.
(7) Does the RAE structure lead to unfair
discrimination against particular types of staff, leading to de-motivation
or a two-tiered system, if they are not submitted (eg staff, particularly
women taking career breaks, staff with high teaching loads)?
(8) Should the financial model be designed
such that departments who improve their ratings will automatically
be rewarded through the financial model. Current view from HEFCE
is that 5* departments will maintain their funding levels but
that the monies allocated to 5, 4 and 3a departments will be reduced
due to "grade inflation". Thus it may be necessary to
improve a grade in order to stand still financially.
(9) Should the RAE be continued?
If so what changes should be made for the next exercise?
If not, what monitoring system for University research
(if any) should be adopted?
2. RESPONSE TO
QUESTIONS
1. Has the RAE been successful in its goal
of improving the quality of research in UK University departments
over the period 1992-2001?
The majority of respondents believed that the
RAE has contributed to an improvement in the quality of research.
Certainly there has been a significant improvement in grades,
but this may not provide an objective indicator of the effectiveness
of the RAE in this context. It is clear that Universities and
their departments have become more adept at providing RAE submissions
which meet the necessary criteria and this may, in part, contribute
to the grade inflation. SfL believe that it is timely for an objective
assessment of the effectiveness of the RAE to be undertaken. This
could be performed by the Royal Society. Possible additional assessment
criteria could include evidence of international acclaim (prizes,
medals etc), science citations, contributions by UK scientists
to international programmes and, from a technological viewpoint,
number of patents.
2. Does the RAE encourage long term forward
planning within departments?
One of the benefits of the RAE is that it does
encourage strategic planning within departments and across University
institutions. This may be a major factor in improving research
quality. In particular preparing for the RAE has meant that Departments
have in general sharpened up their recruitment by requiring international
standards for new staff or the potential for reaching such standards
in young staff. Compared to 20 years ago or more there are, without
doubt, far fewer research-inactive staff. Jobs in the best research
departments have been opened up internationally and there have
been significant appointments from abroad, which can only have
strengthened UK science.
The downside of the procedure is that it results
in a significant degree of selective appointments just prior to
the RAE deadline, which can lead to short-term thinking amongst
departments. This process can detrimentally affect the long term
planning of departments who lose key staff, primarily to bolster
the RAE position of another institution. Certainly any considered
analysis of the RAE process would come to the conclusion that
the time scale to improve research in any area is more than one
RAE period. Critical to long-term planning is recruitment and
retention of new staff who may take many years to reach their
true potential. The current RAE structure may act against this
aspiration.
3. Does the RAE represent a good investment
of time for academics involved in preparing submissions?
Most of the responses submitted expressed concern
at the amount of time and effort which is devoted to the preparation
of RAE submissions. For individual departments this is clearly
beneficial if it results in an increase in funding but the inquiry
should take a strategic view based on a cost-benefit analysis.
Certainly the preparation of submissions is typically done by
the most research-active members of a department and seen from
a national point of view the exercise takes up far too much time.
SfL recommends that the whole process is simplified and streamlined
and that the interval between exercises should be increased, perhaps
to seven years, to concur with the general time-scale for strategic
research developments to come to fruition within departments.
4. Should departments be able to "play
the game" by submitting only those staff they choose or should
all staff have to be submitted?
The broad consensus from our membership was
that departments should be in a position to choose which staff
they submit and that it was essential that University departments
maintained a range of staff, some of whom may concentrate on teaching
and administrational activities. Teaching and administrative jobs
are equally important in a University. In every highly successful
research department one will find staff who are outstanding teachers
or do the admissions or are senior tutors, or who run the exams.
What is critical is that such people are appreciated and not made
into second class citizens. They should be valued for what they
do very well and their contribution to research is as part of
the departmental team. To insist that all staff are submitted,
while maintaining the current scoring system, would force departments
into making appointments which could interfere with other university
missions, such as teaching.
A number of respondents strongly dissented from
the views expressed above and suggested that all staff must be
submitted to prevent departments submitting 50 per cent of their
staff in order to obtain a 5*D, while subsequently advertising
themselves as a 5* department. This issue could be overcome by
amending the grading system to produce a single grade which combines
elements from both grading schemes. For example, the current 1-5*
scale could be reformed as a 1-7 scale and combined with the A-E
scale (5-1) to produce a single 1-12 scale. A 5*A would become
12 while a 5*D would be 9, and this number would be the main published
indicator. A more detailed breakdown would be provided to discriminate
between 5*D and 4B, both of which would be a 9. It would be difficult
to "hide" the percentage of staff included and departments
would be encouraged to submit more staff. Moreover this type of
scale would be flexible enough to permit the specific inclusion
of other factors if these were deemed important. Examples could
include other forms of output (see point 5 below) or the quality
of the research infrastructure (laboratories, technical support
etc).
5. Does the current RAE structure lead to
unfair discrimination against academic output which may not be
in the form of peer reviewed papers (eg contract work, soft-ware
or hard-ware development)? In addition does this provide an undue
bias against the New Universities who may concentrate on this
type of activity?
Generally our respondents were satisfied that
the RAE structure does not unfairly discriminate against non-peer
reviewed output. It was emphasised that the RAE is fundamentally
a method of distributing money from the Funding Councils to support
research. It is, therefore, right to neglect contract research
not because it is unworthy but because it should be attracting
funding into the department through overheads. Moreover, contract
work that involves really interesting innovative results or addresses
intriguing scientific or technological questions almost inevitably
also leads to publications in peer-review journals, although publication
may be delayed due to intellectual property protection priorities.
Moreover panels are able to decide for themselves what sort of
work they are willing to consider, and those in areas in which
much research does not lead to peer reviewed papers can take this
into account.
6. Does the RAE structure lead to unfair discrimination
against groups involved in interdisciplinary research, which may
not fit comfortably within a specific Unit of Assessment, thereby
favouring traditional disciplines?
The consensus of opinion was that this was a
serious issue, but one that the RAE had attempted to address in
the 2001 exercise (following the report "Interdisciplinary
Research and the Research Assessment Exercise" RAE January
1999), albeit not entirely successfully. A number of respondents
believed the RAE inhibits interdisciplinary research and that
institutions and individuals may alter the way they do research
as a result of the RAE. This does not appear to be due to systematic
discrimination by RAE panels but due to a recognition amongst
individual departments and institutions that the presence of large
and coherent interdisciplinary groups will create difficulties
during the preparation of a submission. The division of the RAE
into traditional disciplines (many created in the 19th century)
actively discourages interdisciplinary research, and is perhaps
the single biggest criticism of the RAE. There are sometimes boundary
disputes about whether an individual researcher should be in one
unit or another. More problematical is that each department or
unit of assessment thinks strategically only in terms of its own
result. Thus there is a pronounced disincentive for making appointments
that develop non-traditional or multidisciplinary themes. In addition,
many panels rate research output in terms of publication in high-impact
factor journals but many of these journals tend not to publish
interdisciplinary research. SfL suggest that a major reform of
any future RAE would be to find ways of encouraging rather than
discouraging interdisciplinary endeavours.
7. Does the RAE structure lead to unfair discrimination
against particular types of staff, leading to de-motivation or
a two-tiered system, if they are not submitted (eg staff, particularly
women taking career breaks, staff with high teaching loads)?
This was felt to be an internal management issue
and therefore not directly attributable to the RAE structure.
The RAE does not itself discriminate against staff who are not
submitted. The funding goes to the department or the institution,
not to the staff. The RAE does, and should continue, to allow
justified extenuating circumstances, such as illness or parenthood,
to be considered for otherwise active research staff. There has
always been a tension, as well as a synergy, between teaching
and research within University departments. The RAE has tipped
the balance further towards research in all forms of recognition
and especially in promotion and thus provides a further inducement
to research-active staff to concentrate on their research at the
expense of teaching, while other staff necessarily carry a heavier
teaching load. This does cause de-motivation amongst staff who
concentrate on teaching and administration. The RAE would not
be a cause of such serious de-motivation if the universities were
not so under-funded and the staff so badly paid.
8. Should the financial model be designed
such that departments who improve their ratings will automatically
be rewarded through the financial model?
This question provided the most significant
concerns over the whole RAE process. In the words of one respondent:
"...I think that the RAE is at best a secondary
issue. The real question is whether the Government is willing
to spend enough on university research, and all the indications
are...that it is not. What is `enough'? Well, something like the
average of our competitors would be a start."
The funding that follows from the RAE is the
most critical issue of the moment particularly following the comments
of Margaret Hodge. The lack of extra money will be seen as a huge
snub to the research community and place many Universities in
great difficulty. Most if not all Universities will face an almost
impossible dilemma. Do they reward those who have done well or
invest in those who have not done so well? It seems that without
extra funding attempting both will be impossible. Perhaps the
biggest problem with the 2001 RAE is the huge financial turbulence
that it has created, which is greatly strengthened by the lack
of additional funds. Departments which have not been upgraded
or have been downgraded will be demoralised. Those that have done
well will change quickly from celebration to annoyance and even
demoralisation when they find there is no extra resource and that
an institution has to even reduce their income to support departments
that did not do so well. In this sense the RAE could turn into
a disaster for research. No-one likes to be successful, do what
is asked of them (improve research in the UK) and then find it's
not getting any reward at all. The RAE, without extra funding,
is a recipe for demoralisation of the whole sector.
9. Should the RAE be continued?
If so what changes should be made for the next
exercise?
If not, what monitoring system for University
research (if any) should be adopted?
The majority of respondents believed that the
RAE process should be continued, but in an amended form. Certainly
it is inconceivable, nowadays, that expenditure of several hundred
million pounds can be done without some mechanism of accountability.
The only sensible approach is to recognise that some mechanism
is needed. Taken overall SfL believe that the RAE process has
been beneficial, notwithstanding the drawbacks mentioned above,
but it must be made simpler and cost less in money and time to
universities and their staff.
A significant dissenting body voiced the opinion
that the RAE was not an appropriate mechanism for assessing research
activities and allocating funding. An alternative would be to
allow Universities to monitor themselves but with clear rules
that mean that they have to be objective. Of critical importance
is to have external and in particular international opinion and
judgement involved. As one example the Royal Society or a Research
Council (or other similar body in a particular field) might appoint
a list of internationally distinguished individuals who would
be willing to serve on internal reviews of UK departments in particular
fields.
Dr David Lee
Department of Engineering, Queen Mary College, University
of London
January 2002
| 
