Peer review

Written evidence submitted by the
Higher Education Funding Council for England (PR 57)

1. The Higher Education Funding Council for England (HEFCE) co-sponsors and manages a major periodic national peer review exercise in the form of the Research Excellence Framework. This note sets out some key points about the structure and operation of the exercise that the Committee may find helpful.

2. HEFCE distributes public funds to higher education institutions (HEIs) in England for teaching, research, and related activities. In 2011-12 the Council’s total recurrent grant for research will be £1,558m.

3. The four UK HE funding bodies jointly conduct periodic exercises to assess the quality of research activity in HEIs across the UK. The last of these was conducted, as the Research Assessment Exercise (RAE), in 2008; a further exercise, to be conducted as the REF in 2014, has recently entered its implementation phase. RAE/REF serves several purpose: its primary purpose is to inform allocations of research grant by the funding bodies, but it also has an important role in assurance – demonstrating what excellent research activity has been supported through the funding; in public information, making available the outcomes of expert review of research in HE; and as a tool for managers within HE. The grant allocation methods of the four funding bodies vary but they share a commitment to allocating the funding selectively by reference to judgements of research quality. RAE/REF is the only case where HEFCE makes systematic use of peer review processes.

4. Both RAE and REF are conducted through the expert review of written submissions from the HEIs, describing their research activity over the previous five years or so and identifying certain published outputs for consideration by the panels - up to 4 for each active researcher employed by the HEI and named in the submission. We use the term "expert review" because this is conducted by panels including both leading active researchers ("peers") and people with significant experience in commissioning and using research in the field. The exercises could be described as a form of secondary peer review which is both retrospective (looking back at achievements during the assessment period) and prospective (identifying where the capacity exists to undertake internationally competitive research in the next period).

5. The outcome of the REF will be in a similar form to that of the 2008 RAE: graded quality profiles, showing for each academic discipline how much activity meeting specified quality levels was found in each HEI that made a submission within that discipline. The 2008 exercise looked at all research under 67 subject headings and in 2014 there will be 36 rather larger subject groups.

Selection and role of reviewers

6. For 2014 the panel members are selected through a two stage process: the panel chairs are selected by the funding bodies from a pool of applicants, having regard to their endorsement from within a wide group of academic, user and other interests; and they recommend the membership from a field nominated by this wide group, having regard to the need to cover the full range of academic activity as far as possible. Panel members are appointed by the funding bodies in a personal capacity based on their professional experience and standing.

7. At present the funding bodies have appointed a total of some 750 panel members to 36 sub-panels, working under the guidance of 4 over-arching main panels, and the first round of panel meetings are being held during 2011. These meetings will develop the criteria for assessment to be used in 2014; for the assessment phase of the exercise in that year we envisage appointing additional panel members to broaden and strengthen both the subject coverage and the extent of user engagement in the process.

Multidisciplinary research

8. Each of the sub-panels covers a specified academic field, with the intention that between them they should cover the entire field of research activity. Multi-disciplinary and interdisciplinary research will need to be submitted to one of these panels; the exercise provides for the assessment of such research by suitably qualified people through the range and depth of experience of the panel members including the additional associate members, through the increased breadth of coverage of the individual sub-panels, and through arrangements for work submitted to one sub-panel to be referred to another where necessary.

Use of online resources and statistical data

9. During the extensive consultations leading up to the 2014 REF we established that there is not general support within the academic community for making more systematic use of statistical data within the assessment process, beyond informing the judgements of the panels. In particular, we have ruled out the systematic use of citation data as a key indicator of research quality at present. The sub-panel chairs have mostly indicated that they would not wish to use this data at all (especially in humanities and social science disciplines) or that they would wish to be able to refer to it within a primarily judgement-driven assessment process (most science based disciplines). REF will make considerable use of IT systems to collect and circulate the submissions and to give panel members online access to published outputs for review wherever this is possible.

Peer review process

10. Expert review and the exercise of academic and other professional judgement are at the core of our approach. The sub-panels will review the research outputs that have been submitted by HEIs. The majority of these outputs have been published in peer reviewed journals, but a range of other types of outputs are also submitted, including monographs, conference proceedings and outputs not in conventional print media. The outputs are graded on a four point scale (with the highest grade being ‘world leading’), to produce an intermediate sub-profile that shows the proportion of submitted work at each grade. In 2014, the panels’ assessment of the quality of the cited research outputs will account for 65% of each of the overall quality profiles, with the assessment of research impact accounting for 20% and research environment for 15%.

11. Details of the assessment process, including the names of the panel members, and panel criteria will be published well in advance and there will be a further consultation on the terms of the criteria before these are finalised. For 2014 we are requiring considerable consistency of criteria and approach between the panels, with consistent quality standards applied throughout – including through the role of the 4 main panels in signing off the quality profiles – and as a general rule the sub-panels may vary the common process only where they have a strong reason to do so within their field. Thus the assessment process is demonstrably robust and as transparent as we can reasonably secure compatible with the need for expert judgement to be exercised free from external pressure of any kind.

12. HEFCE regards RAE/ REF as a very strong mechanism both for supporting our funding allocations and for public information and assurance. The framework and processes for the REF have been established following an extensive process of consultation with the academic community and wider public.

Further information

13. Further information about the REF, including material released so far setting out the detail of its implementation, is available online at www.ref.ac.uk.

Declaration of interests

14. There are no relevant interests to declare.

Higher Education Funding Council for England

9 March 2011