Peer review in scientific publications - Science and Technology Committee Contents


Written evidence submitted by Vitae (PR 95)

1.  Vitae is supported by Research Councils UK (RCUK), managed by CRAC: The Career Development Organisation and delivered in partnership with eight regional Hub host universities.

2.  Vitae works in close collaboration with all UK higher education institutions ((HEIs) to embed professional and career development for researchers into the research environment. Vitae provides HE sector leadership, enabling strategic policy interaction between funders and HEIs, building an evidence base of the impact of researchers and career destinations. We play a major role in providing professional training, developing resources and training materials, sharing practice and enhancing the capability of the higher education sector to provide professional development and training for researchers. Our vision is for the UK to be world-class in supporting the personal, professional and career development of researchers. We work with both postgraduate researchers studying for doctoral degrees and research staff employed in institutions primarily to do research.

3.  Within our broader commentary, our response focuses on the implications of the peer review process on the career development of researchers. This includes how early career researchers develop the requisite skills and knowledge to be effective peer reviewers, become involved in the process and the implications for equality and diversity. We comment on the level of recognition associated with peer reviewing in terms of career progression and workload management. In developing this response we canvassed views from senior academics and staff developers through the Vitae network.

THE STRENGTHS AND WEAKNESSES OF PEER REVIEW IN PUBLISHING

4.  Peer review is a critical part of the process of producing research. Overall it has proved to be the most effective system for assuring the quality of research outputs. It provides a mechanism by which the integrity and authority of the research can be assessed by informed reviewers prior to publication, thereby providing a level of confidence to researchers, research users and the public. It is critical for public confidence in research that we are able to demonstrate that the peer review process is fair, inclusive, transparent and robust.

5.  As well as assessing the merit for publication, the peer review process contributes to the rigour of the research; referee reports often providing suggestions to strengthen the presentation of the research. Being a peer reviewer improves critical thinking, skills for giving and receiving feedback, and preparing their own work for publication. These are important aspects of the development of research leaders for the future.

6.  Most researchers will experience both authoring and reviewing papers during their careers and therefore have a vested interest in the system being as robust, ethical and equitable as possible. Engaging in the peer review process is seen as part of being an academic researcher and contributing to the overall health of the sector. It is not a perfect system however: there are tensions between the need for timely publication and the peer review process. The scale and diversity of the process mean that consistency of quality in reviewing is challenging, if not impossible to achieve. There is evidence that questions the objectivity of the process and whether bias and personal views influence academic judgement.[74]

7.  There is an expectation that researchers will contribute to sustaining the peer review system by participating as reviewers. This is predominantly without financial or formal recognition, except for members of editorial boards (or grant review panels). The process of reviewing is time-consuming and seen as an accepted and necessary activity, yet it is rarely acknowledged as part of the formal workload of an academic researcher. Senior researchers may delegate to early career researchers, providing a useful development opportunity, but not always mentoring or acknowledging their contribution. Reviewing is often an "out of normal hours" activity and therefore adds additional burdens on researchers with family and caring commitments. "Good" reviewers are more likely to be invited to do more reviewing, thereby adding to their workloads.

8.  Given the lack of recognition, contributing to the peer review process does not significantly contribute to a researcher's career progression opportunities. Research outputs are critical in achieving and sustaining a research career and engaging in peer reviewing may in practice reduce the opportunity for focusing on producing research outputs. However this is tensioned against the fact that engaging in peer review can help researchers improve and attune their own publications. It is important that researches at all stages in their career are given the opportunity and recognition for their peer review efforts.

MEASURES TO STRENGTHEN PEER REVIEW

9.  The majority of researchers become experienced in peer review by engaging with the process: "learning on the job". This has its strengths and weaknesses. It is a very effective way of learning, provided it is acknowledged as a learning process and appropriate support is provided, such as mentoring and providing feedback on reviews during the process, to improve their expertise. This could be by editors, fellow reviewers or experienced researchers and provided as part of a managed programme of researcher development rather than in an ad hoc manner.

10.  Until recently there were few opportunities for researchers to undertake formal training. The advent of Vitae and government funding through the UK Research Councils for implementing the recommendations of the Sir Gareth Roberts review[75] have significantly increased the opportunity for early career researchers to participate in professional development opportunities, including academic writing for publication and grant applications. These courses generally include experience of the peer review process. There are also examples of universities and other bodies providing structured development opportunities in being a peer reviewer, including encouraging early career researchers to set up and run journal clubs.[76] However, the numbers participating in these activities are fairly small and with the end of "Roberts funding" in March 2011 even this level of provision may be may fall.

11.  In recognition that peer review is important to the development of leading researchers and that peer reviewing is integral to academic publishing, all parties have a responsibility to continually improve and assure the quality and robustness of the system. We present a range of possible activities for different stakeholders.

—  Editors of journals using fair and transparent selection procedures for peer reviewers. Providing more opportunities for training peer reviewers, providing feedback to reviewers and mentoring opportunities for new reviewers. Collectively agreeing and promoting the basic principles of peer reviewing for publication. Considering the use of more double-blind reviews to reduce bias.

—  Universities providing more development opportunities for building peer reviewing skills prior to researchers becoming peer reviewers for journals, e.g. running journal clubs and training courses that include opportunities to review papers and receive feedback. Peer review and its demands on time should be taken into account in implementing equality and diversity strategies, and its accomplishment recognised in performance management and workload models.

—  Senior academics and principal investigators taking active roles in mentoring early career researchers in peer reviewing skill and providing feedback. If delegating reviewing to others, providing critical oversight and acknowledging their contribution. Encouraging and recommending early career researchers to engage in the peer review process.

—  Staff developers and trainers ensuring that researchers are given opportunities to develop peer review skills, especially in multi-disciplinary and international settings which now underpin much of collaborative research publication.

—  Early career researchers taking responsibility for ensuring they understand the peer review process from both the perspective of writing for publication and being a peer reviewer. Taking opportunities to develop their experience and skills relating to reviewing, including asking peers and senior academics to comment on any papers/reviews before submission. The Vitae Researcher Development Framework highlights the need for resources through which researchers can engage in their own professional development, including skills for publishing and peer reviewing.[77]

THE VALUE OF PEER REVIEW IN ADVANCING KNOWLEDGE

12.  From the prospective of potential authors, particularly early career researchers, the peer review process has a valuable role to play in contributing critical comment and feedback that provides useful quality benchmarks. It is one of the ways in which researchers understand the requirements of publication and improve their chances of being published. However, concerns were expressed that reviewers can be seen as conservative and especially, early career researchers may be better at playing it safe, rather than submitting controversial or cutting edge papers. There was also concern that reviewers may not be objective when reviewing papers that conflict with their own views. This impression, correct or otherwise, could stifle innovation, energy and vitality in publication, especially from early career researchers.

IDENTIFYING REVIEWERS WITH THE REQUISITE SKILLS AND KNOWLEDGE

13.  The view of the all the respondents was that the selection of peer reviewers is not set up as a fair or transparent process. Understandably, editors will look for researchers who are experts in their field and from institutions with a strong research profile in the field. Typically, early career researchers will become involved by either recommendation from a senior academic or because they are known to the editor of a journal through their publications. Open calls for journal reviewers exist, but are not the main method, although it is a more common method for setting up grant peer review panels.

14.  In terms of equality and diversity, systems that rely on networks and patronage may disadvantage specific groups or individuals. For example, early career researchers working in the less research intensive universities may find it hard to break into the system unless they have a specialist research niche.

15.  It is likely that the selection of peer reviewers is predominantly on their research record. However, the ability to give and receive feedback constructively is also important. It is not apparent how this is taken into account when identifying potential reviewers. Furthermore, as an important skill for research leaders, it should be an integral part of the development of researchers.

FURTHER INFORMATION

16.  Further information about Vitae and its activities is available online at www.vitae.ac.uk.

DECLARATION OF INTERESTS

17.  There are no relevant interests to declare.

Vitae

May 2011


74   Rees, T (2011) "The Gendered Construction of Scientific Excellence" Interdisciplinary Science Reviews, Special Issue on Gender in Science, Vol. 36, No. 2, pp. 133-45 (in press - June) Back

75   Roberts, G (2001) SET for Success: the supply of people with science, technology, engineering and mathematic skills, DUIS (BIS) www.vitae.ac.uk/roberts  Back

76   Training peer reviewers, Nature 443, 880 (18 October 2006) | 10.1038/nj7113-880b Back

77   The Researcher Development Framework describes the knowledge, behaviours and attributes of successful researchers, including those relating to publication. www.vitae.ac.uk/rdf The associated Researcher Development Statement has been endorsed by the key stakeholders in developing researchers, including RCUK, Universities UK, the funding bodies and the Quality Assurance Agency www.viate.ac.uk/rds  Back


 
previous page contents next page


© Parliamentary copyright 2011
Prepared 28 July 2011