Peer review

Written evidence submitted by the John Innes Centre (PR 64)

Declaration of interests

JIC is an institute of the UK Biotechnology and Biological Sciences Research Council (BBSRC). JIC is a registered charity (No. 223852) and a company limited by guarantee (registered in England No. 511709).

JIC is an international centre of excellence in plant science and microbiology. Our mission is to carry out fundamental and strategic research, to train scientists and to make our findings available to society. JIC undertakes research in the biological and chemical sciences, including cell biology, biochemistry, chemistry, genetics, molecular biology, computational and mathematical biology. All of JIC’s work is carried out with competitively won grant funding that involves peer review. The results of this work are published in peer reviewed international scientific journals.

1. The strengths and weaknesses of peer review as a quality control mechanism for scientists, publishers and the public

1.1 Strengths:

1.1.1 Peer review is an established and internationally accepted method with a good track record of supporting novelty and innovation and of identifying flawed or poor quality work.

1.1.2 Peer review is based on the principle of using multiple, independent experts who possess up to date knowledge. Editorial boards of scientific journals and members of grant awarding bodies are generally able to use a degree of discretion in assessing expert reviews, allowing the balance of opinion to guide the final decision. All reviews and final decisions are recorded.

1.1.3 Peer review by overseas experts allows the competitiveness and value of UK research to be maintained at global standards. However, as overseas researchers may not have a clear understanding of UK funding mechanisms or strategic goals these need to be explained by those seeking reviews.

1.1.4 Peer review is structured to avoid conflicts of interest. Most grant awarding panels and scientific journals exclude peer reviewers when there is a potential for conflict of interest. Most also allow applicants to identify reviewers they believe will have a conflict of interest. Drawing on overseas expertise is (see 1.1.3) is particularly valuable in areas where relevant UK community is small.

1.1.5 Anonymous review is normal procedure. That is, the applicant for grant funding or publication does not know the identity of the reviewers. This has strengths and weaknesses. Its strength is that reviewers can be openly critical without concerns about damage to themselves. However, it can also lead to careless reviews and the blocking of work from competing research groups. Importantly, reviewers are not anonymous to grant awarding bodies or scientific journals, enabling such conflicts of interest to be identified. Double anonymity (see 2.2.1), where the reviewer does not know the identify of the applicant is impractical as it is practically impossible to write a grant or publication that does not reveal the writer’s identity. Anonymity of the application in grant applications does not allow previous track record to be taken into account, which is a disadvantage.

1.1.6 Opportunity for applicant or author of the manuscript to comment and rebut reviewer’s comments is an important quality control that is operated by most grant awarding bodies and by all journals prior to publication

1.2 Weaknesses:

1.2.1 Increasing time pressure means that that appropriate experts may a) decline to review or b) be unable to provide detailed assessment. This could be addressed by giving more recognition to reviewers for time invested.

1.2.2 The increasing and necessary move to larger collaborative and/or multi-disciplinary projects can exclude many potential reviewers. This is particularly a problem in rapidly emerging innovative areas that may have few researchers. In addition, it may be difficult to identify reviewers with the extensive skills needed to assess multidisciplinary projects.

1.2.3 Peer review by overseas experts (1.1.3) is important but they may not have a clear understanding of UK funding mechanisms or strategic goals. Therefore it is important to explain these clearly.

1.2.4 Peer review is not a formal part of the training of PhD students or scientists in the first stage of career (post-doctoral level). However this is increasingly incorporated into transferable skills training.

1.2.5 When funding budgets are restricted, peer review is perceived to favour "safe" grants that are highly likely to succeed . This does not promote the risk taking highly innovative research that will lead to international competitive advantage and world leading UK industry.

2. Measures to strengthen peer review

2.1 Improved training (see 1.2.4) and greater explicitness in guidelines issued by grant funding bodies and scientific publications (see 1.2.3).

2.2 Changes in the process of peer review.

2.2.1 Anonymity could be extended whenever possible to applicants. It is important consider previous track record, but perhaps this could be considered separately from evaluation of the proposal itself. Anonymity has potential drawbacks (see 1.1.5).

2.2.2 More reviewers could be involved dealing with sub sections of multidisciplinary grants. However, there is a risk that increasing the number of reviewers may favour funding "safe" projects and a "race to the bottom" (see 1.2.5).

2.2.3. We think it is necessary for reviewers to be practising scientists.

2.2.4 Open reviews and encouragement to openly discuss publications in journal forums is an option. Twitter and blog sites are emerging as mechanisms of post-peer review evaluation, and these electronic media could have greater use in the assessment of grants or publications prior to acceptance.

2.2.5 Publication of articles in scientific journals is a very important measure for career progression. However, pressures to publish more articles can lead to "quantity over quality", and high quantity imposes pressures on peer-review as there are only so many experts in any given field. The focus should be on quality over quantity which will reduce time pressure and improve the quality of peer-review.

2.3 Improvement of skills databases to identify the best qualified reviewers.

3. The value and use of peer reviewed science on advancing and testing scientific knowledge

3.1 Peer review is essential because it provides expert assessment of progress and the degree to which knowledge can be considered proven and established.

4. The value and use of peer reviewed science in informing public debate

4.1 Public debate is a consideration of evidence in a cultural context (i.e. within a framework of ethical/societal values). Peer review is valuable because it considers evidence and presents the balance of expert opinion and the limits of current knowledge. That is, it provides an essential evidential foundation for the debate.

5. The extent to which peer review varies between scientific disciplines and between countries across the world

5.1 Peer review of research articles differs significantly from discipline to discipline. This is largely driven by the individual pressures of different fields. Reviews in maths and physics often take significantly longer than in biology or chemistry. Physicists also often pre-publish on the archive in an open-access manner prior to seeking a journal for their work. Community sizes impact on the number of journals and the overall volume of papers and the tendency for quick turnaround. This is also linked to the differing career progression metrics that different fields employ.

5.2 For grants, whilst the mechanisms between countries are similar, the ratio of submitted grants to the available resources is critical in how reviewers report. Competition for funding with a decreasing success rate may influence the attitude of reviewers (see 1.1.4 and 1.2.5).

6. The processes by which reviewers with the requisite skills and knowledge are identified, in particular as the volume of multi-disciplinary research increases

6.1 Funding agencies such as BBSRC hold a skills database of UK and international researchers that is able to link key words in grant applications with suitable reviewers. Most journals operate in a similar way. Our experience is that this does not always work efficiently and a proportion of requests to review are inappropriate. Generally these are returned to the agency or journal with alternative suggestions. The accuracy of databases is crucial and greater detail on individual reviewers would be an asset. Mechanisms to reward participation would also help.

6.2 Multi-disciplinary research is a problem as few reviewers will have the skills to assess every aspect of a grant or scientific publication. Having reviewers consider defined sub-sections is an option, but this may risk underrating the overall value and impact of the work.

7. The impact of IT and greater use of online resources on the peer review process

7.1 IT provides greater access to the world science base for peer-reviewers and rapid exchange of information. IT also diminishes the burden on reviewers by making the administrative process more efficient. IT enables greater use of overseas reviewers giving a better assessment of the international competitiveness of UK science. For grant applications, overseas reviewers are unlikely to be conversant with UK costing systems or strategic priorities but this can be addressed by greater clarity in the review requests.

7.2 The availability of online journals and the powerful development in search engines reduces the risk of plagiarism. It also makes information gathering for reviewers a very quick and easy process. Journals now offer search tools to help reviewers get access to relevant publications that aid the peer review process.

7.3 A number of journals, notably Open Access publications such as PLoS ONE, are exploring new mechanisms for improving the peer process using new IT developments. Open online discussion of published articles can be mentioned in this context, something that has recently been taken up by the Nature journal group. Also, the publication of reviewers' and editor's comments with the paper is being explored as a way to present a transparent view of the publication process, whilst encouraging reviewers to take care with the quality of their reports. Many journals now also offer blogs and twitter.

8. Possi ble alternatives to peer review

Peer-review is important because is provides a balance of expert opinion. We do not believe that removing this would be helpful or desirable, either to the development of UK science or to public debate. Augmenting the process may be possible, as described above.

Professor Dale Sanders

Director John Innes Centre

10 March 2011