Peer review

Written evidence submitted by the
British Medical Association (PR 10)

Executive Summary

· Peer review is a highly effective method to quality assure academic research, but has some flaws which warrant scrutiny.

· The strengths of the peer review process include:

o improving research quality

o scrutinising research to ensure that verification and reproducibility are possible

o facilitating scientific discussion and debate.

· However, the BMA believes that there are some concerns:

o the use of information technology to distribute unfiltered research into the public domain presents challenges

o peer review would be considerably improved by formalised training of reviewers

o the current system is unsustainable due to a lack of volunteer reviewers. The BMA suggests professional recognition, accreditation or development of a reward system to encourage participation.

About the BMA

1. The British Medical Association (BMA) is an independent trade union and voluntary professional association which represents doctors from all branches of medicine throughout the UK. It has a membership of over 144,000 doctors.

Introduction

2. The BMA welcomes the opportunity to contribute to the Committee’s inquiry on peer review. The main channel of communication of British research excellence is publication, which at present, as elsewhere in the world, depends on peer review. The British medical publication industry itself is highly respected and highly profitable, and we believe that this can be attributable to the British tradition of peer review.

3. Despite this strong system of peer review, there have been recent but rare major failures of the peer review and editorial system. An example of this was studies on the effect of the Measles Mumps and Rubella vaccine [1] . Other issues such as distribution of unfiltered research, the need for formalised training for reviewers and the shortage of of volunteer reviewers justify some fresh scrutiny of the system.

What is Peer Review?

4. Peer review is the process whereby research is subjected to independent scrutiny by others, who have been selected (e.g. by a journal editor) specifically because they are regarded as experts in the field of study. Hence the experts are "peers" in the same way as used in law. By tradition (albeit a relatively recent one) peer review is expected to occur before the results of research are "made public" by appearing in print.

5. In medicine and related sciences, peer review is generally regarded as an arbiter of scientific quality. The hypothesis is that peer review identifies the most reliable papers and ensures that what does appear is original and important. Exposing research to a peer review process, it is argued, will ensure that the claims made on the basis of the research results, as well as the findings themselves, are verified.

6. Modelled on the high regard in which peer review has been held, similar general principles of scrutiny have also been applied in related areas, such as in the review of grant applications, and in nationwide resource allocation activities, such as the Research Assessment Exercise (RAE) [2] .

Advantages and Disadvantages of Peer Review

7. Advocates of the peer review process believe that it improves research quality, scrutinises research to ensure that verification and reproducibility are possible, and facilitates scientific discussion and debate. In addition, peer review can have an important "policing" role, ensuring adequate acknowledgement and reference to previous research, and helping to identify fraud and plagiarism.

8. In the medical field, the value of peer review has been emphasised as a means of protecting the public against the threat that poorly designed and executed research, with dubious findings, will enter the public domain.

9. "Open access" methods (where authors put their data on-line directly assuming that their peers will comment on-line as well), without peer review, are now the norm in some disciplines, notably mathematics and physics. A concern expressed by many doctors regarding this approach is that in medicine, an additional filter is necessary and that essential "informed understanding" of research may be difficult to achieve without peer review.

10. Counter arguments against peer review include the view that public access to scientific research is increasingly via direct information technology rather than via the filtered selective method epitomised in the medical journal; and the view that peer review itself harbours the potential for bias and prejudice.

Medical Professional Attitudes

11. Although there have been several studies of the peer review process, consultation with the medical profession has not been a prominent feature in any of them. Broadly speaking, however, the BMA believes that most doctors act honestly and sincerely when asked to peer review, and that they support its use, although its organisation could be improved. In particular, the role of editors and their professional responsibility to authors and peer reviewers as well as to publishers needs to be fully considered. The BMA is also very concerned that within the medical field, the peer review system may well be unsustainable as it stands at present; there are too few willing reviewers available to cope with the massive increase in papers that are being submitted for peer review with a view to publication. This has to be seen against a background that (as noted above) peer review remains central not only for papers (research output) but also for research grant funding and as part of assessment under the replacement for the Research Assessment Exercise, the Research Excellence Framework (REF), and of employing institutions increasingly imposing time restrictions on activities which are not deemed to be economically justifiable.

Training

12. Although critical review of published research is now a feature of undergraduate medical education in many medical schools, a different type of critique is required for peer review. Originality, accuracy of methodology, ethical issues, declaration of conflicts of interest, literature citation and statistical accuracy take on more prominence in quality control of research. It is remarkable that there is no formal training process in place for such an important mechanism to ensure scientific quality. Guidance from a publisher alone, who may have parallel but different priorities, is not adequate. The BMA favours a system that provides proper peer review training as an option within postgraduate training.

Anonymity

13. At present much peer review is undertaken anonymously. The editor is the only person who knows the identity of the reviewers. There have been attempts to alter this by making the identity of the reviewers known to the authors. Others wish to take this further by publishing the reviews alongside articles, even when there is selection rather than "open access". This might be attractive, as it would encourage reviewers who would achieve a publication themselves. The BMA position is that there are balancing advantages in retaining a confidential system provided that there are objective (and possibly random) mechanisms in place to check that there has not been abuse of process. In particular, the more junior reviewers are protected by the present system, which mitigates against malicious retribution and bullying. This may help encourage currently under-represented parts of the profession, including women, to come forward as reviewers.

Funding

14. Researchers and academics value the altruistic nature of peer review which supports reviewers to volunteer knowing that when their research is due for publication, others will do the same for them. However, the BMA is concerned that the volunteer system is at risk because there are too few peer reviewers for the growing workload. Reviewing submitted articles can be an extremely time-consuming process. Therefore, in order to maintain the system, it is essential that the undertaking of peer review (both for papers and for grant review and RAE/REF activity) needs to be duly acknowledged within job plans, accredited for continuing professional development, and acknowledged positively in the appraisal and revalidation [3] processes. Limited incentives, such as payment in kind (e.g. subscription), might encourage greater participation in peer review. Such incentives must not be allowed to originate in any way from the author, even if there is a page charge system in place within the journal. Including peer reviewing activity as an activity that is accreditable for the purposes of continuing professional development could be another way of incentivising support.

Information Technology

15. The impact that information technology and online resources has had on the peer review process as represented by "open access" has already been noted above. A notable positive impact of these advances is the increased access to a wider literature, so that peer reviewers can identify related publications instantly. Plagiarism can be recognised far more readily. Peer reviewers can request correction of language more comfortably than in the past. Finally from a UK point of view it has been suggested that the impact of these resources has been greater than it has been on other countries, since it has increased the dominance of the English language within medicine, and hence the increased demands on and for peer reviewers proficient in English.

Impartiality

16. The BMA believes that peer review must be sufficiently broad-based to minimise prejudice against an area of research. At the same time care must be taken that peer review is sufficiently informed about the nature of the research concerned, despite its specialist nature, avoiding rejection of high quality material because it is not understood. There is real concern in small research areas that peer reviewers may condemn papers from rivals and thus reject submissions simply because they are hypothesis-generating rather than being based on experimental or clinical trial data.

17. The converse is also true. For example, there has been considerable disquiet over many years that those who do not accept a viral cause for the Acquired Immune Deficiency Syndrome have been able to publish their hypotheses in non-peer reviewed journals, and thus these ideas have gained more credibility than is justifiable.

18. The BMA believes that the crux of this discussion is that impartiality cannot be guaranteed completely, but that awareness of potential prejudice by peer reviewers and within the system in general is what is required.

Alternatives to Peer Review

19. Although there are flaws in the peer review process, the BMA view is that at best it is highly effective, and that it is the best system available. There does not seem to be a practically viable alternative. A system of assessment by small panels and editors, essentially a triage method, is effectively a peer review with less stringently defined expertise.

Conclusion

20. Peer review can be an extremely useful mechanism for quality assuring research and publications. Removal of peer review would reduce the standing of the published paper. There would be no clear safeguards or checks in place on content. Overall, the BMA supports retention of the system as a primary checkpoint in publication, and believes that giving peer reviewers and the peer review system more respect and support will enhance productivity and performance within it.

British Medical Association

25 February 2011