Peer review in scientific publications - Science and Technology Committee Contents

1 Background

What is peer review?

1.  Peer review is no more and no less than review by experts.[1] It is pervasive throughout all aspects of academic endeavour.[2] The principles of peer review are commonly applied to "the review of grant applications, and in nationwide resource allocation activities, such as the Research Assessment Exercise (RAE)".[3] Peer review is also used in scholarly publishing, in which it is described by the International Committee of Medical Journal Editors as "the critical assessment of manuscripts submitted to journals by experts who are not part of the editorial staff".[4] Those "experts" are commonly referred to as "reviewers" or "referees".

The importance of peer review in scientific publications

2.  Scientific publications are the public face of science; they are the means by which researchers report and explain their findings to the wider world, including other scientists, practitioners, the public, and policy makers. Professor John Pethica of the Royal Society explained that the primary function of peer review in this context is "to improve the process and the coherence of scientific knowledge and its utility".[5] Peer review is used by publishers to help ensure that the scientific record is robust.

The importance of the scientific record to Government

3.  The peer-reviewed literature represents an organised body of knowledge, reviewed by experts. Professor Sir John Beddington, Government Chief Scientific Adviser, summarised the importance of peer-reviewed literature to the Government: "scientific evidence is clearly fundamental to Government policy and peer review is a fundamental part of scientific evidence. […] it is absolutely clear that scientific evidence is essential for […] the evidence-based policy of the Government".[6]

Previous work

4.  On 20 July 2004, the former Science and Technology Committee published the report, Scientific publications: free for all?, which aimed to examine the provision of scientific journals to the academic community and wider public and establish whether the market for scientific publications was working well.[7] On the issue of peer review, the former Committee concluded:

As is the case with any process, peer review is not an infallible system and to a large extent depends on the integrity and competence of the people involved and the degree of editorial oversight and quality assurance of the peer review process itself. Nonetheless we are satisfied that publishers are taking reasonable measures to [maintain] high standards of peer review. Peer review is an issue of considerable importance and complexity and the Committee plans to pursue it in more detail in a future inquiry.[8]

5.  Shortly before the former Committee's report was published, the Sense About Science Working Party on peer review published the discussion paper, Peer review and the acceptance of new scientific ideas.[9] Since then, peer review has become a more mainstream concept outside of the scholarly community. In April 2005, Sense About Science carried out "a series of workshops with educational bodies, patient groups and information providers to produce a user-friendly short guide to the peer review process".[10] This guide, I don't know what to believe… Making sense of science stories, was published in November 2005 and "hundreds of thousands of copies have been downloaded".[11]

6.  In recent years there have been an increasing number of reports and articles assessing the current state of peer review, in some cases questioning whether the peer-review system is "broken".[12] These reports have come at a time when there are big changes afoot in scientific publishing: the total number of peer-reviewed publications has grown by a third since the beginning of the 21st century;[13] the share of publications by countries which are not traditional scientific leaders, for example China and India, is rising;[14] Information Technology has transformed the administration of peer review through, for example, online submission tools and reviewer databases;[15] and the web (including tools such as Twitter) is providing new and immediate ways for rating and commenting on scholarly publications.[16] In this rapidly changing environment, and in view of the importance of evidence-based scientific information to Government, it seemed appropriate to undertake a detailed examination of the current peer-review system as used in scientific publications. Both to see whether it is operating effectively and to shine light on new and innovative approaches. As a consequence, this report examines the issues at length and we set out the bulk of our conclusions and recommendations towards the end of the report.

Our inquiry

7.  We announced our inquiry into Peer Review on 27 January 2011 and issued a call for evidence based on the following terms of reference:

1.  the strengths and weaknesses of peer review as a quality control mechanism for scientists, publishers and the public;

2.  measures to strengthen peer review;

3.  the value and use of peer-reviewed science on advancing and testing scientific knowledge;

4.  the value and use of peer-reviewed science in informing public debate;

5.  the extent to which peer review varies between scientific disciplines and between countries across the world;

6.  the processes by which reviewers with the requisite skills and knowledge are identified, in particular as the volume of multi-disciplinary research increases;

7.  the impact of IT and greater use of online resources on the peer-review process; and

8.  possible alternatives to peer review.

8.  We received 96 submissions in response to our call. We would like to thank all those who submitted written memoranda. We would also like to thanks Dr Irene Hames, the specialist adviser we appointed to this inquiry. Her expert advice was valuable and we are grateful for her contribution.[17]

9.  In May and June 2011 we held four evidence sessions during which we took oral evidence from seven panels of witnesses, to whom we are grateful:

i.  On 4 May 2011 we took evidence from: Dr Nicola Gulley, Editorial Director, Institute of Physics Publishing Ltd; Professor Ron Laskey, Vice President, Academy of Medical Sciences; Dr Robert Parker, Interim Chief Executive, Royal Society of Chemistry; and, Professor John Pethica, Physical Secretary and Vice President, Royal Society.

ii.  On 11 May we took evidence from: Tracey Brown, Managing Director, Sense About Science; Dr Liz Wager, Chair, Committee on Publication Ethics and Board Member, UK Research Integrity Office Ltd; Mayur Amin, Senior Vice President, Research & Academic Relations, Elsevier; Dr Philip Campbell, Editor-in-Chief, Nature and Nature Publishing Group; Robert Campbell, Senior Publisher, Wiley-Blackwell; Dr Fiona Godlee, Editor-in-Chief, BMJ and BMJ Group; and, Dr Andrew Sugden, Deputy Editor & International Managing Editor, Science.

iii.  On 23 May we took evidence from: Dr Rebecca Lawrence, Director, New Product Development, Faculty of 1000 Ltd; Dr Michaela Torkar, Editorial Director, BioMed Central; Dr Mark Patterson, Director of Publishing, Public Library of Science; Dr Malcolm Read OBE, Executive Secretary, Joint Information Systems Committee (JISC); Dr Janet Metcalfe, Chair, Vitae; Professor Teresa Rees CBE, Former Pro-Vice-Chancellor (Research), Cardiff University; and, Professor Ian Walmsley, Pro-Vice-Chancellor, University of Oxford.

iv.  On 8 June we took evidence from: Professor Rick Rylance, Chair-elect, Research Councils UK; David Sweeney, Director for Research, Innovation and Skills, Higher Education Funding Council for England; Sir Mark Walport, Director, Wellcome Trust; Professor Sir John Beddington, Government Chief Scientific Adviser; and, Professor Sir Adrian Smith, Director General, Knowledge and Innovation, Department for Business, Innovation and Skills.

10.  The report begins in chapter two with an overview of the peer-review process in publishing, including common criticisms and new innovations in publishing. Chapter three explores the roles of the editors, authors and reviewers. Chapter four examines the challenges involved in reviewing data associated with submitted work and storing it after publication. Chapter five looks at the growing area of review and commentary after publication. Finally, chapter six explores public debate and trust in science. It also assesses the role of peer review in preventing fraud and misconduct, as well as the broader ways in which research integrity is overseen in the UK.

1   Q 250 [Sir Mark Walport] Back

2   Q 225 [Professor Ian Walmsley] Back

3   Ev w20, para 6 [British Medical Association] Back

4   "Uniform Requirements for Manuscripts Submitted to Biomedical Journals", International Committee of Medical Journal Editors, Back

5   Q 2 Back

6   Q 287 Back

7   Science and Technology Committee, Tenth Report of Session 2003-04, Scientific publications: free for all?, HC 399-I, para 4 Back

8   Science and Technology Committee, Tenth Report of Session 2003-04, Scientific publications: free for all?, HC 399-I, para 207 Back

9   Sense About Science, Peer Review and the Acceptance of New Ideas, May 2004 Back

10   Ev 74, para 3 Back

11   Ev 75, para 3 Back

12   For example: "Nature's peer review debate", Nature Online,; Mark Ware Consulting, Peer Review in Scholarly Journals - perspective of the scholarly community: an international study, January 2008; and, "Is peer review broken?", The Scientist Online, vol 20, Issue 2, February 2006, Back

13   Royal Society, Knowledge, networks and nations: Global scientific collaboration in the 21st century, March 2011, p 16 Back

14   As above Back

15   Ev w59, para 11 [Academy of Social Sciences] Back

16   Ev 73, paras 21-22 [BMJ Group] Back

17   Relevant interests of the specialist adviser were made available to the Committee before the decision to appoint her on 23 March 2011. The Committee formally noted that Dr Hames declared an interest relevant to the Committee's work as a Council member, Director and Trustee, Committee on Publication Ethics (COPE); as a member of the Advisory Board, Sense About Science; as an author of Peer Review and Manuscript Management in Scientific Journals; and as offering advice to the Association of Learned and Professional Society Publishers. During the course of the inquiry as we took evidence Dr Hames declared further interests as an employee (until 31 October 2010) of Wiley-Blackwell; as a member, International Society of Managing and Technical Editors Industry Advisory Board; and as receiving fees for workshops from Roberts' funding for researcher training and career development.  Back

previous page contents next page

© Parliamentary copyright 2011
Prepared 28 July 2011