1 Background
What is peer review?
1. Peer review is no more and no less than review
by experts.[1] It is pervasive
throughout all aspects of academic endeavour.[2]
The principles of peer review are commonly applied to "the
review of grant applications, and in nationwide resource allocation
activities, such as the Research Assessment Exercise (RAE)".[3]
Peer review is also used in scholarly publishing, in which it
is described by the International Committee of Medical Journal
Editors as "the critical assessment of manuscripts submitted
to journals by experts who are not part of the editorial staff".[4]
Those "experts" are commonly referred to as "reviewers"
or "referees".
The importance of peer review
in scientific publications
2. Scientific publications are the public face
of science; they are the means by which researchers report and
explain their findings to the wider world, including other scientists,
practitioners, the public, and policy makers. Professor John Pethica
of the Royal Society explained that the primary function of peer
review in this context is "to improve the process and the
coherence of scientific knowledge and its utility".[5]
Peer review is used by publishers to help ensure that the scientific
record is robust.
The importance of the scientific
record to Government
3. The peer-reviewed literature represents an
organised body of knowledge, reviewed by experts. Professor Sir
John Beddington, Government Chief Scientific Adviser, summarised
the importance of peer-reviewed literature to the Government:
"scientific evidence is clearly fundamental to Government
policy and peer review is a fundamental part of scientific evidence.
[…] it is absolutely clear that scientific evidence is essential
for […] the evidence-based policy of the Government".[6]
Previous work
4. On 20 July 2004, the former Science and Technology
Committee published the report, Scientific publications: free
for all?, which aimed to examine the provision of scientific
journals to the academic community and wider public and establish
whether the market for scientific publications was working well.[7]
On the issue of peer review, the former Committee concluded:
As is the case with any process, peer review is not
an infallible system and to a large extent depends on the integrity
and competence of the people involved and the degree of editorial
oversight and quality assurance of the peer review process itself.
Nonetheless we are satisfied that publishers are taking reasonable
measures to [maintain] high standards of peer review. Peer review
is an issue of considerable importance and complexity and the
Committee plans to pursue it in more detail in a future inquiry.[8]
5. Shortly before the former Committee's report
was published, the Sense About Science Working Party on peer review
published the discussion paper, Peer review and the acceptance
of new scientific ideas.[9]
Since then, peer review has become a more mainstream concept outside
of the scholarly community. In April 2005, Sense About Science
carried out "a series of workshops with educational bodies,
patient groups and information providers to produce a user-friendly
short guide to the peer review process".[10]
This guide, I don't know what to believe… Making sense
of science stories, was published in November 2005 and "hundreds
of thousands of copies have been downloaded".[11]
6. In recent years there have been an increasing
number of reports and articles assessing the current state of
peer review, in some cases questioning whether the peer-review
system is "broken".[12]
These reports have come at a time when there are big changes afoot
in scientific publishing: the total number of peer-reviewed publications
has grown by a third since the beginning of the 21st
century;[13] the share
of publications by countries which are not traditional scientific
leaders, for example China and India, is rising;[14]
Information Technology has transformed the administration of peer
review through, for example, online submission tools and reviewer
databases;[15] and the
web (including tools such as Twitter) is providing new and immediate
ways for rating and commenting on scholarly publications.[16]
In this rapidly changing environment, and in view of the importance
of evidence-based scientific information to Government, it seemed
appropriate to undertake a detailed examination of the current
peer-review system as used in scientific publications. Both to
see whether it is operating effectively and to shine light on
new and innovative approaches. As a consequence, this report examines
the issues at length and we set out the bulk of our conclusions
and recommendations towards the end of the report.
Our inquiry
7. We announced our inquiry into Peer Review
on 27 January 2011 and issued a call for evidence based on the
following terms of reference:
1. the strengths and weaknesses of peer review
as a quality control mechanism for scientists, publishers and
the public;
2. measures to strengthen peer review;
3. the value and use of peer-reviewed science
on advancing and testing scientific knowledge;
4. the value and use of peer-reviewed science
in informing public debate;
5. the extent to which peer review varies between
scientific disciplines and between countries across the world;
6. the processes by which reviewers with the
requisite skills and knowledge are identified, in particular as
the volume of multi-disciplinary research increases;
7. the impact of IT and greater use of online
resources on the peer-review process; and
8. possible alternatives to peer review.
8. We received 96 submissions in response to
our call. We would like to thank all those who submitted written
memoranda. We would also like to thanks Dr Irene Hames, the specialist
adviser we appointed to this inquiry. Her expert advice was valuable
and we are grateful for her contribution.[17]
9. In May and June 2011 we held four evidence
sessions during which we took oral evidence from seven panels
of witnesses, to whom we are grateful:
i. On 4 May 2011 we took evidence from: Dr Nicola
Gulley, Editorial Director, Institute of Physics Publishing Ltd;
Professor Ron Laskey, Vice President, Academy of Medical Sciences;
Dr Robert Parker, Interim Chief Executive, Royal Society of Chemistry;
and, Professor John Pethica, Physical Secretary and Vice President,
Royal Society.
ii. On 11 May we took evidence from: Tracey Brown,
Managing Director, Sense About Science; Dr Liz Wager, Chair, Committee
on Publication Ethics and Board Member, UK Research Integrity
Office Ltd; Mayur Amin, Senior Vice President, Research &
Academic Relations, Elsevier; Dr Philip Campbell, Editor-in-Chief,
Nature and Nature Publishing Group; Robert Campbell, Senior
Publisher, Wiley-Blackwell; Dr Fiona Godlee, Editor-in-Chief,
BMJ and BMJ Group; and, Dr Andrew Sugden, Deputy Editor
& International Managing Editor, Science.
iii. On 23 May we took evidence from: Dr Rebecca
Lawrence, Director, New Product Development, Faculty of 1000 Ltd;
Dr Michaela Torkar, Editorial Director, BioMed Central; Dr Mark
Patterson, Director of Publishing, Public Library of Science;
Dr Malcolm Read OBE, Executive Secretary, Joint Information Systems
Committee (JISC); Dr Janet Metcalfe, Chair, Vitae; Professor Teresa
Rees CBE, Former Pro-Vice-Chancellor (Research), Cardiff University;
and, Professor Ian Walmsley, Pro-Vice-Chancellor, University of
Oxford.
iv. On 8 June we took evidence from: Professor
Rick Rylance, Chair-elect, Research Councils UK; David Sweeney,
Director for Research, Innovation and Skills, Higher Education
Funding Council for England; Sir Mark Walport, Director, Wellcome
Trust; Professor Sir John Beddington, Government Chief Scientific
Adviser; and, Professor Sir Adrian Smith, Director General, Knowledge
and Innovation, Department for Business, Innovation and Skills.
10. The report begins in chapter two with an
overview of the peer-review process in publishing, including common
criticisms and new innovations in publishing. Chapter three explores
the roles of the editors, authors and reviewers. Chapter four
examines the challenges involved in reviewing data associated
with submitted work and storing it after publication. Chapter
five looks at the growing area of review and commentary after
publication. Finally, chapter six explores public debate and trust
in science. It also assesses the role of peer review in preventing
fraud and misconduct, as well as the broader ways in which research
integrity is overseen in the UK.
1 Q 250 [Sir Mark Walport] Back
2
Q 225 [Professor Ian Walmsley] Back
3
Ev w20, para 6 [British Medical Association] Back
4
"Uniform Requirements for Manuscripts Submitted to Biomedical
Journals", International Committee of Medical Journal
Editors, www.ICMJE.org/ Back
5
Q 2 Back
6
Q 287 Back
7
Science and Technology Committee, Tenth Report of Session 2003-04,
Scientific publications: free for all?, HC 399-I, para
4 Back
8
Science and Technology Committee, Tenth Report of Session 2003-04,
Scientific publications: free for all?, HC 399-I, para
207 Back
9
Sense About Science, Peer Review and the Acceptance of New
Ideas, May 2004 Back
10
Ev 74, para 3 Back
11
Ev 75, para 3 Back
12
For example: "Nature's peer review debate", Nature
Online, www.nature.com/nature/peerreview/debate; Mark Ware Consulting,
Peer Review in Scholarly Journals - perspective of the scholarly
community: an international study, January 2008; and, "Is
peer review broken?", The Scientist Online, vol 20, Issue
2, February 2006, www.the-scientist.com Back
13
Royal Society, Knowledge, networks and nations: Global scientific
collaboration in the 21st century, March 2011, p 16 Back
14
As above Back
15
Ev w59, para 11 [Academy of Social Sciences] Back
16
Ev 73, paras 21-22 [BMJ Group] Back
17
Relevant interests of the specialist adviser were made available
to the Committee before the decision to appoint her on 23 March
2011. The Committee formally noted that Dr Hames declared an interest
relevant to the Committee's work as a Council member, Director
and Trustee, Committee on Publication Ethics (COPE); as a member
of the Advisory Board, Sense About Science; as an author of Peer
Review and Manuscript Management in Scientific Journals; and as
offering advice to the Association of Learned and Professional
Society Publishers. During the course of the inquiry as we took
evidence Dr Hames declared further interests as an employee (until
31 October 2010) of Wiley-Blackwell; as a member, International
Society of Managing and Technical Editors Industry Advisory Board;
and as receiving fees for workshops from Roberts' funding for
researcher training and career development. Back
|
