Written evidence submitted by the Wellcome
Trust (PR 55)
SUMMARY
1. Peer review ensures that scientific findings
and research funding proposals are subjected to independent scrutiny
by experts in the field, and as such is a crucial element of the
scientific endeavour. The quality of peer review depends entirely
on the expertise, rigour, generosity and fairness of the researchers
who undertake it. Although this approach is, at times, onerous
for the research community, it does provide a form of continuing
professional development through their involvement. There is
a need to increase levels of understanding of peer review amongst
policymakers and the wider public - including its importance,
its limitations and the added value provided by the researchers
who conduct it.
THE STRENGTHS
AND LIMITATIONS
OF PEER
REVIEW
2. We consider that the process of peer review
is an integral and irreplaceable part of the scientific enterprise
- both at the publication stage and in the context of research
funding decisions. At publication, it ensures that research findings
have been scrutinised by experts in their field (who may be from
academia, industry or other sectors) and are supported by the
underlying data. Similarly, at the funding stage, it helps to
ensure that the allocation of funds is based on appraisal of the
scientific quality of research proposals by independent experts.
Although peer review is not a perfect system and imposes a significant
burden on the research community, the checks and balances it provides
are absolutely core to the scientific endeavour. We do not believe
that any viable alternative model exists.
3. A key concern raised in relation to the current
peer review system is the associated workload for researchers
who provide this review, often on an unpaid basis. Reviewing
submitted papers and funding applications forms an integral part
of the work of these researchers, and does provide benefits in
terms of their ongoing development and breadth of knowledge of
their field. But the burden is considerable. A recent study
by JISC estimated that UK academics spend a total of two to three
million hours per year acting as reviewers, at a cost to the university
sector in terms of academic time of between £110 million
and £165 million.1 Meanwhile, the volume of published
research is continuing to increase. Senior scientists in a given
research field may often be in particularly strong demand to undertake
reviews. The pressure on researchers - especially those leading
their fields - leads to a significant proportion of requests for
review being turned down, and may in turn limit the depth and
quality of review in the system.
4. Other commonly raised criticisms of peer review
are that it can sometimes slow or limit the emergence of new ideas
that challenge established norms in a field; that it has the potential
to be abused by scientists in some cases to protect their own
interests; and that it can lead to undue delays in the dissemination
of scientific knowledge - for example, when reviewers request
additional experiments that may have been beyond the scope of
the original study.
MEASURES TO
STRENGTHEN PEER
REVIEW
5. Whilst we believe peer review remains vital,
it is important that publishers and funders actively explore ways
in which they can help to reduce its burden, whilst not compromising
its quality.
Peer review in publication
6. We believe that the continuing transition
towards open access publishing approaches over recent years offers
some important opportunities. In particular, it can help to ensure
that high quality research can find a route to peer-reviewed publication,
and that the entire published output of research is free at the
point of access for ongoing review and scrutiny by the scientific
community after publication - allowing new ideas to emerge no
matter where they are published, and ultimately providing more
opportunities for the quality of a paper to be judged on its intrinsic
merit (rather than the journal in which it appears).
7. The key challenges in the peer review system
in the context of the open access movement were discussed at a
recent workshop on "Innovation in Scientific Publishing"
which was convened by the Wellcome Trust in partnership with the
Howard Hughes Medical Institute and the Max Planck Society - the
consensus emerging was that:
— the
burden on researchers of reviewing papers is excessive, and we
need to move away from the current system where the same paper
is often reviewed multiple times by different journals;
— whilst
delegates agreed on balance that reviewers should not be paid,
there was wide consensus that their contribution needed to be
recognised; and
— reviewers
should not be encouraged to ask for additional obvious experiments
over and above those reported - if they do, editors need to use
their judgement to curtail this.
8. Several publishers are already adopting innovative
approaches to reduce the burden of peer review that are worthy
of note. The approach adopted by PLOS One - where the
peer review process focuses solely on whether the findings and
conclusions are justified by the results and methodology presented,
rather than on assessment of the relative importance of the research
or perceived level of interest it will generate - has both reduced
the burden on the reviewer and the time it takes to get a paper
published.2 Another interesting model is that adopted
by the Society for Neuroscience - where reviews of rejected papers
are passed onto other journals to which the paper is subsequently
submitted.3
9. Several publishers are also seeking to introduce
more transparency into the peer review process - with some journals
now using fully open peer review, where the identity of reviewers
and their comments are published alongside the article. This
could offer some benefits in terms of increasing quality, ensuring
accountability and potentially enabling greater recognition for
reviewer's contributions. However, it would clearly raise concerns
if reviewers did not feel able to comment openly or fully on the
work, or were discouraged from providing reviews altogether.
We would argue that it would not at this stage be appropriate
for all forms of peer review.
Peer review in funding decisions
10. As a research funder, there are a number
of approaches we are using to attempt to ease the burden on reviewers,
whilst ensuring the quality of review. We actively acknowledge
that there are different forms of peer review (for example, peer
review by external "remote" written referee reports,
and peer review by expert committees, sometimes on the basis of
interview) and we apply each judiciously at the appropriate stage
of the application process.
11. The approaches we are currently using to
obtain the most effective, efficient peer review include making
more active use of a preliminary triage process - both through
independent expert advisory committees or our scientifically-trained
staff (where this is appropriate) so that a smaller subset of
grant applications are sent out for review to external referees.
We have also worked to shorten and focus our application and
review forms. In common with other funders, we have used peer
review "colleges" for some schemes to build communities
of trusted reviewers, who have agreed to be included. In addition,
we actively track the number of approaches to reviewers to reduce
over-burdening particular individuals.
IDENTIFYING REVIEWERS
12. The selection of suitable reviewers relies
to a large extent on the expertise of editorial staff at journals,
and scientifically-qualified staff at funding agencies. As a
funder, our staff typically search the published literature for
suitable reviewers, maintain an overview of the field, seek advice
from members of our expert committees, and develop networks of
contacts. Referee selection is also aided by inviting the applicant
to suggest names of suitable reviewers. Whilst it is of course
not appropriate to select reviewers exclusively on this basis,
it can identify a helpful range of expertise appropriate to the
subject area.
13. Selecting the appropriate balance of expertise
for reviewers for a particular proposal is key. As the research
communities in some areas are fairly small, there is often value
in looking beyond specialists in the area covered, and including
a number of reviewers bringing different perspectives. This is
particularly true where there is benefit in examining a proposal
from several angles (focusing variously, for example, on technical
issues, the use of animals or study design).
COMMUNICATING RESEARCH
14. Peer review is at its heart a very simple
concept - namely that of independent expert scrutiny. We believe
that there is an important need to ensure that peer review is
more widely understood amongst both policy makers and the general
public. This should include acknowledgement of both the benefits
and limitations of peer review, and of the significant role played
by academic researchers and the added value they provide. We
consider that the development of high quality engagement activities
to promote understanding of, and dialogue around, the scientific
process and the key importance of peer review within this should
represent a continuing priority for funders. Clearly scientists
have a key role to play, and we encourage them embrace opportunities
to engage in such dialogue.
15. It is also vital that the importance of peer
review is respected by those communicating scientific findings.
In all cases, it should be made absolutely clear whether a particular
finding has been subject to peer review or not, and appropriate
caution exercised where this is not the case. It is vital that
the scientific community, the media, and other groups involved
in science communication act with responsibility and integrity,
and recognise the inherent value of peer review in terms of ensuring
results and ideas are based on robust research. Failure to do
so will risk undermining public trust in science.
THE WELLCOME
TRUST
The Wellcome Trust is a global charitable foundation
dedicated to achieving extraordinary improvements in human and
animal health. We support the brightest minds in biomedical research
and the medical humanities. Our breadth of support includes public
engagement, education and the application of research to improve
health. We are independent of both political and commercial interests.
REFERENCES
1 See "The
value of UK HEI's to the publishing process" (JISC, Nov 2010)
http://www.jisc-collections.ac.uk/Reports/valueofukhe/ (accessed
4 Mar 2011)
2 See "PLOS
One - editorial and peer review process" http://www.plosone.org/static/review.action
(accessed 4 March 2011)
3 See About the
Neuroscience Peer Review Consortium (NPRC) http://nprc.incf.org/about
(accessed 4 March 2011)
Wellcome Trust
10 March 2011
| 
