Written evidence submitted by The Academy
of Medical Sciences (PR 89)
SUMMARY
The
Academy of Medical Sciences believes that peer review is required
to quality assure scientific publications, to the benefit of all
those who use them. It helps to generate trust and consistent
standards in science.
The
physical sciences communities have adopted a model of pre-publication
community peer review. We believe that some level of peer review
before publication is essential in the biomedical sciences because
of the more subjective and open ended nature of the research.
The
desire to maintain impact factors, to reduce print production
costs and the conservative attitudes of some journals can increase
the rejection rates of papers. Researchers may have to approach
a number of journals before a paper is accepted.
High
rejection rates and the requirement for unnecessary revisions
before publication delay access to the outcomes of publicly and
charity funded science and waste the time of researchers. Ultimately
they slow the progression of science.
We
welcome models that use peer review simply to assess the validity
of the scientific approach taken rather than its potential impactresulting
in a faster rate of publication. To identify the likely high impact
papers, post-filtering mechanisms such as the Faculty of 1000
are helpful to researchers and other users of scientific knowledge.
Digitisation
of scientific publication has facilitated greater use of, and
access to, peer reviewed science. It has also created easier and
quicker automated systems for both the submitting authors and
the reviewers. Online tools for reference checking, plagiarism
monitoring and figure enhancement provide new ways to monitor
unethical behaviour.
INTRODUCTION
The Academy of Medical Sciences welcomes the opportunity
to respond to the Select Committee's inquiry. The Academy is the
independent body in the UK representing medical science that promotes
the advances of medical research and campaigns to ensure these
are converted into healthcare benefits for society. Progression
of scientific knowledge in all science, including medical science,
is dependent on maintaining a firm foundation of information on
which future scientists can base their research. Similarly, public
trust and comprehension of science are better justified by a system
whereby data and scientific ideas have been formally scrutinised
and subsequently endorsed. Peer review of primary research is
essential to both these goals.
Peer review is used to assess research for quality
and potential impact (normally by the journal to which it has
been submitted). It is initially viewed by the journal editors
who can decide to reject the paper outright or send it on to review,
based on the relevance of the manuscript's content to the journal's
scope and on its potential impact. If sent to peer review then
experts in the field will review the contents of the paper and
can either: accept it in its current form; reject it; or send
it back to the authors for further revision and/or experimentation.
In the latter case, the paper is re-reviewed to decide on final
publication or rejection.
Other models of publication use post-publication
peer review as part of a traditional journal or a repository.
Additional variations include the identification of peer reviewers
and the publication of reviews alongside papers.
Peer review is a vital tool in the process of scientific
publication and is ultimately beneficial to both the scientific
community and the public. However, as we outline in this response,
which addresses each of the issues raised by the Committee in
turn, the current system can, and should, be improved to increase
the speed of the dissemination of scientific knowledge.
1. The strengths and weaknesses of peer review
as a quality control mechanism for scientists, publishers and
the public
The key strength of peer review is that it helps
to ensure that weak research is not endorsed through publication
in recognised journals. In this digital age where new information
can be disseminated very easily, peer review acts as an important
filter or "kite mark" to differentiate between research
that has reached an acceptable scientific standard and that which
has not. This is particularly important in the biomedical field
where research is of great interest to many non-scientists. The
peer review process can be important in improving a paper before
publication but, as we outline below, it can also unnecessarily
delay the rate at which new knowledge becomes available.
Scholarly journals use peer review to assess three
main features of potential papers:
The
quality of the research carried out.
The
potential impact of the study.
The
relevance of the research to the remit of the journal.
Peer review is the traditional method of regulating
scientific publication and requires the knowledge of experienced
experts to scrutinise scientific work. Journals vary in nature,
for example by the breadth of their remit. The status of any particular
journal depends on its impact factor, a measure of the impact
its published studies have on science. The various scientific
disciplines use slightly different forms of peer review (see section
5). High impact journals are characterised by having a high rejection
rate, only publishing what they regard to be the highest impact
research.
Publication prestige and quality control for researchers,
publishers and the public
There is increasing pressure on researchers to publish
in high impact journals such as Nature and Science.
A strong publication record is a key determinant in the allocation
of grant funding both to individual researchers and to their universities
via processes such as the Research Excellence Framework (formerly
the Research Assessment Exercise). This has focused attention
on how peer review operates, particularly in high impact journals.
Further, the scrutiny offered by the peer review process helps
to ensure only scientifically sound work is published, providing
a reliable body of information that can be used by other researchers.
This also allows publishers to maintain consistent quality and
scope of the articles published, justifying subscription costs.
Effective peer review is vital in ensuring that the
information subsequently used in the public and policy-making
domains, is accountable and trustworthy. Thus peer review can
help guarantee that knowledge and ideas derived via appropriate
scientific methodology are made available to wider circles of
society. Science has also become a stronger part of public culture.
Whereas previously peer review would occur in the context of communication
between scientists, journals must now consider how new work will
be received in a public setting. Concerns have been raised that
this could influence the timing of publication of work that may
have a public impact (or whether to publish at all) and prevent
potentially important findings being scrutinised, and the experiments
replicated.
Current publishing models can create a risk-averse
publishing environment that can delay progression of scientific
knowledge and lead to wasted research time and money
To justify subscription costs and maintain prestige,
there is increasing pressure on journals to preserve or improve
their impact factor. This in turn places strain on publishers
and peer reviewers to raise the rejection rate of papers, effectively
leading to greater exclusivity of the work published. In addition,
there is an incentive to limit print production costs of hard
copies by reducing the numbers of papers published.
We welcome the fact that some journals have a Fast
Track review and publication process but ultimately, increased
rejection rates and conservative attitudes can delay publication
of valid research, which can hamper progress across disciplines.
After rejection from one journal, submission to another journal
requires not only reformatting of the manuscript, but instigation
of an entirely new round of peer review. Even if the research
is eventually published, this wastes research time and money and
can delay the availability of the findings to the scientists.
The behaviour of peer reviewers who may require revisions that
are of tangential importance can also contribute to unnecessary
time delays associated with publication of research (see below).
Many papers will be published eventually and delays attributed
to the peer review process can sometimes be beneficial in clarifying
a piece of research before publication. However in some of the
biomedical disciplines, the delay between submitting a valid scientific
paper to the first choice journal and having it published (perhaps
in the second or third choice of journal) is causing major concern.
The role of reviewers
Peer review depends on the experts involved having
the experience and knowledge to critically appraise scientific
research. However, reviewing manuscripts is time-consuming, which
can result in delegation of reviewing duties to less senior colleagues,
for example post-doctoral researchers. Younger reviewers, perhaps
due to positions of less seniority, spend on average more time
reviewing a manuscript which can result in a more thorough appraisal
of the work.[55]
If supervised, this can also be a useful training opportunity
for the junior researcher. However, without moderation by the
original reviewer it can result in requests for unnecessary additional
data or revisions that may be superfluous to the key theme of
the study.
Unnecessary changes delay access to new information
for scientists and the public and they waste the time and funds
of researchers, which is especially detrimental insofar as a significant
amount of research is directly (through charitable organisations)
or indirectly (through taxation) funded by the public.
Traditionally, the peer review process is a closed
system, where the reviewer's identity remains anonymous. This
raises the possibility of unfair treatment from competitors and
potentially the "theft" of ideas, but does allow reviewers
to give their honest opinion about a paper without fear of repercussions.
Journals, such as the British Medical Journal have adopted
an open system, whereby the reviewer's details are made available
to authors and the European Molecular Biology Organisation
(EMBO) journal publishes reviewer comments in conjunction
with the publication of the paper.[56],[57]
One randomised study has shown that reviewers that were identified
provided reviews that were of higher quality but took longer to
complete than unsigned reviews.[58]
Reviewers who were identified were more likely to recommend publication.[59]
2. Measures to strengthen peer review
We believe that pre-publication peer review is the
most appropriate method of quality assuring scientific knowledge
in the biomedical sciences, although as outlined earlier, the
practices of some journals can create delays to the progress of
research. One possible method to improve and strengthen how peer
review is utilised is to prioritise the quality of the research
as the key determinant for publication rather than its perceived
impact. This could facilitate quicker access to new knowledge,
while still preventing poor science being published.
Public Library of Science ONE journal approach
An example of a system where peer review places emphasis
on research validity over potential impact is that taken by the
Public Library of Science ONE (PLoS ONE) journal
(also mentioned in section 8). In principle, any manuscript submitted
that presents valid research will be published, with less emphasis
on the potential impact of the paper during the peer review process.
Initially, many researchers used PLoS ONE as a final resort
to ensure their work was published somewhere (after previous rejections),
but the process undertaken by the journals can result in quicker
submission to publication times and more researchers are turning
to PLoS ONE as a first or second port of call for publication
to avoid delays.[60]
The PLoS family of journals (along with an
increasing number of other journals) also use an open access model
where authors pay publication costs, including those incurred
from the peer review process and the papers are free to access
from the date of publication. This also speeds the rate of access
to new knowledge.
3. The value and use of peer reviewed science
on advancing and testing scientific knowledge
Scientific peer review performs vital roles in both
advancing and testing scientific knowledge. If all data were released
in a way that did not involve scrutiny, then the onus would fall
to the reader to judge the relevance of the work. While some may
have the knowledge and capability to do this, many do not, and
peer review ensures the services of experts who can carry out
these roles. This ultimately results in a solid body of information
on which junior scientists can base their own work.
4. The value and use of peer reviewed science
in informing public debate
As highlighted in section 1, peer review is vital
in informing public debate. Journals having access to reviewers
who are deemed experts in the field means they can create a "kite
mark" of quality associated with the papers they are publishing.
This in turn is helpful in informing public trust and reliability
of the particular journal and the work contained therein. Work
that is released in to the public domain without some level of
quality assurance could potentially lead to situations where imperfect
or incorrect science is used by the media and others. Ultimately
this could be detrimental to the public's overall trust in research.
As highlighted above, this is perhaps the most important role
of peer review.
5. The extent to which peer review varies
between scientific disciplines and between countries across the
world
Variation of peer review on the global stage
Science is a global endeavour with an increasing
number of multi-national collaborations. At one level, the country
from which the research originates should have little bearing
on the peer review process. However countries vary in terms of
the conditions under which research is conducted (eg the facilities
and funding available) and journals need to decide whether to
vary their measurements of quality accordingly if they are to
take account of local conditions.
Variation of peer review between disciplines
The physical sciences have been quick to adopt newer
systems for dissemination that use post-publication, rather than
pre-publication, peer review. For example, the database arXiv
acts as an online repository for pre-prints of papers in the physical
and mathematical fields.[61]
Here authors submit original manuscripts to the repository and
anyone can view and comment on the work. Authors can subsequently
use this information to further improve and modify the paper before
submission to the peer review process. Moderators monitor the
discussion of each submission to ensure comments do not go "off
topic" and all comments and discussions are open and transparent.[62]
More formal "Hybrid Peer Review" systems exist whereby
the journals themselves provide the open forum for community discussion
before sending papers off to traditional anonymous peer review
for that specific journal.[63],[64]
This method is utilised by The Journal of Atmospheric Chemistry
and Physics.
We feel that approaches that involve the initial
publication of papers without peer review in an open forum are
less appropriate for biomedical research. In the physical and
mathematical sciences studies are more likely to present a finding
that will be fundamentally right or wrong. However more subjectivity
exists in biomedical research where differing and often competing
experimental systems and approaches are used to answer the same
questions, creating greater scope for "incorrect" or
differing results. The dissemination of non-peer reviewed information,
for example about medical research, to the public domain could
potentially be unhelpful. Hybrid systems and those that "kite
mark" the final, accepted, version of the paper address some
of our concerns but the process as a whole may result in a delay
in overall publication times.
6. The processes by which reviewers with requisite
skills and knowledge are identified, in particular as the volume
of multi-disciplinary research increases
Identification of reviewers with requisite skills
Peer reviewers are generally experts in their field
and thus have the requisite knowledge to review a paper from that
subject area. It has been proposed that perhaps, with the author's
permission, reviewers can use the manuscript as part of a group
discussion with other members of their laboratory or department,
in a similar manner to the universal journal club, where papers
that have already been published are presented at a laboratory
or departmental level.[65]
This can be helpful providing (as outlined in section 1) that
it does not result in authors being asked to make unnecessary
changes to their manuscript. We can see this approach as being
particularly beneficial in contributing to some of the post-publication
filtering methods (see section 8).
Identification of suitable reviewers for multi-disciplinary
research
Multi-disciplinary studies can be potentially problematic
to the standard peer review system. Several approaches may be
required to ensure fair treatment of these types of study, especially
as they become increasingly utilised to address research problems.[66]
Each domain of the submitted work must be scrutinised to the same
degree, which increases the complexity of the peer review process.
It has been suggested that in the future, official
review groups could be set up to act as excellence centres that
could review emerging multi-disciplinary studies. Authors would
have to submit their work under the knowledge that their research
may be reviewed by peer group as opposed to a single reviewer.
Or, less formally, a single peer reviewer could be selected with
the knowledge that they may select someone they feel could help
them review the research.[67]
Ultimately peer review needs to adapt to changes in the way that
scientific research is conducted.
7. The impact of IT and greater use of online
resources on the peer review process
As with many aspects of science, digitisation has
improved how peer review can be conductedsimplifying and
quickening the process. With the majority of reputable journals
publishing online as well as in hard copy, new models of peer
review can emerge due to reduced pressure on number of pages per
paper or issuethe main limiting factor in hard copy publication.
Indeed, all the aforementioned models and further models discussed
in section 8 utilise and are reliant, sometimes entirely, on online
resources.[68]
Impact on the peer review process
Digitisation of submission, tracking and reviewing
of research papers has in some ways hastened the speed of peer
review with many journals using online pro-forma both for authors
to submit their work and for reviewers to access and submit their
comments and decisions.141 It is also now easier for reviewers
to substantiate the author's claims and detect breaches of ethical
behaviour.
Online anti-plagiarism programmes for the ethical
misuse of text, or similar programmes for detecting digitally
modified figures, are routinely available.[69]
Overall therefore, the increased use of online resources aids
in raising confidence levels in the reviewers and in the peer
review process, which can also contribute to the ability to carry
out "light-touch" peer review such as that described
for the PLoS ONE journal.
8. Possible alternatives to peer review
The Academy believes that some form of pre-publication
peer review is a key requirement for any trustworthy and valid
system for scrutinising scientific publication, particularly as
interest in results from those outside the scientific community
increases. The example described in section 2, PLoS ONE model,
represents a variation on the traditional peer review modelwith
quality and validity of the research carried out taking precedence
over expected impact of studies. While one criticism of this approach
is the risk of flooding journals with reams of mediocre studies,
with no "pre-filtering" on the perceived impact of the
study, some modern "post-filtering" approaches can,
and are, fulfilling this requirement.[70]
For example, the Faculty of 1000 (F1000)
is a dedicated post-publication peer review format by experts,
who are elected by peers to determine the paper's impact (as opposed
to the journal's impact factor), generating an "F1000 Article
Factor" (FFa).[71],[72]
A post-filter mechanism like this can identify the most significant
papers more quickly than more accurate impact measures such as
citation indices. While post-publication filters are subjective
estimates of the likely impact of papers, they are helpful when
the speed of publication and the number of papers being published
is increasing.
While we believe that "peer community"
discussions of papers prior to publication are not appropriate
to biomedical research (see section 5), post publication community
review could also be a potential post-filter mechanism to estimate
the impact of papers published through organisations such as PLoS.
A so-called "Amazon-like" process where, like the Amazon
website, readers' reviews can generate a collective opinion on
the research presented, although we are not aware of a working
example of this type of post-filtering.[73]
THE ACADEMY
OF MEDICAL
SCIENCES
The Academy of Medical Sciences promotes advances
in medical science and campaigns to ensure these are converted
into healthcare benefits for society. Our Fellows are the UK's
leading medical scientists from hospitals and general practice,
academia, industry and the public service.
The Academy seeks to play a pivotal role in determining
the future of medical science in the UK, and the benefits that
society will enjoy in years to come. We champion the UK's strengths
in medical science, promote careers and capacity building, encourage
the implementation of new ideas and solutionsoften through
novel partnershipsand help to remove barriers to progress.
THE ACADEMY'S
OFFICERS ARE:
Professor Sir John Bell FRS HonFREng PMedSci (President);
Professor Patrick Sissons.
FMedSci (Vice-President); Professor Ronald Laskey
CBE FRS FMedSci (Vice-President).
Professor Robert Souhami CBE FMedSci (Foreign Secretary);
Professor Susan Iversen CBE.
FMedSci (Treasurer); Professor Patrick Maxwell FMedSci
(Registrar).
17 March 2011
55 Publication Research Consortium (2008). Peer
review in scholarly journals: Perspectives of the scholarly community-an
international study. http://www.publishingresearch.net/documents/PeerReviewFullPRCReport-final.pdf Back
56
See: http://resources.bmj.com/bmj/authors/peer-review-process Back
57
See: http://blogs.nature.com/peer-to-peer/2009/01/embo_journal_introduces_transp_1.html Back
58
Research Information Network (2010). Peer review: A guide for
researchers. http://www.rin.ac.uk/our-work/communicating-and-disseminating-research/peer-review-guide-researchers Back
59
Walsh E et al. (2000). Open peer review: A randomised
controlled trial. The British Journal of Psychiatry 176,
47-51. Back
60
See: http://www.plosone.org/static/information.action Back
61
See: http://arxiv.org/ Back
62
House of Commons Science and Technology Committee (2004). Scientific
Publications: Free for all?
http://www.publications.parliament.uk/pa/cm200304/cmselect/cmsctech/399/399.pdf Back
63
Koop T & Põschl U (2006). Systems: An open,
two-stage peer review journal.
http://www.nature.com/nature/peerreview/debate/nature04988.html Back
64
Bloom T (2006). Systems: Online frontiers of the peer reviewed
literature.
http://www.nature.com/nature/peerreview/debate/nature05030.html Back
65
Lahiri D (2006). Perspective: The case for group review.
http://www.nature.com/nature/peerreview/debate/nature05033.html Back
66
Lee C (2006). Perspective: Peer review of interdisciplinary
scientific papers.
http://www.nature.com/nature/peerreview/debate/nature05034.html Back
67
Lahiri D (2006). Perspective: The case for group review.
http://www.nature.com/nature/peerreview/debate/nature05033.html Back
68
Bloom T (2006). Systems: Online frontiers of the peer reviewed
literature.
http://www.nature.com/nature/peerreview/debate/nature05030.html Back
69
Benos D (2006). Ethics: Detecting misconduct.
http://www.nature.com/nature/peerreview/debate/nature04996.html Back
70
Bloom T (2006). Systems: Online frontiers of the peer reviewed
literature.
http://www.nature.com/nature/peerreview/debate/nature05030.html Back
71
See: http://f1000.com/about/whatis Back
72
Jennings C (2006). Quality and Value: The true purpose of peer
review.
http://www.nature.com/nature/peerreview/debate/nature05032.html Back
73
Arms W (2006). Ethics: Trust and reputation on the web.
http://www.nature.com/nature/peerreview/debate/nature05035.html Back
|