Written evidence submitted by The Royal
Society (PR 69)|
1. The Royal Society is a Fellowship of the world's
most eminent scientists and is the oldest scientific academy in
continuous existence. We aim to expand the frontiers of knowledge
by championing the development and use of science, mathematics,
engineering and medicine for the benefit of humanity and the world.
The Society has three main roles: it is the UK academy of science
promoting the natural and applied sciences, a learned society,
and a funding agency.
2. The Royal Society has used peer review to
make all its publishing decisions since its foundation in 1660
when the Society's Secretary, Henry Oldenburg, first introduced
the concept. As a learned society, the Society publishes eight
peer reviewed journals, including Philosophical Transactions
of the Royal Society, the world's oldest scientific journal.
3. It is important to understand that the scientific
literature is a very large body of analysis and experimental evidence
published in the form of articles over several hundred years.
These articles are the individual reports of scientific studies
on all aspects of the natural world which, together, form a structured
and coherent record of the current state of scientific understanding.
They are not based on mere opinion or assertion, but are the result
of careful observations and experiments which, over time, have
been rigorously tested against each other for consistency in order
to develop increasingly robust and reliable theoretical frameworks.
4. We understand that this inquiry is about the
operation and effectiveness of the peer review process used to
examine and validate scientific results and papers prior to publication.
Peer review is the best mechanism currently available for this
purpose and has stood the test of time. Its use in the evaluation
and validation of scientific research prior to publication is
5. Peer review is the process of subjecting an
author's scholarly work or research to the scrutiny of others
who are experts in the same field. For most journals, the process
generally starts with some form of initial screening of an article
("pre-assessment") to sift out obviously unsuitable
material. Items declined at this early stage would include articles
clearly out of the subject scope of the journal, or that are obviously
non-scientific. The next stage is to select a small number of
reviewers expert in the field or fields covered by the article
and to send the article to them for review (usually electronically).
The reviewers are asked to comment of the scientific validity
of the work, the appropriateness and rigour of the experimental
methods, the quality of any statistical analysis of the data,
and finally to give their opinion of its originality and likely
impact on the field. Their recommendations are used by the journal
editor to inform the decision as to whether the article should
be published or not. In general, there are three categories of
for publication ("as is", or with only minor revisions).
pending major revisions and/or further work.
Some journals choose to treat the second of these
options as a "re-submission" while others consider it
to be a modification of the original submission, but these are
There are three types of peer review in use. In order
of decreasing frequency of use they are: "single blind,"
"double blind" and "open".
By far the commonest system in use is "single
blind" peer review in which the author's name and institution
is known to the reviewer, but the reviewer's name is not provided
to the author.
A number of journals instead choose to operate a
"double blind" peer review system which is fully anonymised
(ie the author(s) are unaware of the identity of the reviewer(s)
and vice versa.
Recently, there have been some experiments with a
third type, "open" peer review, in which the authors'
and reviewers' names are revealed to each other. One major study
in which articles were made publicly available on the web prior
to publication for open peer review concluded "there is a
marked reluctance among researchers to offer open comments."
(Nature, 2006) Open peer review can be reasonably described as
an experimental system at this stage and is far from common.
The strengths and weaknesses of peer review as
a quality control mechanism for scientists, publishers and the
6. Strengths of peer review
The single greatest strength of peer review is the
close scrutiny of new scientific findings using evidence and expertise,
to provide the reader with a mark of value, reliability and authority.
In the absence of peer review some other method of validating
scientific research would need to be used in order to prevent
the proliferation of untested ideas, invalid conclusions, incompatible
theories, pseudoscience and polemics. The progress of science
crucially depends on the interpretation of new findings in the
context of existing understanding and experimental evidence. Without
any means of distinguishing the scientifically valid and coherent
from the unfounded assertion, scientific progress would be severely
(perhaps fatally) compromised.
Journals go to considerable lengths to select the
most appropriate reviewers and to eliminate any potential sources
of bias or conflicting interest in order to select the best material
to publish. The reputation of the best journals is intimately
involved with the rigour of their peer review systems and, in
turn, provides a "kite mark" of authority for the research
they publish. It also makes the journal more attractive for researchers
to both read and to submit papers.
The method in most common use ("single blind")
allows reviewers to provide honest and accurate feedback on an
article without fear of repercussions. This is particularly important
where the author may be "more established," more senior
or have a higher reputation than the reviewer.
Some argue that "double blind" peer review
is fairer in that it removes any potential bias in the mind of
the reviewer resulting from the name, gender and country of origin
of the author. The argument here is that a reviewer might be influenced
into thinking an article is better than it is, if it comes from
a high profile author in a top institution (and vice versa).
However, it can be argued that complete blinding is impossible
as it is often easy to identify the authors or their institution
from other elements of the article (such as the reference list
which is likely to contain references to the authors' own work,
or to unique methodologies.)
It should be noted that the process of peer review
can often provide very useful feedback to authors. Peer review
does not have to produce a simple accept/reject result, and authors
are frequently invited to modify their papers in the light of
critical comment. This is an important way in which peer review
can substantially improve the quality and value of scientific
7. Weaknesses of peer review
Peer review is fundamentally a human operation which
relies on individual expertise and judgement. This means that
reviewers may occasionally miss methodological errors or invalid
conclusions in a research paper. Fortunately, this is relatively
rare and its effect tends to be counteracted by the fact that
several reviewers are usually invited to review a given article.
In the rare event that such errors are not picked up during the
process, they are soon picked up by other experts after publication
and it is then possible to issue a correction, or in severe cases,
a retraction. Where correction is justified there are detailed
and robust publishing processes to ensure the scientific record
is amended. In the long run, the process is self-correcting; the
ultimate test is experimental evidence from the natural world.
Concerns have also been expressed regarding potential
reviewer bias. This may be positive or negative and can arise
from existing relationships or rivalries between reviewer and
author, or from the perceived reputation of the author or their
institution appearing to lend (or remove) credibility from the
article. Such concerns are often cited as a justification for
"double blind" peer review (see above).The risk of an
unfair decision is mitigated by the use of several independent
reviewers for each article.
Some critics of peer review claim that it can be
used maliciously (for example, to suppress the work of rivals
or to damage a competitor's career) and it is this concern that
has promoted experiments with "open" peer review (under
the assumption that reviewers would be less likely to submit malicious
reviews in an open system).
Measures to strengthen peer review
8. Publishers, academies and the wider scientific
community are constantly looking for ways to make peer review
more effective and efficient. It is important to maintain rigorous
standards whilst also minimising the demands on the reviewers
(who, it must be remembered, are themselves busy researchers giving
their time usually without payment). Traditionally peer review
has been a paper-based system involving several communications
between author, reviewer and editorial office and such a process
carries significant financial and time costs.
In the past 10-15 years, however, the process has
evolved away from paper to a fully integrated online process and
all major journals will now only accept online submissions. The
result has been to make peer review quicker and with a far lower
administrative burden. Publishers receive articles, data, references,
tables and illustrations as electronic files, usually via a dedicated
web-based submission system. Manuscripts are then allocated to
reviewers and reviewed online, following which the decision is
communicated to the authors in the same system. Many also accept
supplementary information such as large datasets, video files
etc., which in the case of "paper" journals will only
be available on-line. In addition, online peer review systems
integrate with the major third-party scientific databases such
as PubMed which allow bibliographic reference validation
and assist in selecting appropriate reviewers. Recently many publishers
(including the Royal Society) have introduced integrated plagiarism
Another key improvement has been the development
and widespread adoption of appropriate ethical policies, regulation
and best practice. Like many academic publishers, the Royal Society
has a publishing ethics policy covering such issues as authorship,
dual publication, plagiarism, conflict of interest and openness
in data sharing. Authors, reviewers and editors are required to
read and adhere to this policy which reflects the high standards
we expect in peer review. A number of organisations define codes
of practice and support the "policing" of the peer review
process. These include COPE and the International Committee of
Medical Journal Editors.
The value and use of peer reviewed science on
advancing and testing scientific knowledge
9. Science progresses by testing a hypothesis
against the available evidence obtained through experiment and
observation of the natural world. It is not based on the authority
or opinion of individuals or institutions. In fact, the Royal
Society motto "Nullius in verba" can be roughly
translated as "take nobody's word for it". Scientists
are trained to be sceptical and during peer review they assess
the methods, results and conclusions of a piece of research against
existing evidence. Peer review is particularly good at identifying
where a claim has no evidence, or if the evidence presented has
been arrived at by flawed methods or inappropriate data handling.
The greater the apparent claim or discovery, the greater the amount
of scrutiny the scientific community tends to give it. In general,
an extraordinary claim requires extraordinary evidence.
10. Without peer review, there would be a need
for some other form of validation of scientific findings as it
is crucially important for scientists to be able to use each others'
work with confidence when interpreting the meaning of their last
experiment or designing their next one. The age of the web has
meant that anyone can now "publish" whatever they like
in the form of blogs. The vast proliferation of information (of
hugely variable quality) now accessible would be likely to prove
to be more of an obstacle than a benefit to scientists if there
were there no system of validation helping them to sift the worthwhile
from the worthless.
The value and use of peer reviewed science in
informing public debate
11. We believe that the use of peer review is
valuable in informing the public about science as it acts as a
"kite mark" that a piece of research has been properly
scrutinised and validated by scientists. Barely a day goes by
without a science story being reported in the print and broadcast
media and there is a vast amount of scientific material (with
extremely variable provenance and validity) freely available to
the public via the internet. This often results in apparently
conflicting messages about an issue (such as a nutritional or
medical question) making it very difficult indeed for those with
little or no scientific knowledge to distinguish fact from fiction.
It is important for scientists, policy makers, educators and journalists
to highlight the value of peer-review in providing evidence-based
12. It is also important to put a single piece
of published scientific information into the wider context as
it may agree or disagree with previously published findings. It
is unrealistic to assume that every question will be answered
in one go and the public is often unaware of this level of detail,
or indeed that doubt is an inherent part of science.
The extent to which peer review varies between
scientific disciplines and between countries across the world
13. Science is a truly international enterprise.
The process of peer review is about the scrutiny of factual measurements,
observations and data. Just as the laws of nature are the same
throughout the world, so is the process of peer review. It is
therefore independent of any cultural or legal differences between
nations. Indeed this is vital if the universality of scientific
knowledge and understanding is to be sustained.
The application of peer review is broadly similar
in all scientific disciplines, but there are subtle differences.
In some areas of the physical sciences such as particle physics
articles are usually deposited on the arXiv pre-print database
prior to formal peer-review. This allows the scientists to publish
research quickly and get informal feedback and identify any weaknesses.
This is then followed by formal peer review in a journal. In the
biological sciences there is an imperative to publish research
quickly in a peer reviewed journal. In cross-disciplinary research
care must be taken to ensure peer-review balances the consideration
of all disciplines involved eg both the physical science and biological
science. This often requires the use of many more reviewers for
each article which can increase both time and cost. In the field
of mathematics, the peer review process is not about looking at
experimental evidence and data, but rather the strict and logical
development of a mathematical argument.
The processes by which reviewers with the requisite
skills and knowledge are identified, in particular as the volume
of multi-disciplinary research increases
14. Individuals are selected as reviewers based
on a proven track record of scientific achievement in the relevant
discipline and a thorough knowledge of the existing literature,
and are drawn from across the world. This is vital to effectively
validate new work against the existing body of knowledge. Selection
is supported by electronic databases both within the Society and
from major-third party vendors. Such databases provide great detail
on potential reviewers such as their area of expertise, publishing
and reviewing record.
Reviewing multi-disciplinary research brings specific
challenges. One of our journals is dedicated to science at the
interface of the physical and life sciences. Such articles are
reviewed by both physical and life scientists and require more
referees than for single discipline research. In addition, reviewers
are required to state their level of confidence in assessing the
physical and biological aspects of an article.
The impact of IT and greater use of online resources
on the peer review process
15. IT and other online resources play a vital
role in making peer review as rigorous and as efficient as possible.
When peer review relied on the postal system it typically took
months. With online systems, peer review takes only weeks or sometimes
days. The remaining speed limiting factor is the time required
to carefully read a paper and produce a considered, evidence based
The transition away from paper-based publication
has also facilitated bibliometrics, and an increase in its use
as a proxy for assessment of research quality and output. This
can potentially influence publication behaviour, and further increases
pressure on good reviewers.
Possible alternatives to peer review
16. The scientific publishing community has been
proactive in exploring adjuncts and alternatives to peer review.
Further review and comment after a paper is published ("post
publication review") is already important in testing and
checking the quality of science. As described earlier, new concepts
and results in science are repeatedly tested to validate their
reliability and usefulness.
The online environment has provided the opportunity
for numerous experiments such as commenting, voting and citation
analysis. However, such measures can be crude and are often unproven.
We believe they are important in supporting formal peer-review,
but are not a substitute.
The posting of non peer reviewed content on the web
would have a number of consequences. It would be more difficult
to distinguish evidence-based findings from mere opinion or the
downright false. Public opinion about science might be determined
by opinion and assertion based on what is the most popular blog
or television programme at the time. Formulation of policy requires
a solid, non-transient evidence base. Scientists have a central
role in the better communication of science by highlighting both
the strengths of peer review and its limitations.
Peer review is neither perfect nor infallible, but
we believe that dispensing with it is not an option.
17. The Royal Society is a publisher of journals,
under the imprint Royal Society Publishing.
Nature. (2006). Nature's peer review trial. Nature
The Royal Society
10 March 2011