Written evidence submitted by Philip Campbell
(PR 60)
1. I welcome the enquiry on peer review set up
by the House of Commons S&T Committee. Peer review, after
all, is central to the allocation of research grants and to the
publication of valid reports of research.
2. I am the Editor-in-Chief of Nature
and the Editor-in-Chief of the Nature Publishing Group. In the
first of the roles, I am responsible for the content of Nature,
including the 800-odd research papers that we publish every year.
In the second role, I am responsible for the long-term support
and development of the quality and editorial policies shared by
Nature and all of the other 33 Nature-branded journals
published by the Nature Publishing Group.
3. In this submission I will not attempt a general
definition or overview of peer review. My purpose is to highlight
a number of issues surrounding peer review, and explain our responses
and policies. In some cases these issues relate directly to those
highlighted by the Committee. I hope that my discussion of other
issues will assist the Committee in its thinking.
4. We keep our policies under review, in order
to reflect changing needs within science and changing factors
that affect research. In particular, the Chief Editors of the
journals meet on a monthly basis in order to keep abreast of relevant
developments inside and outside the company, to reflect on researchers'
needs, and to develop policies when required.
5. Nature is the most highly cited multi-discipline
natural science journal. It has a long and distinguished history
(first published in 1869) of covering the most important science
and of rigorously applying peer-review to the papers it publishes.
6. Nature-branded journals for the most part
focus on specific scientific disciplines. Based on their Impact
Factors (a measure of the typical level of citations of a journal),
they are often the most highly cited in their fields or if not,
are one of a handful of top journals in their fields.
7. Nature and the Nature journals are
untypical journals in that they do not have editorial boards of
active researchers. All selection decisions are the responsibility
of the fully independent and Chief Editors of each journal and
their teams, informed (but not instructed) by the advice from
the peer review process. These editorial staff are employees of
the Nature Publishing Group, fully dedicated to the task of selecting
papers for publication. Through their frequent visits to conferences,
laboratories and through constant scrutiny of the literature,
they keep abreast of the state of their disciplines and develop
contacts with existing and new reviewers.
8. Full details of our approach to peer review
and details of our journals are available on the Nature Publishing
Group website. This site includes articles that we have written
about peer review and also community discussions about the topic:
http://www.nature.com/authors/peer_review/index.html and, with
particular relevance to this submission: http://www.nature.com/authors/policies/peer_review.html.
How does our peer review add value?
9. We occasionally survey our authors and are
encouraged by the fact that the substantial majority believe that
peer review has added to the quality of their paper. This reflects
the evidence from published surveys: http://www.publishingresearch.net/PeerReview.htm
and http://www.senseaboutscience.org.uk/index.php/site/project/395.
10. Referees are expected to read a submitted
paper with close attention in order to provide a reasonable assurance
of its validity, checking whether its internal logic is consistent
and also that its content is consistent with firmly established
knowledge of the techniques involved and of the phenomena being
discussed.
11. As well as checking for validity, referees
are expected to suggest improvements where they feel, for example,
that the logic is insufficiently clear, that the previous literature
is inadequately represented, or where certain experiments might
close loopholes or otherwise strengthen the conclusions of a paper.
12. The task of refereeing a paper takes significant
time - anything between several hours and several days.
13. A reviewer is not expected to check the results
of a paper by replication. In exceptional circumstances, referees
will undertake considerable work on their own initiative to replicate
an aspect of a paper.
14. Journal peer review adds value not only to
authors and to publishers, but also in a resulting sense of confidence
amongst stakeholders and publics that the scientific literature's
claims, often based on highly specialized evidence or argument,
have credibility.
15. Even after peer review, a scientific paper
should be considered provisional, either in its detail or even
in its fundamental conclusions. It will ultimately stand or fall
on its subsequent replication or corroboration by other scientists
in the short term, and on the deepening of scientific understanding
in the longer term.
16. It is part of the editor's and peer-reviewer's
responsibilities to ensure that data and materials required for
other researchers to replicate or otherwise verify and build on
the work are subsequently available to those who need it. Such
availability may be problematic eg when expensive materials or
organisms are involved, but we have policies to ensure such availability
where there are facilities such as public databases that enable
it, and in general such availability is a condition of publication.
Anonymity in peer review
17. Nature commissions confidential reviews
of papers on the basis that the identity of the referee will be
kept anonymous. This is a firmly held principle in practice -
we have on occasion declined requests from university enquiries
to reveal referees' identities, for example, though we usually
seek referees' agreement to assist in such circumstances.
18. Our key motivation in peer review is always
to act on behalf of our readers in ensuring that we publish only
the most scientifically significant papers that are submitted
to us. The relationship between an editor and his/her referees,
and their mutual confidence, can become a key element in optimising
the basis for an editor's judgements to that end.
19. Why anonymity? Because otherwise an author
rejected on the basis of a review may resent the reviewer and
subsequently act against him or her either subconsciously or in
deliberate retaliation. This is a particular concern for up-and-coming
researchers, who often provide excellent reviews of papers. Also,
it is not unknown for authors who have identified a referee to
attempt to pressurise them during their assessment of a revised
version.
20. Can anonymity be abused? It is an editor's
responsibility to apply judgement to protect the interests of
authors and readers against such abuse and, if disputes arise,
to seek advice from others, and thereby do whatever can be done
to ensure both ethical and technical integrity.
21. Surveys suggest that anonymity in peer review
is widely accepted and indeed preferred by most researchers (see
http://www.senseaboutscience.org.uk/index.php/site/project/395).
22. Where referees ask to be named, we will usually
identify them.
Peer review by competitors
23. Our policy is that submitting authors may
request that individuals be avoided as peer reviewers on grounds
of competing research interests. Provided the number is not excessive
(the usual limit is three) we almost always abide by such requests,
though reserve the right to choose whoever we need to provide
the best advice.
24. In our letter of invitation to referees,
we ask them to decline our invitation if they have any interests
that they consider to be dangerously conflicting, or at least
to notify us of such interests.
25. It is an editor's responsibility to use their
knowledge of the field and of individuals to take possible competing
interests into account, and to seek additional advice where necessary.
Alleged undue influence by a single referee or
a coterie
26. It is sometimes asserted that a single negative
referee can force the rejection of a paper. While that can indeed
happen on the technical aspects of a paper, our decision-making
is structured in a way that helps prevent undue influence by referees,
whether negative or positive.
27. Editors are expected to apply their own judgements
in selecting what Nature should publish. Nature's
use of referees is not a process of voting, nor a process where
the referees' views are the final arbiter of a decision whether
to publish.
28. Editors will involve their in-house colleagues
where controversies arise or where several scientific disciplines
need to be involved in an assessment of a paper. Except in the
most clear-cut cases, no paper is accepted without also being
considered by the Chief Biology Editor or Physical Sciences Editor.
These senior editors and, in some cases, I will also become involved
where a dispute arises between the authors and editors.
29. We value referees' reports both for their
technical assessments and for the explanations of their judgements
about a paper's significance. Most papers are seen by several
referees of diverse specialist expertise in order specifically
to assess particular aspects of the paper.
30. Our editors will almost always bow to a referee's
expertise on the technical validity of a paper's content. Where
a dispute arises about technical matters, we will seek advice
from other experts.
31. However, we rely on our editors to make the
final decision as to a paper's appropriateness for Nature,
based on its scientific significance as judged by them. A referee's
opinion on this aspect of a paper is often invaluable, but is
in no way binding on our editors. Indeed referees' opinions about
a paper's importance often differ.
32. Thus, in some cases the editor may choose
to accept the paper against a referee's advice about its significance.
There have been occasions where a paper has been accepted despite
negative judgements of its significance by all of the two, three
or (in one case that I am aware of) four reviewers involved. These
are inevitably subjective judgements, quite distinct from judgements
of the technical merits, where the expert views of the referees
take priority.
33. The difference in judgement may operate in
the opposite sense also, resulting in rejection despite a positive
endorsement by one or more referees.
34. Authors may appeal against negative decisions
where they specifically disagree with referees' or editors' comments.
35. Editors frequently try out new reviewers
alongside established reviewers. We also encourage senior reviewers
to include their younger colleagues (whom they must identify)
in the process, in order to grow the reservoir of advice while
also ensuring that our reviews are fully informed of the current
state of the art in evolving experimental techniques.
Alleged influence of where and who a paper comes
from
36. It is a fundamental principle of our processes
that papers are judged on their scientific importance, not on
their authorship or geographical origins. Any other approach ultimately
undermines our perceived value and also may lose us valuable papers.
37. A relevant consideration is that many papers
are co-authored by international collaborations, with different
components of the work being done in different laboratories. We
estimate the average number of authors on Nature papers
to be between 5 and 6.
38. Our decisions frequently require us to reject
authors of high reputation and influence in prestigious institutions,
who may also serve us as referees. Such decisions are a part of
everyday decision-making.
Double-blind peer review
39. It is sometimes suggested that papers being
sent out for peer assessment should be anonymized, in order to
prevent bias.
40. Our current policy is not to offer this facility.
A full discussion of this and our policy can be found at: http://www.nature.com/nature/journal/v451/n7179/full/451605b.html.
41. I would acknowledge that double-blind peer
review is favoured in some surveys (see http://www.publishingresearch.net/PeerReview.htm).
We are therefore keeping this policy under review.
Open peer review
42. In 2006, Nature ran an experiment
in open peer review, in which over a period of four months, submitting
authors were invited to post their papers on an open website for
open assessment by peers. Their papers were also peer-reviewed
in the usual way.
43. The details of the process and the outcome
can be found at:
http://www.nature.com/nature/peerreview/debate/nature05535.html.
44. In brief, the take-up by authors was low,
as was the amount of open commenting. Furthermore it was judged
by the editors that the comments added little to the assessment
of the paper.
45. It is my view, consistent with this outcome,
that scientists are much better motivated to comment on an interesting
paper when directly requested to do so by an editor.
Post-publication review and commenting
46. Nature provides facilities for open
commenting on its research papers. Our experience is that this
facility is little used.
47. Our experience of the open peer review experience
and the lack of commenting on papers in whichever journal provides
it suggests that such unsolicited commenting has yet to take hold
as a part of scientific activity. One can readily speculate why:
there is no prestige or credit attached, there is the risk of
alienating colleagues by public criticism, and everyone is busy.
48. Sometimes public online debate about a paper
may arise after its publication, but that usually happens in the
blogs of scientists and others.
Publication of referees' and editors' reports
and correspondence
49. One journal published by the Nature Publishing
Group, the EMBO Journal, has pioneered the online display
of anonymized referees and editors/authors' correspondence after
publication, alongside the paper. For an account of this policy,
see the article by the publisher of the European Molecular Biology
Organisation (EMBO), Bernd Pulverer at Nature v468, p29-31,
4 November 2010, doi:10.1038/468029a, http://www.nature.com/nature/journal/v468/n7320/full/468029a.html.
50. Nature and the Nature journals have
so far not gone down this route. This reluctance is partly based
on a precautionary fear that it might upset the relationship between
editors and referees. Moreover, the documents reflect only a part
of the process of discussions within the editorial team, between
the editors and the referees, and between the editors and the
authors. There is also a belief that few people will want to wade
through this copious information.
51. Nevertheless, transparency has its own virtues,
and we are keeping this policy under review.
Pressure on peer review
52. There is without question a pressure on peer
review, in that academics are very busy, the number of papers
produced has grown, and so forth. Nature journals are privileged
in that the papers submitted and selected for peer review tend
to be interesting, thus most have not been affected by this.
53. Our editors reject 70-80% of submitted papers
(the exact proportion varies with discipline) on purely editorial
grounds, in order not to burden referees unnecessarily.
54. As explained elsewhere, we are always seeking
new reviewers, and welcome the advice of younger (but fully qualified)
reviewers, who often bring the greatest technical expertise to
bear on newer techniques or fast-moving areas of science given
their very active involvement in research.
Transfer of reports between publishers
55. Since 2008, Nature Neuroscience (NN)
has participated in an experiment involving other publishers in
which, at the request of authors rejected from NN, and with the
explicit agreement of referees, NN's referees' reports and their
identities can be passed to another publishers' journal. (Similar
transfers are routinely offered as a service to authors rejected
by Nature or one of our Nature research journals and wishing
to submit to another Nature journal.)
56. This experiment was conceived in the belief
that it would save referees work - it is quite common for the
same referee to be asked to referee a paper several times by a
succession of journals as an author seeks its publication. Moreover,
an editor might decide to publish or reject a paper immediately,
without troubling referees, on the basis of received reports.
57. The take-up by authors of this facility has
been small - typically amongst the participating journals, a few
percent of rejected authors request it. For those that do, the
processing time at subsequent journals has been reduced, though
I do not have ready access to the statistics.
58. This facility is controversial within NPG.
We invest significant sums of money in our professional editors
spending time both in the office and in visits cultivating contacts
with referees, and fostering insightful refereeing as best we
can. To then hand on the reports and names to another publisher
is to some extent undermining our competitive edge. Indeed, the
principle competitor of NN, Neuron, is not a part of the
experiment, and we might well not have joined the experiment if
it was.
59. However, if there was stronger evidence that
the community wanted this to be happening, and that the burden
on referees would indeed be reduced, we would consider developing
this approach.
Can peer review detect misconduct?
60. Given that editors and peer-reviewers need
to take everything that authors submit on trust, and do not seek
to replicate the work, it is almost impossible for referees to
detect misconduct. There have been occasions where a sharp-eyed
referee has detected an inconsistency or other flaw in reported
results that can only have arisen through inappropriate manipulation,
but these are few and far between.
61. In some of the most severe cases of misconduct,
a problem has arisen because of insufficient critical scrutiny
between co-authors. But that problem lies beyond the scope of
this discussion.
Can misconduct arise during peer review?
62. The unfortunate answer is: yes. It is not
unheard of, for example, for editors to be visiting a lab and
to see lying on a table in an open space a paper that is being
refereed, supposedly confidentially, by someone at the lab.
63. The most egregious type of misconduct is
for a referee to sit on a paper and to steal its results in order
to publish them under his or her own name. Such incidents are
very uncommon, in our experience.
64. We are alert to delays by referees' (which
will usually be for the reason that they have conflicting commitments),
and will cut off a process or seek alternative advice rather than
have a referee unduly delay a paper.
65. This brings to a conclusion my description
of our practice and some key policies associated with our peer
review processes. I hope that the committee find it helpful.
Philip Campbell PhD
Nature
10 March 2011
| 
