Written evidence submitted by Professor
Roger Jones (PR 23)|
I am writing this response to the Inquiry as a primary
health care journal editor with over 30 years' experience of the
peer review system as a researcher, author, reviewer and editor
and also someone who has extensive experience of research grant
funding through peer review and the organisation of scientific
conferences, through peer review of submitted abstracts. I have
seen drafts of one or two other responses to the Inquiry and,
rather than re-iterate points that have already been made by others,
I offer a few additional comments which I hope will be useful.
2. The Place of Peer Review
2.1 Peer review has to be seen as part of a selection
processan important part, but only one of the factors that
journal editors take into account when selecting papers for publication,
such as timeliness/novelty, "fit" with other papers/themes
in forthcoming issues of the journal, demands on space and editorial
resources (notwithstanding the opportunities afforded by web publicationon
line papers still require subeditorial work), newsworthiness and
also commercial considerationsperhaps sometimes implicit
and inappropriate, and related to the needs to attract advertising
and sponsorship. "Hanging committees" are often used
to make final reject/accept decisions on groups of high-quality
papers which all have a claim to publication.
2.2 Judiciously-chosen reviewers are essential
to journal editors, who cannot be expected to possess the expertise
in depth to make selection decisions on everything sent to them;
editors also frequently depend on their reviewers to be sure of
originality and novelty, as well as the veracity and robustness
of the study and the paper, and also to act as monitors of dual/duplicate
and salami publication and sometimes the inappropriate inclusion
of certain authors (gift/ghost authorship).
3. THE CONDUCT
3.1 The way that peer review is carried out varies
widely across journals in relation to almost all it's componentsincluding
blinding of reviewers to the identity and institutions of submitting
authors, the instructions given by the journal to reviewers in
terms of their reports and advice to editors, blinding authors
to the identity of people who have reviewed their work, the availability
and conduct of an appeals procedure for authors who feel hard
done by or misunderstood, the incentives (eg free online journal
access, money or a gift such as a CD) offered to reviewers, feedback
to reviewers about the outcome of submissions and also feedback
to them about the quality/utility of their reviews. As I discuss
later, the evidence base on which editors are able to make decisions
about some of these aspects of peer review is decidedly thin.
3.2 Useful resources for medical journal editors
include the guidance offered by the International Committee of
Medical Journal Editors (aka the Vancouver Group) and COPE,
the Committee on Publication Ethics.
4. PROBLEMS WITH
4.1 Perhaps the most difficult problem with peer
review is that it is not "peer", in the sense that there
is often a big difference between age, experience and seniority
of the reviewer and the reviewed, which immediately introduces
some potentially distorting forces eg when, in an open review
system a junior reviewer is asked to review a bad paper by a senior
with whom he may have an indirect but potentially career-relevant
relationship. In open review, "peerness" itself may
turn out to be a problem, as in the almost inevitable scenario
of competitive experts reviewing each other's papers while wanting
to be first in print. Other perverse incentives to be excessively
hawkish or dove-ish are not hard to conjure up.
4.2 The selection and training of and feedback
to reviewers have been mentionedknowing how best to frame
the review request to get the best out of reviewers and to avoid
disparate advice is not easyand variation between reviewers
in their accept/reject, definitely/possibly recommendations is
commonplace; this does not necessarily represent differences in
the quality of the reviews, but can be related to the precision
or otherwise of the editor's question.
4.3 Peer review of abstracts submitted to (international)
scientific conferences is a particular problem. Abstracts accepted
only as posters may appear in print and be citeable, so quality
assurance is important. Yet conferences are big money-spinners
and there is a considerable commercial incentive to maximise acceptance
of submitted abstracts to maximise attendancemany institutions
will only cover costs of staff going to conferences when their
abstract has been accepted.
5. THE PEER
5.1 There is some researchpeer reviewed
of coursein the literature about the effectiveness and
problems of peer review, but much more needs to be doneby
journals themselves, learned societies and governmentfor
which funding should be identified.
5.2 Some topics from which pertinent research
questions could be developed include:
benefits and dis-benefits of open and blinded review processes.
value of feedback on their performance to reviewers to improve
the quality and usefulness of their reviews in future.
value of training reviewers: clarification of the mission of the
journal and the characteristics of the "ideal" review.
use of structured/unstructured review inventories and structured/unstructured/categorical
advice to editors on quality and suitability for publication.
fate of papers rejected after peer reviewdo reviewers'
comments help authors get published elsewhere and what does that
say about the original "reject" decision?
I am the Editor of the British Journal of General
Practice and have an interest in improving the quality of the
peer review process. I am interested in conducting research in
to peer review.
Professor Roger Jones MA
DM FRCP FRCGP FMedSci FHEA
Editor, British Journal of General Practice
Emeritus Professor of general Practice, King's College London
3 March 2011