Written evidence submitted by the Royal
Society of Chemistry (PR 68)
THE ROYAL SOCIETY OF CHEMISTRY (RSC) WELCOMES
THE OPPORTUNITY TO RESPOND TO THE SCIENCE AND TECHNOLOGY COMMITTEE'S
CONSULTATION ON PEER REVIEW
1. The RSC is the largest organisation in Europe
for advancing the chemical sciences. Supported by a network of
47,000 members worldwide and an internationally acclaimed publishing
business, its activities span education and training, conferences
and science policy, and the promotion of the chemical sciences
to the public.
2. RSC Publishing is one of the largest publishers
of chemical science information in the world. Over 230 people
are employed in the publishing operation. The majority of staff
are located in located in Cambridge (UK), although the RSC also
has modest offices in Philadelphia (USA), Beijing and Shanghai
(China). RSC Publishing is a not-for-profit publisher wholly owned
by the Royal Society of Chemistry. Committed to advancing the
chemical sciences, any surplus is reinvested in supporting the
global scientific community.
3. The RSC Portfolio comprises 32 peer reviewed
high impact journals, two highly acclaimed magazines, approximately
90 new books annually and several databases. The RSC is thus
an established and experienced scientific publisher serving the
chemical science community.
4. This document represents the views of the
RSC. The RSC has a duty under its Royal Charter "to serve
the public interest" by acting in an independent advisory
capacity, and it is in this spirit that this submission is made.
5. The RSC believes that the peer review system:
— Is
beneficial both for the research communities that it serves, as
well as the wider public.
— Successfully
balances the demands of enabling researchers to disseminate their
work quickly and widely, with ensuring that such work maintains
the integrity of the scientific record.
— Whilst
it is not flawless, provides a clearly defined code of ethics
for those that work in it (authors, editors, publishers and reviewers)
together with measures to ensure transparency at all stages.
— Varies
between disciplines, which means a "one-size-fits-all"
approach is inappropriate. Publishers within each discipline have
evolved procedures and guidelines appropriate to each field; an
environment where best practice can be shared within the industry
allows these organisations to develop continuously. However, to
try to adopt uniform guidelines across the industry would be counterproductive
and inappropriate.
— Provides
a mechanism through which the reliability and authority of disseminated
work can be gauged. In an age where information on all topics
is so freely accessible, it has become evermore difficult for
the public to distinguish between different sources and their
respective authority or limitations. However, it is clearly important
that an appreciation of the peer-reviewed process is more widely
understood.
1. The strengths and weaknesses of peer review
as a quality control mechanism for scientists, publishers and
the public
6. The peer review system is a process that benefits
the researchers that it serves, as well as publishers, the wider
scientific community and the public. It does not merely constitute
a process of accepting or rejecting research papers. Rather, it
is a process that examines and refines a piece of research undertaken
by a particular author. Criteria such as methodology and correct
acknowledgement of other research in the field to avoid repeat
publication can be checked. It is a tool that publishers can use
to ensure that the research that they distribute maintains the
integrity of the scientific record to the highest standards.
7. Reports produced by referees as a result of
the peer review process will contain suggestions to strengthen
and further the body of research presented. Many authors find
that the peer review process helps them to improve their work
and so ensure that research in the public domain is of the best
quality. A 2009 survey of over 40,000 researchers, conducted by
Sense About Science revealed that 91% of authors felt that
the peer review process had improved their publication.[16]
In some cases, recommendations through the peer review process
can result in collaborations between academics within a field,
leading to new areas of scientific progress.
8. In most fields, peer review is an activity
undertaken without payment as part of the scholarly system. The
lack of formal accreditation for this activity is seen by some
as a flaw. However, acting as a referee is regarded by most researchers
as a professional activity making active participation in the
peer review process an important part of a scientific researcher's
career. 90% of referees interviewed in the aforementioned survey
cited "playing an active role in the community" as their
primary reason for undertaking peer review. Refereeing falls within
the "code of conduct" of the scientific profession and
the obligations of its members. Such a code of conduct seeks to
maximise the benefits of science to society and the profession.
Most referees are also authors, and so have a vested interest
in contributing to maintaining and raising journal standards.
9. A potential criticism of the peer review process
is that it relies upon the personal judgment of individuals in
a given field. Conflicts of interest can occur and the quality
and experience of referees is inevitably variable. However, mechanisms
to ensure the strength of peer review are extensive (see question
6 also). Publishers employ robust guidelines on ethics and conflict
of interest.[17]
These include guidelines on the ethical use of live subjects (humans
and animals) in research. Publishers, authors and reviewers are
obliged to submit research and review research under these guidelines.
To ensure the strength of peer review, authors can appeal against
the rejection of a paper and there are processes in place for
such requests to be dealt with fairly.
10. The role of the editor is central to the
quality of the peer review process. It is the editor who will
consider the information produced through the process and so ultimately
decide what feedback is communicated to the author and which articles
are published. The judgement applied by the editor to the information
collected in the review process requires knowledge, skill, and
care.
11. It should also be noted that peer review
is only a single part in the "quality control" procedure
for research that is applied prior to the stage of publication.
There are other measures in place to assess research practices
and the authenticity of data collected. Research that is undertaken
as a regulatory study must be undertaken with compliance to good
laboratory practice.[18]
However, a clear distinction must be made between the purpose
of the different parts of the "quality control" process;
it is not the role of peer review to scrutinise laboratory practice.
2. Measures to strengthen peer review
12. Some of the measures employed to strengthen
peer review include variation in the operating methods used. Closed
peer review involves anonymity of either both the author and reviewer
(double-blind) or just the identity of the reviewer (single-blind).
In open peer review, both reviewers and authors are aware of each
others identity.
13. The RSC uses single-blinding in its peer
review process. A number of studies have been carried out regarding
the relative merits and effects of double-blinding on the quality
of reviewing. However, these have largely proved inconclusive.[19]
The success of double-blinding is often hampered by the tendency
of authors to reference their previous work within submitted manuscripts.
Where referee reports for a single paper vary, the view of a third,
usually more senior referee as an adjudicator may also be sought.
14. Open review is a process that has varying
acceptance between disciplines. It is a method that has been used
by the British Medical Journal for more than 10 years.75 However,
many researchers have expressed reservations over this process.
Early-career researchers may be reluctant to use the open review
process to critique the work of more senior figures in the field,
as this may have implications for their career progression at
a later stage. Whilst open review is used in some fields, there
is little active demand for such a shift in methodology; 58% of
reviewers would be less likely to review if their signed referee's
report was available alongside any paper they reviewed.72
15. Core measures that are important in strengthening
the peer review process should focus on appropriate training.
This applies to both reviewers and editors who manage the process.
Within the RSC, editorial duties are handled both internally by
professional editorial staff and externally by associate editors.
Ensuring that consistent procedures are applied in both methodology
and ethics are key to maintaining the integrity of the peer review
process. RSC Publishing staff actively engage with potential authors
and referees in the scientific community. They regularly deliver
workshops on the publishing and peer review process both in the
UK and across the world.
3. The value and use of peer reviewed science
on advancing and testing scientific knowledge
16. Peer review is a valuable mechanism for advancing
scientific research and knowledge, both as a filtering and refinement
tool. Publishing editors must select work that advances scientific
knowledge in an area, but take care to ensure that work is reasoned
and not overly speculative. As outlined in question 1, a referee's
report will usually include improvements to the work presented.
These can include further suggestions or modifications to methodology
to improve the body of work presented. These aspects of the peer
review process ensure the advancement of scientific knowledge
in the broadest sense. For a learned society, such as the RSC
they enable us to fulfil our charter objective of "the
general advancement of chemical science".[20]
Very few papers are published without amendments and the use of
peer review is acknowledged by researchers as an effective method
of improving their work and advancing scientific knowledge.
4. The value and use of peer reviewed science
in informing public debate
17. A sound scientific evidence base is central
in the formulation of policy. As such it is important that society
as a whole has an appreciation and understanding of science. Peer
reviewed research has an important role in advancing the public
understanding of science. Currently, it is possible for peer-reviewed
research and individual's personal opinion to be presented side-by-side
with equal weighting. More critically, the limitations
of each type of information are not usually distinguished by the
public. Often both types of information are interpreted as factual.
Peer review is used to check the methodology used, the accuracy
of reporting with respect to previously published work in the
field and the relevance of the research. The interpretation of
the data gathered from such research cannot be certified using
this process. However, peer review can give an indication of whether
the interpretation of results that is presented is widely accepted
within a particular research community.
18. More should also be done to make the public
aware of the wider context of scientific research. There is still
currently a public preoccupation with scientific research providing
"answers". A single piece of research rarely provides
a definitive answer to a scientific problem. Rather a single piece
of research must be viewed in the overall context of the field,
as it contributes to the overall debate in a given area. Whilst
this distinction is made by other researchers in the field, this
is not often the case when a piece of research is examined in
the public arena.
5. The extent to which peer review varies
between scientific disciplines and between countries across the
world
19. Peer review methodology varies between fields,
with some fields using blinding methods, whilst others widely
use open peer review (see question 2). As in most disciplines,
the RSC assigns experts in the field to carry out reviews.
20. The RSC is one of the largest international
publishers in the field of the chemical sciences. In order to
serve an international research community, referees are sourced
from across the world. As businesses, publishers need to be responsive
to changes in the international research arena. The recent rise
in submissions from institutes in China must be matched in terms
of identifying, training and using referees from there and other
nations with emerging knowledge-based economies. These measures
can help to foster international standards in peer review across
a discipline. The RSC's strong presence in Asia, with offices
in Beijing and Shanghai, reflects the importance of supporting
and developing peer review skills in this region (among other
strategic benefits).
6. The processes by which reviewers with the
requisite skills and knowledge are identified, in particular as
the volume of multi-disciplinary research increases
21. For scientific publishing, a scientifically-literate
workforce is crucial. As mentioned in question 1, the role of
the editor is pivotal in the peer review process. Staff with appropriate
training and expertise within the field are essential in identifying
reviewers with both suitable knowledge and experience. All internal
editors at the RSC have a minimum of a degree in an appropriate
science subject, with many also holding postgraduate qualifications.
Associate editors are experienced researchers in the field who
perform their duties alongside their research, with this commitment
and dedication recognised through payment of an honorarium from
the Publisher. The management of referees is a continuous, skilled
process, with editors building up knowledge and relationships
over time to ensure that manuscripts are reviewed fairly and efficiently.
The RSC employs approximately 80 internal editors in our Cambridge
office, and supports approximately 70 Associate Editors based
throughout the world.
22. The chemical sciences are a field that cover
a range of disciplines, including health, environment and materials.
Increasingly multidisciplinary collaborations across several fields
are becoming common, particularly in areas of "challenge-led"
research. When reviewing research in emerging areas, it is essential
that all aspects of the work are reviewed. This may require the
selection of referees with different specialities to address different
part of the manuscript. Yet again, the role of the editor in managing
the overall process is critical; they must select referees to
strike this balance. They are also crucial in building up knowledge
of emerging fields as they develop and change.
7. The impact of IT and greater use of online
resources on the peer review process
23. The RSC received 34,177 manuscript submissions
during 2010, sending over 28,000 for peer review. This volume
of manuscripts can only be handled effectively through the investment
and utilisation of sound IT systems and processes.
24. The use of IT has greatly extended the pool
of referees accessible to publishers on an international scale.
Of the 32,000 expert referees the RSC consults, the vast majority
are based outside the UK. Changes in technology have also allowed
the reviewing process to be completed with greater speed and ease.
A survey of over 3,500 reviewers showed that more than 70% felt
that it was easier to carry out a review due to technological
advances.72 These improvements have not just benefited reviewers,
but also authors, with shorter receipt-to-publication times.
25. Technological advances have also led to more
sophisticated methods to detect plagiarism. The RSC is planning
to introduce the use of word-overlap detection software to guard
against plagiarism. This software has been developed through collaboration
and in partnership with other science publishers. The use of this
technology is an important part of the ongoing process to ensure
that peer review serves both its community and the public to the
highest standard. Whilst more than 80% of reviewers believe the
peer review process should detect plagiarism, only 38%
believe it is able to do so.72
26. The greater general availability of research
via the internet has made it more difficult for us all
to distinguish the large amounts of information that we encounter
on a daily basis. Whilst researchers in the field and those familiar
with the peer review process are able to discern between peer-reviewed
research and personal opinion, large sections of the public may
not be able to (see question 4 also).
8. Possible alternatives to peer review
27. The integrity of the scientific record is
important, not just to those in the scientific community, but
also for the reliable dissemination of scientific information
to the public. Whilst alternatives to peer review may include
open forums and post-publication review, the established procedures
used in peer review ensure that a consistent process is adopted
for all submitted manuscripts. The filtering process to ensure
that only an appropriate level of research is accepted for publication
is an important advantage of peer review. Alternatives to peer-review,
such as post-publication forum do not have this filter and so
it becomes the responsibility of the reader to determine whether
the content they find an authentic and valid contribution in the
field.
28. As described above (paragraph 17), peer review
cannot guarantee that conclusions drawn by a piece of research
are indubitably correct. However, whilst peer review is not infallible,
it is the most efficient system for the assessment of new research.
Whilst there are variations between fields, publishers are obliged
to set and uphold guidelines on the criteria for submission, review
and publishing of work.
29. There are procedures in place to raise queries
regarding published research that has been subject to peer review.
These procedures acknowledge the transparent nature of peer review.
It is not inscrutable and these mechanisms strengthen its status
as a fair, practical method for the assessment of scientific research.
30. The RSC regularly evaluates the merits and
efficacy of the established peer review system for its high impact
journals. Alternatives do not appear to provide any significant
benefits, and appear to have many shortcomings which could threaten
the integrity and accepted authority of published content. The
RSC fully supports the existing peer review process and is interested
to learn the outcome of this consultation process.
Royal Society of Chemistry
10 March 2011
16 Sense About Science | Peer Review Survey 2009: Preliminary
Findings Back
17
RSC - Ethical Guidelines and Conflicts of Interest Back
18
The Good Laboratory Practice Regulations 1999 Back
19
S van Rooyen, The Evaluation of Peer-Review Quality, Learned
Publishing, 2001, 14, 85 Back
20
RSC Charter and By-laws Back
| 
