Peer review in scientific publications - Science and Technology Committee Contents

Written evidence submitted by the Royal Society of Chemistry (PR 68)


1.  The RSC is the largest organisation in Europe for advancing the chemical sciences. Supported by a network of 47,000 members worldwide and an internationally acclaimed publishing business, its activities span education and training, conferences and science policy, and the promotion of the chemical sciences to the public.

2.  RSC Publishing is one of the largest publishers of chemical science information in the world. Over 230 people are employed in the publishing operation. The majority of staff are located in located in Cambridge (UK), although the RSC also has modest offices in Philadelphia (USA), Beijing and Shanghai (China). RSC Publishing is a not-for-profit publisher wholly owned by the Royal Society of Chemistry. Committed to advancing the chemical sciences, any surplus is reinvested in supporting the global scientific community. 

3.  The RSC Portfolio comprises 32 peer reviewed high impact journals, two highly acclaimed magazines, approximately 90 new books annually and several databases. The RSC is thus an established and experienced scientific publisher serving the chemical science community.

4.  This document represents the views of the RSC. The RSC has a duty under its Royal Charter "to serve the public interest" by acting in an independent advisory capacity, and it is in this spirit that this submission is made.

5.  The RSC believes that the peer review system:

—  Is beneficial both for the research communities that it serves, as well as the wider public.

—  Successfully balances the demands of enabling researchers to disseminate their work quickly and widely, with ensuring that such work maintains the integrity of the scientific record.

—  Whilst it is not flawless, provides a clearly defined code of ethics for those that work in it (authors, editors, publishers and reviewers) together with measures to ensure transparency at all stages.

—  Varies between disciplines, which means a "one-size-fits-all" approach is inappropriate. Publishers within each discipline have evolved procedures and guidelines appropriate to each field; an environment where best practice can be shared within the industry allows these organisations to develop continuously. However, to try to adopt uniform guidelines across the industry would be counterproductive and inappropriate.

—  Provides a mechanism through which the reliability and authority of disseminated work can be gauged. In an age where information on all topics is so freely accessible, it has become evermore difficult for the public to distinguish between different sources and their respective authority or limitations. However, it is clearly important that an appreciation of the peer-reviewed process is more widely understood.

1.  The strengths and weaknesses of peer review as a quality control mechanism for scientists, publishers and the public

6.  The peer review system is a process that benefits the researchers that it serves, as well as publishers, the wider scientific community and the public. It does not merely constitute a process of accepting or rejecting research papers. Rather, it is a process that examines and refines a piece of research undertaken by a particular author. Criteria such as methodology and correct acknowledgement of other research in the field to avoid repeat publication can be checked. It is a tool that publishers can use to ensure that the research that they distribute maintains the integrity of the scientific record to the highest standards.

7.  Reports produced by referees as a result of the peer review process will contain suggestions to strengthen and further the body of research presented. Many authors find that the peer review process helps them to improve their work and so ensure that research in the public domain is of the best quality. A 2009 survey of over 40,000 researchers, conducted by Sense About Science revealed that 91% of authors felt that the peer review process had improved their publication.[16] In some cases, recommendations through the peer review process can result in collaborations between academics within a field, leading to new areas of scientific progress.

8.  In most fields, peer review is an activity undertaken without payment as part of the scholarly system. The lack of formal accreditation for this activity is seen by some as a flaw. However, acting as a referee is regarded by most researchers as a professional activity making active participation in the peer review process an important part of a scientific researcher's career. 90% of referees interviewed in the aforementioned survey cited "playing an active role in the community" as their primary reason for undertaking peer review. Refereeing falls within the "code of conduct" of the scientific profession and the obligations of its members. Such a code of conduct seeks to maximise the benefits of science to society and the profession. Most referees are also authors, and so have a vested interest in contributing to maintaining and raising journal standards.

9.  A potential criticism of the peer review process is that it relies upon the personal judgment of individuals in a given field. Conflicts of interest can occur and the quality and experience of referees is inevitably variable. However, mechanisms to ensure the strength of peer review are extensive (see question 6 also). Publishers employ robust guidelines on ethics and conflict of interest.[17] These include guidelines on the ethical use of live subjects (humans and animals) in research. Publishers, authors and reviewers are obliged to submit research and review research under these guidelines. To ensure the strength of peer review, authors can appeal against the rejection of a paper and there are processes in place for such requests to be dealt with fairly.

10.  The role of the editor is central to the quality of the peer review process. It is the editor who will consider the information produced through the process and so ultimately decide what feedback is communicated to the author and which articles are published. The judgement applied by the editor to the information collected in the review process requires knowledge, skill, and care.

11.  It should also be noted that peer review is only a single part in the "quality control" procedure for research that is applied prior to the stage of publication. There are other measures in place to assess research practices and the authenticity of data collected. Research that is undertaken as a regulatory study must be undertaken with compliance to good laboratory practice.[18] However, a clear distinction must be made between the purpose of the different parts of the "quality control" process; it is not the role of peer review to scrutinise laboratory practice.

2.  Measures to strengthen peer review

12.  Some of the measures employed to strengthen peer review include variation in the operating methods used. Closed peer review involves anonymity of either both the author and reviewer (double-blind) or just the identity of the reviewer (single-blind). In open peer review, both reviewers and authors are aware of each others identity.

13.  The RSC uses single-blinding in its peer review process. A number of studies have been carried out regarding the relative merits and effects of double-blinding on the quality of reviewing. However, these have largely proved inconclusive.[19] The success of double-blinding is often hampered by the tendency of authors to reference their previous work within submitted manuscripts. Where referee reports for a single paper vary, the view of a third, usually more senior referee as an adjudicator may also be sought.

14.  Open review is a process that has varying acceptance between disciplines. It is a method that has been used by the British Medical Journal for more than 10 years.75 However, many researchers have expressed reservations over this process. Early-career researchers may be reluctant to use the open review process to critique the work of more senior figures in the field, as this may have implications for their career progression at a later stage. Whilst open review is used in some fields, there is little active demand for such a shift in methodology; 58% of reviewers would be less likely to review if their signed referee's report was available alongside any paper they reviewed.72

15.  Core measures that are important in strengthening the peer review process should focus on appropriate training. This applies to both reviewers and editors who manage the process. Within the RSC, editorial duties are handled both internally by professional editorial staff and externally by associate editors. Ensuring that consistent procedures are applied in both methodology and ethics are key to maintaining the integrity of the peer review process. RSC Publishing staff actively engage with potential authors and referees in the scientific community. They regularly deliver workshops on the publishing and peer review process both in the UK and across the world.

3.  The value and use of peer reviewed science on advancing and testing scientific knowledge

16.  Peer review is a valuable mechanism for advancing scientific research and knowledge, both as a filtering and refinement tool. Publishing editors must select work that advances scientific knowledge in an area, but take care to ensure that work is reasoned and not overly speculative. As outlined in question 1, a referee's report will usually include improvements to the work presented. These can include further suggestions or modifications to methodology to improve the body of work presented. These aspects of the peer review process ensure the advancement of scientific knowledge in the broadest sense. For a learned society, such as the RSC they enable us to fulfil our charter objective of "the general advancement of chemical science".[20] Very few papers are published without amendments and the use of peer review is acknowledged by researchers as an effective method of improving their work and advancing scientific knowledge.

4.  The value and use of peer reviewed science in informing public debate

17.  A sound scientific evidence base is central in the formulation of policy. As such it is important that society as a whole has an appreciation and understanding of science. Peer reviewed research has an important role in advancing the public understanding of science. Currently, it is possible for peer-reviewed research and individual's personal opinion to be presented side-by-side with equal weighting. More critically, the limitations of each type of information are not usually distinguished by the public. Often both types of information are interpreted as factual. Peer review is used to check the methodology used, the accuracy of reporting with respect to previously published work in the field and the relevance of the research. The interpretation of the data gathered from such research cannot be certified using this process. However, peer review can give an indication of whether the interpretation of results that is presented is widely accepted within a particular research community.

18.  More should also be done to make the public aware of the wider context of scientific research. There is still currently a public preoccupation with scientific research providing "answers". A single piece of research rarely provides a definitive answer to a scientific problem. Rather a single piece of research must be viewed in the overall context of the field, as it contributes to the overall debate in a given area. Whilst this distinction is made by other researchers in the field, this is not often the case when a piece of research is examined in the public arena.

5.  The extent to which peer review varies between scientific disciplines and between countries across the world

19.  Peer review methodology varies between fields, with some fields using blinding methods, whilst others widely use open peer review (see question 2). As in most disciplines, the RSC assigns experts in the field to carry out reviews.

20.  The RSC is one of the largest international publishers in the field of the chemical sciences. In order to serve an international research community, referees are sourced from across the world. As businesses, publishers need to be responsive to changes in the international research arena. The recent rise in submissions from institutes in China must be matched in terms of identifying, training and using referees from there and other nations with emerging knowledge-based economies. These measures can help to foster international standards in peer review across a discipline. The RSC's strong presence in Asia, with offices in Beijing and Shanghai, reflects the importance of supporting and developing peer review skills in this region (among other strategic benefits).

6.  The processes by which reviewers with the requisite skills and knowledge are identified, in particular as the volume of multi-disciplinary research increases

21.  For scientific publishing, a scientifically-literate workforce is crucial. As mentioned in question 1, the role of the editor is pivotal in the peer review process. Staff with appropriate training and expertise within the field are essential in identifying reviewers with both suitable knowledge and experience. All internal editors at the RSC have a minimum of a degree in an appropriate science subject, with many also holding postgraduate qualifications. Associate editors are experienced researchers in the field who perform their duties alongside their research, with this commitment and dedication recognised through payment of an honorarium from the Publisher. The management of referees is a continuous, skilled process, with editors building up knowledge and relationships over time to ensure that manuscripts are reviewed fairly and efficiently. The RSC employs approximately 80 internal editors in our Cambridge office, and supports approximately 70 Associate Editors based throughout the world.

22.  The chemical sciences are a field that cover a range of disciplines, including health, environment and materials. Increasingly multidisciplinary collaborations across several fields are becoming common, particularly in areas of "challenge-led" research. When reviewing research in emerging areas, it is essential that all aspects of the work are reviewed. This may require the selection of referees with different specialities to address different part of the manuscript. Yet again, the role of the editor in managing the overall process is critical; they must select referees to strike this balance. They are also crucial in building up knowledge of emerging fields as they develop and change.

7.  The impact of IT and greater use of online resources on the peer review process

23.  The RSC received 34,177 manuscript submissions during 2010, sending over 28,000 for peer review. This volume of manuscripts can only be handled effectively through the investment and utilisation of sound IT systems and processes.

24.  The use of IT has greatly extended the pool of referees accessible to publishers on an international scale. Of the 32,000 expert referees the RSC consults, the vast majority are based outside the UK. Changes in technology have also allowed the reviewing process to be completed with greater speed and ease. A survey of over 3,500 reviewers showed that more than 70% felt that it was easier to carry out a review due to technological advances.72 These improvements have not just benefited reviewers, but also authors, with shorter receipt-to-publication times.

25.  Technological advances have also led to more sophisticated methods to detect plagiarism. The RSC is planning to introduce the use of word-overlap detection software to guard against plagiarism. This software has been developed through collaboration and in partnership with other science publishers. The use of this technology is an important part of the ongoing process to ensure that peer review serves both its community and the public to the highest standard. Whilst more than 80% of reviewers believe the peer review process should detect plagiarism, only 38% believe it is able to do so.72

26.  The greater general availability of research via the internet has made it more difficult for us all to distinguish the large amounts of information that we encounter on a daily basis. Whilst researchers in the field and those familiar with the peer review process are able to discern between peer-reviewed research and personal opinion, large sections of the public may not be able to (see question 4 also).

8.  Possible alternatives to peer review

27.  The integrity of the scientific record is important, not just to those in the scientific community, but also for the reliable dissemination of scientific information to the public. Whilst alternatives to peer review may include open forums and post-publication review, the established procedures used in peer review ensure that a consistent process is adopted for all submitted manuscripts. The filtering process to ensure that only an appropriate level of research is accepted for publication is an important advantage of peer review. Alternatives to peer-review, such as post-publication forum do not have this filter and so it becomes the responsibility of the reader to determine whether the content they find an authentic and valid contribution in the field.

28.  As described above (paragraph 17), peer review cannot guarantee that conclusions drawn by a piece of research are indubitably correct. However, whilst peer review is not infallible, it is the most efficient system for the assessment of new research. Whilst there are variations between fields, publishers are obliged to set and uphold guidelines on the criteria for submission, review and publishing of work.

29.  There are procedures in place to raise queries regarding published research that has been subject to peer review. These procedures acknowledge the transparent nature of peer review. It is not inscrutable and these mechanisms strengthen its status as a fair, practical method for the assessment of scientific research.

30.  The RSC regularly evaluates the merits and efficacy of the established peer review system for its high impact journals. Alternatives do not appear to provide any significant benefits, and appear to have many shortcomings which could threaten the integrity and accepted authority of published content. The RSC fully supports the existing peer review process and is interested to learn the outcome of this consultation process.

Royal Society of Chemistry

10 March 2011

16   Sense About Science | Peer Review Survey 2009: Preliminary Findings Back

17   RSC - Ethical Guidelines and Conflicts of Interest Back

18   The Good Laboratory Practice Regulations 1999 Back

19   S van Rooyen, The Evaluation of Peer-Review Quality, Learned Publishing, 2001, 14, 85 Back

20   RSC Charter and By-laws Back

previous page contents next page

© Parliamentary copyright 2011
Prepared 28 July 2011