Peer review

Written evidence submitted by the Joint Information Systems Committee, UCL, and the University of Salford (PR 77)

INCREASING THE VALUE FROM PEER REVIEW AND OPEN ACCESS

Introduction

1. This submission to the House of Commons Committee on Science and Technology is from the Joint Information Systems Committee, UCL, and the University of Salford. These three organisations are members of the UK Open Access Implementation Group, whose members also include Universities-UK, Guild-HE, Research Councils UK, the Wellcome Trust, the University of Edinburgh, the Association of Research Managers and Administrators, Research Libraries UK, the Society of College, National and University Libraries (SCONUL), and the Public Library of Science. The Group has agreed to consider this matter at its next meeting in May 2011, with a view to releasing a public statement thereafter.

Declaration of interests

2. The authors of the submission are: Dr Malcolm Read, Executive Secretary, Joint Information Systems Committee; Professor Martin Hall, Vice Chancellor, University of Salford; Professor David Price, Vice Provost for Research, UCL.

General principles of peer review

3. Growth in a knowledge-based economy depends upon the activities of a knowledge community, one of whose functions is quality control "guaranteed because members can each reproduce, test and criticize new knowledge" [1] . This function is allied to concepts of replicability and objectivity in research (particularly in the sciences), full discovery of prior and relevant work, and open verification. These general principles have been systematized by means of anonymous, double-blind reviews (or variants thereof) and formal compliance requirements ahead of publication.

4. As with all systematization, regulatory approaches run the risk of being reductionist and pro forma, and may defeat their original purpose. For example, closed circles of mutual reviewers may develop, anonymity may be misused to attack work, etc. Perhaps more seriously, narrow and formulaic peer reviews may reinforce currently established views by not allowing for paradigm changes.

5. Peer review is intended to counter closed and restricted systems, to reinforce the accepted scientific norms of communalism, universalism, disinterestedness and organised scepticism. In particular here, that is that scientific results are the common property of the scientific community and that scientific claims are subject to critical scrutiny before being more widely accepted.

6. The report of Sir Muir Russell into University of East Anglia's Climatic Research Unit's E-mails found no evidence that attempts had been made by CRU staff to subvert peer review or the editorial process. It further expressed concern that "much of the challenge to CRU‘s work has not always followed the conventional scientific method of checking and seeking to falsify conclusions or offering alternative hypotheses for peer review and publication". Whatever its specific mode of operation (closed or open, before or after publication), it is essential that peer review operates, and operates effectively, within scientific discourse.

The cost and value of peer review

7. The value of peer review in maintaining the integrity of scientific discourse is widely accepted, but this value is being lessened by restricted access to the scientific literature. As well as being the principal means by which science implements "organised scepticism", peer review is also a key component in a system of scholarly communication whose economics are questionable, wherein one part of the academy (libraries) pays for products whose principal value - the research report itself and the peer review that assures its quality - has been given away by another part. The result is that access to research papers is restricted, and scientific results are no longer the common property of the scientific community, let alone the wider economy and society that might make use of them to drive innovation.

8. Peer review is an important stamp of quality upon published publicly-funded research outputs. The cost of peer review has been identified in several reports from UK research organisations, notably by the British Academy in 2007 [2] , which estimated the cost of peer reviewing publications in the humanities and social sciences to be around £900 per published journal article, but made no attempt to calculate the total cost to UK HE of the peer review system. It is a false perception that publishers pay for peer review; they pay the in-house cost of managing the peer review process but not the cost of the peer review itself, which is usually undertaken by researchers as part of their normal work, for which they are paid by their university or research institute. The Houghton Report from JISC [3] in 2009 did estimate the cost to UK HE of undertaking peer review of publicly-funded research outputs to be around £178 million p.a., while a report from JISC Collections in 2010 estimated the cost to be between £110 million and £165 million [4] . Whatever the exact figures, it is clear that peer review represents a high cost to the UK higher education system in terms of time spent by publicly-funded researchers in peer reviewing research publications. Are there any means by which the UK could gain greater value from this investment, without compromising the principles of peer review or increasing public expenditure?

All stakeholders gain some value from peer review but gains are restricted

9. Every stakeholder in scholarly communication gains some value from the peer review system. Researchers gain the assurance that earlier reports upon which they build their own research have been checked by their peers, although they could gain greater value if those research reports were also openly available in journals or in subject or institutional repositories. Users also know that peer-reviewed publications have been subject to a quality-check. In selling publications advertised as being peer-reviewed, publishers gain from the investment the academic community makes in undertaking peer review. Funding agencies and the taxpayer gain the assurance from peer review that their commitment to fund academic research is yielding research reports of good quality, but the value they receive from the dissemination of research outputs may be lower than it could be because most peer reviewed journal articles and monographs are not openly accessible.

The economic case for openness

10. Two strands of research have made a strong case for the economic value of openness. Firstly the encouragement of innovation is understood to be a key pre-requisite for economic growth. Professor Eric Von Hippel of MIT has described the importance of user innovation, making their products freely available through open source systems. [5] Professor Henry Chesbrough (University of California, Berkeley) has noted that "companies that used to rely primarily upon their own internal resources for R&D today must innovate in a more open manner – integrating their internal ideas with the external ideas of many other companies, universities and startups to create new solutions, new systems and new possibilities that no one company could do on its own." Many UK SMEs are trying to do this, and evidence for the potential role of research papers in fostering innovation is provided by Ware / Publishing Research Consortium [6] , which notes that SMEs give "a very high level of importance to research articles, ranking them ahead of other types of information such as technical information, reference work, technical standards or patents." Restricting access to such articles inhibits innovation and growth, and it is in the UK’s economic interest to remove such restrictions.

11. Secondly the economic case for openness has been made by Professor John Houghton and his colleagues, demonstrating the substantial returns to publicly-funded R&D that come through enhanced access to research outputs, estimated at perhaps £172m p.a. for the UK economy. [7] Houghton’s work has been used by Dr Alma Swan to illustrate the financial benefits to UK higher education institutions from a switch to open dissemination of research outputs. [8] The conclusion from both the innovation and the economic research into the value of openness is that the entire scholarly communication process – including the peer review element – would benefit from a full switch to open access models. Because it involves shifts in academic practice and consideration of the role of anonymity, open peer review is likely to be a longer term goal. However, some of the value of UK researchers’ peer review work can be realised more quickly by ensuring that the products of that work, the reviewed papers themselves, are openly available. The full benefit to the economy comes as the volume of open content grows and is used more widely than content for which access is controlled through financial and technical barriers.

Peer review and open access

12. The tenth report of the House of Commons Science and Technology Committee in 2004, Scientific Publications: Free for all?, recommended that "all UK higher education institutions establish institutional repositories on which their published output can be stored and from which it can be read, free of charge, online." Most UK universities now have such a repository. It further recommended that "Research Councils and other Government funders mandate their funded researchers to deposit a copy of all of their articles in this way". The Research Councils and leading charities such as the Wellcome Trust now have such a mandate, and also enable funds to be used for payments to publishers for open access journals.

13. Open access will be a positive step for the UK, but it has not yet been achieved, largely because researchers have not taken the small steps necessary to make their papers openly available. These steps could include:

(i) Putting the final, peer reviewed version of the paper into a repository for immediate reuse by others

(ii) Opting to publish in an open access journal, utilising funds available via their (or their institution’s) grant from research funders where possible

(iii) Stipulating the conditions under which they provide papers or peer reviews to publishers: in the case of papers, this might be by using a ‘license to publish’ rather than passing full copyright to the publisher (thereby retaining the rights they need to put the paper into a repository); in the case of peer reviews, this might be by making it a condition that the reviewed paper be made open access.

The need for a public position

14. While there is evidence that authors benefit when their papers are open access, for example by a higher citation rate, such arguments have not yet convinced most researchers to take the small steps above. This is partly because researchers do not yet feel fully supported in making those steps, and partly because they have so far been insulated from the effects of the economics of scholarly communication. That is, firstly, the effects of their decisions on the overall cost to HE are borne by the library; and secondly, funders are only now beginning to assess research impact. The steps noted above require small changes in practice, whereas science is rightly conservative and takes time to change. However, the evidence is clear that significant benefits will accrue to the UK from a collective move toward open access, and so a public policy position is appropriate.

15. Many research funders already mandate open access, and arrangements are being put in place to monitor compliance. The remaining step for research organisations to take to gain greater value from the peer review work of researchers is to recommend that:

a. researchers funded from the UK public purse should make it a condition of undertaking peer review of research outputs that, when published, those outputs are then available on open access terms with a minimum of delay

b. There should first be a generic study to confirm that the outputs and necessary workflows, which underpin this recommendation, are realistic and deliverable.

Longer term trends

16. In the longer term, it is likely that further experiments in ‘open peer review’ and open peer commentary and annotation will continue, and that ways will be found to address legitimate concerns over anonymity with respect to peer review. The recent launch of BMJ Open, featuring open peer review, the rise of Mendeley, which features shared annotation via an online platform, and the exploration of new publishing models based on repositories, all indicate continued interest in the potential of new technologies to support novel and more effective research and review practices.

17. These trends, and the strains apparent within the current scholarly communication model, make it likely that significant changes are likely to evolve in the structure of associated businesses, such as journal publishing, and the markets for online search and research management systems. In this context, it is possible that peer review will be increasingly de-coupled from the dissemination of research papers, and become a service in its own right.

18. In this rapidly changing domain, it will be important to have a clear articulation of the principles and role of peer review that is independent of particular economic or technological arrangements, and that allows the UK to benefit from the considerable investment made by its researchers in peer review.

Joint Information Systems Committee, UCL, and the University of Salford

10 March 2011