Written evidence submitted by the Joint
Information Systems Committee, UCL, and the University of Salford
(PR 77)
INCREASING THE
VALUE FROM
PEER REVIEW
AND OPEN
ACCESS
Introduction
1. This submission to the House of Commons Committee
on Science and Technology is from the Joint Information Systems
Committee, UCL, and the University of Salford. These three organisations
are members of the UK Open Access Implementation Group, whose
members also include Universities-UK, Guild-HE, Research Councils
UK, the Wellcome Trust, the University of Edinburgh, the Association
of Research Managers and Administrators, Research Libraries UK,
the Society of College, National and University Libraries (SCONUL),
and the Public Library of Science. The Group has agreed to consider
this matter at its next meeting in May 2011, with a view to releasing
a public statement thereafter.
Declaration of interests
2. The authors of the submission are: Dr Malcolm
Read, Executive Secretary, Joint Information Systems Committee;
Professor Martin Hall, Vice Chancellor, University of Salford;
Professor David Price, Vice Provost for Research, UCL.
General principles of peer review
3. Growth in a knowledge-based economy depends
upon the activities of a knowledge community, one of whose functions
is quality control "guaranteed because members can each reproduce,
test and criticize new knowledge".[29]
This function is allied to concepts of replicability and objectivity
in research (particularly in the sciences), full discovery of
prior and relevant work, and open verification. These general
principles have been systematized by means of anonymous, double-blind
reviews (or variants thereof) and formal compliance requirements
ahead of publication.
4. As with all systematization, regulatory approaches
run the risk of being reductionist and pro forma, and may defeat
their original purpose. For example, closed circles of mutual
reviewers may develop, anonymity may be misused to attack work,
etc. Perhaps more seriously, narrow and formulaic peer reviews
may reinforce currently established views by not allowing for
paradigm changes.
5. Peer review is intended to counter closed
and restricted systems, to reinforce the accepted scientific norms
of communalism, universalism, disinterestedness and organised
scepticism. In particular here, that is that scientific results
are the common property of the scientific community and that scientific
claims are subject to critical scrutiny before being more widely
accepted.
6. The report of Sir Muir Russell into University
of East Anglia's Climatic Research Unit's E-mails found no evidence
that attempts had been made by CRU staff to subvert peer review
or the editorial process. It further expressed concern that "much
of the challenge to CRU's work has not always followed the conventional
scientific method of checking and seeking to falsify conclusions
or offering alternative hypotheses for peer review and publication".
Whatever its specific mode of operation (closed or open, before
or after publication), it is essential that peer review operates,
and operates effectively, within scientific discourse.
The cost and value of peer review
7. The value of peer review in maintaining the
integrity of scientific discourse is widely accepted, but this
value is being lessened by restricted access to the scientific
literature. As well as being the principal means by which science
implements "organised scepticism", peer review is also
a key component in a system of scholarly communication whose economics
are questionable, wherein one part of the academy (libraries)
pays for products whose principal value - the research report
itself and the peer review that assures its quality - has been
given away by another part. The result is that access to research
papers is restricted, and scientific results are no longer the
common property of the scientific community, let alone the wider
economy and society that might make use of them to drive innovation.
8. Peer review is an important stamp of quality
upon published publicly-funded research outputs. The cost of peer
review has been identified in several reports from UK research
organisations, notably by the British Academy in 2007,[30]
which estimated the cost of peer reviewing publications in the
humanities and social sciences to be around £900 per published
journal article, but made no attempt to calculate the total cost
to UK HE of the peer review system. It is a false perception that
publishers pay for peer review; they pay the in-house cost of
managing the peer review process but not the cost of the peer
review itself, which is usually undertaken by researchers as part
of their normal work, for which they are paid by their university
or research institute. The Houghton Report from JISC[31]
in 2009 did estimate the cost to UK HE of undertaking peer review
of publicly-funded research outputs to be around £178 million
per annum, while a report from JISC Collections in 2010 estimated
the cost to be between £110 million and £165 million.[32]
Whatever the exact figures, it is clear that peer review represents
a high cost to the UK higher education system in terms of time
spent by publicly-funded researchers in peer reviewing research
publications. Are there any means by which the UK could gain greater
value from this investment, without compromising the principles
of peer review or increasing public expenditure?
All stakeholders gain some value from peer review
but gains are restricted
9. Every stakeholder in scholarly communication
gains some value from the peer review system. Researchers gain
the assurance that earlier reports upon which they build their
own research have been checked by their peers, although they could
gain greater value if those research reports were also openly
available in journals or in subject or institutional repositories.
Users also know that peer-reviewed publications have been subject
to a quality-check. In selling publications advertised as being
peer-reviewed, publishers gain from the investment the academic
community makes in undertaking peer review. Funding agencies and
the taxpayer gain the assurance from peer review that their commitment
to fund academic research is yielding research reports of good
quality, but the value they receive from the dissemination of
research outputs may be lower than it could be because most peer
reviewed journal articles and monographs are not openly accessible.
The economic case for openness
10. Two strands of research have made a strong
case for the economic value of openness. Firstly the encouragement
of innovation is understood to be a key pre-requisite for economic
growth. Professor Eric Von Hippel of MIT has described the importance
of user innovation, making their products freely available through
open source systems.[33]
Professor Henry Chesbrough (University of California, Berkeley)
has noted that "companies that used to rely primarily upon
their own internal resources for R&D today must innovate in
a more open manner - integrating their internal ideas with the
external ideas of many other companies, universities and startups
to create new solutions, new systems and new possibilities that
no one company could do on its own." Many UK SMEs are trying
to do this, and evidence for the potential role of research papers
in fostering innovation is provided by Ware/Publishing Research
Consortium,[34]
which notes that SMEs give "a very high level of importance
to research articles, ranking them ahead of other types of information
such as technical information, reference work, technical standards
or patents." Restricting access to such articles inhibits
innovation and growth, and it is in the UK's economic interest
to remove such restrictions.
11. Secondly the economic case for openness has
been made by Professor John Houghton and his colleagues, demonstrating
the substantial returns to publicly-funded R&D that come through
enhanced access to research outputs, estimated at perhaps £172
million per annum for the UK economy.[35]
Houghton's work has been used by Dr Alma Swan to illustrate the
financial benefits to UK higher education institutions from a
switch to open dissemination of research outputs.[36]
The conclusion from both the innovation and the economic research
into the value of openness is that the entire scholarly communication
process - including the peer review element - would benefit from
a full switch to open access models. Because it involves shifts
in academic practice and consideration of the role of anonymity,
open peer review is likely to be a longer term goal. However,
some of the value of UK researchers' peer review work can be realised
more quickly by ensuring that the products of that work, the reviewed
papers themselves, are openly available. The full benefit to the
economy comes as the volume of open content grows and is used
more widely than content for which access is controlled through
financial and technical barriers.
Peer review and open access
12. The 10th report of the House of Commons Science
and Technology Committee in 2004, Scientific Publications:
Free for all?, recommended that "all UK higher education
institutions establish institutional repositories on which their
published output can be stored and from which it can be read,
free of charge, online." Most UK universities now have such
a repository. It further recommended that "Research Councils
and other Government funders mandate their funded researchers
to deposit a copy of all of their articles in this way".
The Research Councils and leading charities such as the Wellcome
Trust now have such a mandate, and also enable funds to be used
for payments to publishers for open access journals.
13. Open access will be a positive step for the
UK, but it has not yet been achieved, largely because researchers
have not taken the small steps necessary to make their papers
openly available. These steps could include:
(i) Putting the final, peer reviewed version
of the paper into a repository for immediate reuse by others.
(ii) Opting to publish in an open access journal,
utilising funds available via their (or their institution's) grant
from research funders where possible.
(iii) Stipulating the conditions under which
they provide papers or peer reviews to publishers: in the case
of papers, this might be by using a "license to publish"
rather than passing full copyright to the publisher (thereby retaining
the rights they need to put the paper into a repository); in the
case of peer reviews, this might be by making it a condition that
the reviewed paper be made open access.
The need for a public position
14. While there is evidence that authors benefit
when their papers are open access, for example by a higher citation
rate, such arguments have not yet convinced most researchers to
take the small steps above. This is partly because researchers
do not yet feel fully supported in making those steps, and partly
because they have so far been insulated from the effects of the
economics of scholarly communication. That is, firstly, the effects
of their decisions on the overall cost to HE are borne by the
library; and secondly, funders are only now beginning to assess
research impact. The steps noted above require small changes in
practice, whereas science is rightly conservative and takes time
to change. However, the evidence is clear that significant benefits
will accrue to the UK from a collective move toward open access,
and so a public policy position is appropriate.
15. Many research funders already mandate open
access, and arrangements are being put in place to monitor compliance.
The remaining step for research organisations to take to gain
greater value from the peer review work of researchers is to recommend
that:
(a) researchers funded from the UK public purse
should make it a condition of undertaking peer review of research
outputs that, when published, those outputs are then available
on open access terms with a minimum of delay; and
(b) there should first be a generic study to
confirm that the outputs and necessary workflows, which underpin
this recommendation, are realistic and deliverable.
Longer term trends
16. In the longer term, it is likely that further
experiments in "open peer review" and open peer commentary
and annotation will continue, and that ways will be found to address
legitimate concerns over anonymity with respect to peer review.
The recent launch of BMJ Open, featuring open peer review, the
rise of Mendeley, which features shared annotation via an online
platform, and the exploration of new publishing models based on
repositories, all indicate continued interest in the potential
of new technologies to support novel and more effective research
and review practices.
17. These trends, and the strains apparent within
the current scholarly communication model, make it likely that
significant changes are likely to evolve in the structure of associated
businesses, such as journal publishing, and the markets for online
search and research management systems. In this context, it is
possible that peer review will be increasingly de-coupled from
the dissemination of research papers, and become a service in
its own right.
18. In this rapidly changing domain, it will
be important to have a clear articulation of the principles and
role of peer review that is independent of particular economic
or technological arrangements, and that allows the UK to benefit
from the considerable investment made by its researchers in peer
review.
Joint Information Systems Committee, UCL, and the
University of Salford
10 March 2011
29 The concept of a knowledge community within a knowledge
economy is described by Professor Paul David and Dominique Foray
in "Economic fundamentals of the knowledge society"
2002 available at http://ideas.repec.org/p/wpa/wuwpdc/0502008.html Back
30
"Peer Review: the Challenges for the Humanities and Social
Sciences" British Academy 2007 http://www.britac.ac.uk/policy/peer-review.cfm Back
31 "Economic
implications of alternative scholarly publishing models"
Professor John Houghton and others 2009
http://www.jisc.ac.uk/publications/reports/2009/economicpublishingmodelsfinalreport.aspx#downloads Back
32
See the JISC Collections report "The value of UK HEIs contribution
to the publishing process" at
http://www.jisc-collections.ac.uk/News/Value-of-HEIs-to-publishing/
. Back
33
Eric Von Hippel "Democratizing Innovation"MIT Press
2005 freely-available at
http://web.mit.edu/evhippel/www/democ1.htm Back
34
Ware, M / Publishing Research Consortium (2009) Access by UK small
and medium-sized enterprises to professional and academic information:
http://www.publishingresearch.net/SMEaccess.htm Back
35
"Economic implications of alternative scholarly publishing
models" Professor John Houghton and others 2009
http://www.jisc.ac.uk/publications/reports/2009/economicpublishingmodelsfinalreport.aspx#downloads Back
36 A
Swan "Modelling scholarly communication options: costs and
benefits for universities: costs and benefits for universities"
2010 available at http://ie-repository.jisc.ac.uk/442/ Back
| 
