Peer review

Written evidence submitted by the
London Mathematical Society (PR 44)

0. Context of the present submission

0.1 The London Mathematical Society is a membership society for the advancement, dissemination and promotion of mathematical knowledge. It was founded in 1865 for "the promotion and extension of mathematical knowledge" and was granted a Royal Charter in 1965. Today, the LMS is the major UK learned society for mathematics, with over 2300 members.

The aims of the Society are:

· to advance the interests of mathematics;

· to enable mathematicians to research and collaborate to advance mathematical knowledge;

· to make mathematical knowledge available worldwide;

· to promote mathematical research and its benefits to decision-makers, policy advisers, funders and the users of mathematics; and

· to support mathematical education in schools, colleges and universities, and encourage the public and young people to appreciate and engage with mathematics.

0.2 The LMS publishes 400 Peer reviewed articles a year, a total of 8500 pages. This is out of 1200 submitted articles.

1. The strengths and weaknesses of peer review as a quality control mechanism for scientists, publishers and the public;

1.1 Mathematics is distinguished by the fact that the results are not a matter for debate: when an argument is presented, it can be studied by other experts, who will determine whether it is correct and whether it is complete. Although it may take some time for particularly long or difficult arguments, there is no room for disagreement. This gives Peer Review an especially significant role in mathematics.

1.2 Mathematical articles have a long usefulness, with references to 50 year old papers commonplace. This makes the investment of time in the review process worthwhile.

1.2 As a consequence, almost all mathematical research is ultimately published in peer reviewed journals. Often after community scrutiny of preprints, a final formal version of results will be submitted to a journal. The Editor will usually send it to one or two expert Reviewers (more commonly called ‘referees’ in the UK). The Reviewers will subject the work to detailed scrutiny, working through the mathematical arguments in detail as well as testing conclusions against other known results.

1.3 Because of the extreme density of mathematical writing, Editors will usually expect Reviewers to take around two months unless the paper is especially long, difficult or innovative. It is not unusual for assiduous Reviewers to take several times this long. Reviewers are unpaid and must fit the work in with their university duties. Editors usually do their work of finding a Reviewer and evaluating the report outside working hours. Overall, the average time from submission of a paper to receiving a decision is about six months for the journals published by the LMS. Because it is necessary to invest so much effort in reading a single paper, it is extremely valuable to the community that published papers have been declared correct by experts.

1.4 Innovative paradigm-changing articles are rare but extremely significant for the development of the subject. In this case, the Peer Review process can involve detailed scrutiny and reworking by a specially assembled international team of experts for many months. The length of the entire process may be measured in years.

1.5 The main benefits of Peer Review are that it ensures the correctness and clarity of the content. Polish of exposition and appearance are valuable side-effects.

2. Measures to strengthen peer review;

2.1 The response under this heading lists important factors, with the implication that to strengthen Peer Review, resources need to be found to enhance these. Usually the resources required are principally the time of Editors and Reviewers. Pressures on UK universities mean that departments are less sympathetic to academics spending time on editorial work, and reviewing work needs to compete with many other calls on academics’ time. Any formal guidelines need to consider resource implications.

2.2 The most important factors in the effectiveness of Peer Review are the choice of appropriate Reviewers and the amount of time they invest in the task. This means that the reputation of a journal and its editorial board are valuable indicators of quality. Researchers take this into account when assessing the literature.

2.3 It is not usual for the identity of authors to be kept secret from Reviewers. The potential for bias in reviewing is mitigated by the objective nature of the assessment. In any case, the small size of the community means that authorship will usually be apparent to an expert reviewer from the content. There are considerable benefits to the Reviewer from knowing the author’s previous work and thinking.

2.4 The identity of Reviewers is always kept secret from authors. Authors who consider reviews unfair will appeal to the Editor.

2.5 Editors will avoid choosing a Reviewer with a known conflict of interest and, conversely, Reviewers should declare any conflict of interest before agreeing to review an article.

2.6 Articles sent for review are considered confidential.

2.7 Some journals provide guidelines laying out their expectations from Reviewers. In practice Reviewers are more strongly influenced by their interpretations of a shared culture. This interpretation varies: some Reviewers are slow and idealistic and some fast and pragmatic. Editors will take such characteristics into account when selecting Reviewers, and both can be valuable.

3. The value and use of peer reviewed science on advancing and testing scientific knowledge;

3.1 Peer reviewed mathematics is the full record of the outcome of the research process. From it the insight and understanding it records can be rekindled in the mind of other researchers: it is the principal means by which the knowledge and understanding is passed between researchers, although early and informal accounts through lectures and discussion sometimes lead to swifter infection.

3.2 Advancement of mathematical knowledge takes place in the minds of mathematicians in the process of understanding, testing and questioning existing knowledge.

4. The value and use of peer reviewed science in informing public debate;

4.1 Public debate should be based on facts. Peer reviewed science is a source of facts.

4.2 In some experimental science there is room for debate on the interpretation, even of results in peer reviewed publication. This debate should be conditioned on the weight of evidence.

4.3 Except where the system has failed, peer reviewed mathematics is not subject to debate. The obstacle to informing public debate is that it will generally require an expert intermediary to make the content accessible. The challenge is to convey the full strength of certainty to a public who can only access the research at second hand.

5. The extent to which peer review varies between scientific disciplines and between countries across the world;

5.1 The mathematical community is genuinely international, last year the LMS published authors working in 42 countries, 20% from the US and 18% from the UK. The journals were accessed from 209 countries. The form and quality of Peer Review is similar for international journals based anywhere in the world. This is a direct consequence of the fact that authors, Reviewers and Editors at the top level will typically work with journals across the world.

5.2 Such variation as exists between mathematical journals is usually a reflection of the quality and style of the journal.

5.3 Peer Review in the experimental sciences is a different process (timescale, scrutiny, working methods) with somewhat different intentions. This will be apparent from responses from learned societies in these disciplines.

6. The processes by which reviewers with the requisite skills and knowledge are identified, in particular as the volume of multi-disciplinary research increases;

6.1 The choice of Reviewers is critical to the success of the process. A good Reviewer is not just a passive checker of correctness, and will be able to set an article in a context, test against related knowledge and suggest additional connections.

6.2 The mathematical community is relatively small. The number of experts with appropriate knowledge to review an article without prohibitive investment of effort is very small. Removing those with a conflict of interest leads to a handful.

6.3 The web of mathematical contacts of the Editor, augmented by searches of electronic databases of reviews form the basis for selection.

7. The impact of IT and greater use of online resources on the peer review process;

7.1 This impact is all-pervasive.

7.2 At the lowest level, documents are produced electronically, and (since the 1990s) research articles are usually typeset by authors before submission. This allows the Peer Review process to concentrate on mathematical content.

7.3 Since all documents are electronic, the process is swifter, because articles, reports and documents can be exchanged almost instantaneously. Good workflow systems ensure a full, accurate and up to date record of the progress of the review process.

7.4 The availability of massively searchable archives and review systems gives the Reviewer an additional tool. It does not help in the core part of the process of working through the mathematics, but in finding related work it is transformative.

8. Possible alternatives to peer review;

8.1 Any alternative has to involve the scrutiny of experts, who are inevitably peers of the authors. The alternatives may differ in the form and stage of the scrutiny.

8.2 The alternative most often mentioned in mathematics is that of collective scrutiny intermediated by the internet. Preprints are posted on a website, and interested readers post comments and corrections. Some sort of moderation is used, and there are mechanisms for weighting the value of comments by the experience of the contributor. This leads to a rich and valuable source of information, but it is not a substitute for Peer Review. Following the web philosophy it harnesses the energy of enormous numbers of people, and according to evolutionary philosophy the ultimate output can be excellent. The advantages and disadvantages are familiar from the analogies: Wikipedia is a marvellous resource but not a substitute for an authoritative treatise, and the eye is a glorious product of evolution but took many millions of years to arise.

8.3 Occasionally, work of exceptional significance will be scrutinised and accepted entirely outside the Peer Review process (Perelman’s proof of the Poincare Conjecture is a famous example). This relies on the author already having a reputation or strong connections to the establishment, and on prior acceptance of the exceptional importance of the work.

8.4 We are not aware of any alternative to Peer Review that gives rise to a reliable, efficient, durable and accurate literature in mathematics, which can be used as a basis for future progress.

London Mathematical Society

9 March 2011