Peer review

Written evidence submitted by the
British Sociological Association (BSA) (PR 49)

The BSA is the professional association for sociologists in the UK. Founded in 1951, its main charitable purpose is the promotion of sociology. Sociology is its main basis of recruitment and activity. The BSA has over 2,200 members and publishes two wholly owned academic journals: Sociology and Work, Employment and Society.

1. The BSA notes the primacy of peer review in publication and other forms of recognition and validation, and the difficulties presented by hard-to-imagine, alternative systems. Peer review lies at the heart of the validation activity of scientific work i.e. validation by peers is a tried and trusted system of the professionalized academic world and publishing industry. Peer review is integral to the Haldane Principle and takes place at all stages in the research process. It is all-pervasive in the social sciences and humanities: the mechanism by which funders decide whether or not to finance a research project; journal editors decide to publish a research paper; and academics provide post-publication validation of research. The principal value of peer review in informing public debate is providing a quality control mechanism for scientific research.

2. Fundamentally there is no particular difference in the style of peer reviewing between different countries and disciplines. The peer review process may take longer to perform, and may constitute a more burdensome workload in the social sciences and humanities than some other areas, including specifically the short, sharp write-up of experimentation found in some STEM related publications. This is partly due to the sometimes argumentative and discursive style of research and the significant length and complexity of a particular research argument in social sciences and humanities. However, the underlying process is the same.

3. The benefits of peer review are as follows: the editorial filtering and selection of papers attractive and relevant to the readership audiences and the inquiry of the day; the possible of improvement of the quality of work published; maintaining standards; a guarantee to readers, including non specialists, in the sense that the work is ‘certified’ or legitimated - it has in other words been quality assured. There is anecdotal evidence that authors can genuinely find peer review helpful (see individual response at the end of our submission) and that users of published research support and welcome it.

4. We note the distinction between pre-publication and post-publication peer review. Peer review is in fact a layered process in which initial peer review of proposals leads into peer review of publications and thence into post-publication peer review (the latter is sometimes referred to as academic impact). The two are related and equally necessary processes. We would refer the Committee to the writings of Jon Wiener, who, in his book Historians in Trouble, provides case studies demonstrating the vulnerability of post-publication peer review to politically motivated targeting; to Stephen Turner’s writing on how the blogosphere operates to clarify some arguments and is effective in some respects in sorting the wheat from the chaff; and to Michèle Lamont’s writing on the robustness of interdisciplinary peer review in How Professors Think.

5. Peer review is generally conducted by a system involving more than one peer. For example in the research excellence exercise it is conducted by panels composed of experts from the discipline or closely cognate subjects; for journal publication it may be by pairs of reviewers; for grants it may be by a multi-staged process, and so on. Peer review is generally conducted according to a range of conventions. For example it can be anonymised authorship and peer reviewer or ‘double blind’ reviewing; sometimes there may be an option to reveal identity; feedback may be mediated by a third party. Decisions for journal articles may follow a schema, such as ‘accepted’, ‘accepted subject to revision’, ‘rejected but invite to resubmit following substantial revisions’, or ‘rejected’ for example because it is not appropriate to journal. There is always a third party that provides scrutiny and oversight. Payment for taking part in the peer review process may be of a nominal kind or may not occur at all. Rather motivation to perform peer review may include notions relating to professional identity and concepts such as being part of an academic community; employability; seeing new work as it emerges; free subscriptions; acknowledgements; an honour, credibility or reputation enhancing. Peer review is understood as a collegial and professional obligation involving the recognition that any submission of one’s own work requires unpaid peer review which should be returned to the peer review system. Authors and reviewers are generally drawn from a common pool and this provides in part for the development of shared understandings of quality and innovation in the group. The common pool may be developed by targeting and/or by network activity through the normal work of academic professional life (such as conference attendance; dissemination activity; collaborative writing and research activity; employment; professional association and so on). Reviewers might consider originality; rigour of method; timeliness; and the concept of ‘importance’ which we may say in lay language refers to the question ‘why does it matter that this research was done?’.

6. We note that changes to online publication and the publishing industry itself may present changes to timing, methodology and norms. These are difficult to predict at present but open access models with different peer review methodologies are already emerging. We believe that it will be essential for the credibility of academic research, whatever its ultimate mode of final publication, that there are clear quality control mechanisms in place and that it is hard to imagine a more rigorous mechanism than peer review for academic research.

7. For the benefit of your inquiry, we include the particular feedback submitted to us by one of our journal editors below.

7.1 "There has recently been considerable comment on the issue of academic peer review and it has started to attract a bad press. This is unfortunate since, properly conducted, peer review is a vital part of academic research and scholarship. Without it British research would be badly weakened.

7.2 The essence of peer review is that academic work (often anonymised) is sent out for expert comment. When a doctoral candidate submits his/her finished thesis it is read by two experts in the field who then examine the candidate’s knowledge orally (the viva) before passing their verdict on the work. In academic journals, anoymised articles are sent to up to five expert reviewers (whose identity is not revealed to the authors) who write commentaries on the submission. Similarly grant applications (not anonymised) are also sent out to review.

7.3 Because authors and applicants are never told who their reviewers are then junior staff can comment on research done by established professors without fear of the consequences (and some of the best reviews I have seen have been written by junior staff). The idea is that all parties focus on the quality of the argument and the evidence presented rather than the personalities of the writers or the reviewers.

7.4 Over the last twenty years I have been on all sides of the reviewing process; as an author, editor and reviewer I have read and written about countless papers and many of my papers have been read and commented on in their turn. Of course problem reviewers always exist; academics who are so concerned with their particular hobby horse they fail to engage with the paper they should be commenting on; those whose comments simply list publications of their own that the authors must quote without fail; and those who are spiteful, malicious or simply confused (we dine out on such reviews). However an experienced editor can easily cope with this and the vast majority of reviews are constructive – critical certainly but also supportive.

7.5 When the system works well it helps academics tap into a global network of expertise in their area, through which detailed and tailored comments are provided free of charge, often by leading names in the field. Since journal articles are revised, reworked and often re-reviewed the system helps to improve the quality of research and it can be a delight to watch an article progress before publication. The process is an instructive one for reviewers as well since both journals and funding councils distribute the other reviews an article or a grant proposal attracted once the decision is made and reviewers are able to read the comments their peers made.

7.6 I can honestly say that some of the comments I learned most from as a researcher are those that are most critical and even the rejections (I am not arrogant as to assume that anyone who rejects me must be corrupt or incompetent). The critical, constructive advice I received enabled me to improve my work and to progress as an academic.

7.7 The peer review process is a public good since all this reviewing and much of the editing is provided free. Its impact on the quality of UK research is immense. It can, of course, go wrong, but such instances happen on a small of minority of occasions and there are a number of well established structures to safeguard against this, including the role of the editor and ensuring that reviews are written by critical but supportive academics. All journals now give reviewers guidance and some (including the BSA journals) confine most or all of the reviewing to the editorial board, just as the ESRC has appointed a Peer Review College. I find it difficult to imagine another mechanism through which such personalized commentaries could be provided. Before peer review is condemned I recommend that you give it a much closer look. I would be happy to do anything in my power to guide you."

British Sociological Association

8 March 2011