Peer review

Written evidence submitted by Dr Julie Selwyn (PR 21)

1.1 I have been a member of the Department of Education’s Research Liaison Group (RLG) for four years. The group originated in the Department of Health and was transferred when the DfE took responsibility for children’s social care. Members of the group review research proposals and review some of the final research reports funded by the department.

I am also a researcher and Director of the Hadley centre for Adoption and Foster care Studies at the University of Bristol. So, I am a peer reviewer and have also been subject of peer review.

1.2 There are many benefits to peer review. First, in relation to the screening of research proposals submitted as the result of a tendering process, reviewers with research experience can spot proposals that are never likely to be ‘doable’. Sometimes proposers make rather wild claims about what is achievable within the timescales and underestimate the time and the costs involved. Others have a method which will not answer the research question. The peer review system enables the department to get best value for the research it commissions by selecting research proposals with the best chance of being successfully completed within the money and time available.

1.3 I have submitted many tenders for research work and have appreciated the peer review comments on successful bids. These have strengthened the design.

1.4 In relation to peer reviewing the draft final research reports before publication, the in-depth peer review provides a check on whether the ‘findings’ are supported by the data, notices omissions, over-stated findings , what isn’t reported, and any potential bias. There have been a few occasions when final reports have had to be substantial re-written as a result of the peer reviews at the RLG. Researchers have also been asked to go back to their data and do further analysis.

1.5 In my experience the peer review of final research reports is the ONLY point where most of the evidence is available for consideration. Of course, researchers write for peer reviewed journals but they can pick and choose which data they put in an article , the tone, and the overall thrust of the argument. There is insufficient detail to check the claims being made.

1.6 A further benefit of peer review is that it gives the research findings greater credibility. As a researcher I have been faced with some people who have stated

" This is government funded research and therefore there is a political agenda and it is biased". The peer review system enables greater confidence in the research and does not allow it to be ignored in a way that might otherwise occur.

1.7 As a researcher receiving feedback on my final reports, I have always appreciated the huge amount of work that the peer reviewers have undertaken, careful comments and suggestions. These have improved the final product and given me ideas about additional analysis.

1.8 The RLG model has been a very good one. A multi-disciplinary group meeting 3 or 4 times a year , with each report reviewed by 2 members of the group. The group built up expertise, gained helpful knowledge of the research community, and was also able to hear reports on how each study was being disseminated. However, not all the DfE ‘s commissioned work went through the group, and I would support all research being reviewed.

1.9 However, apart from travel expenses all the work for the RLG was freely provided. This could be very substantial. Each report was about 200-300 pages long – with some much more. On average I would take about 2 days to review each report. Outside the RLG, oneoff peer reviews for the department usually had a fee of £100 but I now understand that this is no longer possible because the work has not gone out to tender.

1.10 I would suggest that all commissioned work is peer reviewed; that a framework group of peer reviewers is established from which departments can draw upon and that the RLG model that is currently used in the DfE is considered for wider use. The fee for peer review should also be revised to more accurately reflect the amount of work undertaken.

Dr Julie Selwyn

Hadley Centre for Adoption and Foster care studies

University of Bristol

3 March 2011