The work of the Communites and Local Government Committee since 2010 - Communities and Local Government Committee Contents


3  Research

Overview

27. This chapter focuses on how we have handled research since 2010, including the research that has been cited in written submissions to our inquiries and that which we have commissioned. More than half of the submissions we received in response to our call for evidence in November 2014 discussed the use of research.[35] Four issues were identified:

a)  the sources of information we draw upon in our work, including research that we commission ourselves;

b)  how we identify relevant information for our work;

c)  how we treat the information we receive; and

d)  the role of specialist advisers in our work.

The sources of information we use

28. Most of the submissions considered that we could draw upon wider sources of information than we do currently.[36] Suggestions included consulting organisations such as think-tanks or the media that regularly scrutinise policy or, making greater use of information released through Freedom of Information requests.[37]

29. We were also alerted to a number of organisations working on relevant topics that have been set up explicitly to provide robust research evidence to inform decision-making and enable better use of it by governments at all levels. The new What Works Centres were highlighted in three submissions as a potential source of information.[38] Set up in 2013, the What Works Centres aim to improve the way government and other organisations create, share and use high quality evidence for decision-making by collating existing evidence on policy programmes and practices, producing high quality synthesis reports and systematic reviews in areas where they do not currently exist, and assessing how effective policies and practices are against an agreed set of outcomes.[39] There are currently nine What Works Centres, several of which focus on areas relevant to our work such as the What Works Centre on Local Economic Growth.[40] Other initiatives highlighted include the Local Government Knowledge Navigator funded by the Economic and Social Research Council (ESRC)[41] and the Alliance for Useful Evidence, both of which have published reports that relate to areas of our work for example on the issue of scrutinising the effectiveness of local authorities.[42]

30. Although we have not used organisations such as the What Works Centres or the Alliance for Useful Evidence and cannot comment on their value, the committee in the new Parliament may wish to explore how such organisations might usefully feed into the committee's future work. The committee could, for example, as a pilot ask one of these organisations to review rapidly the available evidence. The potential benefit would be the ability to draw upon information from a range of sources in addition to the submissions received, and the skills and expertise of such organisations in terms of assessing the quality and reliability of this information through methods such as systematic reviews.

31. We were also encouraged to consider broader types of information, for example making more use of statistical data in our work,[43] or drawing upon academic research in the field of social sciences.[44] It was suggested that research councils such as the ESRC would be able to assist us in identifying, or even undertaking, relevant research.[45] Two submissions suggested that we ask DCLG to provide an overview of the research and evidence that they use to inform their policies and decisions.[46] This could be based on the approach taken by the Education Committee in its 'Evidence check' web forum to ask the Department for Education to outline its evidence on nine topics, which was then published by the Committee and views invited on the strength of the evidence provided.[47]

32. We have examined the Education Committee's evidence web forum. The committee in the new Parliament may wish to explore its use with the Department for Communities and Local Government. During this Parliament we have pressed DCLG to explain the evidence base underpinning some of its policies with limited success and an arrangement to elicit better evidence would have been of assistance. It would also be helpful if DCLG was more transparent in sharing the business cases used to inform ministers' policy decisions.

USE OF COMMISSIONED RESEARCH

33. We have commissioned two pieces of research[48] since 2010:

a)  The nature of planning constraints from the Cambridge Centre for Housing and Planning Research;[49] and

b)  Investigation into the British Retail Consortium's Voluntary Agreement Regarding the Sale of Electrical Products from Electrical Safety First.[50]

(In addition, Professor Colin Copus from De Montfort University undertook, on his own initiative but with our encouragement, research in relation to our 2013 follow-up work on our report, Councillors on the frontline;[51] we found his research useful and published it.[52])

34. Our use of commissioned research was welcomed by four respondents to our call for evidence.[53] Commissioning research was seen to be a useful way for a committee to get an overview of a particular topic and to understand the key issues within it. Specifically, it was suggested that it could assist the committee to:

i)  avoid drawing conclusions from a small number of experiences based on the submissions it receives;[54]

ii)  help to identify further lines of enquiry which could inform the call for evidence;[55] and

iii)  provide a basis upon which to question witnesses.[56]

35. We welcome the positive feedback on our commissioned work. In particular, we agree that such research works best when it addresses a clearly defined objective:

i)  in the planning research—relevance to the forthcoming inquiry and a starting point for the inquiry setting out the key issues; and

ii)  in the labelling of electrical products—following up an undertaking given in response to a recommendation in a report.

36. The process of commissioning research requires greater planning ahead and a longer lead time for inquiries, which may make it impractical for inquiries that require a short timetable. We consider that select committees will be assisted in their work if they commission more research and they should build time to commission, carry out and review such research into the timetables for their work.
Reflection: Simon Danczuk MP

I have really enjoyed serving on the Communities and Local Government Select Committee over the past few years and it has been a real privilege to have contributed to the work of the Committee during this time. I believe we have a really good mix of people on the Committee, with different members bringing unique experience and perspectives. This has contributed to a refreshingly non-partisan work environment with members mixing well as a team and using their different strengths to add breadth and depth to the work of the Committee.

In particular I am pleased that the Committee has been involved in some really high profile issues in the recent past. This ranges from huge and complicated policy areas such as fiscal devolution and the future of local authority funding to more sensitive issues such as the Rotherham abuse scandal. I believe the Committee has acquitted itself well under the pressure that accompanies the profiles of these issues and has made a good contribution to shaping policy recommendations and holding people to account for their failures. I would certainly like to see the Committee spend more on such high profile and popular issues going forward.

I have always been of the opinion that the CLG Committee should focus on accountability as much as on policy recommendations and that this scrutiny should apply to individual local authorities as well as the department. In the future I would like to see the Committee take a more wide-ranging approach to scrutinising local government and exposing failings, especially if more powers are to be devolved downwards.

Finally I would like to thank the clerks for all of their hard work and commitment, they have been an invaluable help and ensured that the whole Committee has run effectively.

How we treat the information we receive

37. Several submissions saw greater scope to assess and challenge the evidence we receive. The British Property Federation, for example, said that "more could be done to ensure that the research evidence and statistics submitted to the Committee are challenged with rigour".[57]

38. Distinguishing between the quality and robustness of evidence received as part of select committee inquiries has been highlighted previously.[58] In 2007, the Science and Technology Committee recognised the importance of differentiating between submissions that "are firmly evidence-based" and those that "are primarily opinion pieces".[59] The Committee recommended asking organisations and individuals to declare potentially relevant interests when submitting evidence and give their expertise and experience when stating opinions about particular matters, and it is for our successor committee to explore whether to use different terminology when referring to evidence-based submissions and other submissions.

39. Another approach was to use established checklists or tools to check the quality and appropriateness of evidence submitted as part of our inquiries. One example is the Bond Evidence Principles checklist, designed for use by non-governmental organisations (NGOs) in the international development sector.[60] This checklist can be used to examine a wide range of evidence from statistics, case studies, annual reports and research studies, and assess it according to the extent that it draws upon the views of people actually affected by particular policies (such as park home residents)[61] and the appropriateness of the methods used for what it is trying to find out. The checklist scores the evidence on a scale from 1-4 and then gives it an overall score, and colour (amber, yellow, light green or dark green) based on the quality and robustness of the evidence. The use of a checklist like this might help us to better assess the evidence received, provide an additional tool to help scrutinise and challenge this evidence, as well as be more transparent about how evidence is handled and weighed up.[62] Other suggestions included:

i)  holding seminars or meetings to get an overview of the volume and quality of the evidence available on a particular topic;

ii)  seeking briefings from organisations with specialist and technical expertise such as the National Audit Office at the start of inquiries;[63] and

iii)  training committee staff so they are better able to apply a checklist such as the Bond Evidence Principles checklist to the evidence received—this checklist is used for assessing and improving the quality of evidence in evaluation reports, research reports and case studies.[64]

40. The committee in the new Parliament may wish to review how it handles research evidence—or, as we sometimes found, material that purports to be such evidence. We see scope to improve the process, for example by asking respondents in the call for written submissions if they cite research to explain the methodology and what degree of review the research has been subject to.

41. The committee may wish to pilot an evidence checklist, for example, based on the Bond Evidence Principles. This would give a clear indication to both committee staff and members of the overall quality of the evidence submitted, based on the score and colour it receives.

Role of specialist advisers

42. Several submissions we received discussed our use of specialist advisers.[65] Suggestions included appointing specialist advisers on the basis of their subject knowledge, specialist and technical expertise (such as in the areas of research methodology, finance or performance management) or likely awareness of, and where possible contact with, relevant research. We were surprised by these comments as such considerations are already taken into account when we appoint specialist advisers.

43. Where we did see some scope for change was in the suggestions for adapting the way that we work with specialist advisers. Our usual method of operation was to appoint a specialist adviser for an inquiry with the appointment effectively terminating when the Government responds to our report. Following the lead of the Liaison Committee, we have given greater emphasis to following-up the recommendations in our reports and to monitoring the regular announcements of initiatives in key policy areas. We therefore see a case for appointing advisers for longer terms than just a single inquiry, or appointing a panel of advisers each covering a key area scrutinised by the committee, to provide us with access to expertise. But there will likely still be areas of inquiry, such as procurement, where more specific or in-depth knowledge or expertise may be required.

44. The committee in the next Parliament may wish to appoint a panel of advisers in 2015 to cover the key areas it is scrutinising.


35   Professor Christine Whitehead (WSC 023); Richard Styles (WSC 001); Professor Jane South (WSC 021); Local Government Ombudsman (WSC 020); Local Government Ombudsman (WSC 019); Educe Ltd (WSC 018); Association Of Independent LSCB Chairs (WSC 016); Centre For Public Scrutiny (WSC 015); ACCA (WSC 014); What Works Centre For Local Economic Growth (WSC 013); Local Government Knowledge Navigator (WSC 022); Dr Tim Brown (WSC 011); London Councils (WSC 010); Brethren's Gospel Trusts Planning Group (WSC 009); Core Cities UK (WSC 008); Professor George Jones and Professor John Stewart (WSC 007); Westminster City Council (WSC 006); John Thornton (WSC 005); British Property Federation (WSC 004); Richard Tracey (WSC 002); Richard Bate (WSC 012) Back

36   Richard Styles (WSC 001); Professor Jane South (WSC 021); Educe Ltd (WSC 018); ACCA (WSC 014); What Works Centre For Local Economic Growth (WSC 013); Local Government Knowledge Navigator (WSC 022); Dr Tim Brown (WSC 011); London Councils (WSC 010); Brethren's Gospel Trusts Planning Group (WSC 009); Professor George Jones and Professor John Stewart (WSC 007) Back

37   ACCA (WSC 014) Back

38   Educe Ltd (WSC 018); What Works Centre For Local Economic Growth (WSC 013); Dr Tim Brown (WSC 011) Back

39   For more information on What Works Centres see Cabinet Office, 'What Works Network', accessed 11 February 2015. Back

40   What Works Centre For Local Economic Growth (WSC 013). The What Works Network includes: The National Institute for Health and Care Excellence (NICE); the Education Endowment Foundation (EEF); The Early Intervention Foundation; The College of Policing; The What Works Centre for Local Economic Growth; The What Works Centre for Wellbeing; The Centre for Ageing Better; What Works Scotland; and The Public Policy Institute for Wales. Back

41   Professor Jane South (WSC 021); Local Government Knowledge Navigator (WSC 022); Educe Ltd (WSC 018) Back

42   For example the submission from Educe Ltd (WSC 018) refers to the Alliance for Useful Evidence report Squaring the Circle: Evidence at the Local Level, May 2013, while the submission from Local Government Knowledge Navigator (WSC 022) refers to two reports: Local Government Knowledge Navigator, Collaboration in action: local authorities that are making the most of research, June 2014 and Local Government Knowledge Navigator The role of evidence and research in local government, September 2014. Other examples include Professor Jane South (WSC 021) reference to the Knowledge Navigator Evidence Review on local government and public health, which focuses on some of the same issues that we covered in our Eighth Report of Session 2010-12, The role of local authorities in health issues, HC 694-I. Back

43   London Councils (WSC 010) Back

44   Professor Jane South (WSC 021) Back

45   Professor George Jones and Professor John Stewart (WSC 007) Back

46   Brethren's Gospel Trusts Planning Group (WSC 009); Richard Styles (WSC 001) Back

47   Education Committee, '"Evidence check" web forum', accessed 11 February 2015 Back

48   The process for select committees commissioning external research has several stages:

the committee draws up a specification;

it is assessed in term so of appropriateness, feasibility, timetable and value for money;

invitation to tender and submit bids (if above a threshold); and

bids received are considered against the tender. Back

49   Cambridge Centre for Housing and Planning Research, The nature of planning constraints, March 2014 Back

50   Electrical Safety First, Investigation into the British Retail Consortium's Voluntary Agreement Regarding the Sale of Electrical Products, January 2015 Back

51   CLG Committee, Sixth Report of Session 2012-13, Councillors on the frontline, HC 432-I Back

52   Professor Colin Copus, Report to Communities and Local Government Select Committee: Councillor Workshops: Councillors on the Frontline, August 2014 Back

53   Centre for Public Scrutiny (WSC 015); Dr Tim Brown (WSC 011); Richard Bate (WSC 012); Professor Christine Whitehead (WSC 023) Back

54   Centre for Public Scrutiny (WSC 015) Back

55   Richard Bate (WSC 012) Back

56   Dr Tim Brown (WSC 011) Back

57   British Property Federation (WSC 004); Centre For Public Scrutiny (WSC 015) Back

58   Science and Technology Committee, Thirteenth Report of Session 2006--07, The Last Report, HC 1108 Back

59   Science and Technology Committee, Thirteenth Report of Session 2006--07, The Last Report, HC 1108, para 33 Back

60   Educe Ltd (WSC 018) Back

61   CLG Committee, First Report of Session 2012-13, Park Homes, HC 177-I Back

62   Professor George Jones and Professor John Stewart (WSC 007) Back

63   We consider the position of the NAO at paragraph 54. Back

64   Bond Evidence Principles, accessed February 2015 Back

65   Professor Christine Whitehead (WSC 023); Brethren's Gospel Trusts Planning Group (WSC 009); Professor George Jones and Professor John Stewart (WSC 007); British Property Federation (WSC 004); ACCA (WSC 014) Back


 
previous page contents next page


© Parliamentary copyright 2015
Prepared 25 March 2015