Public engagement in policy-making - Public Administration Committee Contents

6  Measuring success

58.  The importance of understanding and measuring the success of engaging the public in policy-making was summarised by Cesagen:

Evaluation [...] is required not just for public engagement activity but the means and process thereafter where the outputs of public engagement translate into outcomes for policy. This requires substantial investment in tracking and mapping the travel of public engagement outputs and their impact in policy contexts. A cartography [map] of public engagement policy impacts would endlessly improve the value attributed to public engagement among public cohorts and also provide a manual for publics in maximising their influence.[84]

59.  Professor Kathy Sykes suggested that:

Policy-makers need to reflect and record how any piece of public engagement has: helped, or hindered them; what they would have done differently, the costs and time involved; and what might have happened without the activity. These need to be compared systematically.[85]

60.  The Cabinet Office referred to a number of factors that could be used to judge the success of a public engagement exercise, including the number of responses received, perceptions of the Government's responsiveness, and the avoidance of dominance by a single-interest group. It stated:

Insofar as public engagement contributes to a healthy democracy, success will mean more active and meaningful engagement with citizens on the policy in question. Insofar as public engagement makes better policy, success will mean that there are links between engagement activities and better policy outcomes. However, given the many forms that public engagement in policy making can take, it is natural that success or failure can be measured in a number of ways. […] Particular success measures will depend on the tools and techniques adopted to address a particular policy question, and the context in which they are used.[86]

61.  We received little evidence from the Cabinet Office specifying what management data the Government intends to collect in practice to assess the implementation and effectiveness of different approaches to public engagement in policy-making. Indeed when asked whether there should be some baseline data against which to measure the success of open policy-making, the Minister for the Cabinet Office replied "I do not know how you would measure it". When pressed on this issue, particularly as to how success could be determined, the Minister responded "I am not aware of any means of measuring it".[87]

62.  The difficulties of determining measures for success were recognised by our witnesses and in written evidence. Sciencewise argued that:

the assessment of the success or failure of public engagement must be based on the purpose of the exercise. An engagement process which primarily aims to make better informed decisions will have to be judged differently to one which primarily aims to simply provide information[…]The success or failure of engagement goes beyond the choice of method and often depends on the principles that underlie the process.[88]

63.  A number of submissions did provide some suggestion as to how success and impact can be measured. In oral evidence, David Babbs of 38 Degrees added that it was "worth looking at numeric statistics in terms of the number of people who are engaging in different ways. It is also worth being able to point to examples of where public engagement has improved and transformed government policy".[89] In contrast, Tom Steinberg of MySociety suggested that success should be measured through "data recorded around things like the proportion of people who report that they believe they can have some impact on the world around them and that they have any say whatsoever in the country they live in".[90] Stephan Shakespeare of YouGov argued that, ultimately, the success of a public engagement process should be judged by the officials on the "demand side":

The person doing the engagement, who wants to consult and improve their legislation, needs to feel they have actually benefited from a wide range of experience. That is ultimately the purpose.[91]

64.  There are different ways in which the success and impact of public engagement in policy-making can be measured, from the perspectives of both those who have taken part in and those who have conducted the engagement exercise. We are concerned that the Government has not given more thought to measuring the impact of open policy-making, and that it will not be able to demonstrate value for money and improved outcomes in this new approach. Being able to do so is essential, particularly in a time of austerity where spending is rigorously examined and activities judged on the difference they make for citizens. While we recognise that it is not an easy task, some form of measurement or assessment needs to take place. The Government should come forward with details of how the success of engagement efforts across departments will be measured. These indicators or measurements, and the progress against them, should be shared between departments and made available in the Cabinet Office annual business plan.

84   Ev 44 Back

85   Ev 63 Back

86   Ev 69 Back

87   Q 170 Back

88   Ev 49  Back

89   Q 58 Back

90   Q 58 Back

91   Q 58 Back

previous page contents next page

© Parliamentary copyright 2013
Prepared 3 June 2013