6 Measuring success
58. The importance of understanding and measuring
the success of engaging the public in policy-making was summarised
by Cesagen:
Evaluation [...] is required not just for public
engagement activity but the means and process thereafter where
the outputs of public engagement translate into outcomes for policy.
This requires substantial investment in tracking and mapping the
travel of public engagement outputs and their impact in policy
contexts. A cartography [map] of public engagement policy impacts
would endlessly improve the value attributed to public engagement
among public cohorts and also provide a manual for publics in
maximising their influence.[84]
59. Professor Kathy Sykes suggested that:
Policy-makers need to reflect and record how any
piece of public engagement has: helped, or hindered them; what
they would have done differently, the costs and time involved;
and what might have happened without the activity. These need
to be compared systematically.[85]
60. The Cabinet Office referred to a number of
factors that could be used to judge the success of a public engagement
exercise, including the number of responses received, perceptions
of the Government's responsiveness, and the avoidance of dominance
by a single-interest group. It stated:
Insofar as public engagement contributes to a healthy
democracy, success will mean more active and meaningful engagement
with citizens on the policy in question. Insofar as public engagement
makes better policy, success will mean that there are links between
engagement activities and better policy outcomes. However, given
the many forms that public engagement in policy making can take,
it is natural that success or failure can be measured in a number
of ways. [
] Particular success measures will depend on the
tools and techniques adopted to address a particular policy question,
and the context in which they are used.[86]
61. We received little evidence from the Cabinet
Office specifying what management data the Government intends
to collect in practice to assess the implementation and effectiveness
of different approaches to public engagement in policy-making.
Indeed when asked whether there should be some baseline data against
which to measure the success of open policy-making, the Minister
for the Cabinet Office replied "I do not know how you would
measure it". When pressed on this issue, particularly as
to how success could be determined, the Minister responded "I
am not aware of any means of measuring it".[87]
62. The difficulties of determining measures
for success were recognised by our witnesses and in written evidence.
Sciencewise argued that:
the assessment of the success or failure of public
engagement must be based on the purpose of the exercise. An engagement
process which primarily aims to make better informed decisions
will have to be judged differently to one which primarily aims
to simply provide information[
]The success or failure of
engagement goes beyond the choice of method and often depends
on the principles that underlie the process.[88]
63. A number of submissions did provide some
suggestion as to how success and impact can be measured. In oral
evidence, David Babbs of 38 Degrees added that it was "worth
looking at numeric statistics in terms of the number of people
who are engaging in different ways. It is also worth being able
to point to examples of where public engagement has improved and
transformed government policy".[89]
In contrast, Tom Steinberg of MySociety suggested that success
should be measured through "data recorded around things like
the proportion of people who report that they believe they can
have some impact on the world around them and that they have any
say whatsoever in the country they live in".[90]
Stephan Shakespeare of YouGov argued that, ultimately, the success
of a public engagement process should be judged by the officials
on the "demand side":
The person doing the engagement, who wants to consult
and improve their legislation, needs to feel they have actually
benefited from a wide range of experience. That is ultimately
the purpose.[91]
64. There are different ways
in which the success and impact of public engagement in policy-making
can be measured, from the perspectives of both those who have
taken part in and those who have conducted the engagement exercise.
We are concerned that the Government has not given more thought
to measuring the impact of open policy-making, and that it will
not be able to demonstrate value for money and improved outcomes
in this new approach. Being able to do so is essential, particularly
in a time of austerity where spending is rigorously examined and
activities judged on the difference they make for citizens. While
we recognise that it is not an easy task, some form of measurement
or assessment needs to take place. The Government should come
forward with details of how the success of engagement efforts
across departments will be measured. These indicators or measurements,
and the progress against them, should be shared between departments
and made available in the Cabinet Office annual business plan.
84 Ev 44 Back
85
Ev 63 Back
86
Ev 69 Back
87
Q 170 Back
88
Ev 49 Back
89
Q 58 Back
90
Q 58 Back
91
Q 58 Back
|