Annex A
METHODOLOGY
The methodology for this project contained five
key stages:
1. Scope the assignment with the National Audit
Office (NAO).
2. Design the research framework.
3. Implement the research.
4. Analyse and identify best practice.
1. SCOPE THE
ASSIGNMENT WITH
THE NAO
PwC met with the NAO to discuss and refine the
scope of the study and our proposed methodology. The following
were agreed:
— A schedule of dates for regular project
review meetings with the study team;
— How we will work together over the
duration of the project, including responsibilities and outputs;
— The list of questions to be answered
and the associated data requirements;
— The countries that we could focus
on for the desk research, in addition to the two case study countries;
and
— The contents of the final report.
2. DESIGN THE
RESEARCH FRAMEWORK
We devised a research framework based on the
Public Administration Select Committee (PASC) questions from their
Issues and Questions Paper (see Chapter 2 for full list). From
these we identified four sets of questions:
— The first set explores the definitions,
measures, institutional architectures and internal government
standards of good government. The answers to these questions formed
the first part of our output and produced a description of the
different models of good government and how they differ from and
resemble the Westminster model;
— The second set of questions focuses
on how policy is made and delivered, and the factors that influence
this process, such as the role of civil servants and contextual
or policy changes. We explored how the constitutional frameworks
hinder or encourage these processes. These questions contributed
to the second part of the output, looking at how the practices
of the models of good government are influenced by their constitutional
frameworks;
— The third set of questions considers
the performance monitoring and evaluation processes of the different
models of good government. Again, we explored how the nature of
the democracies that use them affects the design and application
of these monitoring frameworks. These questions also contributed
to the second part of the output, looking at how the practices
of the models of good government are influenced by their constitutional
frameworks; and
— Finally, the last part of the research
framework identified good practices. The findings contributed
to the third section of output, looking at how these examples
of good practice can be applied in the Westminster model.
| Step 1. Definition, structures and standards of good government PASC questions 1-2
| Purpose: understand the definition of and performance indicators for good government; understand how constitutions influence institutional architecture.
|
| Definition | Sources of information and analysis
|
—What is the definition of good government used by the World Bank?
—How do experts in the two case study countries
describe good government?
—What are the good government standards used in these countries?
Structures
—What types of constitutional frameworks do the focus countries have?
—What are the power, operational and accountability structures?
—Are the power structures, operational structures and accountability structures balanced?
—Does this structure allow each part to do its work?
| —Literature review
—In-country interviews
—World Bank literature
|
| Step 2. Policy making and delivery PASC questions 5-6
| Purpose: Understand policy making and delivery processes and how constitutional frameworks influence the processes.
|
—How is policy or legislation made? Is it informed by current policy implementation? Could changes to the policy/legislation making process increase the likelihood of successful implementation?
—Is effective policy implementation hampered by too much change?
—How do changes such as new initiatives or wider
structural reorganisations affect public sector workers' ability to deliver policy?
—How are public sector workers incentivised to deliver policy effectively?
| Sources of information
—Desk based review
—Analysis of policy areas to track policy life cycle
—In-country interviews
|
| Step 3. Performance monitoring and evaluation PASC questions 4, 7-8
| Purpose: To understand how governments monitor and evaluate their performance; to understand if and how governments improve poor performance and to consider how constitutional frameworks influence the monitoring and improvement process.
|
—What mechanisms exist for judging performance? How are targets developed? How are reviews undertaken?
—What is done in the face of poor performance?
| Sources of information
—Desk based review
—In country studies, focus on two to three policy areas and discuss performance matrix and monitoring and evaluation processes.
|
| Step 4. Best practices for UK PASC question 9
| Purpose: To identify best practices of good government and consider how they can be applied to the Westminster model.
|
3. IMPLEMENT THE
RESEARCH
This comprised two over-lapping work streams:
The work streams overlap because the in-country research
provided additional sources of literature which enabled us to
focus the desk research more sharply.
Desk review
Our desk review was guided by the agreed research questions.
We focused on a small number of policy areas in the US and France
(welfare, health and education) to provide more specific evidence
of how policy is made, delivered and measured. We also considered
public management methods used in the focus countries and elsewhere.
This approach was framed by the questions from steps 3 and 4 of
the research framework (above). We assessed the underlying causes
of the improvements in specific examples and used this to directly
address the issue of what can we learn about good government from
cases where government has got it right.
In-country research
We conducted a series of one-to-one interviews with leading
academics at the Kennedy School of Government and the Ecole National
d'Administration (ENA). The list of the experts interviewed is
detailed below:
Interviewees from the John F. Kennedy School of Government
— Linda Bilmes, Lecturer in Public Policy
— Akash Deep, Senior Lecturer in Public Policy
— Elaine Kamarck, Lecturer in Public Policy
— Steve Kelman, Weatherhead Professor of Public
Management
— Jeffrey Liebman, Malcolm Wiener Professor of Public
Policy
— Pippa Norris, Paul. F. McGuire Lecturer in Comparative
Politics
Interviewees from L'Ecole Nationale d'Administration (ENA)
— Renaud Dorandeu, Director of Studies
— Lucile Drome-North, Assistant Director of Professional
Studies,
— Frédéric Edel, Assistant Editor
of the Revue Française d'Administration Publique and researcher
for the Centre of Expertise and Administrative Research (CERA),
at the Department of Professional Studies and Research
— François Lafarge, Assistant Editor of the
Revue Française d'Administration Publique and Researcher
of the Centre of Expertise and Administrative Research (CERA),
at the Department of Professional Studies and Research
— Eric Meisse, Consultant, Centre of Expertise and
Administrative Research (CERA), at the Department of Professional
Studies and Research
Based on the findings from the desk review and the research
framework we developed interview tools to frame our discussions
with experts and other stakeholders in the focus countries. These
interviews provided a further collection of literature on specific
topics.
4. ANALYSE AND
IDENTIFY BEST
PRACTICE
A range of data was collected from the two principal evidence
bases: the desk research and interviews. We collated and analysed
the material with respect to the agreed list of questions. This
was a dynamic process in two respects. The research led to further
research and interviewees suggested further reading and specific
ideas. Also, the team met regularly to compare findings with systems
and practice in the UK in order to draw comparisons and pinpoint
international best practice.
5. REPORT
We reported the findings of our research and our analysis
as follows:
Draft report. On 19 August we submitted to the NAO study
team a draft report of our findings which highlighted key points
and allowed for open debate and discussion. We then took feedback
on the findings and deliverables and produced the final report
for the agreed deadline.
Final report. Our report was presented to the NAO on 5 September
and included:
— An executive summary setting out overall conclusions
in plain English; and
— A summary table, giving an overview of the findings
as they relate to some of the questions being considered by the
select committee, clearly referencing the material in the study
to these questions.
|
