• Accessibility
• Email alerts
• RSS feeds
• Contact us

Process for independent evaluation of progress on Government commitments

---

Independent evaluation of progress on Government commitments

Introduction

1.Holding Ministers and their departments to account has always been a core function of select committees.1 In this brief report, we set out a process for a select committee-led independent evaluation of progress on Government commitments in health and social care, designed to develop and enhance that core task of holding the Government to account.

The proposed process

Commissioning the independent evaluation

2.The first step in the process will be to establish an independent panel, comprised of experts. We will commission those experts to evaluate a specific policy area. We will choose a policy area for evaluation based on our knowledge and understanding of the policy context, and our judgement of where such an evaluation could add most value.

3.Initially, we will commission only one evaluation process at a time. We will keep that under review in the light of experience, and in light of the resources which need to be devoted to the evaluation and Committee work arising from it. The first policy area we have chosen is maternity services, and we intend to establish an expert panel to consider Government commitments in that area in the autumn. That first evaluation will act as a pilot, and we will review the process in the light of the experience gained.

Establishing the standards/pledges to judge

4.We will begin the process by inviting the Secretary of State for Health and Social Care, NHS England and Improvement and/or any other relevant Government body to tell us what commitments they have made in the chosen policy area.2

5.From the Government’s response to that invitation, and working with reference to other public documents such as the NHS Long Term Plan,3 we will agree a list of commitments in our chosen topic area. The panel will then evaluate progress against these commitments. The list of commitments to be reviewed will not necessarily be exhaustive, but it will be sufficient for the expert panel to reach a judgement about whether the Government is meeting its commitments across the entire topic area.

Expert panel

6.The expert panel will have a core membership of three people, plus a further membership of three to six people chosen for their expertise in the particular set of commitments being examined.

7.Members of the expert panel will be appointed as specialist advisers to the Committee. They will not be full-time paid roles. The Committee secretariat will act as the secretariat to the independent panel.

Core membership

8.We have appointed Professor Dame Jane Dacre to chair the expert panel and to serve as one of the three core members. Prof Dacre is Professor of Medical Education at University College London, and a consultant physician and rheumatologist at Whittington Health in London. She is a former Director of UCL Medical School and past medical director of the MRCP(UK) exam, and was President of the Royal College of Physicians between 2014 and 2018. Prof Dacre has been a member of the General Medical Council and has an academic interest in assessment systems. She has a research portfolio which includes the development and evaluation of medical curricula and assessments. Her wide experience in medicine, policy and assessment makes her exceptionally well-qualified to take on this role.4

9.We will appoint two further people to join Prof Dacre as the core members of the expert panel:

• one person with experience of representing patient concerns at a high level
• one health and social care policy expert, with wide experience of assessing and analysing Government health and/or social care policy and its implementation.

Those people will be chosen through an open recruitment campaign, which will begin shortly. They will serve for a year in the first instance; their appointments will be renewable.

Additional membership

10.The rest of the membership of the panel considering a particular topic will be decided as best to judge the particular set of commitments being considered, and will include:

• One or two service users
• One or two clinicians
• One or two policy experts/campaigners.

11.Those panel members will be appointed by us in consultation with the core members of the panel, after due consideration of the skills and experience required for the particular topic area subject to evaluation. Their appointment will last for the duration of the specific evaluation they have been appointed to conduct.

Declarations of interest

12.All appointed members of the panel—both the core members and additional members—will be asked to make a declaration of relevant interests, which we will publish.

Principles for the appointment of panel members

13.The following principles will guide our appointment of panel members:

• Individuals of good standing, who are guided by the Seven Principles of Public Life (“the Nolan Principles”)5
• Able to provide impartial evaluation, based on data presented, and not pre-existing belief or political affiliations
• Able to speak truth to power, and to respond effectively to challenge
• Prepared to attend face to face and virtual meetings and to review documents as required
• Provide expertise in the area being examined by the panel, as a clinician, service user or policy expert.

14.The panel will be composed so as to ensure, so far as possible amongst a small group, appropriate diversity in terms of gender, disability, ethnic background and other protected characteristics.

Undertaking the evaluation

Questions for evaluation

15.The following questions will provide the framework for the evaluation of each commitment:6

• Was the commitment met overall? or (in the case of a commitment whose deadline has not yet been reached) Is the commitment on track to be met?
• Was the commitment effectively funded?
• Did the commitment achieve a positive impact for patients?
• Was it an appropriate commitment?

Sub-questions

16.Once the commitments have been agreed for work in a particular policy area, the expert panel will consider what “sub-questions” it will need to consider under each of the questions above in order to make an appropriate assessment of the particular commitments it has been asked to evaluate. Those “sub-questions” will be returned to us for our endorsement.

Method of evaluation

17.The Government will be invited at the start of each evaluation to provide its own evidence of achievement of, or progress towards, the chosen commitments. Analysis and testing of that evidence will be at the centre of the expert panel’s work.

18.We will also issue an open call for written submissions specifically addressing the question of the extent to which the chosen commitments have been met.

19.The evaluation framework used by the expert panel for its evaluation will be derived from published methods, including “realist review”.7

20.The framework will include a “basket” of techniques which may be called upon by the expert panel according to the particular task or subject which they are examining. Those techniques may include—but not be limited to—the following:

• Key word analysis
• Other forms of linguistic analysis of policy documents and/or written submissions to the Committee
• Stakeholder interviews
• Financial analysis, particularly of financial flows in support of chosen commitments
• Analysis of public data, including written submissions to the Committee.

21.In particular, the methodologies used by the National Audit Office (NAO) for its “value for money” work may be used.8

22.The panel will seek appropriate input from the NAO and the Care Quality Commission, as the bodies established by Parliament to take an independent view on health and care matters.

23.The independent panel will decide in each case what documents it wishes to consider to inform its evaluation. They might include (but again, not be limited to):

• The Government’s memorandum
• Written submissions in answer to the Committee’s call
• CCG/Trust/ALB policy documents/reports/Board papers
• Other published data or reports.

Output

24.The panel will produce a report with a rating against each chosen commitment, using the scale used by the CQC (Outstanding/Good/Requires Improvement/Inadequate). It will also provide commentary on its judgements. Draft “anchor statements” describing the standard represented by each rating can be found in the Annex to this report.

25.The panel will also make an overall rating, on the same rating scale, of the Government’s progress towards its commitments in the chosen policy area.

26.We will expect to publish the panel’s report.

Follow-up

27.The key to the success of this exercise, in our view, will be following up the panel’s report. Where the panel reaches the judgement that the Government’s performance against its commitments in a particular area is “inadequate” or “requires improvement”, we expect the publication of that judgement to galvanise the Department and its arm’s-length bodies to action to ensure improvement. We will expect to return to the same topic area and repeat the independent evaluation exercise after an appropriate period, to see whether improvements have been made.

---

• A-Z index
• Glossary
• Contact us
• Freedom of Information
• Jobs
• Using this website
• Copyright