24.We distinguish between two different types of tests for readiness. First, there are operational tests of readiness. These would assess if Universal Credit full service is operating as it should for claimants so that the Department can begin the managed migration pilot. Second, there are tests of readiness to ‘go to scale’. These tests would be applied following the pilot to assess whether the Department is ready to begin managed migration ‘at scale’.
25.Neil Couling told us in October 2018 that the Department has set “entry and exit” criteria for each stage of Universal Credit so far and that it intends to continue to do so:
We have done that at every phase of the programme; we have had a set of entry or exit criteria from the phase we are in, and on the bigger phases we had independent scrutiny to find out whether we are deluding ourselves and what it looks like to a bunch of other project professionals who have run similarly sized projects. I fully accept what the NAO says about having a set of criteria.
26.The Minister for Employment, Alok Sharma MP, also emphasised to us on 18 October 2018 the importance of clear, measured criteria before undertaking managed migration at scale:
You are absolutely right that we will have to have some metrics. Particularly in terms of decision to expand, we will have to make sure that, operationally, we can tick the box for operational readiness, efficiency of the service and being able to support vulnerable claimants and making sure that the functionality is in place.
27.Neil Couling did, however, argue that the criteria for readiness to move to managed migration at scale could only be defined during the pilot. He said this would ensure there was enough flexibility to change the rollout plans in light of the pilot results. This reflects what the Department describes as its “agile” approach to rolling out Universal Credit:
I fully envisage that we will have such criteria, and I would use the testing phase to judge what those criteria will be. I do not want to commit to them now. Universal credit has been going on for quite some time; I think one of the mistakes that was made in 2011, which Mr Mills alluded to, was that people got themselves locked into a plan but did not know if they could deliver it. I do not want to make that mistake again—we talk about learning the lessons of the past. We will set some criteria, but not now.
28.The Minister also suggested that the pilot could reveal the need for additional tests that would be relevant before beginning managed migration ‘at scale’:
We will have things like payment time limits and others, which are already there. What is going to be important is making sure that we learn from this test phase. There may be additional metrics that we think, “Actually, this is quite important, particularly given the cohort of people that we are moving across.”
29.The Department provided further details of its approach to defining success criteria for the pilot to the SSAC. The SSAC noted in its minutes that the Department claimed:
There were simply too many things to learn and too many different types of claimants and their circumstances to enable an accurate prescription of success to be defined at the outset. Success would be defined by the Department’s ability to identify who needed support, how it should be given and its effectiveness in getting them through the process with the correct entitlement (including transitional protection), and that at the end of it claimants understand the new benefit world and how it worked for them.
The implication of this approach, however, is that the Department will have neither set tests before beginning the pilot nor defined what constitutes readiness for moving to UC managed migration ‘at scale’.
30.As a major Government project, the progress of the Universal Credit rollout is overseen by the Infrastructure and Projects Authority (IPA). The IPA is a partnership between the Cabinet Office and the Treasury. As part of its oversight, the IPA carries out Project Assessment Reviews (PARs), based on three to ten days of interviews and a review of documentation. The PARs inform meetings of the Major Projects Review Group (MPRG) which assesses the deliverability, affordability and value for money of major Government projects.
31.The IPA review process nevertheless has significant limitations. We documented some of these in our 2018 report, Universal Credit Project Assessment Reviews. In particular, the reports do not examine Government policy choices, and have not sought to measure how UC is performing for claimants:
The IPA reviews have restricted terms of reference. Their UC reports provide assurance about the finances and delivery of the programme, subject to a scope agreed by the DWP programme team. The overwhelming majority of interviewees were civil servants from DWP and others most closely involved in UC. The reports do not examine Government policy or the consequences for claimants. They do not, for example, include a single statistic of whether people were receiving their payments on time.
32.IPA reviews are therefore not a substitute for robust, transparent “entry and exit” criteria and readiness tests before moving to subsequent stages of UC rollout. Indeed, the IPA have criticised the Department for failing to set such criteria. As we concluded in Universal Credit Project Assessment Reviews:
The IPA has been consistently critical of the Department’s failure to set clear criteria for proceeding to the next stage of the UC rollout. Setting clear performance standards in advance is the best way of ensuring decisions are made objectively. The publication of these would both benefit scrutiny and make it far easier for the Department to explain its decisions.
33.The National Audit Office and SSAC both recommended that tests of operational readiness are needed before the Department begins the pilot. These tests would be to make sure that UC full service is operating as it should for new and naturally migrating claimants, separate from any further tests of readiness for managed migration ‘at scale’. The National Audit Office recommended that the Department:
[…] formally assess the readiness of automation and digital systems to support increased caseloads before migration begins, and ensure the programme does not expand before business-as-usual operations can cope with higher claimant volumes.
Likewise, the SSAC recommended that:
[…] before the testing phase of the managed migration process commences, the Department should publicly define what it considers good operational readiness to be. It should then undertake a rigorous and transparent assessment of whether it has met those criteria (and, if not, what challenges remain). In undertaking this assessment, due consideration should be given to how effectively Universal Credit is currently operating, taking account of the evidence available after the completion of the first phase of the roll-out programme at the end of this calendar year.
34.SSAC proposed that the operational tests of readiness that should be set before the Department begins the managed migration pilot might include:
35.The Secretary of State has argued that, since the NAO and SSAC framed their recommendations on tests for readiness when the Government had an earlier date at which they planned to go ‘to scale’, these recommendations have “less resonance”. But it is hard to see why a longer delay before going to scale would obviate the need for objective tests of readiness. In any case, both the NAO and SSAC recommended that operational readiness tests are applied before the pilot begins, not just before managed migration ‘at scale’.
36.Whilst UC payment timeliness is improving, the Department’s most recent figures show that 14% of new claimants are still not receiving their first UC payment in full and on time. This suggests to us that the Department should not assume the operational readiness of UC before beginning the managed migration pilot.
37.The Department has not provided any convincing reason why it has not accepted the recommendations from expert bodies to set and apply operational tests of readiness before beginning the managed migration pilot. Given that these tests would be separate from tests of readiness to ‘go to scale’, we do not accept the Department’s argument that the NAO and SSAC’s recommendations have any “less resonance” than when they were first framed. If the Department is confident that UC is operationally ready to begin the managed migration pilot, we cannot understand why it will not set tests to demonstrate this. The Department should immediately set tests of operational readiness based on those proposed by the SSAC and ensure that these tests are met before a single claimant is transferred through managed migration. The Department should publish the results of the tests.
38.We wrote to stakeholders in February 2019 asking them for their views on the Department’s decision not to set tests of readiness before beginning the managed migration pilot. We proposed tests of readiness in the following areas, based on the NAO’s outline of data relevant to managed migration:
39.We received responses from Citizens Advice, the Low Incomes Tax Reform Group, Mind, NAWRA, SSAC and the Trussell Trust. Some stakeholders agreed to an extent with the Department that detailed tests of readiness for managed migration, and the benchmark levels at which any tests were set, could not be set in advance of the pilot. Mind told us:
We appreciate that it may be sensible for the Department to establish detailed success measures for the full roll-out of managed migration after it has completed the pilot. We also appreciate that there will be some measures the Department can only meaningfully set once it has designed the detail of how the pilot will work.
Similarly, the Low Incomes Tax Reform Group told us:
We do accept that some of the tests that will be used to inform whether to scale-up managed migration after the pilot may need some refinement following the pilot stage.
40.The same stakeholders, however, emphasised the need for robust and transparent tests to evaluate the pilot, to enable the Department to assess if it is ready to begin managed migration at scale in a way that is safe for claimants. For example, Mind told us:
Unless the pilot is evaluated against some established criteria of success then it is not clear on what basis the Department could make a meaningful decision on whether it is safe to proceed with the wider roll out.
The Department has said that it will frame the tests during the pilot. To do so, however, it must decide what data it will collect during the pilot that might be relevant to defining and applying the eventual tests—for example, whether to collect data on payment timeliness or on claimant vulnerabilities. Inevitably, the Department will have to define some provisional readiness or success criteria in advance of the pilot, even if in very broad terms. Otherwise, any eventual tests that the Department sets will be constrained by whatever data it has collected. The Secretary of State told us that DWP “cannot begin to learn until we actually start”. Equally, the Department cannot ignore existing evidence or properly start a pilot until it knows what it wants to learn.
41.The Government will find it easier to justify its decisions as it proceeds with UC rollout if it commits to transparent, objective tests at each stage. Given the need to rebuild trust in Universal Credit, transparent tests of readiness before proceeding to managed migration ‘at scale’ could offer reassurance both to claimants and to support services. Tests of readiness will also indicate to the Department at an early stage if its timescales for full migration to Universal Credit are realistic.
42.We accept that any tests of readiness to ‘go to scale’, and the threshold levels which would indicate readiness, will need to be refined in light of the results from the managed migration pilot. We are not convinced, however, that the Department can begin the pilot without any provisional view at all on the tests of readiness to ‘go to scale’, for the simple reason that the Department must decide in advance what data it will collect during the pilot.
43.Under the Regulations currently laid before the House, the Government is only seeking powers to transfer up to 10,000 claimants to Universal Credit through managed migration. The Government will therefore need to ask the House for the powers to begin managed migration ‘at scale’ by laying further Regulations, once the pilot is complete. Clear, robust and transparent tests of readiness will be needed for the House to make an informed decision about whether to grant such powers.
44.Published tests of readiness are needed to hold the Government to account at each stage of Universal Credit rollout. Without them, it is unclear how the House could be expected to make an informed decision about whether to grant the powers needed to begin managed migration at scale. If the tests are set only following the pilot, the Government would risk the perception that it is setting the tests only once it knows what the results will be.
27 SSAC, , para 2.4.1 (k)
28 Work and Pensions Committee, Fifth Report of Session 2017–19, , HC 740
29 Work and Pensions Committee, Fifth Report of Session 2017–19, , HC 740, para 11
30 Work and Pensions Committee, Fifth Report of Session 2017–19, ,HC 740, para 35
31 National Audit Office, , HC 1123, June 2018, para 19
32 SSAC, p 25
33 SSAC, p 33
34 , 5 February 2019
35 Universal Credit: 29 April 2013 to 14 February 2019, , 19 March 2019, table 6.1
36 , 20 February 2019
37 , 25 February 2019
38 , 20 February 2019
39 , 5 February 2019
Published: 1 May 2019