In-work progression in Universal Credit Contents

3Building the evidence base

The pilot

18.The DWP’s in-work progression evaluation programme has four key objectives:

  1. identifying the most effective method and level of support;

  2. understanding how employers can best support progression;

  3. identifying the most effective financial levers to drive claimant behaviour without substantial advisor intervention; and

  4. understanding the effect of mandating compliance, setting requirements and imposing conditionality.

The first phase of the programme is a pilot designed to test the effect on UC claimant earnings of interview-based support in Jobcentre Plus (JCP). The pilot will run UK-wide for three years from February 2015, with a final evaluation report in 2018. It has a target sample of 15,000 claimants by autumn 2016, randomly selected by National Insurance number.39

19.The DWP is evaluating its pilot in-work progression scheme using a Randomised Controlled Trial (RCT). An RCT tests participants subject to a policy change against a control group not subject to the change. It is widely thought to provide the clearest evidence of the effect of a policy intervention, particularly where the change is experimental in nature.40 Participants in the DWP pilot are randomly assigned to one of three groups of 5,000 claimants:41

Participants in groups 1 and 2 agree mandatory actions to increase their earnings, whereas the actions of group 3 participants are voluntary.42 The interviews explore options for the claimant to increase their hours, take a second job, or secure a more lucrative job. Failure to meet mandatory actions may result in claimants losing their benefit payments. We consider this conditionality in Chapter 5.

20.Alongside the pilot, the DWP’s business case for a full in-work service, due to be prepared in 2017, will incorporate the findings of externally led trials, run by employers, social enterprises and other agencies.43 These trials target both sectors where progression is typically more limited and specific groups of individuals.44 Organisations responsible for delivering the relevant externally-led trials include:

Transparency

21.Witnesses were concerned about a lack of published information about the pilot.46 The University of Glasgow’s Social Sciences Unit said, “details remain scarce” and said this contravention of standard reporting guidelines (see below) threatened to undermine the credibility of the trial.47 Citizens Advice Scotland told us that any information about the DWP’s approach had emerged only in response to Freedom of Information requests.48

22.The DWP initially told us that the pilot was launched in 10 Jobcentres in April 2015.49 Additional information on the number and characteristics of participants, only made available in response to our requests, showed 500 to 700 participants were included in the initial stage of the pilot. The pilot was then extended to a further 80 JCP offices from December 2015.50 The DWP did not provide us with a breakdown of the participants’ characteristics. Standard trial reporting guidelines state that a trial should include information on what it will measure, the number and characteristics of claimants involved, and interim trial findings.51 ERSA called for the DWP to publish such information.52

23.One year from the start of the DWP’s in-work progression pilot, there is insufficient information in the public domain about the number and characteristics of participants and effects on their pay and employment. Making such information more freely available would strengthen the credibility of the pilot, raise awareness of the in-work service and allow other experts to appraise the results of the trial. Greater transparency will make for better policy. We recommend the DWP regularly publish basic information about the nature and outcomes of its pilot. The first such publication should, at the latest, coincide with the Government response to this Report.

Testing alternative forms of support

24.Though the LWI described the pilot as a “good start” in testing in-work progression,53 we heard varied calls for different approaches to be evaluated. Several witnesses said that in-work claimants would benefit from more flexible in-work support than offered by face-to-face interviews with JCP Work Coaches.54 Emma Stewart, Joint Chief Executive of the Timewise Foundation, suggested that, following an initial face-to-face consultation, email might be a more effective form of communication for in-work claimants.55 Respondents to the DWP’s 2013 call for in-work progression ideas from the private, public and voluntary sector suggested coaching sessions by telephone or video call.56

25.We also considered the role of financial rewards. A smaller 2003 DWP pilot, the Employment Retention and Advancement (ERA), trialled a tri-annual retention bonus of £400, paid for two years to participants who consistently worked more than 30 hours per week, alongside JCP-led in-work advice. It demonstrated that in-work support and financial bonuses could help some groups of claimants to progress in work, effects that were still evident at the end of a five-year follow-up period.57 The DWP told us that it would review the use of financial levers in future.58

26.Other witnesses said that the DWP should more actively test delivery by the voluntary, community and private sectors.59 ERSA suggested that many providers have greater experience than JCP in assisting particular groups of people.60 Centrepoint, a national charity, was already providing successful in-work support for homeless young people. Inclusion London was also performing a similar role for disabled people.61

27.Ross James, Head of UC labour market policy at the DWP, explained that the DWP was seeking to ensure the validity of the trial by isolating the effects of the pilot from those of support from non-JCP providers.62 The DWP emphasised that private and voluntary sector groups had important roles in supporting in-work progression and that the RCT was one part of a wider strategy to build the evidence base.63 Tony Wilson acknowledged this rationale but called for a subsequent evaluation to draw on learning from “broader and deeper” approaches.64

Outcome measures

28.Three measures are used to evaluate the effects of UC:

(1)the number of claimants moving into work;

(2)the number of claimants in sustained employment (in 3, 6, or 12 months); and

(3)the number of claimants whose earnings increase.

The pilot focuses on measuring the number of claimants who earn more.65 Witnesses said this could incentivise an individual to take on a second job or move to a new role when they might be better served staying in their current position.66 Barnardo’s, a national charity for vulnerable children, expressed concern that the earnings focus could prompt individuals to take low-skilled jobs, which “may not be the best decision for their long-term earnings prospects”.67 Other witnesses stressed that undertaking work-related training alongside existing employment may lead to a greater longer-term progression.68 We variously heard that the DWP should consider measuring promotions, job stability, take-up of training, and acquisition of skills and qualifications as part of a “broader and long-term approach” to measuring the effects of in-work support.69

Conclusion

29.There is no comprehensive evidence on which to determine how to deliver an effective in-work service. The DWP will therefore be learning on the job. As the risks and potential rewards are both high, it is vital the DWP carefully builds a broad evidence base. The randomised controlled trial is a commendable start. The pilot will, however, only give a partial picture.

30.If the policy is to be successful, and achieve its great potential, we recommend the DWP direct the best of its welfare reform talent at developing the service. As part of its wider assessment of in-work progression, we recommend the DWP test more flexible forms of contact in addition to face-to-face interviews and draw more widely on voluntary, community and private sector-led support. It should also consider broader measures of progress alongside earnings and clarify any plans or timescales to incorporate financial bonuses in in-work schemes.



39 DWP (IWP0034)

40 HM Treasury, The Magenta Book: guidance for evaluation, April 2011. See also Oakley, M., Employment support for a high-wage economy. Joseph Rowntree Foundation, 2015

41 Q145 (Ross James). Claimants who are unable to work or earn more due to caring responsibilities or because of health conditions or disabilities are not required to participate. Recent victims of domestic violence; prisoners; recently bereaved people; or people undergoing treatment for drug or alcohol addiction are also excluded from the pilot. Self-employed claimants who are eligible for separate support with setting up their own business are also excluded

42 DWP (IWP0034)

43 DWP (IWP0034)

44 DWP (IWP0034)

45 DWP (IWP0034)

46 See, for example, Q76 (Faye Goldman), Reed in Partnership (IWP0001), Employment Related Services Association (IWP0032), Citizens Advice Scotland (IWP0012), Parkinson’s UK (IWP0003), Mind (IWP0028), Remploy (IWP0024), Oxfam GB (IWP0016), Gingerbread (IWP0026), Boycott Workfare (IWP0020), Centrepoint (IWP0005), and University of Glasgow, Social and Public Health Sciences Unit (IWP0021)

47 University of Glasgow, Social and Public Health Sciences Unit (IWP0021)

48 Citizens Advice Scotland (IWP0012). See also Boycott Workfare (IWP0020). For examples of FOI requests see, DWP, In-Work Progression Randomised Control Trial – Lines to Take, April 2015; DWP In-Work Progression Randomised Control Trials – Q&A, April 2015; and DWP In Work progression Randomised Control Trial – Awareness for our people in Service Centres, April 2015

49 DWP (IWP0034)

51 Standard guidelines for trial reporting refer to the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guidance. For further information, see: SPIRIT, 2013

52 Employment Related Services Association (IWP0032)

53 Learning and Work Institute (IWP0031)

54 DWP, In-work progression: supporting information for Work Coaches (IWP0044): section 1.3 states that the preferred method of contact for all Group 1 and 2 claimant interviews is face-to-face. On more flexible in-work support, see, for example, Learndirect Limited (IWP0007), Reed in Partnership (IWP0001), Remploy (IWP0024), PCS Union (IWP0009), University of York (IWP0029), Child Poverty Action Group (IWP0023), Barnardo’s (IWP0037), Working Families (IWP0040), Community Links (IWP0036), and Centrepoint (IWP0005)

55 Q17 (Emma Stewart). See also: Community Links (IWP0036), and Barnardo’s (IWP0037)

57 Hendra, R. et al, Breaking the low-pay, no-pay cycle: Final evidence from the UK Employment, Retention and Advancement (ERA) demonstration, Research report 765, Department for Work and Pensions, August 2011. There were 9,000 participants. Sustained employment and long-term earnings gains were seen among more highly educated lone parents and the previously long-term unemployed parents. The policy did not proceed beyond the trial.

58 DWP (IWP0034)

59 See, for example, Qq 30-31 (Tony Wilson), Association of Employment and Learning Providers (IWP0033), Employment Related Services Association (IWP0032), Prospects Services (IWP0030), Business in the Community (IWP0013), Learndirect Limited (IWP0007), Remploy (IWP0024), Wheatley Group (IWP0008), Mark Cosens MA MIEP (IWP0004), Inclusion London (IWP0011), Mind (IWP0028), Gingerbread (IWP0026), Oxfam GB (IWP0016), Community Links (IWP0036), National Housing Federation (IWP0018), and Centrepoint (IWP0005)

60 Employment Related Services Association (IWP0032)

61 Centrepoint (IWP0005) and Inclusion London (IWP0011)

62 Q145 (Ross James)

63 DWP (IWP0034), Q132 (Ross James)

64 Q30 (Tony Wilson)

65 DWP, In-work progression: supporting information for Work Coaches (IWP0044), see section 5

66 See, for example, Business in the Community (IWP0013), Employment Related Services Association (IWP0032), Barnardo’s (IWP0037), Child Poverty Action Group (IWP0023), and Centrepoint (IWP0005)

67 Barnardo’s (IWP0037)

68 See, for example, Employment Related Services Association (IWP0032), Association of Employment and Learning Providers (IWP0033), Gingerbread (IWP0026), Working Families (IWP0040), Boycott Workfare (IWP0020), National Numeracy (IWP0006), and Professor Jane Millar, University of Bath (IWP0002)

69 See, for example, Qq 24-25, 28-29 (Dr Anthony Rafferty, David Finch, Emma Stewart), Learning and Work Institute (IWP0031), Employment Related Services Association (IWP0032), Learndirect Limited (IWP0007), Child Poverty Action Group (IWP0023), Citizens Advice Scotland (IWP0012), Prospects Services (IWP0030), University of York (IWP0029), National Housing Federation (IWP0018), Crisis (IWP0015), and Professor Nick Bailey, University of Glasgow (IWP0014)




© Parliamentary copyright 2015

Prepared 06 May 2016