Troubled families: progress review Contents

Conclusions and recommendations

1.The Department for Communities and Local Government’s delayed publication of the evaluation of the Troubled Families programme was unacceptable. The Department for Communities and Local Government (the Department)’s evaluation of Troubled Families was originally due to be published in late 2015. However, the Department did not publish it until 17 October 2016, a year later. We requested the evaluation reports on 5 October 2016, but the Department did not provide them to us until they were published on 17 October, just two days before the evidence session. These delays and obfuscation have given a bad impression about the Department’s willingness to be open. The Department informs us that it intends to report to Parliament annually on the progress of the Troubled Families programme, starting in March 2017.

Recommendation: The Department should ensure that its reports to Parliament and the public on the progress of the Troubled Families programme contain the evidence necessary to conclude meaningfully on the programme’s progress. Furthermore, to support effective parliamentary scrutiny, the Department must publish evaluations promptly and supply the Committee with timely information.

2.The Department’s evaluation of the Troubled Families programme was unable to find consistent evidence that it had any significant impact at this stage. Some strands of the Department’s evaluation found evidence of good practice as a result of Troubled Families, such as enabling authorities to support more families and transforming the way they worked with these families. It also found significant evidence that the confidence and attitudes of families participating in the programme had improved. However, the impact evaluation could not directly attribute improvement in a range of outcomes related to employment, crime and health to the Troubled Families programme. The Department was unable to provide assurance at this stage that it would be able to evidence a statistically significant impact of the programme in the future. We are concerned that the Department commissioned the evaluation, and although it is challenging to measure success in the short term, the complex discussions and external reviews suggest that the original commissioning was less than perfect.

Recommendation: The Department must develop a more meaningful longitudinal methodology for evaluating the impact and quality of the Troubled Families programme at both a national and local authority level. This should include an effective annual reporting system.

3.The payment by results framework led to some councils attempting to move families through the programme quickly, potentially at the expense of reduced quality of support. According to the Department’s evaluation of the Troubled Families programme, under the payment by results framework councils were incentivised to move families through the programme quickly in order to draw down payments without providing the support necessary to tackle deep rooted problems. In some cases, local authorities used national administrative datasets to identify families that had achieved positive outcomes and then retrospectively identified these families as ‘troubled’ and drew down reward payments.

Recommendation: The Department should review its payment by results mechanism for Troubled Families to ensure that this is not resulting in local authorities diluting the quality of the support they provide to these families. The Department needs to track what ongoing support is provided to determine what works.

4.The terminology used by the Department overstated the success of the Troubled Families programme in transforming the lives of families. The Department claimed that as many as 116,654 families had been “turned around” by the programme, which is 99% of the 117,910 originally identified by local authorities as eligible for participation in the programme. The implication of ‘turned around’ was misleading, as the term was only indicative of achieving short-term outcomes under the programme rather than representing long-term, sustainable change in families’ lives. While there was some success, by claiming that an outcome achieved meant that a family had been “turned around”, the Department’s use of the term overstated the impact of the Troubled Families programme. The use of this term suggested that long term social problems could be fixed within a few months or years, and risks undermining the entire concept of this work. The Department should not have used such misleading terminology.

Recommendation: The Department should ensure that the terminology it uses to communicate the achievements of the Troubled Families programme gives an accurate depiction of how disadvantaged families make progress.

5.The Department has not demonstrated that the programme has provided genuine financial savings. The Department announced that the programme had saved £1.2 billion. This is an overestimate as it does not take account of the costs of delivering the programme, although these included existing resources at local authority level as well as new money. The Department acknowledged that the £1.2 billion savings figure did not reflect the net savings directly attributable to the Troubled Families programme, and is currently seeking to work with local authorities to develop a better approximation of its financial benefits. Although these claimed net savings cannot be attributed to the programme directly, it also cannot be proven that the programme has had no financial impact.

Recommendation: The Department needs to be able to make more accurate claims about any savings that can be attributed to the Troubled Families programme and, with local areas’ input, develop better ways of measuring whether the programme is achieving value for money.

16 December 2016