Select Committee on Public Accounts Tenth Report

2  The effectiveness of the programme

Performance measurement

8. Key measures used to determine the effectiveness of Jobskills include NVQ achievement, progression between NVQ levels and labour market outcomes. The C&AG drew attention in his report to the need for improved timeliness of performance reporting and greater consistency and comparability in targets. In our view also, the Department's range of performance measures needs to be enhanced. We noted, for example, that it no longer has targets on training outcomes for Access level trainees, or for progression into employment by Jobskills leavers. It looks to us as though targets have been altered to avoid reporting poor performance. The Department must set uniform targets across each strand of the programme and ensure that all key aspects of performance are assessed and reported.[10]

9. Another area of concern which we noted was the absence of performance benchmarking. This is a well-established process that should be firmly embedded within any well-run organisation. We found it astonishing, therefore, that the Department had not carried out regular benchmarking with similar schemes in Great Britain. We were told that this was due to differences between Jobskills and the schemes which operated in other regions. We do not accept this explanation. Subtle differences between schemes is not a justification for failing to benchmark. We have seen this weakness before with Northern Ireland Departments and it is not acceptable. The citizens of Northern Ireland are entitled to know how the standard and cost of their public services compare with elsewhere in the United Kingdom and it is the job of the Department to facilitate this. We expect the Department to initiate a process of formal benchmarking with other regions in Great Britain, both to assess relative performance and to keep abreast of best practice, and to publish the results.[11]

The performance of Training Organisations

10. There is a considerable differential in the performance outcomes achieved by Training Organisations. The C&AG's report showed that, in comparisons of data on NVQ achievement and labour market outcomes, the performance of a substantial proportion of providers fell significantly below the average for each measure. More worryingly, there were significant numbers of providers who had very low success rates, with a small number even recording nil achievements for NVQ attainment and leavers entering employment. Data provided by the Department after the hearing showed that, of the poorest performing 20% of providers on each measure, around one quarter featured on both.[12]

11. We were particularly interested to know whether any of the poorest performers had had their Jobskills contracts renewed and, more tellingly, how many had been sacked. We were told that one organisation had been "indirectly sacked", which, it transpired, meant the non-renewal of vocational training areas with this body. The reality is that no provider has been sacked. It seems to us that the Department is not doing enough to raise the standards of the poorest performers. In our view, it needs to be a lot more proactive and agree action plans for improvements with individual training providers. Where targets are not met, the Department must be prepared to terminate contracts.[13]

Key skills

12. The level of NVQ attainment within Jobskills has been falling for some years. The Department said that this was due to the introduction, in 1999, of 'Key Skills' as a mandatory element in the programme. This is a range of six generic skills, such as communication and numeracy, required in most occupations. According to the Department, this has had a negative effect because many of the participants had left full-time education to avoid this type of classroom-based training. We found, however, that the Department had not been monitoring the achievement of individual Key Skills. Consequently, it had not identified where the main difficulties lay and how these might be overcome. This was yet another example of the Department failing to apply basic, common sense administration. The Department has said that, following the C&AG's recommendation, it has amended its management information system so that the requisite monitoring data can now be collected. We look forward to a considerable improvement in Key Skills attainment levels in the future.[14]

Cost per job

13. One of our main areas of concern was the cost per job created by Jobskills. Since its inception, the programme has catered for some 76,000 young persons, at a total cost of £405 million. With an average of 46% of leavers gaining employment, this suggests a cost per job of around £11,600. However, we noted that that many of those leaving the programme to enter employment do so without completing their NVQ qualification and almost half of all leavers take up a job in a different occupational area from that within which they trained in Jobskills. That suggests to us that Jobskills has had little to do with the securing of those jobs. This is further supported by the Audit Office's estimate of 'deadweight' in the programme (where the jobs would have been gained even without Jobskills) of some 48%.[15]

14. The evidence would suggest, therefore, that the actual cost per job created by Jobskills may be closer to twice the figure of £11,600. In our view, a unit cost of the order of £22,000 is excessive and represents poor value for money. We would like the Department to look closely at this issue, with a view to establishing, and subsequently reducing, the unit cost per job created.

Progression of Access level trainees

15. The purpose of the Access (Jobskills Level 1) strand is to prepare young people with a particular disadvantage to undertake training in the 'Mainstream' (Level 2) strand. Given that this is the grouping with the greatest problems, we found it disturbing that, over the life of the programme, only 40% of Access trainees had progressed to Level 2 and the rate had been falling for some years. The Department told us that it is running a pilot programme aimed at reversing this trend. It said that an interim evaluation has shown a significant increase in the progression rate to 68%. Following a full evaluation later this year, the Department hopes to roll the pilot out across the whole programme.[16]

16. We are very disappointed that almost two-thirds of this particularly disadvantaged grouping has not completed the course, gained a qualification and progressed into mainstream Jobskills training. Nevertheless, the interim results of the pilot scheme are encouraging. We expect to see a marked improvement across the Access strand as a whole and would like the Department to set itself a challenging target in this regard.

Net employment impact of Jobskills

17. One of the most disappointing outcomes of Jobskills is the extremely low net employment impact of the programme. An analysis carried out by the Audit Office estimated that it may be as low as 14%, after allowing for the effects of 'deadweight', 'substitution' and 'displacement'. The Accounting Officer pointed out that the figure for a similar scheme in Scotland was estimated at 12%. This is no defence. The Department has to accept full responsibility for its own scheme and take steps to secure a much better return for the taxpayer. To help achieve this, it should put procedures in place to periodically estimate the net employment effect of Jobskills and set targets for reductions in the levels of deadweight, substitution and displacement.[17]

Early leaving

18. Almost half of all trainees who start Jobskills leave early. The proportion that leaves within four weeks has remained relatively constant over the life of the programme, at around 10% of total starts. Some 40% of the remainder leave after completing more than four weeks but without achieving their NVQ qualification. This is a particularly undesirable outcome in that resources are wasted and the employability of leavers is not enhanced without a qualification.[18]

19. We asked the Department to put a cost on early leaving. It estimated that approximately 4% of programme expenditure (currently £2 million per year) is spent on young people who leave the programme early and do not move to what it describes as a 'positive outcome' (employment, further education or other training opportunities). Given that almost half of all trainees starting Jobskills leave early, this estimate seems very low. We note that a preliminary review by the Audit Office, following the hearing, considered that the actual figure may be substantially higher and possibly around £6 million per year. It is important that the Department establishes a reliable estimate and we have asked the C&AG to review the outcome.[19]

20. We asked why so many trainees leave early. The Department explained that many young people take time to settle into the system and may change their minds about what they want to do. The introduction of Key Skills (paragraph 12 above) had also had a negative impact. The Department was optimistic that around 90% of those who left within the first four weeks were subsequently picked up by the Careers Service and re-integrated into Jobskills. It did admit, however, that it had no formal tracking system. We put it to the Department that the problem did not lie solely with trainees - the Audit Office survey had noted several reasons for early leaving that lay within the Department's own control; for example, the pace of the course was inappropriate and work-placements were of poor quality. The Department accepted this. We were also told that a number of early leavers go to take up a job, although the Department conceded that this was not ideal as the people involved left Jobskills without a qualification.[20]

21. In terms of tackling the problem, the Department said that it worked closely with its colleagues in Great Britain and had learned a lot from their experience. It again referred to the ongoing pilot scheme with Access trainees (paragraph 15 above) which had seen improved retention rates. In addition, it wanted to better focus its Careers Service on making progress in this area.[21]

22. The high level of early leaving has seriously undermined the overall effectiveness of Jobskills. What we find particularly worrying is that the level has remained relatively constant over the life of the programme. This is simply not good enough given that, since 1995, a huge amount of taxpayers' money has effectively 'gone down the drain'. The Department must adopt a specific programme objective to tackle premature leaving and set targets for a radical reduction in the incidence. We want to see a marked improvement in this area and expect the Department to attach a high priority to doing so.

10   Qq 86, 88; C&AG's Report, para 3.5 Back

11   Qq 104-107; C&AG's Report, paras 3.18-3.19; 27th Report from the Committee of Public Accounts, The management of substitution cover for teachers (HC 473, Session 2002-03) Back

12   Qq 89-95; C&AG's Report, paras 3.50-3.52 and Figure 13; Ev 16-17 Back

13   Qq 53-55, 59, 72-74 Back

14   Q 113; C&AG's Report, paras 1.7, 3.11-3.13 and Figure 8; Ev 17 Back

15   Qq 20-36; C&AG's Report, paras 3.26-3.27 and Figure 10, 3.64 and Figure 15, 4.17 Back

16   Q 6; C&AG's Report, paras 3.22 and Figure 9, 3.44 Back

17   Qq 37-39; C&AG's Report, paras 3.59-3.64 and Figure 15 Back

18   C&AG's Report, paras 3.34-3.37 and Figure 12 Back

19   Qq 48-50 Back

20   Qq 7-10, 44-47, 51-52, 60-61 Back

21   Qq 49, 63-67 Back

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2005
Prepared 3 November 2005