2 Whether reported improvements are
calculated accurately
6. Where departments have identified genuine efficiencies,
in most cases the measurement of these efficiencies has not been
sufficiently robust. Departments need to use reliable measures
for calculating their efficiency gains in order both to gauge
the success of their actions and to gain public trust in their
reported statistics.
7. Because efficiency gains are a relative, rather
than an absolute, measure they need to be calculated relative
to a baseline. To know the extent of improvement achieved, it
is essential to select a baseline that fairly represents the level
of performance before the relevant changes were made. For its
initiative to reduce the average length of time patients stay
in hospital, the volatility of the data led the Department of
Health to make an adjustment to the 2003-04 baseline figure. This
adjustment was not statistically sound, and inflated the reported
efficiency gains by £300 million when compared to a calculation
that uses an unadjusted baseline figure.[14]
8. The selection of the baseline period has also
been an issue when reporting headcount reductions. On the basis
of the C&AG's report, the NAO can give substantial assurance
on the headcount reductions reported by the Department for Work
and Pensions and HM Revenue & Customs. However, the Programme
reported over 3,000 reductions which were achieved before the
official baseline period of 1 April 2004.[15]
Using various starting points against which to measure the improvements
of a single programme diminishes confidence in what has been achieved.[16]
9. To provide an accurate reflection of the level
of efficiency gains achieved, it is also necessary to reflect
all of the ongoing costs and benefits associated with an initiative.
Though most efficiency projects have not incurred significant
ongoing costs, the Department for Work and Pensions ignored substantial
additional costs when reporting £300 million of efficiency
gains from an initiative to pay benefits electronically. Because
not all customers have bank accounts in which to receive such
electronic payments, the Government introduced the Post Office
Card Account. The contract for administering this account cost
the Department £164 million in 2005-06, but this cost was
not accounted for in its reported efficiency gains,[17]
thus giving an overly-optimistic picture of what has actually
been achieved and of the true benefit of the initiative to the
taxpayer.[18]
10. Reported efficiencies are also more reliable
if they are based on measures of actual performance, as opposed
to estimates or projections. The Department of Health reported
£93 million of efficiency gains as a result of eliminating
a number of bureaucratic tasks required of GPs. The methodology
used fails to provide a sufficiently robust representation of
the situation either before or after the reforms. In the case
of the baseline position, the Department used a survey based on
the experiences of just 0.15% of GPs.[19]
Similarly, for some areas both the Department for Work and Pensions
and HM Revenue and Customs used projections of staff numbers to
calculate the number of posts they had reallocated to the 'front
line', rather than using actual numbers.[20]
To calculate efficiency gains more accurately, departments should
calculate the actual amount of time, costs or headcount before
and after any changes are implemented, even if this entails a
delay in reporting progress.
14 C&AG's Report, para 2.14 Back
15
Ibid, para 3.9 Back
16
Qq 13, 24 Back
17
C&AG's Report, para 2.21 Back
18
Q 18 Back
19
C&AG's Report, Appendix 3 Back
20
Ibid, para 3.16 Back
|
