3 The proposed payment mechanism
The application of payment by
results to probation services
66. One of the key tenets of the reforms is that
the contracts with providers will include a payment by results
(PbR) element, where providers will be paid according to the rehabilitation
outcomes they achieve. The Ministry set out the key design features
of its proposed payment mechanisms for consultation with potential
providers in May 2013. It proposes two elements: an upfront Fee
For Service for mandated activities, and a payment by result element
based on a series of offender "cohorts".[133]
Unlike other large-scale payment by results initiatives, the former
will form the bulk of the payment to ensure that new providers
can fulfil their statutory functions for the courts. The components
of each element are set out in the box below.
| Fee for Service
· Annual price paid in twelve equal payments made monthly in arrears
· Subject to an annual learning curve discount to drive continuous improvement
· Providers will bid against a predicted baseline volume range, weighted for sentence type and length
· At the end of each contract year, the payment would be reconciled to the actual volumes recorded, with a retrospective payment or clawback applied if actual volume is shown to have been outside the predicted range
· Deductions made for failure to deliver the orders of the court to specified time and quality
Payment by Results
· Binary and frequency measure with a binary "hurdle"
· Quarterly cohorts (to reduce the time lag) with annual top-up payment for genuine improvement against annualised targets
· Monthly 'Foundation Payment' of part of the providers profit component paid upfront for expected achievement of quarterly PbR targets
· Payment only for achieving demonstrable results, with clawback available for underperforming and higher payments for further improvements over minimum requirement
· Large financial deductions / termination for increase in re-offending rates
|
Source: Ibid
67. The Ministry has not provided a full response
to this consultation, which it states resulted in "significant
feedback", but has summarised some of the key points raised
and some of the potential refinements to the model that they are
considering.[134] The
Ministry strongly advised potential providers not to use any information
in the feedback document in any modelling of bids. A final version
of the payment mechanism will be produced during the next stage
of competition (the Invitation to Negotiate stage).
Potential implications for providers'
behaviour
68. Payment by results programmes are very sensitive
to design, in particular to ensure the optimum balance between
cost and quality. Our witnesses have highlighted a number of potential
shortcomings in the proposed model of PbR, both for prime providers
and for subcontractors. As we explored in our probation inquiry,
there is a risk that providers may be incentivised by the performance
metrics included in a payment by results mechanism to 'game' the
system and therefore not provide the same level of service to
all offenders. Accordingly, getting the choice of metrics right
is crucial to the success of these reforms in reducing reoffending,
maintaining public protection, and generating the innovations
and efficiencies required to plug gaps in the system.
BINARY VS. FREQUENCY METRIC
69. One of the key elements of the payment mechanism
that is still to be determined is the nature of the "results"
component. If the Ministry utilises a simple binary measure of
reoffending—which the Secretary of State said in February
was his preferred option as it most aligns with his overall aim
of ensuring that people stop offending—it would pay a provider
when an offender can be proven to have stopped committing crime
altogether.[135] In
contrast, a frequency metric would measure success in terms of
the reduction in the number of times an offender reoffends. The
risk of using a simple binary metric would be that providers direct
the majority of their resources at those offenders who are most
likely to stop reoffending altogether (and indeed might have done
so without any intervention). In recognition of the risks of gaming
from using solely a binary metric the Ministry initially proposed
in its "Straw Man" mechanism a hybrid measure, as set
out in the box below.
|
There will be two measures for re-offending used to calculate the PbR payment:
Binary metric = measures the percentage of offenders that are convicted of an offence within a 12 month period.[136]
Frequency metric = measures the rate of offences committed by offenders within a cohort within a 12 month period.
The MoJ proposed that PbR payments would be allocated on the basis of performance against the binary measure and the frequency measure, with a percentage of the total funding available linked to each. However, to receive any PbR payment, a provider will have to have improved performance on the binary metric to a point of statistical significance within the given CPA, regardless of performance against the frequency metric. This reflects the importance placed on achieving complete desistance from re-offending.
|
Source: Ministry of Justice, Straw Man Payment Mechanism,
May 2013
70. The need for providers to demonstrate statistically
significant improvements in reoffending in order to be eligible
for payments related to performance against the frequency measure
is referred to as the "binary hurdle". Our witnesses
were overwhelmingly against the inclusion of this hurdle.[137]
They felt its inclusionwould create a likelihood that providers
will prioritise this beyond the frequency measure, and not invest
adequately in services for the harder-to reach i.e. those with
complex and entrenched needs which are likely to be challenging,
resource intensive, and require a disproportionate amount of staff
time.[138] It was suggested
that the mechanism should include a severity metric, or differential
or escalated payments.[139]Clinks,
a body which supports voluntary and community sector organisations
working in the criminal justice sector, questioned how the model
would incentivise the provision of services that prioritise intermediate
outcomes i.e. those factors which have an impact on the road to
desistance, even if they do not immediately lead to reduced reoffending.[140]
The Ministry is now testing internally various options for adapting
its approach to the metrics for reduced reoffending to mitigate
these risks, including abandoning or revising the binary hurdle.[141]
INCENTIVISING INVESTMENT AND INNOVATION
71. Another question is whether the payment by results
element will provide a satisfactory return to incentivise providers,
and subcontractors, to deliver the innovations and additional
services that will be required to accomplish decreases in reoffending.
In addition to making efficiencies in existing probation services
to fund the extension of statutory support to short-sentenced
prisoners post-release, new providers are also expected to offer
resettlement services to all offenders coming towards the end
of their custodial sentences, and bolster existing rehabilitative
services for those on community sentences.[142]Required
levels of investment in such provision may well be substantial.
During our probation inquiry we heard that there were longstanding
gaps in provision which hinder the ability of sentencers to use
the full range of requirements that can be attached to community
orders to deal with offending behaviour, particularly for alcohol
treatment and mental health treatment.[143]
The shortage of suitable housing is also an enduring problem hindering
rehabilitation.[144]In
our report Women offenders: after the Corston Report we
raised the question of whether a payment mechanism concentrated
on reducing reoffending was the most appropriate to support the
commissioning of the services that were required for many female
offenders.[145]
72. The Secretary of State told us he wanted to see
a "smart innovative approach" from prime providers which
would be given the operational freedom and flexibility to innovate,
but in return, that they should put some of their own resource
on the line, to carry the risk of failure themselves, rather than
the taxpayer.[146]For
example, he expected that providers would wish to commission housing
provision, which was currently difficult to achieve within the
constraints of the public sector.[147]
73. A range of witnesses expressed reservations about
whether the model and mechanism were calibrated correctly to achieve
the Government's aims.[148]The
proportion of the overall payment to providers that will constitute
the PbR element is unknown, and will depend on the competition
process, although there has been speculation that it will be in
the region of between five and fifteen percent, at least initially.[149]The
public policy experts and one of the former private contractors
that gave evidence to us shared the view that there was insufficient
potential return in the PbR element of the contract and were sceptical
about whether the mechanism could deliver a decrease in re-offending.
Toby Eccles of Social Finance observed:
"The most obvious structural risk...is that
we will end up with a cost-driven production, with no focus on
rehabilitation in anything other than handing out some leaflets
and hoping for the best because the cost envelope will not allow
it [...] In terms of getting in the way of innovation, change
and progress, getting the structure for this wrong from an outcomes
point of view will mean that it becomes more difficult to work
with those people [that are harder-to-reach] than it is at the
moment."[150]
In both Ian Mulheirn's and Richard Johnson's analysis
of the situation, this risk was high in light of the way the payment
mechanism was initially constructed. Mr Mulheirn observed:
Currently, the structure of the payment mechanism
is completely perverse. It encourages cost cutting, and probably,
increases in reoffending, which is the profit-maximising thing
to do. At the moment, the incentives are completely topsy-turvy
[...] The [MoJ's] market feedback and development considerations
that it put out recently it has acknowledged some of those problems.
However, it has not yet gone nearly far enough towards addressing
them. If we do not get that right, the whole payment by results
approach is fundamentally flawed.[151]
Richard Johnson went as far as to say the model was
entirely inappropriate for the behaviour that the MoJ were seeking
to incentivise. He explained:
There are two forms essentially of PBR. This
form is the simple form of cash on delivery. Instead of paying
for a service up front, I am going to make sure that I get the
service by paying for it when it is actually delivered. But people
think what they are buying here is spend to save. In a spend-to-save
model we are tapping into the £5 billion-odd that it costs
us with our current rates of reoffending. That is a completely
different sort of contract, procurement and service. In that model,
you are looking to incentivise risk because there are big rewards,
and those rewards come from, effectively, a profit share between
the public sector purse in reduced reoffending costs and the provider
of that service. What you cannot do in that spend-to-save model
is try and introduce the notion of competition on price, because
competing on price drives you back to this cash-on-delivery model—how
cheaply you can deliver the service for me—rather than how
much you can extend the social impact of this service, because
an increased social impact delivers an increased saving to the
public purse.[152]
The other former private contractor we spoke to,
Max Chambers, explained that the notion that payments must be
attributed to changes of statistical significance could theoretically
result in a situation where providers could allow reoffending
to drift up within that margin. On the other hand he doubted that
this would be the case in practice.[153]
74. It is no
exaggeration to say that the efficacy of the payment by results
mechanism which is finally adopted will be crucial to the prospects
for success of the Government's ambitious plans for a reduction
in reoffending through a rehabilitation revolution. Serious question
marks hang over the design of the PbR mechanism itself, and the
proportion of payment to providers which will depend on the results
they achieve. It is likely that any model introduced at the beginning
of the new system will need to be modified in the light of experience.
We will return to the question of the Ministry's preferred model
and other potential models of payment by results in our final
report in this inquiry.
FINANCIAL SUSTAINABILITY
75. There is a further risk that poor design of the
model will result in contracts that are not financially sustainable.
Our witnesses anticipated high reconfiguration costs, in terms
of developing new delivery models and assembling complex supply
chains, and believed that expectations of financial efficiencies
would be likely to drive providers to focus first on maximising
their gains from the upfront element of the payment by reducing
their cost base.[154]
For example, Richard Johnson thought that providers would focus
on delivering services at the cheapest possible price.[155]Tom
Gash proposed that this would be done "by changing staff
terms and conditions or reducing their staffing within the probation
services."[156]
76. NCVO had recently commissioned an analysis of
some voluntary sector providers' PbR contracts and found that
the financial requirements, in terms of the working capital and
cash flow needed to fund payment in arrears, have meant that providers
have had to: subsidise their PbR work with other income from other
sources, including their reserves; limit the amount of other services
they can deliver; and seek loans to cover payment delays.[157]
Pitching the incentives right will similarly impact on the subcontracting
element of service provision. As spending on rehabilitation within
these contracts was likely to be limited due to the costs of restructuring,
at least within the first few years of delivery, and both Clinks
and the NCVO noted that this might impact on the durability of
existing providers in the sector.[158]
77. The use of payment by results determines the
size of contract package areas, which must be sufficiently large
to identify results with confidence.[159]
Ian Mulheirn believed that there remained risks that providers
might not be able to make a sufficient difference to the numbers
to prove to the Department statistically that they have made a
difference, and that these risks were particularly big for providers
in small areas.[160]
We consider in chapter four the potential implications of provider
failure.
TESTING THE MECHANISM
78. Witnesses expressed appreciation that the Ministry
was consulting on the design of the payment mechanism and considering
various refinements.[161]
NCVO said: "[this] is a promising indication; too often in
designing PbR models, the commissioner proceeds without this kind
of market engagement, and ends up having to restructure a flawed
contract whilst it is operational."[162]
Nevertheless, as described in chapter 1, many witnesses were uneasy
about the limited evidence that exists on the cost-effectiveness
and sustainability of adopting a payment by results approach in
this context, and the lack of testing of aspects of the mechanism
proposed.[163] Regardless
of the final mechanism, DrugScope, among others, urged the Government
to tread cautiously: "There is limited evidence on payment
by results schemes [...] Gradual implementation would enable cross-governmental
coordination of PbR learning and development [including from the
drug and alcohol recovery pilots] and adaptation of programmes
in the light of emerging evidence."[164]
DELAY TO IMPLEMENTATION OF PAYMENT
BY RESULTS
79. There has been some speculation that there will
be a delay in the implementation of the payment by results element
of the contract fee. This possibility is signalled in the Ministry's
response to the "Straw Man" consultation. The Ministry
initially proposed that Tier 1 providers would receive a so-called
Foundation Payment for meeting their PbR targets, which it had
been planning to pay upfront and then claw back if necessary,
to compensate for the "substantial" time lag required
to measure outcomes. Clinks supported this concept as they felt
it was likely to minimise cashflow problems, especially to VCSE
subcontractors for whom delayed payments are particularly problematic.[165]However,
consultation responses suggested that the prospect of clawback
might have the opposite effect to that intended, by making providers
more risk-averse. The Ministry is now instead considering an initial
period at the start of the contracts before the PbR element would
apply or allowing providers to "sculpt" the fee for
service during the bidding process, which presumably means allowing
them to negotiate receiving larger amounts upfront.[166]
80. Much of the detail of the final payment mechanism
will be determined during the next stage of the bidding process.
When we put the possibility of delaying the implementation of
the payment by results element to the Minister he appeared to
have discounted this prospect. He told us that this would kick
in once the cohort of newly sentenced prisoners was large enough
to warrant it, which he estimated would take approximately six
months.[167] As well
as revising the payment mechanism the Minister made it clear that
the Ministrywill expect the prospective providers to:demonstrate
their ability to deliver a quality service; set out a minimum
level of intervention that they will provide for every offender;
and state the level of investment they are going to make in rehabilitation.
He assured us that they would"not accept a situation where
they decide to work with only some offenders and not the rest."[168]
81. The introduction
of payment by results marks a major shift in the commissioning
of rehabilitative services. Few of our witnessesargued with the
premise of providers being rewarded according to their performance,
but the approach remains novel, and the limited experience of
its application, not only in the criminal justice sector but more
widely, suggests that it can be beset with many challenges which
the Government will need to overcome if it is to be a success.
82. We note
that the Ministry appears receptive to comments on the design
of the payment mechanism. In particular Ministers appear to recognise
the hazards of providers "parking" the hardest to engage
offenders and are considering the most appropriate ways of addressing
this. The ultimate design of the mechanism will be vital to the
success of the Government's plans, and an ostensibly small change
in the payments system could lead to a major change in provider
behaviour and hence the outcomes of the programme as whole. In
this context, we understand the motivation of Ministers in wishing
to seek complete desistance from reoffending as an outcome butthe
system has to be one which incentivises providers to work effectively
with all the offenders for whom they are responsible. We therefore
agree with many of our expert witnesses that the binary hurdle
should not be retained in the final payment by results mechanism.
83. We also
note with approval that the Ministry has subjected the proposed
payment by results metrics to internal testing. It appears to
us that officials have appreciated the potential perverse incentives
that must be avoided. At the same time, while a "straw man"
is of course designed to be knocked down, the degree of criticism
encountered by the "straw man" mechanism implies that
the extent of restructuring of the mechanism which may be required
is extensive, especially taking into account the speed with which
the changes are being wrought.
84. It appears
to us that the risk of not achieving sufficient savings relates
more to the level of savings that providers are able to achieve
to reinvest in extending the reach of existing provision, and
the quality of services that might prevail as a result, thanthe
overall costs of the reforms per se. This, and the proposal to
revise the payment mechanism to enable providers to receive more
of the fee for service upfront in return for taking more risks
later on, also suggests that the length of the contracts is the
basis on which the Ministry and Treasury have concluded that the
numbers will add up.
85. We consider it important for the overall success
of the reshaping of the rehabilitation landscape that the final
payment by resultsmechanism, as determined during the contracting
process, should be capable of further refinement and modification
in the light of experience. The mechanism, and the metrics which
it involves, must in addition remain open to parliamentary and
public scrutiny, which must not be deflected by the fact that
it is private sector providers who are delivering this essentially
public service. The Ministry should explain in its response to
this report how it will ensure reliable public accountability
of the performance of providers of rehabilitative services under
the new model.
133 Ministry of Justice, Transforming Rehabilitation Programme - Payment Mechanism: Market Feedback and Development Considerations,
October 2013 Back
134
Ministry of Justice, Transforming Rehabilitation Programme - Payment Mechanism: Market Feedback and Development Considerations,
October 2013 Back
135
Oral evidence taken on 27 February 2013, HC (2012-13) 964, Q30 Back
136 A
proven re-offence will be counted as any offence committed within
a one year follow-up period, following an offender's entry into
the cohort, which then attracts a court conviction or caution
within that one year follow-up period or within a further six
month waiting period to allow for cases to work their way through
the courts. Back
137
See for example Criminal Justice Alliance (PPC 06); DrugScope
(PPC 12); Prison Reform Trust (PPC 13); Magistrates' Association
(PPC 17); NCVO (PPC 26); Clinks (PPC 27) Back
138
Criminal Justice Alliance (PPC 06); DrugScope (PPC 12); Prison
Reform Trust (PPC 13); Magistrates' Association (PPC 17); NCVO
(PPC 26); Clinks (PPC 27); Q150 [Mr Chambers]. See also National
Policing Lead for Integrated Offender Management (PPC 08). Back
139
Criminal Justice Alliance (PPC 06); DrugScope (PPC 12); Prison
Reform Trust (PPC 13); citing CBI and the Social Market Foundation Back
140
Clinks (PPC 27). They noted that many VCSE organisations have
developed services that take account of academic desistance theory,
in which primary desistance, meaning any lull or gap in offending,
however short-lived, is distinguished from secondary desistance,
which refers to a more deep-seated change in an individual where
they develop an identity as a 'non-offender'. Back
141
Ministry of Justice, Transforming Rehabilitation Programme - Payment Mechanism: Market Feedback and Development Considerations,
October 2013 Back
142
Oral evidence taken on 27 February 2013, HC (2012-13) 964. See
Qq 14-16. Back
143 Justice
Committee, Eighth Report of Session 2010-2012, The role of the Probation Service,
HC519-I, paras 128-131 Back
144
Magistrates' Association (PPC 17) Back
145
Justice Committee, Second Report of Session 2013-14, Women offenders: after the Corston Report.
HC 92, para 149. Back
146
Q1; Q242 Back
147
Q194. Longitudinal research has found that prisoners who needed
help with finding somewhere to live after release were more likely
to be reconvicted (65%) in the first year, than those who did
not need this help (45%) (Williams et al., Accommodation, homelessness and reoffending of prisoners: Results from the Surveying Prisoner Crime Reduction (SPCR) survey,
Ministry of Justice, 2012). Back
148 Qq20-22
[Mr Oliver; Mr Johnson]; Qq45-46 [Mr Johnson]; Qq113-119 [Mr Eccles;
Mr Mulheirn]. Criminal Justice Alliance (PPC 06); Local Government
Association(PPC 11) [; DrugScope (PPC 12); Prison Reform Trust
(PPC 13); London Councils (PPC 21); NCVO (PPC 26); Clinks (PPC 27);
Napo (PPC 31) Back
149
See for example Q114 [Mr Eccles]; Napo (PPC 31). Mr Grayling told
us that he hoped to see it higher than 10% and expected organisations
to be willing to ratchet up the proportion of the payment that
would be put at risk over time as knowledge increases Q250-251. Back
150
Q133 Back
151 Ibid.
[Mr Mulheirn] Back
152
Q45 [Mr Johnson] Back
153
Q150 [Mr Chambers] Back
154
See for example Q18 [Mr Johnson]; Q146 [Mr Gash]; Clinks (PPC 27);
NCVO (PPC 26) Back
155
Q17 Back
156
Q146 Back
157
NCVO (PPC 26) Back
158 Clinks
(PPC 27); NCVO (PPC 26). See also Q18 [Mr Johnson], Q170 [Mr Henman].
Back
159
Q122 [Mr Mulheirn] Back
160 Ibid. Back
161
London Councils (PPC 21); NCVO (PPC 26);Clinks (PPC 27); Q133
[Mr Mulheirn]; Qq149-150 [Mr Chambers] Back
162 BWB
and NCVO, Payment by results contracts: a legal analysis of terms and process,
October 2013,p 22 Back
163
Criminal Justice Alliance (PPC 06); DrugScope (PPC 12); Mr Allen
(PPC 21); Mr Underhill (PPC 23) Back
164
DrugScope (PPC 12); Back
165 NCVO
(PPC 26) Back
166
Ministry of Justice, Transforming Rehabilitation Programme - Payment Mechanism: Market Feedback and Development Considerations,
October 2013 Back
167
Qq252-259 Back
168
Q242 Back
| 
