Memorandum by the British Medical Association
(PST 1)
TARGET-SETTING AND PERFORMANCE MEASUREMENT
Introduction
1. Target-setting can be regarded as shorthand
for the wide range of performance measures, guidelines and service
frameworks and agreements, which underpin the delivery of public
services especially health services. The current NHS performance
ratings and high level performance indicators set within the broader
performance assessment framework are the latest in a long line
of performance measures which began in the 1980s with activity
and cost indicators and progressed through efficiency indicators
and patient charters in the 1990s.
The present system
2. At present, most of the debate centres
on the performance ratings and their role in determining access
to "earned autonomy." NHS providers have their performance
assessed against a limited number of key targets and a larger
number and range of indicators. For acute trusts the key targets
are as follows:
— no patients waiting more than 18
months for inpatient treatment;
— fewer patients waiting more than
15 months for inpatient treatment;
— no patients waiting more than 26
weeks for outpatient treatment;
— fewer patients waiting on trolleys
for more than 12 hours;
— less than 1 per cent of operations
cancelled on the day;
— no patients with suspected cancer
waiting more than two weeks to be seen in hospital;
— improvement to the working lives
of staff;
— hospital cleanliness; and
— a satisfactory financial position.
3. The broader range of indicators is intended
to provide a balanced view across three focus areas—clinical,
patient and capacity/capability. Examples of clinical focus indicators
are risk of clinical negligence, post-operative death, speed of
discharge and emergency re-admission to hospital. Examples of
patient focus indicators are inpatient, outpatient and A &
E waits, delayed discharges and patient satisfaction. Examples
of capacity/capability focus include data quality, staff satisfaction
compliance with the New Deal on junior doctors' hours and the
sickness/absence rate for directly employed NHS staff. A Trust,
which has demonstrated high standards of performance against the
key targets and in these three areas, will receive a performance
rating of three stars.
The purpose of target-setting
4. The Government's aims in constructing
its performance ratings can be summarised as follows:
— to monitor progress in what it has
identified as key policy areas;
— to set targets in these areas; and
— to incentivise and support progress
towards these targets.
5. The incentives it uses or intends to
use are the right to autonomy and more direct financial incentives
through changes to staff contracts.
6. In a discussion document published in
March 2000,[1]
the BMA criticised earlier performance indicators largely because
their audience was ambiguous. If the audience was the patient
then they were overly complex and would not satisfy the public's
desire for simple, easy to understand relative information at
a disaggregated level. If their audience was to be the individual
clinician then they were insufficiently specified for local circumstances
including case mix and relative risk. They were probably nearer
to satisfying the necessary criteria for use by commissioning
bodies and/or NHS Trusts to assess relative performance and to
identify areas for further exploration. This could, in due course,
result in dissemination of best practice and as a result lead
to improvements in the overall quality of healthcare delivery.
However, crucially, the measures were probably too insensitive
to variations in inputs. Differences in staff mix and in other
resources were not sufficiently controlled for. These reservations
continue to apply to the present measures.
7. One dilemma faced by successive governments
has been to reconcile local and clinical autonomy with central
control. The present resource allocation arrangements go further
than their predecessors, which were designed to provide equal
resources and thus equal access to healthcare for those in equal
need. They now explicitly include resources aimed at health inequalities
in addition. This argues for tailoring services to specific local
circumstances where these currently contribute to health inequalities.
However, the Government is reluctant to simply set unified budgets
(including this element) for commissioners and leave them to decide
how best to use these to pursue a broad agenda. Earmarked funding
and performance measures are ways of retaining central control.
8. This does not necessarily run counter
to patient wishes. In research underpinning the BMA's Healthcare
Funding Review, we found strong public support for the basic concept
of a healthcare system which is essentially free at the point
of use and aims to provide equal access to the same standard of
care for all. This suggests that the public would react adversely
to local provision that created different expectations in different
areas. Some measure of uniformity is thus necessary.
What sort of approach to performance measurement
could we support?
9. Performance measurement is a desirable
process provided certain basic underlying principles are met:
— there should be support and consensus
for the key policy areas concerned;
— the targets should be relevant to
these policies;
— there should be a small number of
meaningful targets;
— the targets should be within the
capacity of receivers to address;
— any behavioural change stimulated
by the targets should be desirable;
— the measures used to assess progress
should address patient and clinician concerns;
— the measures used should focus on
outcome rather than process; and
— the process itself should be one
of self comparison and benchmarking rather than ranking.
10. The present arrangements fall short
of these principles. By and large there is support for policies
aimed at ending unacceptable variations in health outcomes, but
the present targets are more about process than outcome. Where
outputs and processes are proxies for outcomes, they should be
meaningful ones. The problem is that true outcome measures need
to be risk adjusted and to control for case mix and input variation.
11. The existing measures are, arguably,
overly complex and resource intensive. To influence behaviour,
targets must be personally involving and relevant, such that the
receivers regard the recommendation as applicable to their situation
and needs. Many of the output measures, particularly those in
the public service agreements are beyond the capacity of those
in the service to deliver. For example, of those set out in the
health department public service agreement (PSA), only efficiency
savings, waiting list and prescribing targets together possibly
with the admission rates for elderly and psychiatric patients
fall remotely into this category. Some require inter-agency co-operation
(eg those dealing with delayed discharge) or are better addressed
with through non-health services (eg the prevention of certain
life-threatening conditions).
12. There is a danger in performance management
systems for the process to become more important than the outcome.
Receivers may well manage their workloads to satisfy the narrowly
defined targets, with perverse results. Maximum waiting times
for specific diagnoses may prompt over-referral for example.
13. The Government has indicated that in
general the indicators should take into account data availability
and make use of existing data where possible. There may, however,
be better indicators and the balance between quality and cost
should be borne in mind. The set of indicators should be as small
as possible rather than attempting to cover too much ground. Again,
it is a question of balance—this time between manageability
and coverage.
14. As our discussion document (see above)
pointed out, experience of league tables and their use in the
education sector suggests that consumers and professionals in
the service see them very differently. To the former, relative
position is paramount whereas to the latter performance is best
measured by added value. Good teaching can increase the performance
of individual pupils in a school with little effect on overall
ranking and thus on consumer reaction. This has led to the development
of alternative performance indicators for schools produced by
non-government agencies. These feed back to local education authorities
a wide variety of measures based on added value. These measures
share a number of characteristics. They are multilevel analyses
on a range of indicators and all control for background factors
including most importantly prior attainment. The analogy with
health services is clear. NHS staff need feedback about the effectiveness
of treatments and processes and patients need information to manage
expectations. However, patients are in different states of health
when they access the service and outcomes depend heavily on this.
15. When we produce targets and measure
movement towards them, we should bear in mind the ultimate user
of this information—the patient. Patients tell us that they
require close proximity to high quality health services. Performance
measurement should therefore be aimed at identifying and promoting
best practice using benchmarking and feedback. Choice depends
for its existence on variation. Emphasising performance ratings
and patient choice may well therefore be counter-productive.
October 2002
1 Clinical indicators (league tables): a discussion
document. BMA Board of Science and Education. March 2000. Back
| 
