Memorandum by the British Medical Association (PST
Target-setting and performance measurement
1. Target-setting can be regarded as shorthand for the wide
range of performance measures, guidelines and service frameworks
and agreements, which underpin the delivery of public services
especially health services. The current NHS performance ratings
and high level performance indicators set within the broader performance
assessment framework are the latest in a long line of performance
measures which began in the 1980s with activity and cost indicators
and progressed through efficiency indicators and patient charters
in the 1990s.
The present system
2. At present, most of the debate centres on the performance
ratings and their role in determining access to 'earned autonomy.'
NHS providers have their performance assessed against a limited
number of key targets and a larger number and range of indicators.
For acute trusts the key targets are as follows:
- no patients waiting more than 18 months for inpatient treatment
- fewer patients waiting more than 15 months for inpatient treatment
- no patients waiting more than 26 weeks for outpatient treatment
- fewer patients waiting on trolleys for more than 12 hours
- less than 1% of operations cancelled on the day
- no patients with suspected cancer waiting more than two weeks
to be seen in hospital
- improvement to the working lives of staff
- hospital cleanliness
- a satisfactory financial position
3. The broader range of indicators is intended to provide
a balanced view across three focus areas - clinical, patient and
capacity/capability. Examples of clinical focus indicators are
risk of clinical negligence, post-operative death, speed of discharge
and emergency re-admission to hospital. Examples of patient focus
indicators are inpatient, outpatient and A & E waits, delayed
discharges and patient satisfaction. Examples of capacity/capability
focus include data quality, staff satisfaction compliance with
the New Deal on junior doctors' hours and the sickness/absence
rate for directly employed NHS staff. A Trust, which has demonstrated
high standards of performance against the key targets and in these
three areas, will receive a performance rating of three stars.
The purpose of target-setting
4. The government's aims in constructing its performance ratings
can be summarised as follows:
- to monitor progress in what it has identified as key policy
- to set targets in these areas and
- to incentivise and support progress towards these targets.
5. The incentives it uses or intends to use are the right
to autonomy and more direct financial incentives through changes
to staff contracts.
6. In a discussion document published in March 2000,
the BMA criticised earlier performance indicators largely because
their audience was ambiguous. If the audience was the patient
then they were overly complex and would not satisfy the public's
desire for simple easy to understand relative information at a
disaggregated level. If their audience was to be the individual
clinician then they were insufficiently specified for local circumstances
including case mix and relative risk. They were probably nearer
to satisfying the necessary criteria for use by commissioning
bodies and/or NHS Trusts to assess relative performance and to
identify areas for further exploration. This could, in due course,
result in dissemination of best practice and as a result lead
to improvements in the overall quality of health care delivery.
However, crucially, the measures were probably too insensitive
to variations in inputs. Differences in staff mix and in other
resources were not sufficiently controlled for. These reservations
continue to apply to the present measures.
7. One dilemma faced by successive governments has been to
reconcile local and clinical autonomy with central control. The
present resource allocation arrangements go further than their
predecessors, which were designed to provide equal resources and
thus equal access to health care for those in equal need. They
now explicitly include resources aimed at health inequalities
in addition. This argues for tailoring services to specific local
circumstances where these currently contribute to health inequalities.
However, the government is reluctant to simply set unified budgets
(including this element) for commissioners and leave them to decide
how best to use these to pursue a broad agenda. Earmarked funding
and performance measures are ways of retaining central control.
8. This does not necessarily run counter to patient wishes.
In research underpinning the BMA's Healthcare Funding Review,
we found strong public support for the basic concept of a healthcare
system which is essentially free at the point of use and aims
to provide equal access to the same standard of care for all.
This suggests that the public would react adversely to local provision
that created different expectations in different areas. Some measure
of uniformity is thus necessary.
What sort of approach to performance measurement could we support?
9. Performance measurement is a desirable process provided
certain basic underlying principles are met:
- There should be support and consensus for the key policy areas
- The targets should be relevant to these policies
- There should be a small number of meaningful targets
- The targets should be within the capacity of receivers to
- Any behavioural change stimulated by the targets should be
- The measures used to assess progress should address patient
and clinician concerns
- The measures used should focus on outcome rather than process
- The process itself should be one of self comparison and benchmarking
rather than ranking
10. The present arrangements fall short of these principles.
By and large there is support for policies aimed at ending unacceptable
variations in health outcomes, but the present targets are more
about process than outcome. Where outputs and processes are proxies
for outcomes, they should be meaningful ones. The problem is that
true outcome measures need to be risk adjusted and to control
for case mix and input variation.
11. The existing measures are, arguably, overly complex and
resource intensive. To influence behaviour, targets must be personally
involving and relevant, such that the receivers regard the recommendation
as applicable to their situation and needs. Many of the output
measures, particularly those in the public service agreements
are beyond the capacity of those in the service to deliver. For
example, of those set out in the health department public service
agreement (PSA), only efficiency savings, waiting list and prescribing
targets together possibly with the admission rates for elderly
and psychiatric patients fall remotely into this category. Some
require inter-agency co-operation (e.g. those dealing with delayed
discharge) or are better addressed with through non-health services
(e.g. the prevention of certain life-threatening conditions).
12. There is a danger in performance management systems for
the process to become more important than the outcome. Receivers
may well manage their workloads to satisfy the narrowly defined
targets, with perverse results. Maximum waiting times for specific
diagnoses may prompt over-referral for example.
13. The government has indicated that in general the indicators
should take into account data availability and make use of existing
data where possible. There may, however, be better indicators
and the balance between quality and cost should be borne in mind.
The set of indicators should be as small as possible rather than
attempting to cover too much ground. Again, it is a question of
balance - this time between manageability and coverage.
14. As our discussion document (see above) pointed out, experience
of league tables and their use in the education sector suggests
that consumers and professionals in the service see them very
differently. To the former, relative position is paramount whereas
to the latter performance is best measured by added value. Good
teaching can increase the performance of individual pupils in
a school with little effect on overall ranking and thus on consumer
reaction. This has led to the development of alternative performance
indicators for schools produced by non-government agencies. These
feed back to local education authorities a wide variety of measures
based on added value. These measures share a number of characteristics.
They are multilevel analyses on a range of indicators and all
control for background factors including most importantly prior
attainment. The analogy with health services is clear. NHS staff
need feedback about the effectiveness of treatments and processes
and patients need information to manage expectations. However,
patients are in different states of health when they access the
service and outcomes depend heavily on this.
15. When we produce targets and measure movement towards them,
we should bear in mind the ultimate user of this information -
the patient. Patients tell us that they require close proximity
to high quality health services. Performance measurement should
therefore be aimed at identifying and promoting best practice
using benchmarking and feedback. Choice depends for its existence
on variation Emphasising performance ratings and patient choice
may well therefore be counter-productive.
1 Clinical indicators (league tables): a discussion
document. BMA Board of Science and Education. March 2000