Evidence submitted by the National Centre
for Health Outcomes Development (ISTC 53)
1. This is a brief comment to introduce
our oral evidence.
2. The National Centre for Health Outcomes
Development (NCHOD) is an independent research centre based jointly
at the London School of Hygiene and Tropical Medicine, University
of London and the Department of Public Health, University of Oxford.
It is involved in the design and development of measures of health
outcome, production of comparative clinical and health indicators
for the National Health Service using available routine data,
and electronic publication of extensive statistical and bibliographic
information about health outcomes. NCHOD provides an ISTC Performance
Management Analysis Service to the Department of Health under
contract.
3. A preliminary report presenting an overview
of retrospective performance, with regards to 26 contractual Key
Performance Indicators (KPIs), of four ISTC schemes covering five
providers, was submitted in October 2005 and published by the
Department of Health.[9]
This report has been referred to a number of times in the evidence
presented to the Health Select Committee on 9 March and is attached
here for completeness. Analysis for a second report is currently
under way.
4. The first report made constructive comments
about some shortcomings in the process of data collection, collation
and reporting, acknowledging that these were first steps in an
evolving service. The following are some of the points we made
and progress to date in addressing them:
(a) The specifications of some of the KPIs
made interpretation of performance difficult. The specifications
have since been reviewed by the Department of Health and tightened,
in the light of our comments;
(b) There was substantial variation in the
interpretation of the definitions of the KPIs and their component
parts between ISTC schemes, between component parts of the schemes
and over time within schemes/component parts. The template and
instructions for collection of data on the KPIs have since been
revised by the Department of Health, in order to ensure consistency
of interpretation;
(c) There was variation between schemes in
the completeness and quality of data submitted, rendering attempts
at commenting on trends and comparisons between schemes and with
any external benchmarks difficult. The Department of health has
made efforts since to ensure higher levels of completeness and
quality of data. We have just received the data for the next round
of reports;
(d) Missing from what is potentially a robust
and ambitious performance monitoring system were data on clinical
outcomes, for example the extent to which hip replacement operations
actually lead to the expected improvements in mobility and reduction
in pain, the extent to which a cataract operation improves vision
etc. Data collection on such indicators is likely to commence
in April and should lead to a better assessment of the quality
of clinical care;
(e) In making our assessments and judgements
we are entirely dependent on data supplied to us by the ISTCs
via the Department of Health. We have no way of judging the accuracy
of the data submitted and have to take them at face value. We
recommended independent validation, for example an audit of a
sample by comparison with case notes. The Department of Health
is exploring ways of doing this.
5. One of the terms of reference of this
inquiry is whether ISTCs are providing care of the same or higher
standard as that provided by the NHS. In order to assess this,
both the NHS and the ISTCs would need to collect the same data,
in the same way, for the same kinds of patients. In the absence
of such data, it is important to resist drawing conclusions from
anecdotes. The example below, while strictly not applicable, is
illustrative of what is needed and the complexities involved in
addressing this term of reference.
6. This study, based on existing routinely
collected data, shows the proportion of patients readmitted to
hospital as an emergency within 28 days of previous discharge,
and compares patients treated in NHS hospitals and NHS patients
treated by the independent sector (see Table). These data cover
all types of patients aged 16 years and over (except those with
cancer), not just those types of patients treated by ISTCs. Within
the independent sector category, the figures cover all NHS patients
treated by the independent sector, not just those treated by the
ISTCs. For comparability, the percentages have been adjusted to
take into account differences between Trusts in terms of patient
age, gender, method of original admission, diagnoses and operations.
However, the results may still reflect other differences in the
types of patients treated by the two sectors. The percentages,
as a reflection of what typically happens in such settings, are
subject to random variation. The 95% confidence interval provides
a measure of the uncertainty created by such variation and gives
the range within which the true percent is most likely to be (with
95% probability) given the number of patients involved. The relatively
smaller number of patients treated by the independent sector results
in wider margins. Any comparison of percentages and the identification
of differences should be done with consideration of the confidence
intervals. These unpublished data show that there is no statistically
significant difference in the proportion of patients readmitted
between patients treated in NHS hospitals and NHS patients treated
by the independent sector. The study was undertaken as part of
work we have done for the Healthcare Commission, in the context
of NHS Performance Ratings and is quoted here for illustrative
purposes.
EMERGENCY READMISSIONS TO HOSPITAL WITHIN
28 DAYS OF PREVIOUS DISCHARGE FROM HOSPITAL, ENGLAND, 2004
|
| % readmissions
| % lower limit of 95% confidence interval
| % upper limit of 95% confidence interval
|
|
All NHS Trusts in England | 9.25
| 9.22 | 9.28
|
Small Acute Trusts | 8.77
| 8.68 | 8.85
|
Medium Acute Trusts | 9.18
| 9.12 | 9.24
|
Large Acute Trusts | 9.15
| 9.10 | 9.20
|
Independent Sector | 9.39
| 7.87 | 11.12
|
|
Source of data: Hospital Episodes Statistics, Department
of Health.
National Centre for Health Outcomes Development
14 March 2006
9
National Centre for Health Outcomes Development. Report to the
Department of Health: ISTC Performance Management Analysis Service-Preliminary
Overview Report for Schemes GSUP1C, OC123, LP4 and LP5. October
2005. Back
|