Select Committee on Health Written Evidence


Evidence submitted by the National Centre for Health Outcomes Development (ISTC 53)

  1.  This is a brief comment to introduce our oral evidence.

  2.  The National Centre for Health Outcomes Development (NCHOD) is an independent research centre based jointly at the London School of Hygiene and Tropical Medicine, University of London and the Department of Public Health, University of Oxford. It is involved in the design and development of measures of health outcome, production of comparative clinical and health indicators for the National Health Service using available routine data, and electronic publication of extensive statistical and bibliographic information about health outcomes. NCHOD provides an ISTC Performance Management Analysis Service to the Department of Health under contract.

  3.  A preliminary report presenting an overview of retrospective performance, with regards to 26 contractual Key Performance Indicators (KPIs), of four ISTC schemes covering five providers, was submitted in October 2005 and published by the Department of Health.[9] This report has been referred to a number of times in the evidence presented to the Health Select Committee on 9 March and is attached here for completeness. Analysis for a second report is currently under way.

  4. The first report made constructive comments about some shortcomings in the process of data collection, collation and reporting, acknowledging that these were first steps in an evolving service. The following are some of the points we made and progress to date in addressing them:

    (a)  The specifications of some of the KPIs made interpretation of performance difficult. The specifications have since been reviewed by the Department of Health and tightened, in the light of our comments;

    (b)  There was substantial variation in the interpretation of the definitions of the KPIs and their component parts between ISTC schemes, between component parts of the schemes and over time within schemes/component parts. The template and instructions for collection of data on the KPIs have since been revised by the Department of Health, in order to ensure consistency of interpretation;

    (c)  There was variation between schemes in the completeness and quality of data submitted, rendering attempts at commenting on trends and comparisons between schemes and with any external benchmarks difficult. The Department of health has made efforts since to ensure higher levels of completeness and quality of data. We have just received the data for the next round of reports;

    (d)  Missing from what is potentially a robust and ambitious performance monitoring system were data on clinical outcomes, for example the extent to which hip replacement operations actually lead to the expected improvements in mobility and reduction in pain, the extent to which a cataract operation improves vision etc. Data collection on such indicators is likely to commence in April and should lead to a better assessment of the quality of clinical care;

    (e)  In making our assessments and judgements we are entirely dependent on data supplied to us by the ISTCs via the Department of Health. We have no way of judging the accuracy of the data submitted and have to take them at face value. We recommended independent validation, for example an audit of a sample by comparison with case notes. The Department of Health is exploring ways of doing this.

  5.  One of the terms of reference of this inquiry is whether ISTCs are providing care of the same or higher standard as that provided by the NHS. In order to assess this, both the NHS and the ISTCs would need to collect the same data, in the same way, for the same kinds of patients. In the absence of such data, it is important to resist drawing conclusions from anecdotes. The example below, while strictly not applicable, is illustrative of what is needed and the complexities involved in addressing this term of reference.

  6.  This study, based on existing routinely collected data, shows the proportion of patients readmitted to hospital as an emergency within 28 days of previous discharge, and compares patients treated in NHS hospitals and NHS patients treated by the independent sector (see Table). These data cover all types of patients aged 16 years and over (except those with cancer), not just those types of patients treated by ISTCs. Within the independent sector category, the figures cover all NHS patients treated by the independent sector, not just those treated by the ISTCs. For comparability, the percentages have been adjusted to take into account differences between Trusts in terms of patient age, gender, method of original admission, diagnoses and operations. However, the results may still reflect other differences in the types of patients treated by the two sectors. The percentages, as a reflection of what typically happens in such settings, are subject to random variation. The 95% confidence interval provides a measure of the uncertainty created by such variation and gives the range within which the true percent is most likely to be (with 95% probability) given the number of patients involved. The relatively smaller number of patients treated by the independent sector results in wider margins. Any comparison of percentages and the identification of differences should be done with consideration of the confidence intervals. These unpublished data show that there is no statistically significant difference in the proportion of patients readmitted between patients treated in NHS hospitals and NHS patients treated by the independent sector. The study was undertaken as part of work we have done for the Healthcare Commission, in the context of NHS Performance Ratings and is quoted here for illustrative purposes.

EMERGENCY READMISSIONS TO HOSPITAL WITHIN 28 DAYS OF PREVIOUS DISCHARGE FROM HOSPITAL, ENGLAND, 2004


% readmissions
% lower limit of 95% confidence interval
% upper limit of 95% confidence interval

All NHS Trusts in England
9.25
9.22
9.28
Small Acute Trusts
8.77
8.68
8.85
Medium Acute Trusts
9.18
9.12
9.24
Large Acute Trusts
9.15
9.10
9.20
Independent Sector
9.39
7.87
11.12

  Source of data: Hospital Episodes Statistics, Department of Health.

National Centre for Health Outcomes Development

14 March 2006




9   National Centre for Health Outcomes Development. Report to the Department of Health: ISTC Performance Management Analysis Service-Preliminary Overview Report for Schemes GSUP1C, OC123, LP4 and LP5. October 2005. Back


 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2006
Prepared 25 July 2006