House of Commons - Patient Safety

Patient Safety - Health Committee Contents

3 Measurement and evaluation

34. The DH's policies for eliminating avoidable harm to patients have to be judged by their effectiveness. The measure of this is the overall amount of harm done to patients, and its cost to the NHS, which diverts resources that could be spent on other care. In this chapter we examine the available evidence.

How much harm is done to patients?Box 4: Types of evidence about the scale of harm done to patients, and their respective strengths and weaknesses

Type of Evidence Strengths Weaknesses

Reported incidents Provides a minimum number Under-reporting

Case note reviews Provides an adverse event rate Only provides a snapshot of a particular time and place; methodologies differ (limiting value of comparisons) and may not be applied consistently by reviewers

Patient surveys Captures the voice of the patient Under-reporting (patient may not be aware an incident occurred, or may not appreciate what counts as an incident)

Reported data on specific problems (e.g. healthcare-associated infections) Provides a minimum number Under-reporting; only captures one facet of safety

35. When we asked Mr Fletcher how much harm the NHS in England does to patients, he referred us to data on reported incidents.[26] Of the roughly 850,000 incidents in the NHS in England reported annually through the NRLS, over 65% are "No harm" incidents (including "near misses", i.e. prevented incidents), about 25% are "Low harm" incidents, around 5% involve Moderate harm, less than 1% (about 7,500) involve serious harm and less than 0.5% (around 3,500) involve the death of the patient.[27]

36. However, as the NPSA itself acknowledges, there is very significant under-reporting of safety incidents (as is discussed further below). So figures for reported incidents are not a reliable indicator of how many incidents actually take place in the NHS.

37. When we asked the Chief Nursing Officer (CNO), Dame Christine Beasley, the same question, she could only refer us to patient survey data collected by the Healthcare Commission (HCC), showing that "92% of people say their care is very good, good or excellent".[28] This answer, however, bypasses the fact that patients may well be unaware of failings in their treatment. On the same occasion, the NHS Medical Director, Professor Sir Bruce Keogh, told us only "I have nothing to add".[29]

38. We sought to clarify what is known about the harm done to patients. Richard Thomson, Professor of Epidemiology and Public Health at the University of Newcastle, told us that the best available data on the overall extent of harm to NHS patients in England consist of "hospital case note review studies,[30] the studies that have looked at medical records of patients", which "demonstrate that around 10% of admissions to hospital suffer some form of harm".[31] These data show that, in between a third and half of admissions associated with harm, that harm was preventable. The DH's memorandum of written evidence for this inquiry states that these findings are consistent with those of similar studies in other developed countries.[32]

39. There have only been two, very limited, such studies in England (relating to admissions in 1998 and 2004, respectively), each covering around 1,000 admissions in one and two hospitals respectively.[33] Other countries in which studies have been conducted include the USA, Australia, Denmark, France, Canada, the Netherlands, Spain and New Zealand; some of these involved up to 50 hospitals and 30,000 admissions (see Annex 2 for a more detailed analysis).

40. All such studies are associated with important methodological problems. Firstly, it can be difficult to decide whether or not an adverse event was avoidable. Making such a judgement carries the danger of "hindsight bias": not making sufficient allowance for the fact that those involved in the event will not have had the advantage of perceiving the situation with full (retrospective) knowledge or the luxury of time to reflect. (However, it is arguably not "hindsight bias" to regard harm as avoidable when it occurs due to lack of foresight, i.e. ignorance of factors that are known to contribute to unsafe care.)

41. Secondly, the criteria used to determine avoidability can vary between studies and consistency can be hard to achieve even among researchers in the same study. One study concluded that "preventability is in the eye of the reviewer".[34] A study of a hospital, oriented towards discovering cases of potential medical negligence, in Massachusetts found an adverse rate of 2.7%; when the same notes were examined using a different approach to assess all forms of harm to patients the rate was put at 11%.

42. Thirdly, as well as the problems in determining preventability, there are particular issues relating to deaths associated with unsafe care. Often, these are the deaths of patients who were very ill with an already poor prognosis regarding their original illness or condition. In such circumstances, determining whether a death counts as an "excess death" (one attributable wholly to the adverse event) is difficult, and sometimes, impossible.[35]

43. At various times over the past decade, the NPSA, the DH and others have cited estimates of the extent of harm to NHS patients in England, extrapolated either from the case note review studies in England or from similar studies conducted overseas. These extrapolations have suggested that the NHS inadvertently kills tens of thousands of patients each year. Figures generated in this way are questionable given the problems discussed above.

44. Despite the limitations of case note review data, and the Department's failure to ensure the production of better data, it is clear that a great deal of harm is done to patients in the NHS—as it is in all healthcare systems. The international studies mentioned above put the rate of adverse events in acute care at between 2.7% and 16.6%. Even at the lower end of the range this represents a huge amount of harm given the millions of patients treated.

45. Case note reviews are also an important source of data at the local level. During our visit to Luton and Dunstable NHS Foundation Trust we learned how the Trust had made effective use of case note review data to monitor its progress in improving patient safety. Each month, the Trust scrutinises a random sample of 20 case notes to compute an adverse event rate, which is then reported to the Board and to staff as an indicator of how far care is safe.

46. There are various streams of data on particular aspects of patient safety in acute hospitals, for example, on medication errors and healthcare-associated infections (HCAIs). Cases of MRSA and Clostridium difficile are now reported and the Office for National Statistics produces statistics regarding deaths in which these two diseases are recorded causes. These data do not provide a reliable, comprehensive picture of the problem since there is likely to be underreporting of cases and under-registration of deaths; they do, nevertheless, show that HCAIs are a major concern.

47. Regarding harm to NHS patients that occurs outside NHS acute hospitals, Professor Thomson told us that the available evidence is even thinner. There have been no case note review studies in primary care. A review of the international literature on safety incidents in primary care found that studies showed "there might be errors occurring in anywhere between five and 80 in every 100,000 consultations". But this cannot be taken as a reliable indication of harm to patients in primary care in the NHS in England. According to Professor Thomson:

However, this information too is inadequate for determining exactly how many patients are harmed and in what ways.

48. In respect of harm to NHS patients from treatment by independent-sector providers, Professor Thomson told us that nothing is known: "That is an area really where there is very little reliable data at all at the moment. I do not think we have anything that could give us an overarching figure for that at all."[37] The HCC told us, regarding data on quality that Independent Sector Treatment Centres were expected to provide, that "the level of returns and data quality were very poor" in 2007 although "there has been some improvement since". Also, for reports on adverse events, "data quality can be poor and the level of reporting from different providers of the same type varies greatly".[38]

What does harm to patients cost the NHS?

49. When we asked about the cost to the NHS in England of harm to patients, the NHS Medical Director referred us to the sum of £633 million[39] paid out by the NHS Litigation Authority (NHSLA) in settlement of clinical negligence claims in 2007-8.[40] It is clear that this figure does not indicate the true cost, since it relates only to litigation over the NHSLA threshold and omits so many of the costs relating to avoidable harm—not least the cost of additional medical treatment.

50. Other figures which attempt to give truer estimates of the cost to the NHS are also incomplete and unreliable but, even so, leave little doubt that the cost is huge. Mr Fletcher of the NPSA referred to work done by the NAO in which Trusts were asked to estimate the cost of unsafe care; this had yielded figures for individual Trusts ranging between £88,000 and £400,000 per year. He also cited work by the NPSA on patient falls, which were estimated to cost the NHS in England and Wales around £15 million per year.[41]

51. Alastair Gray, Professor of Health Economics and Director of the Health Economics Research Centre in the University of Oxford, told us that, according to an extrapolation from the first hospital case note review study in England, additional bed-days due to preventable adverse events could be costing "approximately £1bn" per year.[42] Such data are subject to the limitations relating to case note reviews referred to above, but are at least based on a systematic assessment of medical notes.

52. The CNO informed us that the annual cost of additional bed-days due to healthcare-associated infections was in the region of £1 billion,[43] although this is an extrapolation from a single-hospital study that used data from 1994-5.[44]

53. Dr Alison Holmes, the Director of Infection Prevention and Control, and a Consultant in Infectious Diseases, at Imperial College Healthcare NHS Trust, spelt out to us the need for better data on the cost of healthcare-associated infections:

54. Another area of cost is that relating to adverse events due to medication errors, which the NPSA has estimated at £774 million per year.[46] Professor Gray explained that this too is an extrapolated figure the validity of which is far from certain:

Conclusion

55. The evidence, particularly that from case note reviews, both in England and internationally, indicates that the extent of medical harm is substantial, even on a conservative estimate, and that much is avoidable. International studies suggest that about 10% of all patients who are admitted to hospital suffer some form of harm. Judging how far patient safety policy has been successful requires more reliable data regarding how much harm is done to patients. Unfortunately, neither the NPSA nor the DH was able to provide us with that. Government estimates of avoidable harm and the attendant financial costs are extrapolations from old, very limited, data; and no attempt has been made to produce reliable up-to-date figures.

56. We remind the Department of the value of the random case note review that was a part of Royal College inspections for accreditation for training of junior doctors. We commend to all hospitals the practice of conducting regular sample case note reviews, as is done at Luton and Dunstable Hospital, to provide a clear indicator of local performance in making services safer. We recommend that the NPSA monitor progress by the NHS in improving patient safety, using local sample case note peer review data and other sources of information on harm to patients.

26 Q 3 Back

27 National Reporting and Learning System quarterly data, Issue 12, May 2009 Back

28 Q 4 Back

29 Q 5 Back

30 Hospital case note review studies are also referred to as "record review" studies. They are usually retrospective, i.e. based on a study of records relating to completed treatment episodes. Back

31 Q 102 Back

32 Ev 2 Back

33 Charles Vincent et al., "Adverse events in British hospitals: preliminary retrospective record review", British Medical Journal, vol 322 (2001), pp 517-519; Ali Baba-Akbari Sari et al., "Extent, nature and consequences of adverse events: results of a retrospective casenote review in a large NHS hospital", Quality and Safety in Health Care, vol 16 (2007), pp 434-439 Back

34 Rodney Hayward and Timothy Hofer, "Estimating hospital deaths due to medical errors: preventability is in the eye of the reviewer", Journal of the American Medical Association, vol 286 (2001), pp 415-420 Back

35 PS 80 Back

36 Q 102 Back

37 Q 103 Back