Memorandum submitted by Sir Liam Donaldson
(SAGE 44)
You asked me to write to elaborate my comments,
made during evidence to the Committee, that modelling data were
of less value in guiding the response than they had been in the
advance planning phase.
During the pandemic, a team of clinical advisors
under my direct supervision, carefully investigated and documented
every death attributable to swine `flu. This was vital work. It
allowed me to present accurate information to the public and media.
It helped show that we had a good grasp on the evolving pandemic
in the UK. The work has subsequently been published in the Lancet
and the British Medical Journal.
Combining our emerging data with case estimates,
produced by the Health Protection Agency, allowed my team to estimate
a case fatality rate (the percentage of people who develop swine
`flu and subsequently go on to die). From an early stage in the
UK pandemic it was apparent that the case fatality rate was much
lower than we had feared.
The first planning assumptions, released on
16 July 2009 from scientific modelling work, suggested a much
higher case fatality rate than our own emerging data. At that
time our data indicated a case fatality rate of around 0.05%,
compared with the published figure of 0.1-0.35%. Extrapolating
our figures to the UK population, using SPI's assumptions around
attack rates, would have led to a "best" estimate of
9,000 deaths, and a "worst case" scenario of 25,000
deaths (see attached table). Such estimates made intuitive sense.
The virus was being described as "mild", and the past
two pandemics, prior to recent advances in intensive care medicine,
had seen around 30,000 to 35,000 deaths in the UK.
It was important to make the right judgement
on a sensible upper case fatality rate. It was also important
not to give too much weight to the Mexico experience, with uncertainties
about their data and a different healthcare system.
Modelling scientists will always quite properly
say that their estimates depend on the nature and quality of the
input data. Thus, they become more refined as time goes on and
more real data accumulate. The problem is that the public and
media perception is different. They take estimates at face value
so that even with the caveats expressed, the failure of early
estimates to match with later actual figures leads to criticism
and sometimes ridicule of those communicating the data. In the
pandemic I found the deaths data I gathered, used to provide insights
into the behaviour of the pandemic, on a rule of thumb basis,
was giving me a fairer idea about severity from quite early on.
The very high deaths scenarios seemed to me implausible. Such
deaths data would not normally be available since ONS works in
arrears and death certificates are not reliable on their own so
the approach was novel. Understandably the modelling methods that
had been carefully worked out over many years in the preparatory
phase held scientific sway.
None of this is a criticism of the distinguished
modellers who did the work just something that I feel needs to
be reflected on. That is why before I left as CMO I established
a Statistical Legacy Group whose report might be available soon.
Sir Liam Donaldson
Chairman
National Patient Safety Agency
23 November 2010
APPENDIX
COMPARING THE CMO'S TEAM'S EMERGING DATA
WITH SPI ESTIMATES OF MORTALITY.
Predictions of overall mortality for the UK
have been estimated by applying the SPI attack rate and the case
fatality estimated from the CMO dataset to the UK population.
These are shown in table 1.
The case fatality rate (CFR) for the CMO data
set has been calculated from the "best data" available
at the time. The total deaths in England as published in the CMO
media brief on the Thursdays of 16 July 2009, 3 September 2009
and 22 October 2009 were used. These data were combined with the
Health Protection Agency cumulative case estimates for the previous
week, to produce case fatality estimates. The mid—and low—case
estimates give a best and worst case estimate for overall mortality
respectively. These calculations are shown in table 2.
The attack rates taken from the SPI documents
to produce overall population mortality estimates are shown in
table 3.
Table 1
COMPARISON OF THE PREDICTED OVERALL MORTALITY
BASED ON THE CASE FATALITY RATE FROM THE CMO DATA SET WITH THE
PREDICTIONS PUBLISHED BY SPI
| | Worst Case
| Best Estimate |
| | SPI
| CMO | SPI |
CMO |
| 16 July | | 65,000
| 25,000 | 23,000 | 9,000
|
| 3 September | | 20,000
| 9,000 | 5,000 | 2,000
|
| 22 October* | | 1,000
| 3,500 | 80-500 | 800
|
* 22 October estimates refer to the second wave only
Table 2
CALCULATION OF CASE FATALITY RATE FROM CMO DATASET
| Cases |
| Case Fatality Rate |
| Deaths | Low
| Mid | Worst |
Best |
| 16 July | 26 | 50,000
| 20,000 | 0.13 | 0.05
|
| 3 September | 61 | 280,000
| 120,000 | 0.05 | 0.02
|
| 22 October | 93 | 435,000
| 195,000 | 0.04 | 0.02
|
Table 3
ATTACK RATES ESTIMATED BY SPI
| Attack Rate |
| |
| | Worst Case
| Best Estimate |
| 16 July | | 30%
| 25% (20-30%) |
| 3 September | | 30%
| 15% (10-20%) |
| 22 October* | | 12%
| 6% (less than 12%) |
* 22 October estimates refer to the second wave only
|
