Annex
THE BADGER CULL: ISSUES IN THE DESIGN OF
THE FIELD TRIAL
Dr Fiona Mathews, Royal Society Dorothy Hodgkin
Research Fellow, Department of Zoology, University of Oxford
SUMMARY
1. It is increasingly recognised that many
ecological experiments suffer from inadequate statistical power.
Real and biologically important differences between groups are
therefore likely to go undetected. A "statistically insignificant"
result could reflect either the true lack of a treatment effect,
or the inability of the study to detect an effect of this magnitude.
2. Although the results of a priori
power and sample size calculations are only estimates, they do
act as useful guides. However, as with any statistical method,
the results will be compromised if the underlying assumptions
do not hold true. Ecological studies are often complex to execute
and analyse. For example, sampling units frequently border each
other. Power and sample size calculations must reflect this reality
as far as possible. There is a danger that erroneous calculations
give "scientific" credence to studies that are too small
or poorly designed.
3. The badger culling trial aims to determine
whether the removal of badgers can decrease the incidence of bovine
tuberculosis in cattle. Although the trial is controversial, there
has been little scrutiny of the project's design. In part this
may be due to sample size calculations which appear to show that
the project will be able to detect important reductions in the
incidence of TB in cattle within five years (p = 0.05, power =
0.9). However, the sample size calculations may have seriously
overestimated the study's statistical power, by failing to account
for important aspects of the project's design.
4. The trial uses cluster randomisation,
whereby groups of adjacent farms are allocated to the same treatment.
Each unit within the cluster therefore cannot be treated as independent:
farms within groups are likely to be more similar than farms in
different treatment groups. Calculations using simulated data
illustrate the potential impact on sample size: even were the
variation between clusters only moderate, the sample size would
need to be doubled to maintain the same statistical power. If
the sample size remains unchanged, the chance of detecting a 20
per cent reduction in TB incidence would only be 66 per cent.
With greater betweencluster variation, the sample size increase
required is even greater.
5. The trial also assumes that TB occurs
in accordance with a Poisson distribution: repeated breakdowns
on the same farm, and breakdowns on adjacent farms are treated
as independent. This assumption is unlikely to be justified. Under
more realistic assumptions, the effective sample size is further
reduced by up to a third.
6. In summary, the original sample size
calculations appear incorrect. Rather than having a power of 90
per cent to detect a reduction in TB incidence of 20 per cent
within five years, the true power is likely to be nearer to 50
to 60 per cent. This means that if culling badgers reduces cattle
TB by up to 20 per cent, there is a 40 to 50 per cent chance that
it would not be detected. Even were the effect of the badger cull
greater, and cattle TB was reduced by as much as 25 per cent,
there would still be less than an 80 per cent chance that the
trial would give a "positive" result within five years.
7. Given the economic importance of bovine
tuberculosis, the ethical implications of largescale removal
of protected wildlife, and the cost of the study, the badger culling
trial requires urgent review.
