Further memorandum submitted by the Former
Independent Scientific Group on Cattle TB
RESPONSE TO
QUESTIONS FROM
THE ENVIRONMENT,
FOOD AND
RURAL AFFAIRS
SELECT COMMITTEE,
[8 NOVEMBER 2007]
FJ Bourne, CA Donnelly, DR Cox, G Gettinby, JP Mclnerney,
WI Morrison and R Woodroffe
Former members, Independent Scientific Group
on Cattle TB
1. Sir David King makes specific criticism
of the ISG's interpretation of confidence intervals for the detrimental
effect of proactive badger culling outside the specified area.
Specifically, King's paper notes that several of these confidence
intervals extend below zero, and are fairly wide (para 43)—with
the implication that they should be treated with caution. Can
you explain the ISG's thoughts on King's analysis in greater detail?
We discuss this in detail in our response to
Sir David King's report.[13]
Here is the relevant section of this response:
(i) Interpretation of statistical confidence
intervals in subgroup analysis
ISG13 King et al (2007) place great
weight on interpretation of results presented in Figure 5.2B of
our final report (Bourne et al, 2007). This figure shows
the results of a subgroup analysis, stratifying overall RBCT results
into beneficial and detrimental effects recorded at different
distances from the boundaries of proactive trial areas. As is
often the case in scientific studies, each effect was presented
as a point estimate, associated with 95% confidence limits. The
latter give a measure of the uncertainty associated with the point
estimate; there will be substantial uncertainty when small sample
size prevents a precise estimate from being obtained from a subset
of the data.
ISG14 In paragraph 43 of King et al
(2007), the detrimental effect of badger culling on cattle TB
incidence outside culling area boundaries is dismissed since "Three
out of four [confidence intervals] go through zero (ie one cannot
be confident that the overall effect is detrimental)". This
interpretation is incorrect, as it omits one very crucial proviso
in the interpretation of our subgroup analysis. The limits attached
to any subgroup of data concern what can be learned from that
subgroup of data on its own. While it is useful to know this,
it is rarely the primary focus of analysis, and was not so in
this case. Indeed if the data are broken into a large number of
small subgroups, each on its own will have substantial uncertainty
and be indecisive on any issue of concern, even if the overall
picture is entirely clear on the point under study, as was the
case here. The overall picture must be studied, not fragments
of it. This is what we have done in all our analyses, subject
to tests of the uniformity of the effect under study.
ISG15 In the interests of consistency, we
note that the same analysis showed that all five of the subgroup
estimates of beneficial effects of culling inside proactive areas
had confidence intervals which included zero. Thus, were the same
(erroneous) interpretation to be placed on those findings, evidence
of the beneficial effects of culling should likewise have been
dismissed.
2. Table 5.10 of the ISG's Final Report shows
that the point estimates from the initial cull and the first follow-up
culls (29.3% and 27.3%) are both significant at the 5% level (0.015
and 0.044 respectively). What were the null and alternative hypotheses
in these cases?
The null (Ho) and alternative (HA) hypotheses
were in each case:
Ho: There was no effect of proactive badger
culling on the incidence of all (confirmed and unconfirmed) TB
breakdowns among cattle herds up to 2 km outside RBCT trial areas.
HA: There was an effect of proactive badger
culling on the incidence of all (confirmed and unconfirmed) TB
breakdowns among cattle herds up to 2 km outside RBCT trial areas.
[Note that the alternative hypothesis is two-sided.
In other words, the null hypothesis would be rejected in light
of sufficient evidence of either an effect which increased incidence
or an effect which decreased incidence.]
So the conclusions are that there was evidence
of a detrimental effect of proactive culling on the incidence
of all TB breakdowns among cattle herds up to 2 km outside RBCT
trial areas both from the initial cull (p=0.015) and from the
first follow-up cull (p=0.044), based on RBCT location data (Table
5.10, page 101 of the Final Report). Similarly, there was evidence
of a detrimental effect of proactive culling on the incidence
of confirmed TB breakdowns among cattle herds up to 2km outside
RBCT trial areas both from the initial cull (p=0.0l6) and between
the initial and first follow-up culls (p=0.021), based on RBCT
location data (Table 5.8, page 98 of the Final Report).
It is worth noting that the estimates based
on all (confirmed and unconfirmed) breakdowns were (in 5 of the
6 analyses) smaller in magnitude than those obtained in analyses
of confirmed TB breakdowns only (compare Table 5.8, page 98 and
Table 5.10, page 101 of the ISG Final Report). This was similarly
the case within RBCT trial areas (compare Table 5.3, page 92 and
Table 5.5, page 95 of the ISG Final Report). While our analyses
revealed consistent and statistically significant effects of badger
culling on the incidence of confirmed cattle herd breakdowns,
there were no similar effects on unconfirmed breakdowns. For this
reason we considered it appropriate, from a biological perspective,
to focus primarily on effects on confirmed breakdowns, rather
than all breakdowns. However, disruptions and costs result from
both confirmed and unconfirmed breakdowns, although unconfirmed
breakdowns are typically shorter in duration. Both confirmed and
unconfirmed breakdowns result in the compulsory slaughter of reactor
cattle, movement restrictions on the herd, and additional testing
of cattle. Hence, from an economic point of view preventing unconfirmed
breakdowns would be desirable, whether or not they indicate the
presence of disease.
We would also wish to draw the Committee's attention
to further analyses of the incidence of confirmed breakdowns,
presented in paragraph A20 of our response to Sir David King's
report.
3. Appendix D of the ISG's report (p205) provides
summary data on triplets recruited to the RBCT. Some of the sample
sizes recorded (for total badgers culled, culled badgers with
TB) look to be fairly small once the total number of culls is
taken into account. Does this affect the robustness of the RBCT
statistical results and the conclusions that can be drawn from
them?
The numbers of badgers culled per year under
the proactive and reactive strategies are detailed in Tables 2.4
and 2.6 (pages 49 and 50), respectively. Regarding the numbers
of badgers culled in reactive areas, fewer badgers were taken
in reactive areas due to the more limited geographic scale of
the reactive culling strategy, compared with the repeated widespread
culling in proactive trial areas. Furthermore, the suspension
of reactive culling in November 2003 meant that reactive culling
was undertaken for fewer triplet years than proactive culling.
The statistical precision of the RBCT results
concerning TB incidence in cattle herds was a function of:
(a) the per-annum incidence levels in trial
areas,
(b) the number of years of observation,
(c) the number of triplets (10) and
(d) the consistency of culling effects between
the triplets.
The precision was not a function of the numbers
of badgers removed.
The number of badgers taken from a trial area
on a particular cull is a function of the underlying density of
badgers within that area and the proportion of badgers removed.
Underlying density will be influenced by the suitability of the
environment for badgers, suppression of the population by previous
culls (both before and during the RBCT), immigration into the
trial area from outside, and the natural growth rate of the population
due to reproduction and survival of badgers. The proportion of
badgers removed is influenced by the trapping method used, as
well as by season, weather conditions, land access, disruption
and occupation of traps by non-target species; however our published
analyses (Donnelly et al 2007; Woodroffe et al 2007)
detected no evidence of an effect of the last three factors on
overall capture rates.
We discussed the impact of culling efficiency
in paragraphs 10.10-10.14 (pages 164-5 of the ISG Final Report),
and concluded that "improvements in culling efficiency—if
implemented in isolation from other changes—are unlikely
to generate benefits [in terms of reducing TB incidence in cattle
herds] greater than those recorded in the RBCT". This is
primarily because, in the absence of geographical barriers to
badger movement, improvements in trapping success would be largely
offset by badger immigration into culled areas. We went on to
discuss, in paragraphs 10.15-10.24 (pages 165-167 of the ISG Final
Report), alternative approaches based on proactive culling and
concluded that there was, at present, no practicable culling method
likely to generate benefits substantially greater than those achieved
by RBCT proactive culling.
While the "sample size" of badgers
did not influence the precision of analyses concerning the incidence
of TB in cattle herds, these sample sizes did directly affect
the precision of estimates obtained in analyses concerning the
prevalence of M bovis infection in badgers. While the sample sizes
were large enough for us to detect various important effects (for
example, both proactive and reactive culling were associated with
elevated M bovis infection prevalence in badgers), it is of course
very possible that additional effects would have been found to
be statistically significant had sample sizes (ie numbers of culled
badgers) been substantially larger. This is, of course, always
the case in such a study.
4. Medical research trials sometimes use significance
tests at 1%. Can the ISG's reasons for testing at the 5% level
of significance be further explained?
In assessing the evidence in our data we did
not use any specific significance level but reported the precise
p values achieved (rather than "p0.05", for example).
One quite widely used interpretation is that if p is roughly 0.05
there is quite strong evidence of an effect beyond the play of
chance and if p is roughly 0.01 very strong evidence and so on.
But we did not set hard and fast rigid rules for the interpretation
of isolated effects. (This is a widely used approach for interpretation,
to be contrasted with rules of hard and fast decision making.)
A brief reference discussing these issues is given in Cox and
Snell (1981) with a more extended account written for non-statisticians
presented in Cox (1982). Furthermore, while p values are a way
of reporting the results of statistical tests, they do not define
the practical importance of the results.
Although initial power calculations were based
on the likelihood of detecting a significant effect (this in turn
depending on the assumed significance threshold), later discussions
focussed on the precision with which any effect could be estimated
(paragraph 2.18, page 37 of the ISG Final Report). We typically
reported both the confidence interval and the p-value associated
with estimates, where the confidence interval quantifies the precision
of that estimate.
5. If economic costs were excluded from your
analysis or calculated in a way that might lead to reduced expense,
would the Committee be correct to conclude that the ISG might
have reached a different recommendation concerning the desirability
of culling?
Our analyses indicate that modest overall benefits
could be obtained by culling badgers in a simultaneous and coordinated
fashion across large areas of the countryside, and repeating such
culls over several years. However, our experience of overseeing
such culling in the course of the RBCT—though over only
ten 100 km2 areas—leads us to very seriously doubt that
culling could be performed in the manner that would be required,
particularly following dissolution of the Defra Wildlife Unit.
Our results provide consistent and robust evidence that failing
to meet any of the necessary criteria—that is, if culls
were conducted asynchronously, or in a patchy manner, or not repeated—their
effect would be to increase, rather than reduce, the incidence
of cattle TB, and to contribute to its geographical spread. This
was our primary reason for recommending against a badger culling
policy.
These conclusions were not dependent on the
economic assessments included in Chapter 9, and would not have
been changed by any plausible adjustments to the cost figures
used there. The economic appraisal simply reinforced our view
that culling could not contribute meaningfully to control policy,
but introduced the added perspective that, to be sustainable and
defensible, measures also needed to have some economic justification.
The evidence showed that culling along the lines of the RBCT could
not meet this requirement
In considering other culling methods it was
clear their impact on the incidence of cattle TB would need to
be incomparably greater than that achieved by cage trapping if
they were to approach economic justification. For example, licensed
culling by farmers using snares (by far the least expensive means
of culling) would need to be 10 times more effective at reducing
cattle TB incidence than was RBCT culling (para 9.24). Our discussion
of culling options in Chapter 10 of our final report led us to
conclude that such enhanced performance is highly improbable.
While we did not formally discuss shooting or gassing setts using
tractor exhaust, which would undoubtedly be less costly, we consider
it unlikely that these could be conducted in the widespread, simultaneous,
coordinated and repeated manner that would be needed to achieve
benefits. Hence the use of such methods would be likely to increase,
rather than reduce, the incidence of cattle TB. Moreover, we are
concerned that such practices would not meet necessary welfare
concerns. Individual farmers, or small groups of farmers, might
feel inclined to try them in pursuit of their own personal benefit,
but analysis suggests even so that their financial gains may be
illusory while the effect of badger social group perturbation
implies potentially severe financial costs to their neighbours.
December 2007
REFERENCES
Bourne, J, Donnelly, CA, Cox, DR, Gettinby, G, McInerney,
JP, Morrison, WI, and Woodroffe, R (2007) Bovine TB: the scientific
evidence, Defra www.defra.gov.uk/animalh/tb/isg/pdf/final report.pdf,
London.
Cox, DR (1982). Statistical significance tests. British
Journal of Clinical Pharmacology, 14, 325-331.
Cox, DR and Snell, EJ (1981). Applied statistics.
London: Chapman and Hall, pp 37-39.
Donnelly, CA, Wei, G, Johnston, WT, Cox, DR, Woodroffe,
R, Bourne, FJ, Cheeseman, CL, Clifton-Hadley, RS, Gettinby, G,
Gilks, P, Jenkins, HE, Le Fevre, AM, McInerney, JP, and Morrison,
WI (2007) Impacts of widespread badger culling on cattle tuberculosis:
concluding analyses from a large-scale field trial. International
Journal of Infectious Disease, 11, 300-308.
King, D, Roper, TJ, Young, D, Woolhouse, MEJ, Collins,
DA, and Wood, P (2007) Tuberculosis in cattle and badgers: a report
by the Chief Scientific Adviser, Sir David King http://www.dius.gov.uk/publications/Expert%20advice%20on%20badgers%20and%20cattle%20TB.pdf,
London.
Woodroffe, R, Gilks, P, Johnston, WT, Le Fevre, AM,
Cox, DR, Donnelly, CA, Bourne, FJ, Cheeseman, CL, Gettinby, G,
Mclnerney, JP, and Morrison, WI (2007) Effects of culling on badger
abundance: implications for tuberculosis control. Journal of
Zoology, doi:10.1111/j.1469-7998. 2007.00353.x.
13 See Ev 101. Back
|
