Select Committee on Agriculture Appendices to the Minutes of Evidence


Memorandum submitted by Mr Trevor Lawson (J17)

  As a freelance journalist I have been covering the issue of bovine TB and the subsequent Krebs' experiment for several years. I write for The Guardian, Daily Express and other publications, and I am currently completing a feature on the issue for BBC Wildlife magazine. I write to draw the attention of the Committee to a number of concerns I have been unearthing about the current Krebs' trial.

  Last year, I wrote to draw the Committee's attention to the problems I had had in securing adequate information about the statistical validity of the trial. During the course of this year, my concern has grown, but key questions remain unanswered. Below, I recount for the Committee's interest some of my e-mail exchanges with Dr Christl Donnelly and Sir David Cox who helped develop the statistics of the current trial.

  On 19 May 2000, Dr Donnelly stated that:

    Our assumption is that the number of observed cattle breakdowns follows a Poisson distribution. Such a distribution would arise if the cattle breakdowns occurred totally at random.

  I was struck by this. It is clear from the evidence available that bovine TB incidents are anything but random. They occur in the same places—TB hotspots—and consistently strike certain farms over time. On 22 May, I asked Dr Donnelly the following questions (this e-mail is edited down):

    My impression of the bTB outbreaks is that they are not random. Indeed, the trial focuses on areas known as hotspots. Am I therefore right in thinking that this assumption is the basis for your null hypothesis?

    Second, if this is so, I am uncertain how you address the issue of repeat breakdowns. I note you state that: "the key quantities are the total number of TB breakdowns in the survey only (control) areas". The Poisson distribution seeks to show random (and therefore unrelated?) events. But in hotspots breakdowns are not unrelated. They can occur repeatedly on the same farms. They are anything but random. Do you treat repeat breakdowns on the same farm as new events which are unconnected with previous breakdowns, which add to this key "total number of TB breakdowns"? If so, what effect does this have on the power of the experiment?

    It also occurs to me that the hotspots are sometimes related spatially. Some trial areas are virtually adjacent to one another, though they may be some distance from other trial areas which are adjacent to one another. So, how do you deal with the spatial relationship between adjacent trial areas in Cornwall/Devon and Hereford/Gloucester, versus the non-clustered areas elsewhere. The reason I ask is that clustering could reflect an underlying factor, such as geology or soil type, and to discount it could be discounting a significant contributor to the disease.

  [Dr Donnelly had also said the statistics were able to cope with "various types of non-compliance". I asked what she meant by this:]

    But what about other issues which I shall call "non-compliance", such as significant shifts in weather patterns, changes in husbandry practices (new feedstuffs, grazing regimes, antibiotics, and changes in the number of cattle movements (per week, month or year—pick a period)? What are your underlying assumptions here? Do you assume they are constant or simply not relevant?

  Having asked these questions, I was then astonished to receive this response from Sir David Cox in Oxford's Nuffield College and Department of Statistics, via Dr Donnelly. I have numbered his paragraphs. He wrote on 23 May 2000:

    (1)  Dear Christl, Thank you very much for keeping me in touch with your correspondence with Trevor Lawson. I don't know Mr Lawson. He asks some interesting questions but I do wonder if he really understands the nature and purpose of power calculations. As I see them they are important indeed essential assessments beforehand of the size of study that seems sensible and, especially as in the present case there was no comparable study to work from, they are absolutely inevitably based on simplifying working assumptions. The answer is in any case general guidance; who is to say what level of power is appropriate anyway other than by general judgement. The assumption of a Poisson distribution is surely the correct one to make although to some extent probably a best case scenario is unlikely to be sufficiently far off to matter much.

    (2)  The really crucial point is that the calculation shows that 10 triplets will give a reasonable level of precision, neither absurdly over precise implying a waste of recourses nor something so imprecise as to be useless. Another crucial aspect is that the power calculations are assessments beforehand. The precision actually achieved will be found from the data when we have them; the power assumptions and calculations play no role in that. Maybe the initial calculations will prove a shade pessimistic and higher precision than expected will be achieved or maybe the contrary which will be disappointing but not disastrous. It is a quite pointless waste of time fretting now over details of the power calculations, important though they were two years ago. No doubt the forthcoming statistical audit will look at power but hopefully spend most time on analysis methods.

    (3)  The use of hot spots was essential for reasoned economy; the Poisson assumption applies to the comparison of the three regions forming a triplet not to comparisons across a broader region and the purpose of the randomisation across the three areas forming a triplet is precisely to exploit inter-area differences to enhance precision.

    (4)  Non-compliance in these contexts has a reasonably precise technical meaning any failure of an individual trial area to follow the regime to which it has been allocated.

  I am about to follow up these comments with Sir David, but he was right to say that I did not understand the nature and purpose of power calculations. My understanding now gives me greater cause for concern.

  It appears from Sir David's first paragraph (1) the underlying assumptions on which the statistics are based are simple. Let us take one of those assumptions, as specified in the Krebs report: all badgers will be killed in proactive areas. It is clear that not all badgers are being killed. It is also apparent that there is no certain way of assessing badger density before culling or afterwards, and therefore what impact culling is making on badger numbers. I suggest that these errors in assumptions will affect the statistical reliability of the trial.

  Sir David's second paragraph reveals that (2) the statisticians have no idea how accurate the trial's results will be until they get the data. I find this incredible. From the outset, members of the press and the public have been led to believe that the trial would last five years and that would be that. I suspect that Ministers have been told the same thing. The trial had a budget and a finite lifespan. There has been some slippage in delivery, but there has been no suggestion that the trial might need to continue for many more years if the data gathered are inadequate. If the assumptions weaken the data too much, MAFF will either have to abandon the trial with no useful results, or carry on culling for an unknown period of time until enough data are gathered. The implications for the public purse are massive.

  Sir David's third paragraph is interesting, since it suggests (3) that the data gathered will only give us meaningful answers about bovine TB in the triplet areas. Because hot spots have been selected for reasons of economy, the data are specifically skewed and, I imagine, will tell us nothing reliably about the wider countryside. Given that a supposed fear is that the TB "epidemic" will continue to grow, it is surprising that the trial will not be able to confirm this.

  I regret that Sir David's fourth paragraph still fails to clarify what non-compliance means for the trial. However, if we take the literal words of the Krebs report, with the aim in proactive areas being to cull all badgers, then I suggest it is certain that (4) the entire trial is non-compliant, because not all badgers are being killed.

  I apologise to members of the Select Committee for this long presentation of evidence. However, if you are able to throw any light on the true statistical reliability of the trial it would be welcomed by member of the press and, no doubt, the farmers whose future depends on a solution to bovine TB.

27 October 2000

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2001
Prepared 10 January 2001