Select Committee on Agriculture Minutes of Evidence

Examination of Witnesses (Questions 1 - 19)




  1. Welcome to the Committee for the return bout, and thank you for your memorandum. We are flattered that you quoted so extensively from our report in your article in the Veterinary Record of February 19—a word-for-word reference to our report—and regret only that we were not acknowledged. I suppose we will simply be content with the implied approval. Let us start with the obvious question, if we may. How confident are you that the programme you are overseeing is going to deliver results, and when will we know what they are?

  (Professor Bourne) I think at the last Select Committee meeting you did not have our second report available to you, but you subsequently had that and, of course, you would have read that and noted that the timetable that we recommended should be put in place and wished to adhere to has been met by MAFF with respect to the proactive culls. We recommended that seven should be completed by the end of this trapping year, which terminates at the end of January, with the final three being completed next year, and all 10 triplets have now been identified. We are up to the time-scale on that. We went into some detail in the second report on the likely timing of the emergence of results. You will gather from the report that we do not envisage a time when one can open a window and see everything clearly. What we will see is the accumulation of analyses of data which starts now, from which will emerge knowledge on which policy options will certainly, be based in future. However, the impact that knowledge will have on policy options in the short term, of course, is not known. For instance, we have started looking at TB99 data. Our intention is that an annex will accompany the next report we write, which will be in March, and from that we will gain a little more information with respect to some of the risk factors. There will be nothing earth-shattering about that, but it will be the start of knowledge being gained and made available to the general public and, of course, to Government, which may or may not influence policy. With respect to the trial itself, we again went into some detail and, as you know, the original trial was based on the 50 trial triplet years. Again, my colleague Dr Donnelly may wish to enlarge on this. A number of assumptions were taken into account. Those assumptions may, in fact, lead us to have useful data before the end of the 50 triplet-year period or the 50 triplet-year period may be truncated depending upon the incidence of cattle TB breakdowns in our trial areas. So we suggested in that document that it may be as early as 2002 that we have some useful hard data that can be usefully translated into policy options. It is more likely to be into 2004, but depending upon the strength of the data it could be beyond that time. Do you wish to expand on that, Christl?
  (Dr Donnelly) As is illustrated in the report, the key thing is the number of breakdowns that are observed, and so our ability to tell and to detect robustly a reduction in TB incidence due to the interventions—either of a proactive or a reactive culling—will depend on what the baseline breakdown rate is, the incidence in the control survey on the area. So if TB increases in the underlying rate then that will mean we accumulate data more quickly and are more quickly able to detect a difference; whereas a somewhat conservative estimate would be a rate of eight breakdowns per triplet per year, and that would bring us up to the 50 triplet years, which would end at the end of 2005.

  2. You said, in your second report, you expected to begin analysis of the results from the trial in November this year. I take it from your reply that has started. In your memorandum, referring to MAFF, there are two areas where you expressed some concern: one is about the delay in starting the road traffic accident survey and the second one is the interruption in administering the TB99 form. Are you confident that MAFF is putting sufficient into those activities? Is it concentrating too heavily on the trial itself?
  (Professor Bourne) The trial is but one part of the programme of work in place, as you well realise, and we have a whole raft of research activity looking at cattle issues, particularly cattle pathogenesis, a better understanding of disease in cattle and having improved diagnostic tests. With respect to the RTA that you mentioned, and also the application of TB99 on farms, no we are not totally reassured and MAFF are aware of that. The RTA is being delayed because of a number of issues, one being health and safety and, latterly, of course, the swine fever outbreak in East Anglia, which has taken a lot of MAFF staff away from other activities including some of the TB post mortem activities. RTA is now in place and we are relieved and reassured that the thing actually is on the road, but we are concerned that its scope, at present, is quite different to the scope that we envisaged initially. We have told ministers we will accept the situation as it stands, but we would wish to review the situation in January when we expect the swine fever episode to come to an end, and expect a more rigorous input from MAFF into the road traffic accident survey. With respect to TB99 and the application of these on farms, we again accepted that MAFF were in some difficulty with respect to staff being transferred to swine fever duties. We were extremely concerned, nonetheless, that TB99 was not being carried through in trial areas. We now have the reassurance of MAFF that this is being done. We understand there will be some delay in getting these done outside the trial areas, but we are less concerned about that. We have also expressed concern that even within trial areas it is being done by animal health staff rather than veterinary staff, and we sought reassurances on the quality of staff input into TB99. We have received from MAFF those assurances.

  3. Do these concerns amount to a, in your view, serious compromise of what you are trying to do? Or are they a sort of niggling detail?
  (Professor Bourne) With respect to TB99 I would hope it is a niggling detail, but with respect to the road traffic accident survey, clearly, it would have been better if we had had this in place two years ago and not now. It is the only handle we have on getting some information on the prevalence of TB in badgers outside trial areas. It could be, at the end of the day, that the RTA proves to be a useless tool, but unless it is done properly we will not know that. So we are more concerned about the RTA at the moment. As I said, we insist on coming back to review this in January next year, and we would look then to more effort being directed to the RTA.

Mr Todd

  4. The Committee commented in its initial report on the doubts there were about the statistical power of the trial. The response that you gave dismissed those doubts. Would you like to expand on why you feel that was right?
  (Professor Bourne) I do not think it is true to say we dismissed those doubts. We took it very, very seriously, and continue to take it very seriously.

  5. I think it was said to be based on a "total misunderstanding" of the role of the power of calculation. I must admit, I use dismissive language myself sometimes, and that is the kind of thing I might say.
  (Dr Donnelly) There have been people who raised concerns about the statistical power and when we were last here we discussed that and the Committee recommended that an independent statistical auditor be appointed and review what was going on in the trial from a statistical basis. That has since happened, he was appointed in August of this year and he has provided a preliminary report that has gone to MAFF. That was supportive of what we were doing. The concerns that had been raised looked at the assumptions that we have made underlying the power of calculations; that is the assumption of a Poisson distribution for breakdowns. The Poisson distribution is appropriate for a series of events where you are dealing with incidence data, but it assumes that all breakdowns are occurring independently once you have taken account of predictive factors. Such as, if you had it you could account for badgers in a particular area exposing two different farms to the same potential risk. However, beyond things that you have accounted for they are independent. We have acknowledged that that will not be absolutely the case; any time you do power calculations you have to make assumptions about things and they will not be absolutely independent. I have looked at the distribution of breakdowns in farms—how many farms had multiple breakdowns over a period of years prior to enrolment in the trial—and they were consistent with the Poisson distribution. The other thing that relates to the power to detect differences of certain magnitudes is the underlying incidence rate. The 50 triplet years that we have talked about is actually quite conservative in assuming that the underlying breakdown rate will be eight breakdowns per triplet per year. So to the extent that there is some non-independent that will reduce a little bit the power that we have, the fact that we have made this very conservative assumption about what the incident rate is means we will actually have greater power.

  6. What is your estimate of the power, based on your current knowledge?
  (Dr Donnelly) It depends on what sort of breakdown rate you are actually looking at. If you look at—

  7. This is a bit critical, is it not? If it is not going to produce strong enough results which give a clear enough indication of the relationship, then people will feel that they have wasted a huge number of badger lives and a large amount of money to demonstrate something which may be possible but not a strong enough relationship to establish policy. So it is critical to get this rather tedious and complex—which I cannot claim to fully understand—statistical methodology right to ensure that we are going to be guided by policy which is founded on science and a proven incidence of this particular problem.
  (Dr Donnelly) We considered as an example a 20 per cent reduction in incidence and found that we have a 90 per cent power, under the assumptions that are made to detect that at the end of the 50 triplet year period, which would be the end of 2005.

  8. Have you checked back on the information you have gathered so far from the early data to check that estimate?
  (Dr Donnelly) You mean since the trial has started?

  9. Yes.
  (Dr Donnelly) I have done one interim analysis which I was given blinded, so I did not actually know which treatments were allocated to which—information from three triplets that have had proactive culling prior to this. I would say the data appeared to be consistent with the assumptions that we have made, but I would also caution that that data is based on 3 triplets.

  10. Also, presumably, if the triplets that you drew data from were from traditional hot-spots, there is an argument for saying that the data collected may not apply in other places where infection has been much more recent.
  (Dr Donnelly) We have taken into account in all our selection of triplet areas the incidence in the past year and the past three years, at the time the selection was made. So at that point that is the best incidence data that we have, and that gives us the best prediction of what will happen in future.

  11. What is the minimum reduction in TB incidence that you would consider it important to detect, and within what time-scale in this trial?
  (Dr Donnelly) It is extremely difficult to answer given that we do not yet have the information, but we have talked about doing an economic analysis and talking about that in the broader sense of economic, taking into account a number of concerns, to look at the cost-benefit analysis of doing a sort of culling intervention. So if it turns out that a 20 per cent reduction in breakdowns is possible and we are able to detect that, but that it actually costs more to complete the intervention than you would save by doing that, then a 20 per cent reduction would not be sufficient to suggest that that was a sensible policy. So as yet we cannot actually say, but there are going to be research programmes looking at what the economic impacts are to help us make that judgment.

  12. You mentioned Professor Mollison's scrutiny of the analysis and that he had given that to MAFF. Could you say a little bit more on what he has said so far, or should we ask the Minister that question?
  (Professor Bourne) We have seen a draft copy of that report and the full report, I believe, has gone to MAFF and it would be appropriate for you to ask them about that. From our perspective it is certainly reassuring that his assessment of the statistical validity of the trial agrees with our own.

  13. Are you prepared to make data available to outside sources to validate your assumptions?
  (Professor Bourne) You saw the document we prepared for Professor Mollinson, which I believe is part of your papers.

  14. Is that more widely available?
  (Professor Bourne) I think it is in the public domain, is it not, as indeed are all the papers related to this?

  15. Not yet if it has been given to us, but it may well be.
  (Dr Donnelly) Our submission is available on the MAFF website.

Mr Mitchell

  16. Can I return to this point of the power of the calculations and the results? We have had a memorandum from a Dr Fiona Mathews, a Hodgkin Research Fellow at the Department of Zoology in Oxford. She says that the original sample sizes appear incorrect. "Rather than having a power of 90% to detect a reduction in TB incidence of 20% within 5 years, the true power is likely to be nearer to 50-60%. This means that if culling badgers reduces cattle TB by up to 20% then there is a 40-50% chance that it would not be detected. Even were the effect of the badger cull greater, and cattle TB was reduced by as much as 25%, there would still be less than an 80% chance that the trial would give a `positive' result within 5 years." How do you react to that?
  (Dr Donnelly) I think you were given the summary and I have seen the paper; Fiona Mathews provided it to us, actually, a year ago. So I have seen her concerns, and the concerns that she has are the clustering of TB breakdowns of the non-independence that we suggested and, also, variations between triplets in incidence. She is taking concerns that are valid to the extent that we had to consider non-independence, we had to consider clustering and repeat breakdowns, and so there is a valid area for concern. We have examined those, we have looked in detail at the data on past incidence in the trial areas, and we found that the heterogeneity that she is assuming to get those sort of reductions is not evident in the data. What she is saying is if those sorts of difference appear as the trial went on, then the power would be reduced. We cannot say absolutely what will happen in the data—what will arise—because we have not seen the data yet. However, on all the data that we have looked at so far, including the past, we have no reason to think that we are getting that sort of level of non-independence and that it would have such a reduction on our power. Those are the exact issues that we discussed with them as well.

  17. So she is wrong?
  (Dr Donnelly) She has taken a very extreme view, which I do not think is representative of what is going on in the trial.

  18. She is, in fact, saying that your original calculations on the power of the study were incorrect and that you really need more triplets.
  (Dr Donnelly) Well, more triplets or more triplet years. Yes, she is saying that.

  19. You are defending the correctness of your original calculation—90 per cent power.
  (Dr Donnelly) Yes.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2001
Prepared 10 January 2001