National Institute for Health and Clinical Excellence - Health Committee Contents

Examination of Witnesses (Questions 525 - 539)



  Q525  Chairman: I welcome the witnesses to the fifth evidence session in our inquiry into NICE. Perhaps for the record the witnesses will introduce themselves and the positions they hold.

  Professor Devlin: I am Nancy Devlin. I am professor of economics at City University where I work at the City Health Economics Centre. I am also a senior associate at The King's Fund.

  Professor Appleby: I am John Appleby, visiting professor at City University and chief economist at The King's Fund.

  Q526  Chairman: To start, I ask a question of both witnesses: what does your research reveal about the cost-effectiveness threshold used by PCTs and how do you believe it relates to that used by NICE?

  Professor Appleby: Perhaps I may first explain what the study is about. It is a very small feasibility study conducted jointly with NICE. First, could we identify decisions made by PCTs either to invest or perhaps disinvest in new services? From that, could we estimate some sort of implied cost per QALY for the decisions taken? That is the background to the study. Clearly, it is linked to concerns of NICE, this Committee and many others about what the threshold should be and how much we are willing to pay to get health benefit from the NHS. We looked at only seven or eight PCTs and identified the decisions they made. We then chose some of the decisions to invest or disinvest and proceeded with the work. We found that in some cases we could estimate a cost per QALY; in other cases the decisions were based on the notion that a service which was to be introduced or not introduced was clinically more effective than anything else and cheaper, in which case it was dominant. We had very few observations and so I am not sure we can arrive at anything too conclusive about what the threshold is out in the real world in NHS decisions.

  Q527  Chairman: Do you agree with that, Professor Devlin?

  Professor Devlin: Yes. It was a feasibility study. The purpose of commissioning the study was to determine whether it would be possible in a larger study with more observations to identify the threshold that applied at PCT level. I think it demonstrated that that was feasible. It is not easy to do it, however. A lot of the decisions one observes are those where PCTs take what may be seen to be eminently sensible decisions either to disinvest in services which are very poor value for money or to invest in services which are extremely good value for money. It is easy to identify newly-added or newly-eliminated services, but that will not necessarily reveal marginal services and the willingness to pay at PCT level to gain a quality adjusted life year. One can certainly identify the services and assign to them a cost per quality adjusted life year gained, but a much larger-scale study would be required in order to come up with precise estimates of the thresholds that arise from PCT budget constraints.

  Professor Appleby: We have to understand the position of PCTs here. They are going for relatively easy and more obvious things in terms of disinvestment. The sort of information we received was a fairly wide spread of values, which is what we would expect. In a larger study we would hope to start to identify more decisions and narrow that range, but it will always be a range; there will not be a single number out there on which PCTs operate.

  Q528  Dr Naysmith: Professor Appleby, why do you think they are going for the easier things? Is it lack of information or do they just want to chop something quickly?

  Professor Appleby: It is the low-hanging fruit argument, that is, if the evidence is fairly strong and something is clinically pretty ineffective there will not be much argument about it. Like NICE, PCTs can find themselves in a difficult position with their public and residents if they stop funding some service or whatever, so that is in the back of their minds.

  Professor Devlin: I think that investing in services that are dominant and disinvesting from ones that are dominated is an entirely logical thing to do.

  Q529  Dr Naysmith: One of the things we have recommended many times is that that is something NICE should do more of than in the past?

  Professor Devlin: The other matter that has been observed and makes the research very challenging is that PCTs are not always well informed of some of the data that could support them in decisions regarding either investment or disinvestment. What slightly confuses the picture somewhat is that cost-effectiveness is not the only criterion that matters to PCTs, in the same way that it is not the only criterion that matters to NICE. Therefore, unpicking the basis of a decision that is made is also important.

  Q530  Chairman: In evidence to us Professor Peter Smith suggested that the PCT threshold was likely to be nearer £20,000 than £30,000. Do you believe there is any robust evidence? Do you agree with what he says?

  Professor Appleby: I think we have to be very careful. I emphasise that our study was very small and involved only eight PCTs. We looked at about nine different sorts of decisions that those PCTs had made. By the way, they were often the same decision. Some co-ordination going on between PCTs in going for some of the same things. I do not believe from the evidence we have we can say one way or the other whether Peter Smith's figure is the right one or is correct in some sense.

  Professor Devlin: There is no doubt in my mind that this is the sort of research that needs to be done to inform what NICE's response should be.

  Q531  Chairman: In your evidence you more or less suggest to that in taking these types of decisions the NHS should have the equivalence of the committee at the Bank of England that sets the interest rate. Should NICE thresholds be more explicit, and who should set them?

  Professor Devlin: There are two aspects to explicitness in a sense. One is the way in which the threshold is stated, whatever it is. What we have noted in our evidence is that there are a number of statements, which are inconsistent in very subtle ways, about what the NICE threshold is. They also emphasise the range and it is not always clear what the interpretation of that range is or, from the perspective of somebody who sits outside the NICE context, how that range is being applied in decision-making. Therefore, one aspect of explicitness is how the prevailing threshold is expressed. The other aspect of explicitness is that the threshold has no explicit basis or location in evidence. I believe that an explicit threshold should be both evidence-based and clearly expressed.

  Professor Appleby: Together with David Parkin, who is also a professor at City University, we wrote a BMJ editorial on this matter. In our evidence to you we have said this is a fundamental issue that touches on what services the NHS should and should not provide and it goes to the heart of decisions about the willingness to pay to get benefit from the health service. As my colleague has said, there is some fudging going on as far as NICE and others are concerned not just about what the threshold is but how it is applied. There is no empirical evidence to support the threshold as stated by NICE; there is no real theory behind it, and yet it is such a fundamental issue. It is not just a technical issue in the sense of finding out some numbers and then deriving a threshold, crank the handle and that is it; it involves social value judgments and so on. We believe that either a separate independent organisation or at least an independent process can be set up by which the issues with which NICE and this Committee have been grappling are discussed, the research is commissioned and carried out and it is transparent. The public should also be involved because in the end it is an issue for them; it is their money that goes into the NHS and these are decisions to be made about how best that money is used. It is a finite budget. This decision will never go away no matter how much money we have.

  Q532  Chairman: The article in the BMJ suggests in a sense that NICE is not as independent as it should be.

  Professor Appleby: I hope we did not suggest that.

  Q533  Chairman: I believe this is important in terms of this inquiry. What independence do people perceive NICE has?

  Professor Devlin: I do not think this was intended to be critical of NICE. In a sense I think that NICE has been placed in an invidious situation. It does an excellent job at assessing the clinical and cost-effectiveness evidence in an enormously rigorous and diligent way. What we say is that the threshold that must be applied to make a judgment on the basis of that evidence is an extremely difficult issue. We are suggesting that NICE's threshold is not just a matter for NICE alone: it is not just NICE's business. If NICE makes a mandatory decision that PCTs must implement it completely alters the bundle of services which PCTs can afford to deliver. That affects the services that all patients can potentially consume or benefit from, so NICE's threshold should have an input from the sector and a much wider range of expertise.

  Professor Appleby: NICE does an amazing job given the task it has been handed, but that job is more of a technical nature in collating and assessing evidence, commissioning research to fill gaps in that evidence and so on. But when it comes to the threshold that should be a separate job, as it were. We argued that maybe NICE should be handed a threshold to which it should operate. At its inception it realised it could not do its job unless it had some sort of threshold at which point it said something was or was not cost-effective or that it did or did not recommend it. But at the moment that is buried in the work NICE does. Our argument is that it should be recognised as a separate matter and there are different ways to deal with it.

  Q534  Dr Naysmith: What is the answer to the question put by the Chairman? Who should set the threshold and where should the decision-making process lie? If we are agreed that it should not be the Department, where should it lie?

  Professor Appleby: We suggested that the parallel should be the MPC. There could be a separate organisation, group or at least process and that would set the threshold. It is not just a one-off exercise; it will have to be updated every year. New evidence will emerge. The public's values—how much they are willing to pay and so on—will also change from time to time, so it includes a whole of things and it is an ongoing process.

  Q535  Dr Naysmith: It is a good idea to say that somebody else should make the decision, but working out exactly who that should be is quite difficult.

  Professor Appleby: Yes, but I am sure it is not beyond the wit of us to devise something.

  Q536  Dr Naysmith: The Monetary Policy Committee is full of economists, bankers and so on, but do we want former trust executives or former Ministers on this body? Who should it be?

  Professor Devlin: Perhaps it should be a range of expertise: economists and people involved in budget-constrained decisions in PCTs as well as people in NICE. I do not believe that the question of the appropriate expertise on such a committee is insurmountable.

  Q537  Dr Naysmith: It is quite difficult to do.

  Professor Devlin: It is not an easy question, but what we suggest is that it really requires a concerted effort that perhaps stands slightly outside NICE's remit. It is also a dynamic issue. A lot of the discussions around the threshold have tended to focus on either a single figure or a statement about a range. Those statements have been made since 1999. It is completely unfeasible that whatever the appropriate threshold or range was in 1999 is the same now. Regardless of what determines or drives that threshold, whether it is society's willingness to pay for a QALY or the threshold revealed by budget-constrained decisions, those factors will change year on year.

  Chairman: In the end are we not talking about the terms of reference used by NICE? Clearly, if the thresholds were set outside and handed to NICE it would work to those thresholds.

  Dr Naysmith: It does at the moment.

  Q538  Chairman: Yes, but it does not admit it. What would be the change in reality?

  Professor Appleby: I was trying to illustrate that in terms of what my colleague said originally about how the threshold to which NICE operates is difficult to define. There is no real evidence to support the threshold it uses, and the practice of using it is pretty fuzzy. I cannot now remember the numbers, but quite a lot of decisions are beyond the £30,000 QALY range that it recommends. I cannot remember how many but it is a minority. As far as I understand, NICE says that those are the circumstances where special factors come to bear. There appear to be an awful lot of special cases here which throws into doubt the range that NICE talks about. Therefore, there is an issue about making that more transparent. I suppose that it could be done through NICE. Perhaps one method is to change NICE's terms of reference, but we were thinking of taking it a step further and saying that NICE has another job to do, that is, just evaluations. It then applies a threshold which it is handed. I suppose that as a society we will never agree a single number, but at least we can get closer to what in a sense the public would like to see.

  Chairman: What the public would like to see is an interesting concept. I would have thought that the wider range adopted by NICE is a case for the defence. If it decides that it is £50,000 a year it is quite happy that people should have it.

  Q539  Dr Stoate: Whatever debate we may have about what the threshold actually is, there must be one. You have said that it is a range and it is interpreted in different ways. Is there any research evidence about whether the current threshold as it is applied is affordable? Do we know whether or not we can carry on like this?

  Professor Devlin: Answering that question empirically is surprisingly difficult.

previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2009
Prepared 28 October 2009