Examination of Witnesses (Questions 525
- 539)
THURSDAY 18 OCTOBER 2007
PROFESSOR NANCY
DEVLIN AND
PROFESSOR JOHN
APPLEBY
Q525 Chairman: I welcome the witnesses
to the fifth evidence session in our inquiry into NICE. Perhaps
for the record the witnesses will introduce themselves and the
positions they hold.
Professor Devlin: I am Nancy Devlin.
I am professor of economics at City University where I work at
the City Health Economics Centre. I am also a senior associate
at The King's Fund.
Professor Appleby: I am John Appleby,
visiting professor at City University and chief economist at The
King's Fund.
Q526 Chairman: To start, I ask a
question of both witnesses: what does your research reveal about
the cost-effectiveness threshold used by PCTs and how do you believe
it relates to that used by NICE?
Professor Appleby: Perhaps I may
first explain what the study is about. It is a very small feasibility
study conducted jointly with NICE. First, could we identify decisions
made by PCTs either to invest or perhaps disinvest in new services?
From that, could we estimate some sort of implied cost per QALY
for the decisions taken? That is the background to the study.
Clearly, it is linked to concerns of NICE, this Committee and
many others about what the threshold should be and how much we
are willing to pay to get health benefit from the NHS. We looked
at only seven or eight PCTs and identified the decisions they
made. We then chose some of the decisions to invest or disinvest
and proceeded with the work. We found that in some cases we could
estimate a cost per QALY; in other cases the decisions were based
on the notion that a service which was to be introduced or not
introduced was clinically more effective than anything else and
cheaper, in which case it was dominant. We had very few observations
and so I am not sure we can arrive at anything too conclusive
about what the threshold is out in the real world in NHS decisions.
Q527 Chairman: Do you agree with
that, Professor Devlin?
Professor Devlin: Yes. It was
a feasibility study. The purpose of commissioning the study was
to determine whether it would be possible in a larger study with
more observations to identify the threshold that applied at PCT
level. I think it demonstrated that that was feasible. It is not
easy to do it, however. A lot of the decisions one observes are
those where PCTs take what may be seen to be eminently sensible
decisions either to disinvest in services which are very poor
value for money or to invest in services which are extremely good
value for money. It is easy to identify newly-added or newly-eliminated
services, but that will not necessarily reveal marginal services
and the willingness to pay at PCT level to gain a quality adjusted
life year. One can certainly identify the services and assign
to them a cost per quality adjusted life year gained, but a much
larger-scale study would be required in order to come up with
precise estimates of the thresholds that arise from PCT budget
constraints.
Professor Appleby: We have to
understand the position of PCTs here. They are going for relatively
easy and more obvious things in terms of disinvestment. The sort
of information we received was a fairly wide spread of values,
which is what we would expect. In a larger study we would hope
to start to identify more decisions and narrow that range, but
it will always be a range; there will not be a single number out
there on which PCTs operate.
Q528 Dr Naysmith: Professor Appleby,
why do you think they are going for the easier things? Is it lack
of information or do they just want to chop something quickly?
Professor Appleby: It is the low-hanging
fruit argument, that is, if the evidence is fairly strong and
something is clinically pretty ineffective there will not be much
argument about it. Like NICE, PCTs can find themselves in a difficult
position with their public and residents if they stop funding
some service or whatever, so that is in the back of their minds.
Professor Devlin: I think that
investing in services that are dominant and disinvesting from
ones that are dominated is an entirely logical thing to do.
Q529 Dr Naysmith: One of the things
we have recommended many times is that that is something NICE
should do more of than in the past?
Professor Devlin: The other matter
that has been observed and makes the research very challenging
is that PCTs are not always well informed of some of the data
that could support them in decisions regarding either investment
or disinvestment. What slightly confuses the picture somewhat
is that cost-effectiveness is not the only criterion that matters
to PCTs, in the same way that it is not the only criterion that
matters to NICE. Therefore, unpicking the basis of a decision
that is made is also important.
Q530 Chairman: In evidence to us
Professor Peter Smith suggested that the PCT threshold was likely
to be nearer £20,000 than £30,000. Do you believe there
is any robust evidence? Do you agree with what he says?
Professor Appleby: I think we
have to be very careful. I emphasise that our study was very small
and involved only eight PCTs. We looked at about nine different
sorts of decisions that those PCTs had made. By the way, they
were often the same decision. Some co-ordination going on between
PCTs in going for some of the same things. I do not believe from
the evidence we have we can say one way or the other whether Peter
Smith's figure is the right one or is correct in some sense.
Professor Devlin: There is no
doubt in my mind that this is the sort of research that needs
to be done to inform what NICE's response should be.
Q531 Chairman: In your evidence you
more or less suggest to that in taking these types of decisions
the NHS should have the equivalence of the committee at the Bank
of England that sets the interest rate. Should NICE thresholds
be more explicit, and who should set them?
Professor Devlin: There are two
aspects to explicitness in a sense. One is the way in which the
threshold is stated, whatever it is. What we have noted in our
evidence is that there are a number of statements, which are inconsistent
in very subtle ways, about what the NICE threshold is. They also
emphasise the range and it is not always clear what the interpretation
of that range is or, from the perspective of somebody who sits
outside the NICE context, how that range is being applied in decision-making.
Therefore, one aspect of explicitness is how the prevailing threshold
is expressed. The other aspect of explicitness is that the threshold
has no explicit basis or location in evidence. I believe that
an explicit threshold should be both evidence-based and clearly
expressed.
Professor Appleby: Together with
David Parkin, who is also a professor at City University, we wrote
a BMJ editorial on this matter. In our evidence to you
we have said this is a fundamental issue that touches on what
services the NHS should and should not provide and it goes to
the heart of decisions about the willingness to pay to get benefit
from the health service. As my colleague has said, there is some
fudging going on as far as NICE and others are concerned not just
about what the threshold is but how it is applied. There is no
empirical evidence to support the threshold as stated by NICE;
there is no real theory behind it, and yet it is such a fundamental
issue. It is not just a technical issue in the sense of finding
out some numbers and then deriving a threshold, crank the handle
and that is it; it involves social value judgments and so on.
We believe that either a separate independent organisation or
at least an independent process can be set up by which the issues
with which NICE and this Committee have been grappling are discussed,
the research is commissioned and carried out and it is transparent.
The public should also be involved because in the end it is an
issue for them; it is their money that goes into the NHS and these
are decisions to be made about how best that money is used. It
is a finite budget. This decision will never go away no matter
how much money we have.
Q532 Chairman: The article in the
BMJ suggests in a sense that NICE is not as independent
as it should be.
Professor Appleby: I hope we did
not suggest that.
Q533 Chairman: I believe this is
important in terms of this inquiry. What independence do people
perceive NICE has?
Professor Devlin: I do not think
this was intended to be critical of NICE. In a sense I think that
NICE has been placed in an invidious situation. It does an excellent
job at assessing the clinical and cost-effectiveness evidence
in an enormously rigorous and diligent way. What we say is that
the threshold that must be applied to make a judgment on the basis
of that evidence is an extremely difficult issue. We are suggesting
that NICE's threshold is not just a matter for NICE alone: it
is not just NICE's business. If NICE makes a mandatory decision
that PCTs must implement it completely alters the bundle of services
which PCTs can afford to deliver. That affects the services that
all patients can potentially consume or benefit from, so NICE's
threshold should have an input from the sector and a much wider
range of expertise.
Professor Appleby: NICE does an
amazing job given the task it has been handed, but that job is
more of a technical nature in collating and assessing evidence,
commissioning research to fill gaps in that evidence and so on.
But when it comes to the threshold that should be a separate job,
as it were. We argued that maybe NICE should be handed a threshold
to which it should operate. At its inception it realised it could
not do its job unless it had some sort of threshold at which point
it said something was or was not cost-effective or that it did
or did not recommend it. But at the moment that is buried in the
work NICE does. Our argument is that it should be recognised as
a separate matter and there are different ways to deal with it.
Q534 Dr Naysmith: What is the answer
to the question put by the Chairman? Who should set the threshold
and where should the decision-making process lie? If we are agreed
that it should not be the Department, where should it lie?
Professor Appleby: We suggested
that the parallel should be the MPC. There could be a separate
organisation, group or at least process and that would set the
threshold. It is not just a one-off exercise; it will have to
be updated every year. New evidence will emerge. The public's
values—how much they are willing to pay and so on—will
also change from time to time, so it includes a whole of things
and it is an ongoing process.
Q535 Dr Naysmith: It is a good idea
to say that somebody else should make the decision, but working
out exactly who that should be is quite difficult.
Professor Appleby: Yes, but I
am sure it is not beyond the wit of us to devise something.
Q536 Dr Naysmith: The Monetary Policy
Committee is full of economists, bankers and so on, but do we
want former trust executives or former Ministers on this body?
Who should it be?
Professor Devlin: Perhaps it should
be a range of expertise: economists and people involved in budget-constrained
decisions in PCTs as well as people in NICE. I do not believe
that the question of the appropriate expertise on such a committee
is insurmountable.
Q537 Dr Naysmith: It is quite difficult
to do.
Professor Devlin: It is not an
easy question, but what we suggest is that it really requires
a concerted effort that perhaps stands slightly outside NICE's
remit. It is also a dynamic issue. A lot of the discussions around
the threshold have tended to focus on either a single figure or
a statement about a range. Those statements have been made since
1999. It is completely unfeasible that whatever the appropriate
threshold or range was in 1999 is the same now. Regardless of
what determines or drives that threshold, whether it is society's
willingness to pay for a QALY or the threshold revealed by budget-constrained
decisions, those factors will change year on year.
Chairman: In the end are we not talking
about the terms of reference used by NICE? Clearly, if the thresholds
were set outside and handed to NICE it would work to those thresholds.
Dr Naysmith: It does at the moment.
Q538 Chairman: Yes, but it does not
admit it. What would be the change in reality?
Professor Appleby: I was trying
to illustrate that in terms of what my colleague said originally
about how the threshold to which NICE operates is difficult to
define. There is no real evidence to support the threshold it
uses, and the practice of using it is pretty fuzzy. I cannot now
remember the numbers, but quite a lot of decisions are beyond
the £30,000 QALY range that it recommends. I cannot remember
how many but it is a minority. As far as I understand, NICE says
that those are the circumstances where special factors come to
bear. There appear to be an awful lot of special cases here which
throws into doubt the range that NICE talks about. Therefore,
there is an issue about making that more transparent. I suppose
that it could be done through NICE. Perhaps one method is to change
NICE's terms of reference, but we were thinking of taking it a
step further and saying that NICE has another job to do, that
is, just evaluations. It then applies a threshold which it is
handed. I suppose that as a society we will never agree a single
number, but at least we can get closer to what in a sense the
public would like to see.
Chairman: What the public would like
to see is an interesting concept. I would have thought that the
wider range adopted by NICE is a case for the defence. If it decides
that it is £50,000 a year it is quite happy that people should
have it.
Q539 Dr Stoate: Whatever debate we
may have about what the threshold actually is, there must be one.
You have said that it is a range and it is interpreted in different
ways. Is there any research evidence about whether the current
threshold as it is applied is affordable? Do we know whether or
not we can carry on like this?
Professor Devlin: Answering that
question empirically is surprisingly difficult.
| 
