HOW DOES NICE ARRIVE AT ITS DECISIONS?
100. A lack of understanding of NICE's analytical
methodology was a common theme in our evidence. North Tyneside
Health Authority argued that the credibility of guidance has been
undermined where it is unclear how the Institute reached its decision.[131]
Ealing Hammersmith and Hounslow Health Authority felt that "The
2001 Guide to the Technology Appraisal Process says very little
about how the Appraisal Committee should come to its decision.
Issues such as these could be given more prominence by making
the process of weighting the strength of evidence, relative (cost)
effectiveness and other factors more explicit".[132]
Confusion about evidence-weighting seemed widespread: Clara
McKay, principal policy adviser to the Consumers' Association,
argued that a wide-ranging independent survey of patient organisations
conducted by the Association revealed "a real concern across
the board of patient organisations not having a sense of the impact
they make on the outcome of the guidance", and this uncertainty
was not confined to patient organisations. The NHS Centre for
Reviews and Dissemination asserted that:
"Research findings at times appear to receive
less weight than submissions from other stakeholders. The lack
of clarity and transparency in the decision making process may
lend itself to accusations of bias in favour of certain aspects
of the process (e.g. input by pharmaceutical companies). The independence
of NICE is threatened by this lack of clarity."[133]
101. NICE told us quite categorically that "we
regard the evidence submitted by all of our stakeholders as of
equal value, and equal value also with the assessment report that
we receive ... from the independent academic organisations".[134]
It had recognised a lack of clarity about how its decisions are
reached, and its reports now included a "Consideration"
section which sets out how the Appraisal Committee interpreted
its evidence, and how different types of evidence influenced its
conclusions.[135] However,
we feel that this section is very brief, and has yet to offer
the sort of information that will assure readers of the validity
of NICE's processes.
102. Many concerns were raised about cost-effectiveness,
and in particular NICE's use of cost per Quality Adjusted Life
Years (QALYs), a method which attempts to provide a comparative
measure of the benefits of a treatment against its cost, considering
the extra years of life the treatment or intervention is likely
to yield per person treated, adjusted to take account of quality
of life. Dr Crayford of Croydon Health Authority was one of many
who criticised NICE for its inconsistent approach to the use of
Quality Adjusted Life Years, which he estimated NICE had employed
in only about half of its assessments.[136]
The ABPI argued that the true costs and benefits of treatments
sometimes took years to establish, and several memoranda also
made the point that NICE's calculations of cost and benefit failed
to take account of the wider societal costs and advantages of
particular treatments, including, for example, reductions in benefit
dependency and improved ability to work, both for patients and
their carers.[137]
The difficulties surrounding the new and imprecise science of
cost-effectiveness were recognised by NICE, who told us that "QALYs
are not the answer, definitely not. They are, however, the only
thing we presently have which is internationally recognised and
gives us a handle on comparisons across technologies, across disease
states and across individuals and age groups".[138]
Whether or not QALYs are used, we recommend that NICE should consider
the wider societal costs and advantages of particular treatments
and in particular the wider costs and benefits to the public purse
of reduced benefit dependency and improved ability to work both
for patients and their carers.
103. Glyn McDonald of the MS Society raised concerns
about NICE's decision-making process, describing the "creeping
establishment of a threshold for cost-effectiveness".[139]
Professor Barnett told us that "The Appraisal Committee does
not consider the threshold [for cost per Quality Adjusted Life
Year] and has not been given instructions about a threshold and
has not discussed a threshold per se at all".[140]
However, the King's Fund argued that, whether it is arrived at
implicitly or explicitly, "NICE has a responsibility explicitly
to justify the threshold that appears to have emerged as its decision
rule".[141] The
decision about how much money the NHS should be prepared to spend
to improve an individual's health or save their life is fraught
with difficult judgements which go beyond the technical, as illustrated
by Professor Walley:
"... it may be that a new therapy is uniquely
effective but constitutes very poor value for money, i.e. a conflict
between clinical effectiveness and cost-effectiveness. Whether
it should be funded is therefore a matter for judgement or appraisal,
to include the financial, social, ethical and legal implications
of its use or non-use. The rules or values for such appraisal
are not clear. We require open debate about how we value such
trade-offs."[142]
104. This was clearly understood by Professor Rawlins:
"We certainly recognise that some of our decisions, or the
basis upon which they are made, require value judgements. We fully
recognise with those value judgements that we are no more competent
than anybody else to make them".[143]
We note NICE's plans to establish a Citizens' Council composed
of 'ordinary men and women around the country' to advise on these
value judgements. [144]
We agree with the many witnesses who argued for a review of NICE's
appraisal methodology, and the publication of clear criteria.[145]
We therefore recommend that NICE, aided by the Department of Health,
should conduct a review of its methodologies for assessing clinical
and cost-effectiveness, which should result in the publication
of a set of clear and consistent criteria for the assessment of
both aspects. This should include a description of the weighting
given to different types of evidence, a detailed argument for
its use of QALYs, and the impact of both cost- and clinical effectiveness
on the final determination, including any cost-effectiveness
'thresholds'. In tandem with this, NICE should work to strengthen
its cost-effectiveness evidence base by encouraging pharmaceutical
companies to collect this type of data routinely.
COST-EFFECTIVENESS AND AFFORDABILITY
105. A further area of confusion and concern over
NICE's remit and methodology relates to the issue of affordability.
Although Lord Hunt and NICE were clear that affordability was
not an issue for NICE, but was rather a matter for government,
some witnesses felt that the boundary between cost-effectiveness
and affordability was very blurred. Mr Cayton of the Alzheimers'
Society argued that: "one of NICE's biggest problems was
being given this second task, which in a sense people have referred
to as really a political decision, of doing cost-effectiveness
as well as clinical effectiveness".[146]
Dr Cunningham of Lambeth, Southwark and Lewisham Health Authority
suggested that:
"In a sense NICE was set up to give robust guidance
on the basis of evidence, and it appears now to be almost being
asked to do resource allocation by the Department of Health. It
seems sensible to separate these two functions more clearly, particularly
the selection of the portfolio which NICE considers and the decisions
which are made when NICE has made its pronouncements."[147]
106. As the Government has now issued directions
for mandatory implementation of all NICE guidance, the funding
and provision of treatments would appear to "follow automatically"[148]
from a positive NICE appraisal, or be automatically withheld following
a negative one. This may create the impression that, because
it is likely that NICE recommendations will be implemented whatever
the cost, NICE is essentially making decisions about how money
is spent in the NHS, or ministers are somehow making decisions
through NICE. As Professor Walley put it, "because it is
not apparent when the Minister has made a decision that the funding
will be available, there is a perception, which I do not think
is actually true, that at some point the Minister has influenced
the process before it has got to the final stage".[149]
This confusion may be consolidated by the wording of NICE's Framework
Document, which states that NICE should have regard to "the
effective use of available resources", and by the fact that
NICE calculates and publishes the net budget impact of its recommendations,
although NICE is clear that this plays no part in the appraisal
process.[150]
107. Decisions about the affordability of NICE's
recommendations are, we believe, properly a matter for government.
We also believe that there is a need for decisions about the affordability
of NICE's recommendations to be seen to be made entirely separately
from NICE's decisions about clinical and cost-effectiveness. While
we have received no compelling evidence that these decisions are
not being taken independently, the widespread perception that
the three elements are linked urgently needs to be dispelled.
We therefore recommend that the Government should take steps to
clarify its own role in taking decisions about whether or not
individual pieces of NICE guidance will be funded. One solution
would be for the Department of Health to issue a separate
document alongside each piece of NICE guidance, stating the costs
(both financial and staffing) of implementation of the guidance,
and on the basis of this indicating whether or not, and in
what circumstances, the guidance was to be implemented.
105 Ev 128, citing Raftery, J, Analysis of guidance
on health technologies, BMJ 2001; 323:1300-1303. Back
106
Ev 47 (National Cancer Alliance). Back
107
Ev 236. Back
108
Ev 54. Back
109
Ev 5. Back
110
Ev 196. Back
111
Ev 199; Ev 244. Back
112
QQ290-93; QQ348-55. Back
113
Ev 131. Back
114
Ev 247. Back
115
For example, Ev 4 (Consumers' Association); Ev 248 (Aventis). Back
116
Ev 91. Back
117
Q134 . Back
118
Ev 44. Back
119
Ev 131. Back
120
Ev 17. Back
121
Q9. Back
122
Q165. Back
123
Q261. Back
124
Ev 54-55. Back
125
Q259. Back
126
Ev 222; Ev 243. Back
127
Ev 107. Back
128
Ev 166. Back
129
For example, Ev 239 (Professor Maynard); Ev 54-55, Ev 56 (The
Alzheimer's Society). Back
130
Q450. Back
131
Ev 77. Back
132
Ev 203. Back
133
Q6; Ev 8; Ev 222. Back
134
Q356. Back
135
Q356. Back
136
Ev 72. Back
137
ABPI give the example of the medicine EPO, used to treat patients
with kidney failure, which had an estimated QALY at launch in
1990 of £103,000 but which, following changes in the management
of patients, now has a QALY of around £3-9,000 (Ev 92). For
discussion of wider costs and benefits see, for example, Ev 233
(The Pituitary Foundation), Ev 231 (RCGP). Back
138
Q447. Back
139
Q110. Back
140
Q447. Back
141
Ev 210. Back
142
Ev 23. Back
143
Q315. Back
144
Q315. Back
145
Ev 2 (Consumers' Association); Ev 246 (Aventis). Back
146
Q123. Back
147
Q146. Back
148
Q82. Back
149
Q82. Back
150
Q312. Back
