Examination of Witnesses (Questions 340
- 359)
WEDNESDAY 30 JANUARY 2002
PROFESSOR SIR
MICHAEL RAWLINS,
MR ANDREW
DILLON AND
PROFESSOR DAVID
BARNETT
340. Who is on the NCCHTA? Who decides who goes
on that board? There are panels and boards all over the place.
I am getting slightly confused as to who overlaps with whom and
who appoints to which group. It is all becoming very incestuous
almost.
(Mr Dillon) No, I do not think it is. In fact I think
the fact that NICE operates in the way that we do—the example
of the external commissioning of our evidence—is an example
of the robustness that is built into the process. NCCHTA consists
of staff who are employed by the NHS Research and Development
Programme. The office is led by Professor Kent Woods at—
341. They are the same people as run the Horizon
Scanning Group.
(Mr Dillon) No. No, they are a different group. They
are quite different. The Horizon Scanning staff have academic
contracts at the University of Birmingham and they work on the
basis of a contract that they receive from the NHS Research and
Development Programme.
342. Yes. I am sorry, that is where I was coming
to.
(Mr Dillon) Yes, that is the link.
(Professor Sir Michael Rawlins) The Health Technology
Assessment Units are also all part of the NHS R&D Programme,
so in one sense are a free good to NICE.
Dr Naysmith
343. We were told by a representative from Glaxo
WellcomeSmithKline that industry pharmaceutical firms are out
to check original technology assessment reports. It must be part
of that procedure you first outlined. Are you entirely happy with
this? Could it be done better by independent people such as the
people who run the British National Formulary or the Drugs and
Therapeutics Bulletin.
(Mr Dillon) I am not quite sure I follow the introduction
to the question. We do not expect the industry to—
344. Let me say a little bit more. They are
going to look through it. Do they always confine themselves purely
to factual errors—numbers in the wrong place, decimal points,
perhaps—or do they ever make comments on other things in
the report?
(Mr Dillon) They have exactly the same opportunity
as any other stakeholder to make whatever comments they want but
generally their comments fall into three categories. The first
is where they believe there are factual errors, and that might
be numbers that are misquoted—
345. Not just factual errors. That can be a
chance to comment on other people's comments.
(Mr Dillon) The second category is where they may
say that there is evidence which exists and which has not been
included, and they draw the attention of the Institute to that.
The third is where they believe that the way in which the evidence
has been laid out is unhelpful. But they are perfectly entitled
to make any and all of those observations, and it is a matter
for the Institute and the Appraisal Committee to decide whether
or not to take them into account.
346. So it is actually much more than checking.
(Mr Dillon) Yes. We do not attempt to constrain them,
because to do that might potentially lead to some genuine error
or omission in one of the assessment reports not being drawn to
our attention which would clearly disadvantage the appraisal.
347. Purely statistical or factual or scientific
mistakes could be looked at and corrected by some other body,
could they not?
(Mr Dillon) Well, they already are. If they get through
to that stage, then it means that they have been missed by the
authors, they have been missed by the peer reviewers and they
have been missed by the National Coordinating Centre for Health
Technology Assessment. So the number of occasions when that occurs
are actually quite small, and it is really at that stage that
if factual errors have crept in we want to make sure that we know
about them, because we are very close at that stage to the Appraisal
Committee seeing the document for the first time.
Andy Burnham
348. I would like to take forward Dr Naysmith's
line of questioning, to think about the original data set and
evidence base that you get provided from the pharmaceutical companies
themselves when you originally submit data on a technology or
a treatment. Sir Iain Chalmers, who is a fairly eminent commentator
on these matters, has stated in evidence to the Committee that
"important data having been withheld from NICE by companies
whose interests would not be served by submitting it". Do
you think that happens?
(Mr Dillon) When we invite companies to make submissions,
we ask them to sign a declaration that they have submitted to
us all relevant information. If they sign that declaration, we
accept it. We have no means of challenging it. If we ask someone
a question and they give us an answer, unless we are presented
with clear evidence from whatever source that that declaration
is wrong then we have to operate on the basis of trust that we
have received all the relevant information.
349. Might I suggest that that might sound a
little too trusting. Sheila Bird, the statistician on your own
Appraisal Committee, said in evidence "NICE is not afforded
full access [to information] by all pharmaceutical companies".
It seems that this is something that people are suggesting to
us, and yet when we put this point to the ABPI last week they
categorically said it does not happen. Why the difference in view?
(Mr Dillon) I do not think it is a difference of view.
I think the reference that you are quoting there was a reference
to the appraisal that we conducted on the beta-interferons and
glatiramer acetate for multiple sclerosis. As I think the Committee
knows, we commissioned an additional economic model as part of
that appraisal. We asked a consortium of British universities
to construct independently an economic model. They would have
preferred to have had access to what is described as "patient
level data"; in other words, the original data which resulted
from the clinical trials conducted by companies which is then
summarised and presented into the public domain. The companies
we asked for that information from decided not to release it to
us. Their reasons for doing so were that it was confidential information;
that to release it to us, in their view, would breach the confidentiality
agreement that they had in effect entered into with the patients
who were the subject of those clinical trials.. They also argued
that the data was not relevant to the inquiries that we were making
or the process of constructing the economic model.
350. Were you happy with those arguments?
(Mr Dillon) No. The view that was being put to us
by the consortium of universities doing the modelling was that
they would have preferred to have had that data and so on that
basis we would have preferred that the company supplied it.
351. Has Sir Iain Chalmers made these concerns
known to you?
(Mr Dillon) Yes, I had a conversation with him at
the Institute's conference in December and he said that he had
heard that things had been said that led him to suspect that this
was the case.
352. Have you taken any action on that?
(Mr Dillon) Unless we are presented with very clear
evidence that effectively challenges the statements that are made
by companies that they have submitted all relevant evidence, I
cannot take action. I have to have the name of the appraisal and
the name of the company and the details of the data that the individual
who is expressing the concern thinks has been withheld from us.
If I am supplied with that information, then I will make inquiries
with the company concerned.
353. When you say presented, could you not go
out and seek that? If people suggest that there is a problem,
could you not, rather than waiting—
(Mr Dillon) If I am provided with the details of the
appraisals and the companies concerned, I will make those inquiries.
But I cannot just randomly approach companies and say, "Have
you done what in effect you have already told me you have done
by virtue of the declaration that you have made?"
354. Do you think there might be a flaw in the
system here that it is taken on trust, that there may be the situation
where one side is saying one thing and one another? Would you
welcome greater powers to be able to take information, if you
deemed it was necessary to the work you were doing?
(Mr Dillon) In one sense we are in the same position
as the licensing bodies for pharmaceuticals, who seek all the
relevant information from the companies and they seek commitments
from the companies that they have provided that information. But
I think it is the case—Professor Rawlins will know better
than I—that in the case of the Medicines Control Agency
that request is backed up with the force of statute. The Institute's
position is different. Our request is not backed up with the force
of statute.
355. Would you welcome that?
(Mr Dillon) Yes, it would obviously be better if it
was.
Julia Drown
356. Some forms of therapy and some patient
groups do not have the lobbying capacity that the pharmaceutical
companies do that we have just been discussing. How do you make
sure that their views are fed in and are regarded in your processes?
(Mr Dillon) By making sure, first of all, that we
do what we say, that we act what we set out as being the way in
which the Institute approaches taking evidence; that is that we
regard the evidence submitted by all of our stakeholders as of
equal value and equal value also with the assessment report that
we receive, as we have discussed, from the independent academic
organisations. I mentioned earlier that we worked with patient
organisations to identify not just the best way in which they
can engage with the Institute but also the nature of the evidence
that they can supply to us. It has been the case that patient
organisations have expressed concern since NICE was established
about two things . . . or perhaps three. One is that they are
uncertain about what we want to get from them, what is it that
they know and that only they know, in other words, that we do
not get from the work done by the academic organisations or from
other sources that we need to have, specifically that the Appraisal
Committee needs to have, to interpret the evidence. Secondly,
how do they get involved? How can they be certain that the Appraisal
Committee is taking what they have said into account? They attend
part of but not all of the Appraisal Committee meeting. They can
go to the Committee and talk through their evidence, they can
enter into discussions with committee members, but there is a
point at which they withdraw to allow the committee to consider
the evidence further and to debate. How can they be certain that
their evidence is being taken into account in that part of the
committee meeting? Finally, they are concerned about resourcing.
Some of the patient organisations we work with are very small,
they have very limited resources, and they are concerned about
the cost of engaging with the Institute, particularly where we
are looking at a series of appraisals in which they might be interested.
We have worked with them to try to deal with as many of those
concerns as we can, partly through the process, partly through
supporting them when they need it directly, when they come and
ask us for assistance in understanding our processes, partly by
making a decision to provide some financial support to their work—it
is only a small contribution, in effect a contribution to the
costs they incur, £400 per organisation per appraisal. It
is the best that we can do with the money we have got available,
but we recognise that there is a need for it and we have tried
our best to help them. We have tried to reassure them that what
they provide us with is used every time particularly to inform
the way in which the committee interprets the other evidence that
it has received.
357. Particularly with regard to the patient
groups, why can they not sit through the whole process, so they
can see if their particular points are being raised and continued
through the whole process?
(Mr Dillon) Partially, because we take and use evidence
which is submitted to us as commercial in confidence.
358. What if they were willing to sign up to
the same commercial in confidence criteria that you have to sign
up to?
(Mr Dillon) The way in which the Institute operates
at the moment is on the basis that if companies come in complete
confidence and give us data on which they may already have decided
on a publication strategy for or on which they have decided that,
for whatever reasons, they do not wish to release into the public
domain—and, as the owners of the data, they are entitled,
it seems to me, to do so- then we having given them a commitment
that we will respect the confidentiality of that data and expose
that in relation only to working of the Appraisal Committee. Unless
in some way that can be changed . . . it is a limiting factor
in terms of the extent to which we can expose the work of the
Appraisal Committee to the general public. But the members of
the patient organisations that come to Committee meetings themselves
see the data that is submitted to the Institute, so there is a
distinction here between the general public having access to the
committee and patient organisations, who are formal consultees,
who have signed a confidentiality agreements. Those patient and
clinical representatives sit with the committee for a significant
proportion of the discussion, but there comes a point at which
the committee needs to have the time to consider everything that
it has heard, including all the representations that have been
made directly to it, and to consider its decision This is a short
period out of the whole appraisal.
(Professor Barnett) I want to reinforce what has just
been said. First of all, I made a career of listening to patients,
which is what I do in my day job, and I think we try to make it
abundantly clear to the patient and carers who arrive, that we
will do so and that is exactly what we do. They are also present,
as Andrew has just pointed out, for a substantial proportion of
the time during which that technology is being discussed. They
engage directly with committee members; they ask questions and
are asked questions of. I think that is something of which most
of the patient and care organisations who engage with us are fully
aware and quite happy with.
359. We have heard it was the cancer charities
who felt they did not have evidence that their views had been
fed in, that they had been dismissed, and they personally had
not had that feedback to know why they had been overridden.
(Mr Dillon) Part of the problem there—and we
began to realise this in the past as a result of the discussions
that we were having with patient organisations and also other
stakeholders—was that after the first meeting of the Appraisal
Committee, where the draft guidance is prepared (it was called
a provisional appraisal determination), we set out the evidence
that we received, we describe the technology and we write the
draft conclusions in the form of the draft guidance, and what
we realised we were not doing in the earlier appraisals effectively
in the document was connecting the two things; in other words
setting out in writing how the evidence that we put in the document
allowed us to reach the conclusions that we set out in the guidance.
What we have started to do is to write a section that we call
"Consideration". It is in that section that we describe
how the committee has interpreted the evidence and that gives
us the opportunity to describe how patient evidence influenced
the conclusion.
| 
