Memorandum by Sir Iain Chalmers (PI 29)
PERSONAL STATEMENT
I am editor of The James Lind Library, a public
website explaining and documenting the evolution of fair tests
of treatments in health care (www.jameslindlibrary.org). I am
also a co-convenor of the James Lind Alliance, a coalition of
organisations representing patients and clinicians collaborating
to confront important, shared uncertainties about the effects
of treatments in health care (www.jameslindlibrary.org/jla.html).
I have a longstanding interest in improving the quality of evidence
about the effects of treatments in health care, an interest that
was prompted initially by realising that I had been harming my
patients by relying on "eminence-based" rather than
"evidence-based" guidance. I have previously submitted
evidence to the committee for its enquiry into the National Institute
for Clinical Excellence (NICE). I am submitting this memorandum
in an individual capacity. I would be willing to give oral evidence,
if the Committee decides that this might be helpful.
SUMMARY OF
MAIN POINTS
AND RECOMMENDATIONS
IN THIS
EVIDENCE
Many patients are suffering and dying unnecessarily,
and resources for health care and research are being wasted, because
of:
(1) perverse influences on the clinical research
agenda;
(2) failure to take systematic account of
existing knowledge when planning clinical research;
(3) biased under-reporting of research;
(4) insufficient public and professional
access to trustworthy evidence about treatment effects and information
about relevant ongoing research;
(5) government-endorsed secrecy about the
effects of licensed drugs; and
(6) the introduction of inadequately assessed
treatments within the NHS.
The pharmaceutical industry inevitably contributes
substantially to these serious problems because of the very prominent
role it plays in clinical research and information about the effects
of treatments.
This unsatisfactory state of affairs can be
addressed by:
(1) assessing the extent to which the clinical
research agenda reflects the priorities of patients and practitioners
for developing and evaluating healthcare innovations;
(2) ensuring that systematic reviews of existing
evidence are the basis for planning new research;
(3) requiring public registration of clinical
trials with unique identifiers and outlawing biased under-reporting
of health research;
(4) increasing the capacity for preparing,
maintaining and disseminating systematic reviews of the effects
of treatments, and for providing public information about ongoing
research;
(5) promoting public access to the results
of all clinical studies of the effects of licensed drugs; and
(6) restricting the use of inadequately evaluated
drugs within the NHS to the context of controlled evaluative studies,
thus reducing unacceptable uncertainties about their cost-effectiveness.
1. DRUG INNOVATION
Over the past half century, better health care
has been responsible for between a third and a half of the increase
in life expectancy, and an average of five additional years free
of chronic health problems (Bunker et al 1994). Compared
with all previous eras of health care, this is a remarkable record
of success, and drug treatments have made their contribution to
it. Indeed, the extent of these achievements over the 50 years
before the human genome was sequenced will be the yardstick against
which we will come to judge the validity of current claims that
the genetic revolution promises even greater benefits to patients.
In spite of these heartening estimates of the
effects of health care, the public could have obtained better
value for the substantial private and public resources invested
in research intended to improve health, particularly during the
past 20 years. During this time, very few newly approved drugs
have offered worthwhile advantages to patients over previously
available options (International Society of Drug Bulletins 2001),
a problem that has been noted within the pharmaceutical industry
(for example, Horrobin 2003) as well as outside it (for example,
Garattini and Bertele 2002).
Over the past two decades, industry and academia
have increasingly worked in partnership to promote commercially
dictated objectives. This has had perverse influences on the clinical
research agenda (Chalmers 2000). For example, most clinical trials
relating to osteoarthritis of the knee are commercial studies
of drugs. When patients, rheumatologists, physiotherapists and
general practitioners were asked to identify their priorities
for research, they made clear that they had little need for any
more such studies, but instead wanted more rigorous evaluations
of physiotherapy and surgery, and assessments of the educational
and coping strategies that might help patients to manage this
chronic, disabling and often painful condition more successfully
(Tallon et al 2000).
Studies evaluating the effects of out-of-patent
drugs and non-drug interventions cannot expect to attract support
from industry. The opportunity costs of acquiescing in industry's
pervasive influence on the clinical research agenda is that many
questions of importance to patients, professionals and the NHS
more generally, are not being addressed.
Recommendation
Independent evaluations should be commissioned
to assess the extent to which research reflecting the priorities
of the pharmaceutical industry is promoting or jeopardising the
development and evaluation of innovations regarded as important
by patients, practitioners and the NHS itself.
2. THE CONDUCT
OF MEDICAL
RESEARCH
An editorial in the British Medical Journal
entitled "the scandal of poor medical research" has
called for "less research, better research and research done
for the right reasons" (Altman 1994). In part, the scandal
reflects the perverse influences of commercial interests on the
research agenda. For example, there is evidence suggesting that
some studies sponsored by industry have been designed deliberately
to yield results favourable to particular drugs (Djulbegovic et
al 2000). This can be done by withholding a comparison treatment
known to help patients, or giving comparison treatments in inappropriately
low or high doses.
The scandal of poor medical research would be
reduced if researchers were required to take systematic account
of existing knowledge when planning and reporting their research.
Because this job is not done well, patients have been denied treatments
which could have benefited them, while other patients have received
treatments that were either of no material benefit, or actually
harmful (Antman et al 1992).
For example, after reviewing the experience
of thousands of patients who had participated in controlled trials
of new calcium-blocking drugs given to people experiencing a stroke,
a Dutch team found no evidence to support the increasing use of
these drugs in practice, or for the large numbers of clinical
trials that had been performed (Horn and Limburg 2001). Furthermore,
when they subsequently prepared a systematic review of the relevant
animal studies they found that these had never suggested that
the drug would be useful in humans (Horn et al 2001).
These serious problems would be reduced if all
funders of research followed the lead of the Medical Research
Council in requiring those seeking support for new research to
refer explicitly to systematic reviews of existing evidence, showing
that proposed additional research is necessary and that it is
building on previous experience. Because research ethics committees
have tended to ignore this principle, patients and the public
have not been protected effectively from unethical research (Savulescu
et al 1996).
Recommendation
As a precondition for funding and ethical approval,
funders and research ethics committees should be required to ensure
that proposals for new research are supported by systematic reviews
of existing evidence.
3. THE PROVISION
OF DRUG
INFORMATION AND
PROMOTION
Reliable evidence about the effects of drugs
on patients depends on systematic reviews of all relevant, unbiased
studies. Such systematic reviews are now used extensively by the
National Institute for Clinical Excellence (NICE) and in clinical
guidelines. The increased amount of evidence being assembled in
systematic reviews is a major advance. However, these reviews
may overestimate the beneficial effects of drugs and underestimate
their adverse effects because the results of some key studies
are not publicly available: studies that have not found beneficial
effects of drugs, or which have found adverse effects, are less
likely than others to be published (Dickersin 1997).
Biased under-reporting of clinical trials kills
patients and wastes resources. For example, had all the completed
studies of drugs to reduce heart rhythm abnormalities in patients
having heart attacks been reported, tens of thousands of deaths
from these drugs could have been avoided. To their credit, Dr
Cowley and his colleagues in Nottingham pointed out how an unpublished
study done there 13 years previously might have provided an early
warning of this iatrogenic disaster. "When we carried out
our study in 1980", they reported, "we thought that
the increased death rate was an effect of chance—The development
of lorcainide was abandoned for commercial reasons, and this study
was therefore never published; it is now a good example of "publication
bias" (Cowley et al 1993).
Recommendations
All clinical trials done within the NHS should
be registered publicly at inception, and assigned an International
Standard Randomised Controlled Trial Number (www.controlled-trials.com).
Pharmaceutical companies wishing to perform
trials in the UK should be required to publicly endorse and comply
with the Good Publication Practice for Pharmaceutical Companies
(Wager et al 2003 and www.gpp-guidelines.org).
Consideration should be given to introducing
legislation—comparable to that in Spain—requiring
pharmaceutical companies to publish all results of all randomised
clinical trials involving NHS patients.
4. PROFESSIONAL
AND PATIENT
EDUCATION
Health professionals and patients alike face
an overload of information about the effects of treatments. This
problem is compounded by the fact that it is usually impossible
to interpret the significance of the findings of individual research
studies because researchers rarely set the results of new research
in the context of systematic reviews of all other relevant data
(Clarke et al 2002).
Welcome support for the training and other infrastructure
needed to prepare and maintain systematic reviews has been provided
by all UK governments since 1991, largely through the NHS R&D
Programme and NICE. The results of these analyses have also been
increasingly successfully made available for professionals and
patients, for example through the Cochrane Library, The Centre
for Reviews and Dissemination, the NHS Health Technology Assessment
Programme, NICE, NHS Direct, and the National electronic Library
for Health.
In spite of this encouraging progress, however,
current levels of support for preparing and maintaining systematic
reviews remains dangerously inadequate. Professionals and patients
have ready access to only a small proportion of existing evidence
about the effects of drugs and other treatments. As systematic
reviews of existing evidence are an essential (albeit not sufficient)
component of the foundation for informing professional practice,
patient choice, and policies within the NHS, support for this
work must be increased substantially.
Furthermore, when systematic reviews of existing
evidence reveal that there are important uncertainties about the
merits of treatments, healthcare professionals and patients should
be able to find out more easily whether any ongoing research is
addressing these uncertainties, and how they may be able to contribute
to these studies.
Recommendations
The Department of Health, the Medical Research
Council and the medical research charities should increase substantially
their current levels of investment in systematic reviews.
The Department of Health should ensure that
patient-friendly information about ongoing clinical trials within
the NHS is made readily available.
5. REGULATORY
REVIEW OF
DRUG SAFETY
AND EFFICACY
Continued lack of public access to information
about the effects of licensed drugs is incompatible both with
current public expectations and with the government's professed
commitment to promoting informed patient choice and rational use
of resources within the NHS. Denial of access to information held
by the Medicines and Healthcare products Regulatory Agency (MHRA)
puts the interests of pharmaceutical companies ahead of those
of patients and prescribers. This is particularly indefensible
in the light of evidence of that regulatory agencies, supposedly
established to protect the public, are acquiescing in biased later
publication of the information they hold (Melander et al
2004).
For example, after the safety of human albumin
solution had been called into question in a systematic review
of published and unpublished controlled trials, the Medicines
Control Agency (MCA, which preceded the MHRA) refused to disclose
the evidence that had led it to grant a licence for the product
(Roberts et al 1998). The government colluded with the
manufacturers of evening primrose oil in suppressing information
suggesting that the drug had no useful effect in eczema. Five
years later, the MCA withdrew the marketing authorisation for
evening primrose oil on grounds of lack of efficacy, but without
revealing the evidence upon which it had based its decisions initially
to award and then to withdraw a licence for the drug (Chalmers
2004).
Recommendation
There should be easy public access to all information
from clinical studies of the effects of licensed drugs, including
that held by the Medicines and Healthcare products Regulatory
Agency.
6. PRODUCT EVALUATION,
INCLUDING ASSESSMENTS
OF VALUE
FOR MONEY
During the 2nd World War, a drug was produced
which was thought to promote faster recovery from common colds—a
substantial cause of lost work and reduced national efficiency.
Because of its potential national importance, the Ministry of
Supply asked the Medical Research Council to evaluate the drug.
The pharmaceutical industry provided supplies of the drug and
placebos for this research, and, within a few months, strong evidence
had been produced showing that the drug was highly unlikely to
have any useful effect (Clarke 2004). This showed how concerted
common purpose can reliably answer questions of national importance
about the effect of a drug. It is relevant today, and is reflected
in the creation of NICE.
NICE exists because the information supplied
by pharmaceutical manufacturers to the MHRA is inadequate for
assessing the extent to which the costs of drugs represent good
value to the NHS. As I noted in evidence submitted to the Committee
for its enquiry into NICE, the organisation cannot be expected
to serve the interests of the public effectively while it does
not have full access to relevant evidence, and while there is
continued acquiescence in biased and incomplete public access
to the results of relevant clinical research.
NICE cannot make adequate assessments of cost-effectiveness
without information about the relative merits of alternative treatments
(including drugs in the same class), and evidence gathered after
wider use and longer follow up of new products showing their effects
on the outcomes that matter to patients. In these circumstances
of significant ignorance about the effects and cost-effectiveness
of drugs, the NHS should consider adopting recommendations made
in respect of the Australian Pharmaceutical Benefits Scheme (Glasziou
1995): promising but inadequately evaluated drugs should be used
by the NHS only within the context of research designed to reduce
uncertainties about their value, and thus provide the basis for
informed choices and decisions.
Recommendation
Selected promising but inadequately evaluated
drugs should be used in the NHS only within the context of controlled
evaluative studies until enough is known to judge their cost-effectiveness.
14 August 2004
REFERENCES
Altman DG (1994). The scandal of poor medical
research. BMJ 308:283-4.
Antman EM, Lau J, Kupelnick B, Mosteller F,
Chalmers TC (1992). A comparison of results of meta-analyses of
randomized control trials and recommendations of clinical experts.
JAMA 268:240-48.
Bunker JP, Frazier HS, Mosteller F (1994). Improving
health: measuring effects of medical care. Milbank Quarterly 72:225-258.
Chalmers I (2000). Current Controlled Trials:
an opportunity to help improve the quality of clinical research.
Current Controlled Trials in Cardiovascular Medicine 1:3-8. Available:
http://cvm.controlled-trials.com/content/1/1/3.
Chalmers I (2004). In the dark. Drug companies
should be forced to publish all the results of clinical trials.
How else can we know the truth about their products. New Scientist,
6 March:19.
Chalmers I, Hedges LV, Cooper H (2002). A brief
history of research synthesis. Evaluation and the Health Professions
25:12-37.
Clarke M. The 1944 patulin trial of the British
Medical Research Council: an example of how concerted common purpose
can get reliable answers to important questions very quickly.
The James Lind Library (www.jameslindlibrary.org).
Clarke M, Alderson P, Chalmers I (2002). Discussion
sections in reports of controlled trials published in general
medical journals. JAMA 287:2799-2801.
Cowley AJ, Skene A, Stainer, Hampton JR (1993).
The effect of lorcainide on arrhythmias and survival in patients
with acute myocardial infarction. International Journal of Cardiology
40:161-166.
Djulbegovic B, Lacevic M, Cantor A, Fields KK,
Bennett CL, Adams JR, Kuderer NM, Lyman GH (2000). The uncertainty
principle and industry-sponsored research. Lancet 356:635-638.
Garattini S, Bertele V (2002). Efficacy, safety,
and cost of new anticancer drugs. BMJ 325:269-71.
Glasziou PP (1995). Support for trials of promising
medications through the Pharmaceutical Benefits Scheme. Medical
Journal of Australia 162:33-36.
Horn J, Limburg M (2001). Calcium antagonists
for acute ischemic stroke (Cochrane Review). In: The Cochrane
Library, Issue 3, Oxford: Update Software.
Horn J, de Haan RJ, Vermeulen M, Luiten PGM,
Limburg M (2001). Nimodipine in animal model experiments of focal
cerebral ischaemia: a systematic review. Stroke 32:2433-38.
Horrobin DF (2003). Modern biomedical research:
an internally self-consistent universe with little contact with
medical reality. Nature Reviews/ Drug Discovery 2:151-154.
International Society of Drug Bulletins (2001).
ISDB Declaration on therapeutic advance in the use of medicines.
Melander H, Ahlqvist-Rastad J, Meijer G, Beermann
B (2003). Evidence b(i)ased medicine—selective reporting
from studies sponsored by pharmaceutical industry: review of studies
in new drug applications. BMJ 326:1171-3.
Roberts I, Li Wan Po A, Chalmers I (1998). Intellectual
property, drug licensing, freedom of information, and public health.
Lancet 352:726-729.
Savulescu J, Chalmers I, Blunt J (1996). Are
research ethics committees behaving unethically? Some suggestions
for improving performance and accountability. BMJ 313:1390-1393.
Tallon D, Chard J, Dieppe P. Relation between
agendas of the research community and the research consumer. Lancet
2000;355:2037-40.
Wager E, Field EA, Grossman L (2003). Good Publication
Practice for pharmaceutical companies. Current Medical Research
and Opinion 19:149-54, and at www.gpp-guidelines.org.
| 
