3Improving clinical trials transparency
Developments since 2013 and the need for high-level political leadership
22.Since our predecessor’s Report, clinical trials transparency has been discussed “at the highest levels”,50 with a number of international and intergovernmental organisations releasing statements on this issue:
- In April 2015, the World Health Organization published a statement on the public disclosure of clinical trial results. The WHO statement defined reporting timeframes, called for results-reporting of older but still unpublished trials, and outlined steps to improve linkages between clinical trial registry entries and their published results.51
- In September 2016, the UN Secretary-General’s High-Level Panel on Access to Medicines placed the responsibility for making progress on this issue with national governments. It recommended that “Governments should require that the unidentified data on all completed and discontinued clinical trials be made publicly available in an easily searchable public register established and operated by existing mechanisms such as the WHO Clinical Trials Registry Platform, clinicaltrials.gov or in peer reviewed publications, regardless of whether their results are positive, negative, neutral or inconclusive”.52
- In May 2017 the WHO produced a joint statement on public disclosure of results from clinical trials, with 21 signatories committing to “develop and implement a policy with mandated timeframes for prospective registration and public disclosure of the results of clinical trials” that they fund, co-fund, sponsor or support within 12 months of completion. Moreover, the signatories committed to “monitor registration and endorse the development of systems to monitor results reporting on an ongoing basis”, with outputs from the monitoring process being made publicly available. UK signatories include the Medical Research Council, the Department for International Development and the National Institute for Health Research.53
23.The issue was also previously discussed at the highest level within the UK. Following the Ebola epidemic, in 2015 the then Prime Minister David Cameron told a G7 summit that:
The UK will be the first country in the world to require clinical trials and disease control operations to be fully transparent. From now on any UK-funded research, data or operation will be made openly available and the UK will look to develop an international agreement—via the G7—that would see the publication of results of all clinical trials of vaccines for relevant diseases.54
However, the focus for this commitment was specifically trials relating to potential global pandemics, and there does not appear to be any tangible progress even on this more limited aspect. In particular, Dr Goldacre told us that “there are many trials of both treatments and vaccines for Ebola that have completed but not reported results, including I believe some from the UK”.55
24.We were told that further political attention in the UK was required to make progress on this issue. TranspariMED argued that “so far, the systematic distortion of evidence generated by clinical trials has been largely framed as a technical issue, rather than as a political issue”.56 Dr Goldacre warned that the public were “increasingly aware that serious problems [with transparency] have been left unaddressed: that trial results are routinely withheld, that there has been little serious effective effort to fix the issue over decades, that the biggest players in the ecosystem of scientific research are not taking adequate action”. He argued that an ongoing failure to address these problems “[laid] fertile ground for quacks, anti-vaccination conspiracy theorists, and climate change denialists”.57
25.Our predecessor Committee concluded in 2013 that it had “not been impressed” by the Government’s efforts to resolve the problem of un-registered, non-reported and mis-reported clinical trials. We believe that while there have been some improvements there is still much more to be done.
26.We welcome the recent statements and recommendations from the WHO and the UN on clinical trials transparency aimed at improving registration and reporting rates. The Ebola epidemic prompted political attention in the UK to the risks to public health of non-publication of clinical trial results, with the then Prime Minister David Cameron making commitments to clinical trials transparency in 2015. Since then, progress has slowed in the UK at a political level. Clinical trials transparency is as much a question of political will as it is a technical issue. We recommend that the Government explicitly re-commit to tackling clinical trials transparency, perhaps through a focused ministerial speech on this issue. This should set a clear time limit for institutions to fully comply with clinical trials transparency requirements and make clear what the consequences will be of failing to meet that deadline.
Improving registration and reporting rates through auditing
27.An approach to improving clinical trials transparency emphasised by our witnesses was publishing better information on which clinical trials have reported and which have not. Dr Goldacre explained that “we need to know which researchers are the best and the worst, and which institutions are the best and the worst, because that is information we can act on. Assuming good faith, I hope that the institutions that have fallen behind would want to learn from those doing well at reporting their clinical trials”.58
28.Recent attempts to provide such information are described in Chapter 2. Surprisingly, information on whether an individual trial has published results is not readily available online without secondary analysis. Such work has been undertaken by Dr Goldacre and others to match information in clinical trial registries with publications records, with information on sponsor compliance rates published online.59 Despite the scope for using audits to drive improvements in compliance rates, Dr Goldacre found that his attempts to put trial-by-trial information in the public domain were seen as subversive. He told us that:
When you finish an audit showing which institutions in the UK are best and worst at reporting their clinical trial results at all, and which are best at reporting their clinical trial results on time, people sometimes respond as if you are doing something that somehow is transgressive or confrontational, which shows how far we have to go. It would be a very straightforward thing to fix. This Committee could write to the Health Research Authority and say, “We want you to audit every trial that you approve; we want you to publish line by line; we want you to identify the individual trials and trialists who have not published their results”.60
29.Our witnesses argued that an appropriate official provider of trial-by-trial information on registration and reporting compliance was the Health Research Authority (HRA), drawing on its oversight and coordination of research ethics committees (RECs) in the UK.61 Under both the EU Clinical Trials Directive 2001 and UK governance arrangements, in order to obtain clinical trial authorisation, all UK trials must first be evaluated and approved by an accredited REC. RECs therefore hold large amounts of information on proposed trials, albeit with some aspects being confidential for commercial reasons.62 Indeed, our predecessor Committee recommended in 2013 that “Research Ethics Committees should have a role in considering and monitoring compliance with transparency policies”.63
30.We asked Professor Jonathan Montgomery, the non-executive Chair of the HRA, whether there were any barriers to the HRA publishing all the information it holds on which clinical trials have reported and which have not. He told us that:
We hold data that comes through in relation to projects, some of which is commercially sensitive, and within our processing systems we need to respect the basis on which people have given that to us. […] However, going forward in relation to whether trials have resulted in publication, I see no reason why we cannot match publicly available data.64
He also told us that he was “not in favour of naming and shaming” organisations with poor compliance records but would nevertheless be “very keen to make transparent who has and has not published” as a prompt to encourage publication.65
31.Non-compliance with reporting rules is not currently documented by public bodies on a trial-by-trial basis. Official publication of such information would expose where there are weaknesses in compliance and where best practice within the sector could be found and shared. We welcome recent efforts by Dr Goldacre and the AllTrials campaign to make information on reporting rates available online. However, it should not be left to researchers and campaigners to collate and publish this information themselves. We turn to the issue of who should produce this information later in this chapter.
Models for auditing compliance
32.We heard about a range of possible models for auditing compliance. A joint submission from STOPAIDS, Healthwatch UK, Universities Allied for Essential Medicines UK and TranspariMED described various options for questions that an audit could seek to address, and the relative difficulty of conducting such an exercise.
Table 1: Types of audit
|
Audit question |
Audit methodology |
|
Was this clinical trial prospectively registered? |
Easy to audit. There are only 16 WHO-recognised primary registries that the HRA needs to search for each trial. |
|
Were the summary results of this clinical trial published on all the registries where it is registered within 12 months of study completion? |
Very easy to audit. The HRA can quickly check the already identified registry entries for trial results. |
|
Has this clinical trial published results in an academic journal following global best practices [including referencing the unique ID number of the trial issued by the registry in the abstract of the corresponding journal article]? |
Very easy to audit. The HRA can quickly search journal databases for the trial number. |
|
Are the registration and results data for this trial consistent across different registry entries? |
For trials registered in more than one registry the HRA can check whether key trial data is consistent across different registries. This requires no specialist skills, but does require manual comparison. |
|
Has this trial ever reported its results anywhere? |
Time intensive to audit. The HRA would need to follow a lengthy search protocol for every trial. No specialist skills required. |
|
Has this trial accurately reported its results? |
Time intensive and difficult to audit. The HRA would need to conduct a trial-by-trial analysis. This requires specialist skills. |
Source: STOPAIDS et al (RES0036)
33.We explored some of these options and the relative costs with our witnesses. Dr Goldacre argued that the HRA could use automated tools which searched databases for publications to answer the question of whether a trial had published results in an academic journal. However, Dr Kolstoe argued that this approach would “solve less than half of the problem”, since “determining whether results have been posted does not address whether they appropriately represent the study that was actually conducted”.66 He argued that, since the HRA had access to all of the confidential research protocols, it could also support the research ethics committees to compare outcomes reported in publications with the objectives specified in the original protocols to check for mis-reporting or selective non-reporting.
34.Dr Kolstoe told us that while it could be “relatively cheap and easy” to audit non-publication using automated software and data held by the HRA, in order to address “outcome reporting bias” a manual comparison would need to be made of the final study reports against the original proposal.67 He had conducted such an audit for the Hampshire A Research Ethics Committee to demonstrate that it was possible to do this. He argued that those with access to REC records were “in a particularly powerful position to detect publication and reporting bias in contrast to similar attempts conducted by research funders or systematic review organisations who do not have immediate access to such a wide range of otherwise confidential protocols”.68
35.Dr Kolstoe noted that “extra resources would be needed if individual RECs or those managing them were to take on this role more comprehensively”.69 The HRA claimed that the total cost of undertaking this work would be £2.4m, but did not provide a basis for this estimate.70 They warned that “without a significant increase in our funding, any active monitoring of all research projects approved by the HRA using this methodology would we believe have a detrimental impact on other areas of our services, such as approval timelines”.71 Jonathan Montgomery from the HRA told us that he thought it would be unlikely to ever secure such additional funding, and that it would not be “a sufficient priority against all the other things we are doing if we did have that money to deal with it”.72 He contrasted this cost with “about £250,000” for an IT-based solution, albeit one which would not address the question of whether the matched publication addressed the full aims of the trial or was a partial report.73
36.We recommend that the Health Research Authority (HRA) should be provided with funding to establish a national audit programme of clinical trials transparency, including the publication of a single official list of which UK trials have published results and those which are due to but have not. In the first instance this should focus on providing information on whether any results have been published in an academic journal following global best practice, building on the automated methods already developed by others. We recognise that there are other dissemination routes for clinical trials results beyond academic journals that automated methods might not capture. Where alternative means have been used to publish information the HRA can use this process to prompt lead investigators to provide details of where the results have been posted so that the entry for that trial can be corrected as necessary.
37.We are disappointed that the HRA does not believe it can secure funding for a more comprehensive form of audit and does not see this as a priority. Even if the cost of fully assessing reported trial outcomes against the original specification in the application for ethical approval amounts to £2.4m per year, as the HRA suggested in evidence to us, this is a small price to pay compared with the sums of money involved in policy decisions that draw on clinical trials evidence, such as the £424m the Government spent on stockpiling Tamiflu without full access to evidence on its effectiveness. We recommend that the HRA undertake further work to determine an accurate figure for the cost of such an audit and prepare a funding proposal for the Government to consider. The cost should be weighed against the potential public savings made by tackling mis-reporting, in terms of reduced ‘research wastage’ and the scope for better procurement decisions. If this model is pursued, then the results should be published trial-by-trial rather than simply at the summary level.
38.The Government should direct the HRA to publish information on trials that have received ethical approval but are not registered in a publicly-accessible register, on a trial-by-trial basis.
Sanctions for non-compliance
39.Our predecessor Committee in 2013 recommended that the HRA should introduce “penalties for non-compliance” with registration and reporting rules.74 We asked Professor Montgomery, non-executive Chair of the HRA, whether the HRA had the statutory power to impose meaningful sanctions for non-compliance. Professor Montgomery told us that “we do not have sanctions as part of the HRA, other than refusing permissions […] we do not employ the researchers so our only real powers are to withhold [ethical] permissions or refer to those who do have those powers”.75
40.Naturally the ethical permission for a trial cannot be withheld after the research has taken place as a response to non-reporting. However, the REWARD Alliance, an international alliance focused on reducing wastage in research, suggested to us that research ethics committees could refuse to approve proposals for further research unless the sponsor or researcher can show that their previous research had been reported.76 Dr Goldacre agreed that “ethics committees should not allow researchers to have access to patients unless they can show that they have published the results of all the trials they have previously conducted”.77 We asked Professor Montgomery whether this would be possible:
I think that would be tricky. […] We took to our research ethics committee members the question of whether we should make it a condition of a new application that results had been published within 12 months of completion of the previous ones. There was quite strong resistance to that. There was a group that simply told us it did not think it should happen; a group that thought it would not be possible to deliver on it; and a group that thought it was too ambiguous.78
Instead he argued that others could make decisions based on non-compliance information if it were made available, as we recommend (at paragraph 36):
I am not convinced that the ethics committee is the right bit of the system to do that. […] I would expect sponsors to be interested in whether their money had been well spent previously and led to the reported outcomes. I would expect the hosts of research to want to know it was worth hosting that research, so we should be providing them with the opportunity to ask whether they trust a particular research group or sponsor to deal with it.79
41.The HRA appears to be reluctant to enforce its rules, or to make previous compliance with transparency legislation a pre-requisite for ethical approval of future trials. As a result, there are currently no sanctions imposed on sponsors or investigators who fail to comply with HRA rules, or even on those who fail to respond to the HRA when their non-compliance is queried. Echoing our predecessor Committee’s conclusions from 2013, we recommend that the HRA introduce a system of sanctions to drive improvements in clinical trials transparency, such as withdrawing favourable ethical opinion or preventing further trials from taking place. The Government should consult specifically on whether to provide the HRA with the statutory power to fine sponsors for non-compliance.
Allocating responsibility for driving improvement
42.Our predecessor Committee recommended that the Government make the promotion of research transparency a statutory objective of the Health Research Authority.80 As a result, in 2013 the Government tabled an amendment to the Care Bill at Report stage which made it explicit that the HRA’s main objective of “protecting and promoting the interests of participants, potential participants and the public by facilitating the conduct of safe, ethical research” included “promoting transparency in research”.81 The Bill which became the Care Act 2014 clarified that ‘transparency’ included the registration of research and the publication and dissemination of research findings and conclusions.82 Our Chair was the lead Minister for the Bill at the time.
43.The HRA’s 2017/18 annual report listed “being a champion for transparency in research” as part of its strategic objectives, and noted that a “key risk” for the HRA related to “expectations around transparency and the HRA’s ability to deliver within its current remit”.83 Nevertheless, the HRA told us that “as far as our regulatory remit is concerned, we feel that we have sufficient scope to act within the existing legislation”.84
44.The annual report also stated that further work to consider transparency was underway, “with closer links with key partners being made to support the delivery of the transparency agenda together with refreshed key performance indicators for 2018/19”.85 There are currently no measurable performance indicators relating to transparency in the report. The two achievements listed for the year in its “performance scorecard” are that the HRA had “performed an audit of clinical trial registration and published the findings report on [the HRA’s] website” (as it has done for several years, at a summary level), and “compiled a list of accepted clinical trial registers with [its] Transparency Forum and published this list on our website”.86
45.The Health Research Authority has been explicitly responsible for “promoting research transparency” as part of its statutory objectives since 2014, but this does not appear to have brought about significant change in this area over the last four years. We recommend that the Government ask the HRA to publish, by December 2019, a detailed strategy for achieving full clinical trials transparency, with a clear deadline and milestones for achieving this. We also recommend that the Government write to the HRA to clarify that it should interpret the Care Act 2014 to mean that it is responsible for driving improvements in clinical trials transparency—as opposed to ‘promoting’ transparency as a virtue. The performance of the HRA should then be explicitly measured on this basis through its annual report, including through specific measurable performance indicators. If further financial resource for the HRA is required to tackle clinical trials transparency then the Government should consider favourably such requests.
46.We recommend that the Government consult further with the HRA on whether it is capable of delivering the improvements to clinical trials transparency needed within its current remit. If necessary its remit should be extended through introducing legislation which amends the provisions of the Care Act 2014.
51 World Health Organization, ‘WHO statement on public disclosure of clinical trials results’, accessed 23 August 2018
52 United Nations Secretary-General’s High-Level Panel on Access to Medicines, Promoting innovation and access to technologies (September 2016), para 4.3.5
53 World Health Organization, ‘Joint statement on public disclosure of results from clinical trials’, accessed 23 August 2018
54 Gov.uk, ‘Prime Minister calls for ‘wake-up to the threat from disease outbreak’’, 7 June 2015
61 Responsibility for regulation of most health and social care research in Wales, Scotland and Northern Ireland remains with each Devolved Administration, since health and social care are themselves devolved matters. However, the HRA performs various functions relating to research ethics committees on behalf of the Devolved Administrations by arrangement, as per the Care Act 2014 Schedule 7 paragraph 15.
63 Science and Technology Committee, Third Report of Session 2013–14, Clinical trials, HC 104, para 110
74 Science and Technology Committee, Third Report of Session 2013–14, Clinical trials, HC 104, para 110
80 Science and Technology Committee, Third Report of Session 2013–14, Clinical trials, HC 104, para 109
81 Department of Health, Government response to the Science and Technology Committee inquiry into clinical trials, Cm 8743, November 2013, para 27
82 Care Act 2014, section 110 (7)
83 Health Research Authority, Health Research Authority Annual Report and Accounts 2017/18, HC 1284
85 Health Research Authority, Health Research Authority Annual Report and Accounts 2017/18, HC 1284
86 Health Research Authority, Health Research Authority Annual Report and Accounts 2017/18, HC 1284
Published: 30 October 2018