Science and Technology CommitteeWritten evidence submitted by Cochrane Collaboration Individual Participant Data Meta-analysis Methods Group

Summary

Although consideration of individual participant data (IPD) relates to access to trial data rather than access to trial results, we anticipate that others will interpret “results” as extending to IPD. Although requiring access to trial IPD would afford greatest opportunity for scrutiny and contribution to further research, this is more complex than access to summaries of trial results. We believe it important to separate issues around disclosure of trial results and aggregate data from disclosure of IPD.

Prospective registration of all clinical trials carried out in, or which recruit human participants from, the UK should be compulsory and there should be a requirement that the (suitably defined) results of these trials be placed in the public domain, with open public access to this information. However, open access to trial IPD would pose risks to patient confidentiality and has potential to unintentionally damage clinical trial recruitment and conduct. De-identification of IPD to permit public access would retain some risk of disclosure and would render the data less useful for research purposes.

Whilst developing mechanisms to increase access to clinical trial results and aggregate data should begin immediately, increasing access to trial IPD should be preceded by considered debate, and by investigation into the potential impact on clinical trials.

About us

As co-convenors of the Cochrane Collaboration Individual Participant Data Meta-analysis Methods Group, each of us has considerable experience over many years of obtaining clinical trial results and trial datasets (comprising data from each trial participant from published and unpublished trials) for inclusion IPD systematic reviews and meta-analyses. We also have been involved in running and sharing data from clinical trials and in setting up and running a clinical trials register.

Declaration of Interests

Professor Lesley Stewart is Director of the NIHR Centre for Reviews and Dissemination (CRD) at the University of York. She is responsible for delivering programmes of work that include systematic reviews of both aggregate and individual participant data. She is co-convenor of the Cochrane Collaboration IPD Meta-analysis Methods Group. She recently instigated development of PROSPERO an international prospective register of systematic review protocols and has previously overseen the development and management of a web-based national register of cancer clinical trials. She has been involved previously in the design of publicly funded clinical trials. She is currently a member of the advisory board of Current Controlled Trials which registers clinical trials and issues international standard randomised clinical trials numbers. She is Co-Editor in Chief of a journal that publishes systematic review protocols.

Dr Jayne Tierney is Meta-analysis Lead of the MRC Clinical Trials Unit (CTU) and Deputy Director of the MRC CTU Hub for Trials Methodology Research, London. She is responsible for the conduct of a programme of systematic reviews and meta-analyses of aggregate and particularly individual participant data. This involves collaborating with trial organisations worldwide to obtain IPD from their trials, and acting as custodian for these collated data. She also works closely with CTU clinical trials on systematic reviews to inform trial design, conduct and reporting, and was previously involved in the development of a web-based national register of cancer clinical trials. She is a co-convenor of the Cochrane Collaboration IPD Meta-analysis Methods Group.

Professor Mike Clarke is Director of the MRC-funded All Ireland Hub for Trials Methodology Research at Queen’s University Belfast, Northern Ireland, which is establishing a programme of research into ways to improve the quality and relevance of clinical trials and to ensure that their findings are available to patients, practitioners, policy makers and the public when making decisions and choices about health and social care. He is involved in the conduct of several randomised trials and systematic reviews. Some of these systematic reviews include the use of IPD. He is a co-convenor of the Cochrane Collaboration IPD Meta-analysis Methods Group.

Professor Maroeska Rovers is professor of evidence-based surgery at Radboud University Medical Center Nijmegen, The Netherlands. She received a promising VENI grant to study the challenges of IPD meta-analyses in which she showed that IPD meta-analyses provide valuable opportunities to study subgroups. Dr Rovers has been involved in many IPD meta-analysis. The results of her IPDMA into the effectiveness of antibiotics in subgroups of children with acute otitis media, which was published in The Lancet (2006), have been incorporated in various international guidelines. She has performed several randomized controlled trials. She is a co-convenor of the Cochrane Collaboration IPD Meta-analysis Methods Group and co-editor of the Cochrane ENT-group.

Factual Information

1.0 The Cochrane Collaboration is separately submitting a response (to which we have contributed) which describes evidence of withheld data leading to harm to public health, and describes some lessons that can be learned from experience outside the UK. The bulk of that submission relates to the availability of summary results from trials, but part of it sets out the Collaboration’s position regarding access to IPD. Here, although we touch on other aspects, we focus on providing some additional views on access to trial IPD, which is a more complex and difficult issue than access to summaries of trial results. We do not aim to be comprehensive in this.

2.0 Systematic review is a research technique that uses transparent methods to identify, critique and, where possible and appropriate, synthesise the results of all relevant studies that have addressed a pre-specified and clearly defined research question. Systematic reviews provide the best means of informing health decision making, but rely on the integrity of the underlying research evidence in reaching fair and unbiased assessments. If unfavourable trial results are withheld from systematic reviews, this evidence base will be biased, undermining the decisions that depend on them. It is therefore vital that the results of all clinical trials are made available.

3.0 Systematic reviews using IPD involve central collection, validation and re-analysis of data from individual participants rather than using aggregate data for each trial. They are widely regarded as a “gold standard” approach and strengthen the quality of systematic reviews in a variety of important ways.

4.0 IPD are the data gathered for each trial participant, which are subsequently analysed to generate trial results. In our view, consideration of IPD relates to access to trial data rather than access to trial results, and so strictly speaking is outside the scope of this consultation. However, we anticipate that other respondents will interpret “results” as extending to IPD, particularly as the current consultation by the European Medicines Agency on access to trial data includes IPD.

Question 4: Disclosure of the existence of clinical trials

5.0 The occurrence of all clinical trials should be made open to scrutiny by making prospective registration of all trials compulsory. This could be brought about by legislative requirement that any clinical trial carried out in, or which recruits human participants from, the UK is registered prospectively in a designated trials register. This should include requirement for the full trial protocol to be made available either through the register or by linking to relevant publications (increasing numbers of healthcare journals publish trial protocols). A single national register of clinical trials (linking to the WHO international clinical trials platform) could be achieved by re-developing an existing registry or by establishing a new one.

Disclosure of trial results

6.0 Any requirements to improve access to clinical trial results should relate to making available the results of all the planned analyses set out in the trial protocol, statistical analysis plan or other prospective plan.

7.0 Going forward, it will be important to define what is meant by results, to avoid ambiguity or misunderstanding. Trial results could be interpreted as meaning the summary statistics describing the results of statistical analyses investigating the effects of interventions. Alternatively, results could be interpreted as a descriptive summary of the characteristics of trial participants, aggregate data summarising how often particular events, such as recurrence of cancer, or measures, such as blood pressure, are observed in each of the trial treatment groups, as well as full details of all of the statistical analyses.

8.0 Ensuring access to full statistical results of all analyses planned in the trial protocol or other plan would pose no risk to patient confidentiality, but should provide full details of the potential harms and benefits of treatments as assessed in the trial. There would be no need to moderate or control access. However, this information would permit only limited scrutiny of results by third parties and would not be sufficient to allow independent re-analysis of a trial.

9.0 Extending results to include aggregate data would also pose little risk to patient confidentiality, but would afford greater opportunity for scrutiny. For example, allowing some third party re-analysis and cross checking of results against the data provided. However, depending on the detail of the data presented, the level of scrutiny would likely still be somewhat limited, as would be the opportunity to contribute to detailed additional research using the trial data. Given the low risk, there would be no need to moderate or control access.

10.0 Full disclosure of clinical trial results could be achieved by making it a requirement that any clinical trial carried out in, or which recruits human participants from, the UK must place the (suitably defined) results of all analyses described in the trial protocol or other plan in the public domain.

10.1This could be implemented using existing mechanisms to formally publish in scientific or medical journals, or to make trial reports publicly available eg on company or institutional websites. Formal publishing would be preferable to website publishing because of the more permanent nature of journals and advantage of peer review of submitted reports, but may impose restrictions on the level of detail that can be included and full accessibility to all parties.

10.2Alternatively, a registry or repository for aggregate trial results could be developed, as has been done in the USA with clinicaltrials.gov, which is described in the main response from The Cochrane Collaboration. This centralised, standardised approach might be preferable to the more haphazard approach of availability through numerous, diverse channels, but would come with associated financial and operational costs, for both those managing such a registry, and also those involved in the conduct of trials. If such a repository were to be developed it should be linked closely with trial registration, and this could be achieved through a single national registry responsible for both functions.

11.0 A reasonable time frame is required between trial completion and provision of trials results, and this may need to vary from trial to trial, depending on the nature of the disease or condition, and the outcomes being collected. Those conducting clinical trials should have a reasonable (but not unlimited) period of exclusive use of their data. Clarity is needed on what constitutes trial completion, as many trial data and results accrue long after the recruitment of participants has stopped. Ideally, a time frame would be described in the trial protocol.

12.0 Operationally, it would be important that publications and postings are linked to the trial registration record so that one can be found from the other. Trials should therefore use a unique registration/identifying number. This will also help identify multiple publications of the same trial (which can lead to a different type of bias if favourable trials are covertly reported many times).

13.0 With suitable redaction of any patient identifiers, clinical study reports produced by manufacturers for regulatory trials could be published using existing publishing mechanisms or deposited with a registry/repository. This could offer a quick solution to increasing access to trial results, but would apply only to the subset of trials that are developed or submitted for market authorisation.

Access to individual participant data

14.0 Access to trial IPD would afford the greatest opportunity for scrutiny and the greatest opportunity to contribute to further research, as data can be re-analysed in ways that link patient characteristics and outcomes, which are not otherwise possible. Many trial funders already have data sharing polices and many trials organisations already share trial IPD for research purposes at the request of other organisations.

15.0 An open-access model would pose risks to patient confidentiality. If adopted, data would need to be de-identified before being made public. That is, all variables that might either on their own or in combination with other variables, lead to potential identification of an individual trial participant would need to be removed. The level of de-identification that would be required to allow public access could render the data much less amenable to scrutiny and further research, and might make it impossible to replicate the original analyses. Therefore, we suggest that a risk-dependent approach to the de-identification of IPD would be required, depending on who would be able to access it and for what purpose.

15.1The risk of a researcher identifying an individual trial participant would be much lower than someone who knows a trial participant and/or whose intention is to use open access as a means of obtaining personal information. For example, knowing that an individual has entered a clinical trial, the hospital at which they were treated, their age, and sex could be sufficient information to gain knowledge to sensitive information such as someone’s depression score, history of self-harming or other aspects of their health or lifestyle. If information about hospital, age, and sex were to be removed from the trial dataset, then it would mean that analyses (using the open access IPD) could not take account of these potentially very important factors and would be weakened as a result.

15.2These concerns would be accentuated for trials in rare conditions where it might be much easier to identify individual participants because of their rarity of their condition.

15.3Some trials and some outcomes are potentially more sensitive than others, for example, those dealing with sexual behaviour in the context of sexually transmitted infections.

16.0 Open public access to clinical trial IPD would also have the potential to unintentionally damage trials. We do not know how potential participants would react to the prospect of personal details being made available to anyone for any purpose and whether this would impact on their willingness to consent to join a trial. Investigation and research on this would be required prior to implementation of any requirement for public access to IPD.

17.0 Depositing and subsequent use by others of IPD may not be as straightforward as it might seem. Our experience of obtaining IPD directly from those responsible for trials has highlighted the difficulty of understanding datasets at face value. A detailed dialogue with the trial investigators is often required to reach a full understanding of the trial and its data. This understanding is necessary to avoid inappropriate or naive analyses.

18.0 We agree with the main Cochrane Collaboration submission that ultimately access to and scrutiny of IPD for research purposes, and with suitable safeguards in place to protect patient confidentiality and to protect trials, is desirable. However, we do not support open public access to clinical trial IPD. We believe that while open public access to trial results and aggregate data is in the public interest, open public access to IPD is not. The potential harms outweigh the benefits.

19.0 Before mandating access to IPD, there should be serious and considered debate. Increasing access to IPD is more complex than access to aggregate data. Consideration should include: ethical issues including protecting patient confidentiality; the potential impact on clinical trials (including on patient recruitment); resource and funding issues; and practical issues around data formats and curation. Consideration should also be given to how these issues might be handled in an international context.

20.0 Should, after due deliberation, mandatory access to IPD for research purposes be pursued, then various models might be considered. In a reactive approach, trial data would need to be supplied in response to appropriate and legitimate requests. This would align with many funders current policies on clinical data sharing, and is likely to be a less resource-intensive approach. However, it may be difficult to monitor. An alternative would be the development of a national repository of IPD (with access restricted to those undertaking legitimate research in the interest of public health).This would require mechanisms to ensure appropriate use, such as: registration of research protocols relating to use of data; compulsory deposition of final reports/publications from data analyses and transparency around potential conflicts of interest. Although, such a repository would be more resource intensive to manage and populate with trial data, it would permit monitoring of data provision and of subsequent access and use.

Recommendations

21.0 The government to introduce legislation to ensure that any clinical trial carried out in, or which recruits human participants from, the UK is registered prospectively in a designated trials register.

22.0 The government to introduce legislation to ensure that any clinical trial carried out in, or which recruits human participants from, the UK must place the (suitably defined) results of all analyses described in the trial protocol in the public domain.

23.0 Government agencies to consider developing a single linked national register of clinical trials and repository of trial results for any clinical trial carried out in, or which recruits human participants from, the UK.

24.0 Such a registry to ensure that full trial protocols are made publicly available, free of charge and in an accessible format within a specified period.

25.0 Public funding agencies to recognise the resource implications of 23.0 and 24.0 for those conducting trials and providing results, and to ensure that these costs are met within awards of research grants and programmes.

26.0 Developing mechanisms to increase access to clinical trial results and aggregate data should begin immediately.

27.0 Developing mechanisms to increase access to IPD should not, be pursued immediately but be preceded by informed discussion and by detailed investigation into the potential impact on clinical trials.

28.0 Open public access to trial IPD should not be pursued.

29.0 Any future mandatory access to clinical trial IPD should be restricted to legitimate research purposes for the good of public health, should include mechanisms to prevent misuse and be transparent about conflict of interest.

February 2013

Prepared 16th September 2013