House of Commons - The Reviews into the University of East Anglia's Climatic Research Unit's E-mails

The Reviews into the University of East Anglia's Climatic Research Unit's E-mails - Science and Technology Committee Contents

4 Key findings of the reviews

Disclosure of data and methodologies

52. The lack of transparency in disclosing raw data and the methodologies it used has been at the heart of many of the allegations against Professor Jones and others at the CRU at UEA. The specific allegations that most concerned our predecessor Committee were: that CRU inappropriately withheld data and methodologies used to create its temperature dataset, CRUTEM3; that CRU deliberately misrepresented its data; and that this amounted to scientific fraud. Our predecessor Committee examined these issues in March 2010 and recommended that the Independent Climate Change E-mails Review (ICCER) should "reach specific conclusions" on the issue of data availability.[70]

53. On the allegations concerning temperature data, the ICCER concluded:

Regarding data availability, there is no basis for the allegations that CRU prevented access to raw data. It was impossible for them to have done so.
Regarding data adjustments, there is no basis for the allegation that CRU made adjustments to the data which had any significant effect upon global averages and through this fabricated evidence for recent warming.
We find that CRU was unhelpful in dealing with requests for information to enable detailed replication of the CRUTEM analysis.
Crucially, we find nothing in the behaviour on the part of CRU scientists that is the subject of the allegations dealt with in this Chapter to undermine the validity of their work.[71]

RAW DATA

54. Lord Oxburgh explained the importance of making raw data available in his oral evidence:

In making data available, it seems to me that a number of issues arose […] none of the data that they [CRU] were really using today did they collect. So for all of those data, if people wanted them, the thing to do was to go to the people who generated them and actually get permission if they were not in the public domain already.[72]

55. The ICCER recommended to UEA the provision of a formal metadata repository:

Whilst we recognize and accept that CRU relies on other bodies both nationally and internationally to provide and to archive basic weather station data, we believe that a formal approach to the storage and archiving of metadata is required. Such a repository would, for example, have made it far easier to respond quickly to requests for the list of station identifiers associated with particular CRUTEM datasets. Where a University is hosting a unit of such international significance, we believe that it should ensure funding is available for such a repository either through the research grant process or from central resources.[73]

56. Professor Davies acknowledged that CRU did have "a case to answer, as many other climate scientists do—indeed, as all scientists in other areas do—of doing rather more to make [...] previous versions of data series, previous to publication, available for scrutiny".[74] He later went on to announce that UEA would be investing in posts to help ensure that CRU's data archive was efficient and that all previous versions of data series or metadata were readily accessible when requests come through.[75]

57. Sir Muir Russell made the point to us that disclosure of data was necessary "for people to challenge science in the conventional way of building on the work that people have done, replicating it, challenging it, coming forward with new hypotheses".[76]

58. The disclosure of raw data and sufficient details of the computer programmes is paramount in encouraging people to question science in the conventional way, challenging existing work, enabling validation of it and coming forward with new hypotheses. We welcome the ICCER's recommendation to UEA on the provision of a formal metadata repository, and are pleased that CRU is investing in posts to archive their data efficiently. We hope that no obstacles, financial or otherwise, will get in the way of CRU pursuing this.

CRU METHODOLOGIES

59. Lord Oxburgh described the importance of disclosing enough detail of the methodologies used in coming to a conclusion, to allow others to validate the work:

One of the things that anyone handling large amounts of data would have wanted to do was to use their own techniques, which may be proprietary techniques which they have developed as part of their research. I think that when the observations, when the conclusions are published, there must be enough explanation and enough material […] in order to allow another expert to come to the same conclusion or to disagree. You have got to have that availability, otherwise it doesn't stand.[77]

60. It is equally important for scientists to be able to replicate the work that they themselves have already carried out. However, when asked whether the scientists at CRU were able to make accurate reconstructions from the publication back to the raw data that they themselves had used, Lord Oxburgh answered "Not in every case. Not with the old material."[78] When we pressed Professor Davies, he explained that:

On the question about replication, it is perfectly true, during the time that the Oxburgh Panel were at the Climatic Research Unit, that it was not possible to replicate all of the work that had been undertaken. Some of this work was undertaken 20-plus years ago and the data were not immediately accessible. I have spoken to colleagues in CRU and they assure me, and I am confident, that given time, a number of weeks or days, understandably, given the fact that this work goes back 20 or 30 years, then they can replicate their work.[79]

61. Lord Oxburgh said that CRU was not able to make accurate reconstructions in every case, particularly of old material. Professor Davies from UEA confirmed this but said CRU scientists would be able to do this given a number of weeks. This is precisely the sort of work we would have expected the Scientific Assessment Panel to conduct—had it been less concerned about rushing to publish its report—during its inquiry into methodologies and the integrity of research at CRU.

The allegation of scientific fraud

62. In written evidence to our predecessors and to us, Mr Douglas Keenan made allegations of scientific fraud against a researcher at the University of Albany, Professor Wei-Chyung Wang, in relation to a 1990 paper on 'Urban heat islands in China'.[80] Professor Jones had produced his own important paper on the effect of urbanization on temperature in 1990, including data from Professor Wang. This paper was cited in the Intergovernmental Panel on Climate Change (IPCC) Fourth Assessment Report.[81]

63. The basis of the allegation was that meteorological stations located in rural areas can, over time find themselves in expanding urban areas. This potentially affects the temperature measurements made at those stations over a period of time, as urban areas could be warmer than rural areas. Mr Keenan said that the station location histories were not as reliable as suggested by Professor Jones and Professor Wang. The Global Warming Policy Foundation (GWPF) report, The Climategate Inquiries, stated that it has been claimed by Mr Keenan that, "even once Jones became aware of problems with Wang's data, he failed to issue a correction to his paper and even continued to cite it, including in the IPCC's Fourth Assessment Report [...] this amounts to scientific fraud".[82]

64. When we raised this allegation of fraud, Sir Muir Russell explained to us that he covered this issue in the ICCER, and that:

Fraud implies that it was all deliberately set out to be done to conceal or to fabricate. I don't think that's the evidence of what people thought they were doing in 1990. Whether it should have taken 17 years for someone to spot it and whether, at the end of the day, the records weren't right so that you could set it right is another question. But, remember, Jones did another paper […] to check the thing through.[83]

The paper that Sir Muir refers to was written by Jones et al. in 2008.[84] This paper "verified the original conclusions [of the 1990 paper] showing that the precise location of weather stations was unimportant to the outcome".[85] Professor Davies added that the State University of New York had fully investigated this allegation and that Professor Wang had been entirely exonerated.[86]

65. Sir Muir Russell acknowledged it was "difficult" to investigate accusations of scientific fraud but added that making data available so that others could go through the process of checking scientific findings was "the best way" to "guard against fraud".[87]

66. We consider that data disclosed in publications should be accompanied by sufficient detail of computer programmes, specific methodology or techniques used to analyse the data, such that another expert could repeat the work. Providing the means for others to question science in this way will help guard against not only scientific fraud but also the spread of misinformation and unsustainable allegations.

Peer review

67. Following the disclosure of e-mails from the CRU, allegations were made that the scientists there had made improper attempts to influence the peer review process. Our predecessor Committee concluded that the operation of peer review was a critical issue and that the ICCER team needed to ensure that they had the relevant experience to deal with this.[88]

68. Sir Muir Russell told us that he agreed with the Committee and had addressed this by "commissioning work from the editor of a leading peer review journal [Richard Horton, Editor of the Lancet] and from the Chair of the Committee on Publication Ethics [Elizabeth Wager] to ensure that Review members have a clear understanding of the relevant issues as they consider the evidence presented to them."[89]He added that the scientific members of the Review team are fully aware of the importance and practice of peer review through their own extensive work.[90]

69. Richard Horton provided A brief history of peer review to the Review team, in which he explained that:

Editors send manuscripts to reviewers based on a principle of confidentiality. The author expects the editor to maintain a covenant of trust between the two parties. The editor will not misuse the author's work by circulating it outside of the confidential peer review process. The editor expects that covenant of trust to be honoured by the peer reviewer. No manuscript should be passed to a third party by a reviewer without the permission of the editor, usually on the grounds of improving the quality of the critique of the manuscript by involving a colleague in the review process. A disclosure to a third party without the prior permission of the editor would be a serious violation of the peer review process—a breach of confidentiality.[91]

CONFIDENTIALITY OF PEER REVIEW

70. The GWPF report, The Climategate Inquiries, put forward an example of a leaked document from CRU, which "appears to be a breach of peer review confidentiality".[92] This was the e-mail on 26 February 2004, in which Professor Jones contacted Professor Michael Mann at Pennsylvania State University, to discuss a paper he had apparently been shown by his CRU colleague, Dr Tim Osborne:

Can I ask you something in CONFIDENCE—don't email around, especially not to Keith and Tim here. Have you reviewed any papers recently for Science that say that MBH98 and MJ03 have underestimated variability in the millennial record—from models or from some low-freq proxy data. Just a yes or no will do. Tim is reviewing them—I want to make sure he takes my comments on board, but he wants to be squeaky clean with discussing them with others. So forget this email when you reply.

Cheers

Phil.[93]

71. Sir Muir Russell told us that, although this was not one of the e-mails he investigated in detail, the author of the GWPF report used "tentative comments" such as this "it appears to be a breach" and that "you could equally read that e-mail and say there was a question put to [Professor] Jones rather than [his colleague] showing him the paper".[94] Furthermore, Sir Muir denied the allegation in The Climategate Inquiries report that the ICCER "ignored the recommendation of their own [peer review] adviser that they investigate the possibility that CRU staff had breached the confidentiality of the peer review process".[95]

72. We pressed Professor Davies on whether it was common practice for CRU scientists to discuss publications for peer review with colleagues. He told us:

I think all reviewers are well aware of the important issues of confidentiality, as Sir Muir has indicated. There will be a number of occasions, and I think this is accepted within the peer review community, where an individual reviewer is not expert in all areas which are covered by a particular publication. Certainly conversations will go on with colleagues in confidence, without revealing any details of data or results, about advice, about whether this, for example, is an appropriate methodology. I think that is within the spirit of peer review and doesn't break the conventions of peer review and confidentiality.[96]

SUBVERSION OF PEER REVIEW

73. The broader allegation investigated by the ICCER was that CRU made improper attempts to influence the peer review system, pressuring journals to reject submitted articles that did not support a particular view of climate change. The ICCER team assessed three specific examples of this, which we summarise as follows:

i. The Soon and Baliunas affair and Climate Research

Soon and Baliunas published a paper in the journal, Climate Research, reviewing 240 previous papers on temperature trends over the last millennium. Its claim, that recent temperatures were not unprecedented, was welcomed by those sceptical of anthropogenic global warming, but dismissed by other researchers on scientific grounds. The paper was accepted by one of the journal's Review Editors (Chris de Freitas) after peer review, but a number of other Review Editors resigned as a reaction against what they considered to be a seriously flawed paper. The Editor in Chief also resigned on being refused permission by the publisher to write an editorial on the failure of the peer review system. E-mails disclosed from CRU referring to this affair included the comment from Professor Jones to a colleague stating that he would have "nothing more to do with it [the Climate Research journal] until they rid themselves of this troublesome editor [de Freitas], a well known sceptic".[97] This led to allegations that normal procedures of publication were being undermined by CRU scientists.[98]

ii. The conflict with Dr Boehmer-Christiansen

Dr Boehmer-Christiansen was editor of the scientific journal Energy and Environment (E&E). She claimed that the hacked e-mails revealed attempts by CRU to manipulate peer review to Energy and Environment's disadvantage, and showed that libel threats were considered against its editorial team. She stated that "The emailers expressed anger over my publication of several papers that questioned the 'hockey stick' graph and the reliability of CRU temperature data. The desire to control the peer review process in their favour is expressed several times".[99]

iii. Professor Briffa's editorship of Holcene

Professor Briffa's conduct as Editor of the journal, Holcene, was called into question by the disclosure of e-mails between him and various reviewers discussing whether or not a paper submitted to him should be rejected. The e-mail extract, "confidentially, I now need a hard and if required an extensive case for rejecting" has been widely discussed. This led to allegations that he attempted to reject submitted articles that did not support a particular view of climate change.[100]

74. Having investigated these three examples, the ICCER concluded:

In our judgement none of the above instances represents subversion of the peer review process nor unreasonable attempts to influence the editorial policy of journals. It might be thought that this reflects a pattern of behaviour that is partial and aggressive, but we think it more plausible that it reflects the rough and tumble of interaction in an area of science that has become heavily contested and where strongly opposed and aggressively expressed positions have been taken up on both sides. The evidence from an editor of a journal in an often strongly contested area such as medicine suggests that such instances are common and that they do not in general threaten the integrity of peer review or publication.[101]

75. On questioning Sir Muir Russell about the choice of the three examples, he informed us that "they were the three that had been at the top of the head [...] in the comments that were made when the whole story broke […] We couldn't do everything but we looked at three very solid accusations".[102] He explained that the ICCER team were advised by their peer review adviser, Richard Horton, that it was "entirely natural that people should take a robust view about their own work" when peer reviewing the work of others with a different disposition.[103]

76. Our predecessor Committee concluded that the evidence they saw did not suggest that Professor Jones was trying to subvert the peer review process and that academics should not be criticised for making informal comments on academic papers.[104]

77. The conclusions reached by the Independent Climate Change E-mails Review (ICCER) are in line with our predecessor Committee's findings that "the evidence they saw did not suggest that Professor Jones was trying to subvert the peer review process and that academics should not be criticised for making informal comments on academic papers". We stand by this conclusion and are satisfied with the detailed analysis of the allegations by the ICCER.

Freedom of Information

78. It was alleged, by a number of correspondents who sent submissions to the ICCER team, that requests under the Freedom of Information Act 2000 (FoIA) and the Environmental Information Regulations (EIR)[105] had been inappropriately dealt with by UEA.

79. The FoIA, creating new rights of access to information, and the EIR, a statutory instrument providing access to environmental information, came into operation on 1 January 2005. CRU, as part of UEA, is classed as a "public authority" for the purposes of the FoIA and EIR. In his evidence to our predecessor Committee, Mr Richard Thomas, who was Information Commissioner from 2002 until June 2009, explained the application of the FoIA to scientific data held by UK universities:

the public must be satisfied that publicly-funded universities, as with any other public authority in receipt of public funding, are properly accountable, adopt systems of good governance and can inspire public trust and confidence in their work and operations [...] The fact that the FoIA requests relate to complex scientific data does not detract from this proposition or excuse non-compliance.[106]

80. Mr David Holland was the author of several requests for information to UEA, some of which were allegedly mishandled. On 7 July 2010, the Information Commissioner's Office (ICO) announced that the UEA "breached regulation 14(2) of the EIR by failing to provide a response to a request within 20 working days and breached regulation 5(2) by failing to provide a response to other requests".[107] As Mr Holland indicated that he was "content not to proceed" with his complaint in relation to UEA's failure to provide him with the information he had requested, "the Commissioner requires no further steps to be taken with regard to these requests".[108]

81. Mr Holland also made an allegation that the information he requested had been deleted, an offence under regulation 19 of the EIR.[109] The ICO decided that "although the emails referred to [...] indicated prima facie evidence of an offence, the Commissioner was unable to investigate because six months had passed since the potential offence was committed, a constraint placed on the legislation by the Magistrates Court Act 1980".[110]

82. The Deputy Information Commissioner, Mr Graham Smith, previously explained to our predecessor Committee that the FoIA and EIR made it an offence for public authorities to act so as to prevent intentionally the disclosure of requested information.

83. Our predecessors concluded:

There is prima facie evidence that CRU has breached the Freedom of Information Act 2000. It would, however, be premature, without a thorough investigation affording each party the opportunity to make representations, to conclude that UEA was in breach of the Act. In our view, it is unsatisfactory to leave the matter unresolved simply because of the operation of the six-month time limit on the initiation of prosecutions. Much of the reputation of CRU hangs on the issue. We conclude that the matter needs to be resolved conclusively—either by the Independent Climate Change Email Review or by the Information Commissioner.[111]

84. This matter was investigated by the ICCER, which found:

On the allegation that CRU does not appear to have acted in a way consistent with the spirit and intent of the FoIA or EIR, we find that there was unhelpfulness in responding to requests and evidence that e-mails might have been deleted in order to make them unavailable should a subsequent request be made for them.[112]

85. The ICCER also stated "there seems clear incitement to delete emails, although we have seen no evidence of any attempt to delete information in respect of a request already made."[113] We questioned Sir Muir Russell about what this statement meant. He explained that the e-mails "do still exist";[114] then went on to explain why the ICCER did not come to a conclusion on deliberate deletion of e-mails that had been requested:

The reason we didn't do that was something that I think I made clear to [the former Committee] [...] in March. I said I wasn't going to put the review into the position of making the sort of quasi-judicial prosecutorial, investigative judgments that Mr Thomas […] had spoken about. That was an ICO's job. That was the position that we took. So, had we been going to get into this, we would have had to start asking questions under caution. We would have been doing the sort of investigative stuff, because you're getting to the point where you're alleging that there might have been an offence, and that really wasn't the thing that my inquiry was set up to do, especially when there is a parallel entity called ICO that has the investigative skills, the training and the background with its personnel.

So that, in short order, was why we didn't go down the road of saying, "And did you delete things that had been requested?", because we felt that that would take us into an area where we would have had to operate under caution, and it wasn't actually relevant to where we had got to on the issue that all this is about, which is what was the end product of the influence that this process had on what was said in the IPCC report.[115]

86. Professor Acton had no such qualms questioning the scientists at CRU about whether or not they had deleted e-mails subject to FoI requests. He told us "Can those e-mails be produced? Yes, they can. Did those who might have deleted them say they deleted them? No. They say they did not".[116]

87. Despite not reaching an unequivocal conclusion about whether or not e-mails were deleted by CRU scientists in response to FoI requests, the ICCER did make specific recommendations to UEA on best practice in dealing with FoI requests in the future. These included: "promotion of the University's formal publication policy; incorporating more information on FoIA/EIR/DPA [Data Protection Act] responsibilities in the induction processes for new staff members; developing a rolling awareness campaign to focus the attention of established staff [...] and issuing annual reminders of the importance of transparency and of key FoIA/EIR/DPA responsibilities".[117]

88. Professor Acton acknowledged that there had been weaknesses in UEA's system, and in pockets of their culture, for dealing with requests for information but that he was keen for UEA to be "an exemplar" in improving the culture related to dealing with FoIA and EIR.[118]

89. We are concerned that the Independent Climate Change E-mails Review did not fully investigate the serious allegation relating to the deletion of e-mails. We find it unsatisfactory that we are left with a verbal reassurance from the Vice-Chancellor that the e-mails still exist. On the basis of the ICO's announcement made on 7 July 2010, it is reasonable to conclude that there was a breach of EIR by a failure to provide a response within 20 working days. On the allegation that e-mails were deleted to frustrate requests for information, a firm conclusion has proved elusive. UEA have accepted that there were weaknesses in their system, and in pockets of their culture, for dealing with requests for information. We are pleased that they are working towards rectifying this.

APPLICATION OF FOI TO SCIENTIFIC RESEARCH

90. Lord Oxburgh noted that "there are very interesting questions to be asked about the interface between the Freedom of Information Act and scientific research in progress";[119] an issue also highlighted by the ICCER. The Review explained that there was confusion about how FoI legislation should be applied in terms of the materials developed during a research process, and that the American experience was instructive here:

The so called "Shelby Amendment" in 1998 directed the US "Office of Management & Budget (OMB)" to produce new standards requiring all data produced under Federally funded research to be made available under the US Freedom of Information Act. This resulted in great concern within the US Scientific community, expressed through Congressional testimony, that a very broad interpretation of this requirement could seriously impair scientific research and collaboration. In the final OMB guidelines, recognising these concerns, "research data" is defined as: "the recorded factual material commonly accepted in the scientific community as necessary to validate research findings, but not any of the following: preliminary analyses, drafts of scientific papers, plans for future research, peer reviews, or communications with colleagues".[120]

The Review recommended that the ICO should hold consultations on a similar distinction for UK FoI legislation.[121]

91. The current Information Commissioner, Mr Christopher Graham, wrote to us on 21 October 2010 explaining the interaction between the ICO and the ICCER, as well as progress made towards dealing with the problems presented in applying the FoIA to scientific research.[122] Following a preliminary meeting on 22 July 2010 between the ICO and representatives from the Higher Education (HE) Sector, they agreed to organise a roundtable to begin work towards enhanced guidance for the sector on working within the FoIA and EIR. The roundtable discussion took place on 29 September 2010, and a note of the discussions was provided by the ICO.[123] The absence of a definition for "research data" in law was discussed, as well as the US experience. The outcome was that:

It was agreed to establish a working group representing the HE sector to work with the ICO in developing sector-led and sector-specific guidelines around the issues of research data, teaching materials and [Intellectual Property Rights].

After this work has been concluded, then the group might work with the ICO to inform any proposed amendments to the existing sector publication scheme in order to consider whether a framework for proactive dissemination of research data and /or teaching materials which still protected universities' necessary interests might be feasible. Research council initiatives around open access might provide a model in these discussions.

The ICO and Universities UK, working with JISC [Joint Information Systems Committee], RIN [Research Information Network] and other key stakeholders, have agreed to take this process forward.[124]

92. In addition to the work of the ICO, the Government Chief Scientific Adviser, Professor Sir John Beddington, told us that the Minister for Universities and Science, Rt Hon David Willetts MP, has asked him to examine the application of FoI to scientific research in more detail.[125]

93. The broader confusion about how FoI legislation should be applied to scientific research must be resolved. The Information Commissioner's Office has made some progress, but this should now be pursued as a matter of urgency. The Government Chief Scientific Adviser will also be looking at this issue. We regard this matter as sufficiently serious that we want to see it resolved. We hope the Information Commissioner's Office will provide clear guidance on the application of FoI to scientific research by the start of the new academic year in September 2011.

70 HC (2009-10) 387-I, para 39 Back

71 ICCER, section 6.7 Back

72 Q 43 Back

73 ICCER, section 11.4 Back

74 Q 83 Back

75 Q 124 Back

76 Q 61 Back

77 Q 43 Back

78 Q 34 Back

79 Q 74 Back

80 HC (2009-10) 387-II, Ev 181; Mr Keenan repeated the allegations in a memorandum-Ev W 12 Back

81 Intergovernmental Panel on Climate Change, Climate Change 2007: The Physical Science Basis, 2007, p 124 Back