Memorandum submitted by Steven Mosher (CRU 33)
Memoranda: How independent are the other two national data sets?
1. Declaration of interests. My name is Steven Mark Mosher. I am a resident of the United States. My interests in this matter are as follows. I am a published author of the first book that covered the email release from the University of East Anglia. That is currently my only source of income. I have been unemployed since June of 2009. Since 2007 I have been involved in the climate debate on the web. I have received no funding or financial support in any way from any person or organizing for my activities these past 3 years. I volunteer my insights and analysis free of charge.
2. During the years of 2007 to 2009 I was employed in the Open source movement. That experience gave me an appreciation for the power of transparency. Throughout 2007 and into 2009 I followed the quest of Steve McIntyre and others as they tried to get climate data and climate code released to the public. As a former data analyst for Northrop Aircraft , former statistican and former software engineer I was interested in having access to the data and the code so that I could perform basic quality checks on the science that was being used to combat global warming
3. On my view Global warming is a potential threat to our planet and consequently the evidence in support of global warming and the software analyzing that data should be of the highest quality. During the course of 2007 to 2009 the papers I had read and the limited data I had reviewed gave me reason to doubt the accuracy of underlying data, the robustness of the calculations described in papers, and the reliability of the results.
4. In July of 2009 I was witness to a series of FOIA requests made by Steve McIntyre and others requesting the underlying data of the global temperature index . In addition I myself sent in an FOIA request to CRU. That request asked to see the confidentiality agreements that CRU had made contradictory claims about.
5. Recently CRU have been in contact with various agencies requesting temperature data. I have a pending FOIA appeal at CRU with respect to the procedures CRI is obligated to follow when acquiring confidential data. According to FOIA guidelines CRU is not allowed to use confidential data unless it is deemed necessary or essential to their mission. In my FOIA request I requested any supporting information CRU had to support their decision to contract for confidential data. They had performed no such analysis.
6. The question in front of the inquiry pertains to the 'independence" of the CRU temperature data or rather to the CRU temperature series. According to CRU publications the data they use is largely take from the repository at NCDC. This database is known as GHCN. The other two series, NASA's series and NOAA's series also claim to take their data from GHCN. On the surface if we take the agencies at their word they are largely dependent on the same sources.
7. There is a fundamental difficulty facing anyone who wants to validate the claim that all three series use largely the same data. In order to validate this claim with certainty , one must have access to the data as used. That is, one must have the various copies each of the agencies used in constructing their series. The point is a fine one, but is the typical first test in any quality assurance test. CRU claim to copy some of their data from GHCN. To verify this claim one must compare the data that GHCN has with the copy that CRU claim to have made.
8. This effort to conduct a simple quality check was thwarted by CRU's unwillingness to share the data as they used it. It was also complicated by the fact that they failed to keep good records on which data could be released. Simply, they comingled confidential data with open data. Consequently they refused any release. Even a release of data which they claim to have acquired from GHCN.
9. Since the state of the question regarding the dependence or independence of the data is undetermined, that is, since one cannot verify the claim or refute the claim without access to the actual data as used by CRU, any claim made about the dependence or independence is a speculation based solely on the claims made in papers.
10. This is an easy matter within the scientific community if the scientific method is followed. What follows is a series of recommendations which will allow the community of researchers to settle the questions associated with the series.
11. CRU should be required to follow the guidelines of FOIA legislation. Those guidelines require that they only acquire confidential data if it is necessary to their mission. The two other agencies do not use confidential data and they claim similar results to CRU. It is recommended that CRU avoid the use of confidential data altogether.
12. If CRU are in the process of acquiring confidential data or restricted data or have already acquired it, it is recommended that they justify the necessity of holding this data. The guidelines require that they do this. However, my FOIA request indicates that they have not objectively determined that this data is necessary to their mission.
13. If it is shown to be necessary to their mission it is recommended that they institute data control procedures which keep the confidential data segregated from the open data.
14. It is recommended operate an open access policy with regard to the data that is not covered by confidentiality agreements. Both the NASA series and the NOAA series provide public access to their data.
15. It is recommended that CRU publish it computer source code connected to all aspects of calculating the Global temperature series under the appropriate open source license. Currently, NASA provide the source code for the calculation of their series. This will allow interested parties to compare the calculations that both agencies perform on the databases.