Intergovernmental Panel on Climate Change Fifth Assessment Report: Review of Working Group I contribution - Energy and Climate Change Contents

2  Production of Working Group I contribution to the Fifth Assessment Report


7. The process by which the IPCC's Assessment Report is produced and agreed has improved considerably since the release of the Fourth Assessment Report (AR4) in 2007, mostly as a consequence of a review carried out by the InterAcademy Council (IAC) in 2010. The IAC was created by the world's science academies in 2000. It mobilises scientists and engineers to provide advice to international bodies.[9] The IAC review of the IPCC was commissioned by the United Nations Environment Programme (UNEP) and conducted by twelve experts who were not climate scientists. It concluded that "significant improvements [to the IPCC] are both possible and necessary for the fifth assessment and beyond".[10] In total, the IAC made 22 recommendations on matters ranging from the review process, treatment of uncertainty and governance and structure.

8. The Department of Energy and Climate Change (DECC) and the Met Office spoke particularly favourably of the IPCC's updated procedures for dealing quickly with errors post-publication.[11] Furthermore, the introduction of a common language to describe uncertainty should help to maintain consistency between Working Groups and Assessment Reports.[12] Sir Peter Williams, former Vice President and Treasurer of the Royal Society, who was a member of the IAC review panel, was also pleased with changes to the management and structure of the IPCC, particularly the introduction of an Executive Team:

    One recommendation that has been implemented is the way in which the 194 governments operate through the bureau of 30-odd strong, which meets very infrequently. It was crying out for some form of sub-committee in a PLC sense to look after the shop between meetings. The so-called e-team, the executive team, that had been tried previously and did not meet and was not effective, has given way to a properly-appointed executive team that does meet.[13]

However, he was disappointed that:

    In the executive committee, the one element that has not been brought about is the appointment of three independent non-climate scientists to sit alongside the IPCC professionals. That, in a typical PLC-type context, would also improve transparency, openness and good governance.[14]

9. A number of contributors commented that the IPCC review does not represent "peer-review" in the traditional sense.[15] Dr Ruth Dixon, Leverhulme Trust Postdoctoral Researcher at the University of Oxford, suggested it is more like a system of "public comment".[16] Under the IPCC system, a large number of "Expert Reviewers" are invited to comment on a particular chapter (though they are free to choose which sections they do and do not critique) and the authors-who can see the identity of the reviewer-must then respond to each of these comments in turn. For the WGI contribution to AR5, some 50,000 comments were received and responded to. Dr Emily Shuckburgh, Head of the Open Oceans research group at the British Antarctic Survey and Fellow of the Royal Meteorological Society, explained:

    It is to be commended that this process has become very transparent now that all these comments have been published. The second thing is to note that it is a very open process. Anybody was able to submit reviews to that process and the way that has now been documented, as was demonstrated in the released documents, is that every single one of those reviewer comments has been looked at, taken account of and then a Review Editor has come in and overseen that each one of those different comments has been addressed. In that sense, it is a very thorough review process. The third point to make is that, of course, the IPCC Working Group 1 primarily draws on the published literature that itself has been peer-reviewed, so what we are talking about is a peer review of a peer review. As a society, we feel that that is a very robust mechanism.[17]

10. The IPCC has continued to strengthen and improve its Assessment Report procedure. The IPCC has put a series of measures in place to help to minimise the risk of errors creeping in, and quickly rectify them if they emerge. The IPCC has responded extremely well to the constructive criticism of the InterAcademy Council (IAC). With regard to the IAC's recommendations, we would like to see the appointment of non-climate scientists to the Executive Committee.

11. Donna Laframboise, journalist, founder of and author of two books about the IPCC, was concerned that WGI authors can too easily dismiss troublesome comments without good reason, a concern shared by the IAC.[18] The IPCC has since brought in measures to help safeguard against this, primarily through increasing the number of Review Editors and inviting them to write a summary of their impressions of how well the comments have been dealt with. However, Dr Dixon and Marcel Crok, a freelance science writer, still had concerns that it was theoretically possible for critical comments to slip through without being fully addressed, which could damage the image and authority of the IPCC's reports.[19] They recommended:

    If the IPCC reports are to be seen as truly authoritative, the IPCC should institute some sort of 'Red Team review' by scientists, statisticians and other experts from outside the climate field, to bring the rigour and expertise of other scientific disciplines to bear on reviewing these important reports. Such a team would have the task of challenging the reports in order to identify significant weaknesses in balance or content, if they can. If it was not practicable to review the whole of the three Working Group reports in this way, certain particularly policy-relevant chapters could be chosen.[20]

Dr Dixon suggested that these scientists could act in a similar manner to the IAC, but across the entirety of the Assessment Report. The prevailing view was that the IPCC review process for the WGI contribution to AR5 has largely been successful.[21] As Professor Myles Allen, Professor of Geosystem Science at the University of Oxford and former IPCC Lead Author, commented:

    [The review process] means that we draw in comments and thoughts from directions where we might not have done so otherwise. That is very healthy and that is very helpful to the science.[22]

It is worth stressing the point made to us by a number of climate scientists, that the strength and authority of the reports does not lie in the complexity of the assessment process, but in the quality of the underlying scientific evidence presented.[23]

12. For future Assessment Reports the Government should recommend to the IPCC that they recruit a small team of experts who are not climate scientists to observe the review process from start to finish. The team would not constitute an extra stage of review, but rather oversee the process and arbitrate when controversies arise. The testimony of this independent team would improve the credibility of the report when it is released, and potentially protect it from any unnecessary and unfounded criticism. The team could also feed back to the IPCC in order to facilitate continuous improvement.

Inclusivity and objectivity

13. The IPCC does not employ climate scientists or conduct any original research in the preparation of its Assessment Reports. Instead, a team of authors is nominated by different United Nations (UN) member countries and invited to volunteer their time to compile an assessment of the peer-reviewed literature on climate change science. It is vital that these authors are acting in an objective manner, and in a manner that is inclusive towards the full range of scientific viewpoints. In order for the report to be credible, it must be an unbiased representation of the views of the scientific community.

14. Though the IPCC is a scientific body, it operates through the endorsement of the UN, and is therefore subject to "understandable and inevitable political pressures".[24] Some respondents saw this as a major flaw in the IPCC process.[25] Donna Laframboise remarked that the IPCC generates "science for politics' sake".[26] However, Professor Allen informed us:

    I think I would emphasise that, certainly at the chapter level, the scientists involved are contributing based on their expertise entirely and that is all. There is no Government interference at all at that point.[27]

This sentiment was further reinforced by Professor Sir Brian Hoskins, Director of the Grantham Institute for Climate Change at Imperial College and a former IPCC author, who told us he "certainly did not feel the weight of a Government of any country telling [him] what [he] should say".[28] It has been argued that the perception of interference may serve to undermine the scientific conclusions of the finalised report.[29] We consider the political pressures are isolated to the drafting of the Summary for Policymakers (see paragraph 27) rather than the authoring of the full report.[30]

15. We heard concerns that the personal persuasions of individual contributors may have led to some views being unjustly excluded from the WGI contribution to AR5. A number of possible problems were cited:

·  IPCC authors may display a bias towards including papers that agree with their own theory in a process of self-confirmation (so-called "confirmation bias" was also raised by the IAC in its review of the IPCC). This tendency could lead to papers being excluded that disagreed with the prevailing consensus view and could unwittingly lead to a "group-think" mentality. [31]

·  IPCC authors may be concerned about threats to their funding or research programmes if they include work that is critical of consensus theories of man-made climate change.[32]

·  IPCC authors may let their political persuasions (especially towards environmental activism) cloud their judgement as to what should and should not be included in the reports.[33]

16. These potential problems were, however, generally not well supported, especially given the changes made by the IPCC since AR4. Sir Peter Williams was particularly robust on this point:

    Bearing in mind the number of scientists involved in something like the Working Group I report you are considering today, it is simply impossible for some sort of herd instinct group-think with funding bias to take root in that diverse spectrum of people. All scientists are different. It is a general term that conjures up images of white coats and stereotypical conformity. The truth could not be further from that. The very idea that some form of funding bias, because of the popularity of the topic, could itself skew the observations of what are natural phenomena, which in turn could somehow skew the interpretations that are apt to mislead policymakers, does not bear examination.[34]

17. Dr Peter Stott, head of the Climate Monitoring and Attribution team at the Met Office and IPCC lead author said, when asked about objectivity in the IPCC:

    [I] do not see it as a problem at all. There is a conflict of interest statement that we sign and there are examples given on there. For example, if we are part of some non-governmental organisation or something, there is a recommendation that we should resign. We have signed a conflict of interest statement or we signed it as part of this process. We are first and foremost scientists, good sceptical scientists, scrutinising the science and the discussions that we had were purely on the basis of the science. They are not on the basis of political considerations or activism or anything else. They are on the basis of the science.[35]

18. Professor Allen further described to us the "very open and collegial attitude" within climate science, evidenced by the shared analysis and scrutiny of data between different groups.[36] Professor Allen explained:

    […] the discussions in IPCC author meetings are extremely heated. These are the meetings where I see the science tested most critically by other IPCC authors, trying to establish whether statements are robust. That process is tremendously useful for the science.[37]

The ethos of ensuring robustness, comprehensive assessment and using traceable evidence was emphasised by the leadership team of WGI throughout the writing process.[38]

19. Professor Allen described the process by which the IPCC took account of dissenting views as "painstaking".[39] The views of climate scientists (and non-climate scientists) who are not in agreement with the core conclusions of the IPCC have been included through direct engagement as contributing authors, consideration of their work in the academic literature, and engagement through the review process. It is obviously necessary for the IPCC reports to be selective about which views are included, and with a vastly growing body of evidence between each report, there will inevitably be parties who continue to be dissatisfied.[40] Scientists from the Department of Meteorology at the University of Reading stated that they were "unaware of any barrier that prevents scientists wishing to question the IPCC assessment in the peer-reviewed scientific literature".[41]

20. The Assessment Report procedure depends to a large extent on the integrity of the authors and editors involved, but we have found no evidence to suggest that this should give cause for concern. The authors drew upon a wide pool of peer-reviewed literature, highlighting areas of disagreement as readily as areas of agreement. We are satisfied that there was no systemic bias of any kind, be it financial, political or otherwise that would jeopardise the accuracy of the reported scientific conclusions. The procedures in place to safeguard against the influence of such biases appear to be sufficiently robust.

21. The majority of scientists who responded to our inquiry were understandably uneasy about claiming that any area of science is "settled", as this is contrary to the principles of sceptical inquiry under which science operates.[42] However, in response to the question of whether the IPCC is an accurate representation of the current views amongst climate scientists, the answer was overwhelmingly that it is. Some were keen to stress that a direct measure of "consensus" is difficult to quantify, and that each researcher may agree with some parts of the report more than others but, regardless of this, it is clear that the WGI contribution to AR5 is reflective of the prevailing majority opinion currently held within the climate science community.[43] Professor Sir Brian Hoskins provided a valuable summary of this view:

    There are thousands of scientists and you will get a range of views, but across the vast majority of that they would say, "Well, my view has been taken account of. Perhaps I might have written it slightly differently, but I can see it has been taken account of and essentially it is in the range there" […] It is not a natural process for scientists to go through this but, given that, I think it is a remarkable job that has been done.[44]

22. Although the terms "consensus" and "settled science" with regards to climate change were generally not thought to be helpful, as uncertainty and debate are required to drive research forward, we conclude that there is clearly strong agreement that the IPCC has captured the prevailing scientific opinion, notwithstanding some disagreement from a number of reputable scientists.


23. The process for producing the WGI contribution to AR5 was an enormous effort for those involved. The Assessment Review process took six years and involved a scoping meeting, two rounds of drafting and expert review, preparation and editing of the Summary for Policymakers (SPM) before final publication.[45] Having now run through this cycle five times since 1990, we asked witnesses whether they felt the timescale and level of commitment required by the IPCC was appropriate. The responses were clearly mixed. Professor Richard Tol, Professor of Economics at the University of Sussex and an IPCC lead author , told us:

    The IPCC process assesses scientific knowledge according to a political time-scale. That implies that parts of the literature are assessed too frequently while other parts of the literature are not assessed frequently enough. Instead of a mega-report every 6-7 years, it would be better to have an IPCC Journal with frequent updates where the literature moves fast and infrequent updates where little new is written.[46]

Others agreed with Professor Tol that the six to seven year timeframe was too long.[47] For example, the National Environment Research Council (NERC) claimed:

    The long gaps between reports and consensus approach does lead to delays in being able to incorporate the very latest scientific understanding into the assessments.[48]

The need for more frequent assessments was echoed in a statement made by DECC to the IPCC following the release of WGI contribution to AR5:

    The assessment cycle should not be more than 6-7 years as presently. However there is great demand from policy-makers for more frequent updates so ways should be considered to bring these into the cycle.[49]

The very large commitment required to contribute to these six to seven year "mega-reports" prompted Professor Sir Mark Walport, Government Chief Scientific Adviser, to describe the process as "undoubtedly burdensome".[50] This sentiment was repeated by many of the witnesses who had contributed to the reports.[51] Professor Richard Lindzen, Alfred P. Sloan Professor of Meteorology at Massachusetts Institute of Technology (MIT) and a former IPCC lead author, told us that:

    For most of us, it was an extraordinary bother dealing with the IPCC. For about four pages that I was involved in, I had to circumnavigate the globe two or three times and, like most of my fellow participants, I never participated again. It was just too much of a drag on one's time and effort.[52]

Professor Sir Brian Hoskins suggested that changes to the procedure from previous reports may have made the process "even more burdensome than it was before".[53]

24. It was clear, however, that the exhaustive and periodic nature of the Assessment Reports holds a number of advantages, including providing a focus for research activities such as climate modelling.[54] Professor Sir Brian Hoskins told us that the benefit of the Assessments reports is that they produce a large body of results that can be analysed in great detail by scientists at the same time.[55] Additionally, the advantages of the all-encompassing nature of the reports were emphasised by Nicholas Lewis, a climate researcher and mathematician:

    I think there is an advantage to having a fairly comprehensive assessment at the state of advice put down at one time because it enables you to examine it as a whole and see what implications the various bits of it have for other parts and see whether they are consistent or not. […] To expect these academic scientists to do this on top of their normal workload is not the ideal by a long way, but I think it is useful.[56]

25. Consequently, there was no clear consensus on how the IPCC should adapt and evolve. As Dr Shuckburgh testified, there is a "spectrum of views", some wish to develop a much less burdensome, policy-relevant and frequent assessment procedure, but others think that this could jeopardise the comprehensive and authoritative nature of the current process.[57] What is clear however, is that the final publication of AR5 marks a good opportunity for the IPCC to gather views and take stock on its future direction.[58] Though the work of the IPCC is targeted towards policymakers, it is essential that the scientists that underpin the Assessment Reports are well supported and their efforts are adequately acknowledged to ensure their continued commitment and enthusiasm for a challenging but important task.[59]

26. There are mixed views regarding the frequency and size of IPCC Assessment Reports. Transition to smaller, more frequent reports would arguably relieve the burden on contributing authors and ensure policymakers were kept up to date, but the finished document would lack the comprehensive and authoritative nature of the current Assessment Reports. Any revision of the tried and tested IPCC formula should only be introduced after careful consultation with both the governments who use the IPCC reviews and the scientists who write them. The aftermath of AR5 is an optimum time for this period of reflection to take place.

Summary for Policymakers

27. In September 2013, following the second round of drafting and expert review of the WGI contribution to AR5, the authors of the report gathered with representatives from United Nations (UN) member governments in Stockholm. During this four-day meeting, the 1,536 pages of the Full Technical Report, was reduced to a 33 page Summary for Policymakers (SPM) and agreed line-by-line. The meeting was held in private and the notes were not released following the session (unlike, for instance, the reviewer comments which were published in full). The SPM is the only part of the report that many people will read. It is unsurprising that it is the focus of many of the criticisms of the IPCC's alleged lack of transparency.[60]

28. The lack of transparency in this stage of the writing process, and the direct involvement of government representatives, has led to a great deal of speculation about the degree to which the scientific report is politicised before emerging as the finalised SPM.[61] Donna Laframboise was particularly critical of the process:

    Delegations from more than 100 countries were involved in the four day, behind closed doors, barred to the media meeting. Politicians, diplomats, and bureaucrats argued about phrasing and about which tables, graphs, and illustrations should be included. When they were done, the Summary for Policymakers was five pages longer than the draft but contained 700 fewer words.[62]

In support of this, Christopher Walter, Viscount Monckton of Brenchley, claimed that phrases critical of the performance of climate models had been removed from the SPM on the request of political agents.[63] Mr Lewis was also critical of the absence of technical information that he argued was of importance to the understanding of the conclusions of the SPM.[64]

29. Sir Peter Williams suggested that there was no particular cause for concern about the process through which the SPM is prepared:

    The one risk that remains, which the IPCC is very resilient against, is that the immensely complex science is ultimately digested by the plenary, which is the government delegations from around the planet. I think it does remarkably well not to be conservatised as a result of that process and nor, from my experience of having attended the plenary, does the message acquire a degree of interpretation or, as you might term it, a spin. From what I can see, the IPCC has done a very good job of resisting those tendencies.[65]

Furthermore, we were informed by WWF that some "observer organisations" are allowed to sit in on this plenary:[66]

    Throughout AR5 WGI, WWF observed constructive debate by all governments. Points of intervention were factual, science-based or focusing on improving the clarity of message for policy-makers. WWF did not observe a single intervention by any government that questioned the overall substance and/or general message of the report.[67]

Professor Allen emphasised the importance of involving policymakers in that final stage:

    The reason I think we need that process is that if you send scientists away and tell them just to write a report entirely in isolation, it is quite difficult to predict how people will interpret what they say because we end up just spouting jargon and nobody knows quite what we mean. You need that dialogue to make sure that the conclusions of the scientists have been correctly understood.[68]

30. Other IPCC authors were keen to re-iterate the fact that they have ultimate authority on the final content of the SPM and that all statements are directly traceable back to the relevant evidence given in the Full Technical Report.[69] Professor Sir Brian Hoskins suggested that this level of traceability in a summary of a technical subject designed for policymakers, was unusual when compared to other disciplines:

    On the traceability of this and the review of the whole process, Members of Parliament can be more content over what goes on in this case than, say, much of the information they get in the economic sphere, I suspect, where they would read the summary but not know the details and the traceability would be rather less.[70]

31. Including policymakers in the final stage of the report writing process does not seem to have had any substantial negative effects on the Summary for Policymakers (SPM) and very likely serves to improve the relevance and accessibility of the finished document.

32. It is inevitable that the distillation of such a complex and lengthy report will lead to the omission of some technical detail, but the traceability of the SPM to the full report adequately compensates for that. Any further technical detail that may be required for policymaking, such as in the setting of carbon budgets, is readily obtainable. The SPM succeeds in its purpose of keeping policymakers informed on issues surrounding climate science.

33. We recommend that the Government call on the IPCC to introduce a greater level of transparency in the plenary meetings to agree future Summaries for Policymakers (SPM). This may be through the admission of the independent team of observers to oversee the discussions (see paragraph 12). The feedback from the team would then serve to provide reassurance that the summary-writing process has been carried out objectively.

9   InterAcademy Council, 'About the IAC,' accessed 15 July 2014 Back

10   InterAcademy Council, Climate Change Assessments: Review of the Process and Procedures of the IPCC (August 2010), p26 Back

11   Department of Energy and Climate Change (IPC 025), Met Office (IPC 026) Back

12   Q23 [Dr Stott] Back

13   Q115 [Sir Peter] Back

14   Q116 [Sir Peter] Back

15   University of Reading (IPC 035) Back

16   Dr Ruth Dixon (IPC 023) Back

17   Q113 [Dr Shuckburgh] Back

18   Q98 [Ms Laframboise], Q100 [Professor Lindzen], Christopher Walter (IPC 005), Dr Ruth Dixon (IPC 023), Marcel Crok (IPC 041)  Back

19   Dr Ruth Dixon (IPC 023), Marcel Crok (IPC 041) Back

20   Dr Ruth Dixon (IPC 023) Back

21   Q46 [Professor Allen], Q114 [Dr Shuckburgh]  Back

22   Q46 [Professor Allen] Back

23   Q3 [Professor Allen], Myles Allen (IPC 037), WeatherAction (IPC 059) Back

24   Q114 [Sir Peter] Back

25   Christopher Walter (IPC 005), Alex Henney (IPC 006), John McLean (IPC 016), Donna Laframboise (IPC 039) Back

26   Donna Laframboise (IPC 039) Back

27   Q44 [Professor Allen] Back

28   Q44 [Professor Hoskins] Back

29   Ian Strangeways (IPC 022) Back

30   Q35 [Dr Stott], DECC (IPC 025), Grantham Institute for Climate Change (IPC 032) Back

31   Judith Curry (IPC 052) Back

32   Conor McMenemie (IPC 014), John McLean (IPC 016) Back

33   Q105 [Ms Laframboise], Donna Laframboise (IPC 039)  Back

34   Q150 [Sir Peter] Back

35   Q45 [Dr Stott]  Back

36   Q48 [Professor Allen] Back

37   Q39 [Professor Allen] Back

38   Q39 [Dr Stott] Back

39   Myles Allen (IPC 037) Back

40   Friends of Science Society (IPC 015), Nongovernmental International Panel on Climate Change (IPC 042) Back

41   University of Reading (IPC 035) Back

42   DECC (IPC 025), Royal Meteorological Society (IPC 029), Grantham Institute for Climate Change (IPC 032), Myles Allen (IPC 037), Professor Pierre Darriulat (IPC 049), WWF (IPC 054) Back

43   Q34 [Professor Allen, Professor Hoskins, Dr Stott], Q89 [Professor Lindzen],Ian Strangeways (IPC 022), Robin Guenier (IPC 024), Royal Meteorological Society (IPC 029), Grantham Institute for Climate Change (IPC 032), Jonathan Cowie (IPC 033), Natural Environment Research Council (IPC 036), Corinne Le Quéré and Andrew Watkinson (IPC 050), Mark Richardson (IPC 077) Back

44   Q34 [Professor Hoskins] Back

45   IPCC, 'IPCC Process,' accessed 15 July 2014 Back

46   Richard Tol (IPC 040) Back

47   University of Reading (IPC 035), Myles Allen (IPC 037) Back

48   Natural Environment Research Council (IPC 036) Back

49   DECC, Future of the IPCC Review - UK Government 2014 Response to IPCC, 25 February 2014 Back

50   Q230 [Professor Walport] Back

51   Q117 [Dr Shuckburgh], Royal Meteorological Society (IPC 029), University of Reading (IPC 035), Natural Environment Research Council (IPC 036), Myles Allen (IPC 037) Back

52   Q61 [Professor Lindzen] Back

53   Q35 [Professor Hoskins] Back

54   Corinne Le Quéré and Andrew Watkinson (IPC 050) Back

55   Q40 [Professor Hoskins] Back

56   Q101 [Mr Lewis] Back

57   Q41 [Professor Hoskins], Q117 [Dr Shuckburgh], Q289 [Mr Barker] Back

58   Q41 [Dr Stott], Q231 [Professor MacKay] Back

59   Qq287-290 [Mr Barker, Mr Warrilow] Back

60   Qq58-59 [Ms Laframboise, Professor Lindzen], Q63 [Mr Lewis], Christopher Walter (IPC 005), John McLean (IPC 016), Madhav Khandekar (IPC 019), Donna Laframboise (IPC 039), Professor Pierre Darriulat (IPC 049) Back

61   Qq58-59 [Ms Laframboise, Professor Lindzen], Donna Laframboise (IPC 039) Back

62   Donna Laframboise (IPC 039) Back

63   Christopher Walter (IPC 005)  Back

64   Qq63-69 [Mr Lewis] Back

65   Q142 [Sir Peter] Back

66   List of IPCC observers organisations (April 2014) Back

67   WWF (IPC 054) Back

68   Q44 [Professor Allen] Back

69   Q4 [Professor Hoskins, Dr Stott], Q11 [Dr Stott], Q35 [Dr Stott], Q44 [Dr Stott], Q124 [Sir Peter], DECC (IPC 025) Back

70   Q4 [Professor Hoskins] Back

previous page contents next page

© Parliamentary copyright 2014
Prepared 29 July 2014