2 Production of Working Group I contribution
to the Fifth Assessment Report |
7. The process by which the IPCC's Assessment Report
is produced and agreed has improved considerably since the release
of the Fourth Assessment Report (AR4) in 2007, mostly as a consequence
of a review carried out by the InterAcademy Council (IAC) in 2010.
The IAC was created by the world's science academies in 2000.
It mobilises scientists and engineers to provide advice to international
bodies. The IAC review
of the IPCC was commissioned by the United Nations Environment
Programme (UNEP) and conducted by twelve experts who were not
climate scientists. It concluded that "significant improvements
[to the IPCC] are both possible and necessary for the fifth assessment
In total, the IAC made 22 recommendations on matters ranging from
the review process, treatment of uncertainty and governance and
8. The Department of Energy and Climate Change (DECC)
and the Met Office spoke particularly favourably of the IPCC's
updated procedures for dealing quickly with errors post-publication.
Furthermore, the introduction of a common language to describe
uncertainty should help to maintain consistency between Working
Groups and Assessment Reports.
Sir Peter Williams, former Vice President and Treasurer of the
Royal Society, who was a member of the IAC review panel, was also
pleased with changes to the management and structure of the IPCC,
particularly the introduction of an Executive Team:
One recommendation that has been implemented
is the way in which the 194 governments operate through the bureau
of 30-odd strong, which meets very infrequently. It was crying
out for some form of sub-committee in a PLC sense to look after
the shop between meetings. The so-called e-team, the executive
team, that had been tried previously and did not meet and was
not effective, has given way to a properly-appointed executive
team that does meet.
However, he was disappointed that:
In the executive committee, the one element that
has not been brought about is the appointment of three independent
non-climate scientists to sit alongside the IPCC professionals.
That, in a typical PLC-type context, would also improve transparency,
openness and good governance.
9. A number of contributors commented that the IPCC
review does not represent "peer-review" in the traditional
sense. Dr Ruth Dixon,
Leverhulme Trust Postdoctoral Researcher at the University of
Oxford, suggested it is more like a system of "public comment".
Under the IPCC system, a large number of "Expert Reviewers"
are invited to comment on a particular chapter (though they are
free to choose which sections they do and do not critique) and
the authors-who can see the identity of the reviewer-must then
respond to each of these comments in turn. For the WGI contribution
to AR5, some 50,000 comments were received and responded to. Dr
Emily Shuckburgh, Head of the Open Oceans research group at the
British Antarctic Survey and Fellow of the Royal Meteorological
It is to be commended that this process has become
very transparent now that all these comments have been published.
The second thing is to note that it is a very open process. Anybody
was able to submit reviews to that process and the way that has
now been documented, as was demonstrated in the released documents,
is that every single one of those reviewer comments has been looked
at, taken account of and then a Review Editor has come in and
overseen that each one of those different comments has been addressed.
In that sense, it is a very thorough review process. The third
point to make is that, of course, the IPCC Working Group 1 primarily
draws on the published literature that itself has been peer-reviewed,
so what we are talking about is a peer review of a peer review.
As a society, we feel that that is a very robust mechanism.
10. The IPCC has continued to strengthen and improve
its Assessment Report procedure. The IPCC has put a series of
measures in place to help to minimise the risk of errors creeping
in, and quickly rectify them if they emerge. The IPCC has responded
extremely well to the constructive criticism of the InterAcademy
Council (IAC). With regard to the IAC's recommendations, we would
like to see the appointment of non-climate scientists to the Executive
11. Donna Laframboise, journalist, founder of NOconsensus.org
and author of two books about the IPCC, was concerned that WGI
authors can too easily dismiss troublesome comments without good
reason, a concern shared by the IAC.
The IPCC has since brought in measures to help safeguard against
this, primarily through increasing the number of Review Editors
and inviting them to write a summary of their impressions of how
well the comments have been dealt with. However, Dr Dixon and
Marcel Crok, a freelance science writer, still had concerns that
it was theoretically possible for critical comments to slip through
without being fully addressed, which could damage the image and
authority of the IPCC's reports.
If the IPCC reports are to be seen as truly authoritative,
the IPCC should institute some sort of 'Red Team review' by scientists,
statisticians and other experts from outside the climate field,
to bring the rigour and expertise of other scientific disciplines
to bear on reviewing these important reports. Such a team would
have the task of challenging the reports in order to identify
significant weaknesses in balance or content, if they can. If
it was not practicable to review the whole of the three Working
Group reports in this way, certain particularly policy-relevant
chapters could be chosen.
Dr Dixon suggested that these scientists could act
in a similar manner to the IAC, but across the entirety of the
Assessment Report. The prevailing view was that the IPCC review
process for the WGI contribution to AR5 has largely been successful.
As Professor Myles Allen, Professor of Geosystem Science at the
University of Oxford and former IPCC Lead Author, commented:
[The review process] means that we draw in comments
and thoughts from directions where we might not have done so otherwise.
That is very healthy and that is very helpful to the science.
It is worth stressing the point made to us by a number
of climate scientists, that the strength and authority of the
reports does not lie in the complexity of the assessment process,
but in the quality of the underlying scientific evidence presented.
12. For future Assessment Reports the Government
should recommend to the IPCC that they recruit a small team of
experts who are not climate scientists to observe the review process
from start to finish. The team would not constitute an extra stage
of review, but rather oversee the process and arbitrate when controversies
arise. The testimony of this independent team would improve the
credibility of the report when it is released, and potentially
protect it from any unnecessary and unfounded criticism. The team
could also feed back to the IPCC in order to facilitate continuous
Inclusivity and objectivity
13. The IPCC does not employ climate scientists or
conduct any original research in the preparation of its Assessment
Reports. Instead, a team of authors is nominated by different
United Nations (UN) member countries and invited to volunteer
their time to compile an assessment of the peer-reviewed literature
on climate change science. It is vital that these authors are
acting in an objective manner, and in a manner that is inclusive
towards the full range of scientific viewpoints. In order for
the report to be credible, it must be an unbiased representation
of the views of the scientific community.
14. Though the IPCC is a scientific body, it operates
through the endorsement of the UN, and is therefore subject to
"understandable and inevitable political pressures".
Some respondents saw this as a major flaw in the IPCC process.
Donna Laframboise remarked that the IPCC generates "science
for politics' sake".
However, Professor Allen informed us:
I think I would emphasise that, certainly at
the chapter level, the scientists involved are contributing based
on their expertise entirely and that is all. There is no Government
interference at all at that point.
This sentiment was further reinforced by Professor
Sir Brian Hoskins, Director of the Grantham Institute for Climate
Change at Imperial College and a former IPCC author, who told
us he "certainly did not feel the weight of a Government
of any country telling [him] what [he] should say".
It has been argued that the perception of interference may
serve to undermine the scientific conclusions of the finalised
report. We consider
the political pressures are isolated to the drafting of the Summary
for Policymakers (see paragraph 27) rather than the authoring
of the full report.
15. We heard concerns that the personal persuasions
of individual contributors may have led to some views being unjustly
excluded from the WGI contribution to AR5. A number of possible
problems were cited:
authors may display a bias towards including papers that agree
with their own theory in a process of self-confirmation (so-called
"confirmation bias" was also raised by the IAC in its
review of the IPCC). This tendency could lead to papers being
excluded that disagreed with the prevailing consensus view and
could unwittingly lead to a "group-think" mentality.
· IPCC authors
may be concerned about threats to their funding or research programmes
if they include work that is critical of consensus theories of
man-made climate change.
· IPCC authors
may let their political persuasions (especially towards environmental
activism) cloud their judgement as to what should and should not
be included in the reports.
16. These potential problems were, however, generally
not well supported, especially given the changes made by the IPCC
since AR4. Sir Peter Williams was particularly robust on this
Bearing in mind the number of scientists involved
in something like the Working Group I report you are considering
today, it is simply impossible for some sort of herd instinct
group-think with funding bias to take root in that diverse spectrum
of people. All scientists are different. It is a general term
that conjures up images of white coats and stereotypical conformity.
The truth could not be further from that. The very idea that some
form of funding bias, because of the popularity of the topic,
could itself skew the observations of what are natural phenomena,
which in turn could somehow skew the interpretations that are
apt to mislead policymakers, does not bear examination.
17. Dr Peter Stott, head of the Climate Monitoring
and Attribution team at the Met Office and IPCC lead author said,
when asked about objectivity in the IPCC:
[I] do not see it as a problem at all. There
is a conflict of interest statement that we sign and there are
examples given on there. For example, if we are part of some non-governmental
organisation or something, there is a recommendation that we should
resign. We have signed a conflict of interest statement or we
signed it as part of this process. We are first and foremost scientists,
good sceptical scientists, scrutinising the science and the discussions
that we had were purely on the basis of the science. They are
not on the basis of political considerations or activism or anything
else. They are on the basis of the science.
18. Professor Allen further described to us the "very
open and collegial attitude" within climate science, evidenced
by the shared analysis and scrutiny of data between different
] the discussions in IPCC author meetings
are extremely heated. These are the meetings where I see the science
tested most critically by other IPCC authors, trying to establish
whether statements are robust. That process is tremendously useful
for the science.
The ethos of ensuring robustness, comprehensive assessment
and using traceable evidence was emphasised by the leadership
team of WGI throughout the writing process.
19. Professor Allen described the process by which
the IPCC took account of dissenting views as "painstaking".
The views of climate scientists (and non-climate scientists) who
are not in agreement with the core conclusions of the IPCC have
been included through direct engagement as contributing authors,
consideration of their work in the academic literature, and engagement
through the review process. It is obviously necessary for the
IPCC reports to be selective about which views are included, and
with a vastly growing body of evidence between each report, there
will inevitably be parties who continue to be dissatisfied.
Scientists from the Department of Meteorology at the University
of Reading stated that they were "unaware of any barrier
that prevents scientists wishing to question the IPCC assessment
in the peer-reviewed scientific literature".
20. The Assessment
Report procedure depends to a large extent on the integrity of
the authors and editors involved, but we have found no evidence
to suggest that this should give cause for concern. The authors
drew upon a wide pool of peer-reviewed literature, highlighting
areas of disagreement as readily as areas of agreement. We are
satisfied that there was no systemic bias of any kind, be it financial,
political or otherwise that would jeopardise the accuracy of the
reported scientific conclusions. The procedures in place to safeguard
against the influence of such biases appear to be sufficiently
21. The majority of scientists who responded to our
inquiry were understandably uneasy about claiming that any area
of science is "settled", as this is contrary to the
principles of sceptical inquiry under which science operates.
However, in response to the question of whether the IPCC is an
accurate representation of the current views amongst climate scientists,
the answer was overwhelmingly that it is. Some were keen to stress
that a direct measure of "consensus" is difficult to
quantify, and that each researcher may agree with some parts of
the report more than others but, regardless of this, it is clear
that the WGI contribution to AR5 is reflective of the prevailing
majority opinion currently held within the climate science community.
Professor Sir Brian Hoskins provided a valuable summary of this
There are thousands of scientists and you will
get a range of views, but across the vast majority of that they
would say, "Well, my view has been taken account of. Perhaps
I might have written it slightly differently, but I can see it
has been taken account of and essentially it is in the range there"
] It is not a natural process for scientists to go through
this but, given that, I think it is a remarkable job that has
the terms "consensus" and "settled science"
with regards to climate change were generally not thought to be
helpful, as uncertainty and debate are required to drive research
forward, we conclude that there is clearly strong agreement that
the IPCC has captured the prevailing scientific opinion, notwithstanding
some disagreement from a number of reputable scientists.
23. The process for producing the WGI contribution
to AR5 was an enormous effort for those involved. The Assessment
Review process took six years and involved a scoping meeting,
two rounds of drafting and expert review, preparation and editing
of the Summary for Policymakers (SPM) before final publication.
Having now run through this cycle five times since 1990, we asked
witnesses whether they felt the timescale and level of commitment
required by the IPCC was appropriate. The responses were clearly
mixed. Professor Richard Tol, Professor of Economics at the University
of Sussex and an IPCC lead author , told us:
The IPCC process assesses scientific knowledge
according to a political time-scale. That implies that parts of
the literature are assessed too frequently while other parts of
the literature are not assessed frequently enough. Instead of
a mega-report every 6-7 years, it would be better to have an IPCC
Journal with frequent updates where the literature moves fast
and infrequent updates where little new is written.
Others agreed with Professor Tol that the six to
seven year timeframe was too long.
For example, the National Environment Research Council (NERC)
The long gaps between reports and consensus approach
does lead to delays in being able to incorporate the very latest
scientific understanding into the assessments.
The need for more frequent assessments was echoed
in a statement made by DECC to the IPCC following the release
of WGI contribution to AR5:
The assessment cycle should not be more than
6-7 years as presently. However there is great demand from policy-makers
for more frequent updates so ways should be considered to bring
these into the cycle.
The very large commitment required to contribute
to these six to seven year "mega-reports" prompted Professor
Sir Mark Walport, Government Chief Scientific Adviser, to describe
the process as "undoubtedly burdensome".
This sentiment was repeated by many of the witnesses who had contributed
to the reports. Professor
Richard Lindzen, Alfred P. Sloan Professor of Meteorology at Massachusetts
Institute of Technology (MIT) and a former IPCC lead author, told
For most of us, it was an extraordinary bother
dealing with the IPCC. For about four pages that I was involved
in, I had to circumnavigate the globe two or three times and,
like most of my fellow participants, I never participated again.
It was just too much of a drag on one's time and effort.
Professor Sir Brian Hoskins suggested that changes
to the procedure from previous reports may have made the process
"even more burdensome than it was before".
24. It was clear, however, that the exhaustive and
periodic nature of the Assessment Reports holds a number of advantages,
including providing a focus for research activities such as climate
Sir Brian Hoskins told us that the benefit of the Assessments
reports is that they produce a large body of results that can
be analysed in great detail by scientists at the same time.
Additionally, the advantages of the all-encompassing nature of
the reports were emphasised by Nicholas Lewis, a climate
researcher and mathematician:
I think there is an advantage to having a fairly
comprehensive assessment at the state of advice put down at one
time because it enables you to examine it as a whole and see what
implications the various bits of it have for other parts and see
whether they are consistent or not. [
] To expect these academic
scientists to do this on top of their normal workload is not the
ideal by a long way, but I think it is useful.
25. Consequently, there was no clear consensus on
how the IPCC should adapt and evolve. As Dr Shuckburgh testified,
there is a "spectrum of views", some wish to develop
a much less burdensome, policy-relevant and frequent assessment
procedure, but others think that this could jeopardise the comprehensive
and authoritative nature of the current process.
What is clear however, is that the final publication of AR5 marks
a good opportunity for the IPCC to gather views and take stock
on its future direction.
Though the work of the IPCC is targeted towards policymakers,
it is essential that the scientists that underpin the Assessment
Reports are well supported and their efforts are adequately acknowledged
to ensure their continued commitment and enthusiasm for a challenging
but important task.
26. There are
mixed views regarding the frequency and size of IPCC Assessment
Reports. Transition to smaller, more frequent reports would arguably
relieve the burden on contributing authors and ensure policymakers
were kept up to date, but the finished document would lack the
comprehensive and authoritative nature of the current Assessment
Reports. Any revision of the tried and tested IPCC formula should
only be introduced after careful consultation with both the governments
who use the IPCC reviews and the scientists who write them. The
aftermath of AR5 is an optimum time for this period of reflection
to take place.
Summary for Policymakers
27. In September 2013, following the second round
of drafting and expert review of the WGI contribution to AR5,
the authors of the report gathered with representatives from United
Nations (UN) member governments in Stockholm. During this four-day
meeting, the 1,536 pages of the Full Technical Report, was reduced
to a 33 page Summary for Policymakers (SPM) and agreed line-by-line.
The meeting was held in private and the notes were not released
following the session (unlike, for instance, the reviewer comments
which were published in full). The SPM is the only part of the
report that many people will read. It is unsurprising that it
is the focus of many of the criticisms of the IPCC's alleged lack
28. The lack of transparency in this stage of the
writing process, and the direct involvement of government representatives,
has led to a great deal of speculation about the degree to which
the scientific report is politicised before emerging as the finalised
SPM. Donna Laframboise
was particularly critical of the process:
Delegations from more than 100 countries were
involved in the four day, behind closed doors, barred to the media
meeting. Politicians, diplomats, and bureaucrats argued about
phrasing and about which tables, graphs, and illustrations should
be included. When they were done, the Summary for Policymakers
was five pages longer than the draft but contained 700 fewer words.
In support of this, Christopher Walter, Viscount
Monckton of Brenchley, claimed that phrases critical of the performance
of climate models had been removed from the SPM on the request
of political agents.
Mr Lewis was also critical of the absence of technical information
that he argued was of importance to the understanding of the conclusions
of the SPM.
29. Sir Peter Williams suggested that there was no
particular cause for concern about the process through which the
SPM is prepared:
The one risk that remains, which the IPCC is
very resilient against, is that the immensely complex science
is ultimately digested by the plenary, which is the government
delegations from around the planet. I think it does remarkably
well not to be conservatised as a result of that process and nor,
from my experience of having attended the plenary, does the message
acquire a degree of interpretation or, as you might term it, a
spin. From what I can see, the IPCC has done a very good job of
resisting those tendencies.
Furthermore, we were informed by WWF that some "observer
organisations" are allowed to sit in on this plenary:
Throughout AR5 WGI, WWF observed constructive
debate by all governments. Points of intervention were factual,
science-based or focusing on improving the clarity of message
for policy-makers. WWF did not observe a single intervention by
any government that questioned the overall substance and/or general
message of the report.
Professor Allen emphasised the importance of involving
policymakers in that final stage:
The reason I think we need that process is that
if you send scientists away and tell them just to write a report
entirely in isolation, it is quite difficult to predict how people
will interpret what they say because we end up just spouting jargon
and nobody knows quite what we mean. You need that dialogue to
make sure that the conclusions of the scientists have been correctly
30. Other IPCC authors were keen to re-iterate the
fact that they have ultimate authority on the final content of
the SPM and that all statements are directly traceable back to
the relevant evidence given in the Full Technical Report.
Professor Sir Brian Hoskins suggested that this level of
traceability in a summary of a technical subject designed for
policymakers, was unusual when compared to other disciplines:
On the traceability of this and the review of
the whole process, Members of Parliament can be more content over
what goes on in this case than, say, much of the information they
get in the economic sphere, I suspect, where they would read the
summary but not know the details and the traceability would be
policymakers in the final stage of the report writing process
does not seem to have had any substantial negative effects on
the Summary for Policymakers (SPM) and very likely serves to improve
the relevance and accessibility of the finished document.
32. It is inevitable
that the distillation of such a complex and lengthy report will
lead to the omission of some technical detail, but the traceability
of the SPM to the full report adequately compensates for that.
Any further technical detail that may be required for policymaking,
such as in the setting of carbon budgets, is readily obtainable.
The SPM succeeds in its purpose of keeping policymakers informed
on issues surrounding climate science.
33. We recommend that the Government call on the
IPCC to introduce a greater level of transparency in the plenary
meetings to agree future Summaries for Policymakers (SPM). This
may be through the admission of the independent team of observers
to oversee the discussions (see paragraph 12). The feedback from
the team would then serve to provide reassurance that the summary-writing
process has been carried out objectively.
9 InterAcademy Council, 'About the IAC,' accessed 15
July 2014 Back
InterAcademy Council, Climate Change Assessments: Review of the Process and Procedures of the IPCC
(August 2010), p26 Back
Department of Energy and Climate Change (IPC 025), Met Office
(IPC 026) Back
Q23 [Dr Stott] Back
Q115 [Sir Peter] Back
Q116 [Sir Peter] Back
University of Reading (IPC 035) Back
Dr Ruth Dixon (IPC 023) Back
Q113 [Dr Shuckburgh] Back
Q98 [Ms Laframboise], Q100 [Professor Lindzen], Christopher Walter
(IPC 005), Dr Ruth Dixon (IPC 023), Marcel Crok (IPC 041) Back
Dr Ruth Dixon (IPC 023), Marcel Crok (IPC 041) Back
Dr Ruth Dixon (IPC 023) Back
Q46 [Professor Allen], Q114 [Dr Shuckburgh] Back
Q46 [Professor Allen] Back
Q3 [Professor Allen], Myles Allen (IPC 037), WeatherAction (IPC 059) Back
Q114 [Sir Peter] Back
Christopher Walter (IPC 005), Alex Henney (IPC 006), John McLean
(IPC 016), Donna Laframboise (IPC 039) Back
Donna Laframboise (IPC 039) Back
Q44 [Professor Allen] Back
Q44 [Professor Hoskins] Back
Ian Strangeways (IPC 022) Back
Q35 [Dr Stott], DECC (IPC 025), Grantham Institute for Climate
Change (IPC 032) Back
Judith Curry (IPC 052) Back
Conor McMenemie (IPC 014), John McLean (IPC 016) Back
Q105 [Ms Laframboise], Donna Laframboise (IPC 039) Back
Q150 [Sir Peter] Back
Q45 [Dr Stott] Back
Q48 [Professor Allen] Back
Q39 [Professor Allen] Back
Q39 [Dr Stott] Back
Myles Allen (IPC 037) Back
Friends of Science Society (IPC 015), Nongovernmental International
Panel on Climate Change (IPC 042) Back
University of Reading (IPC 035) Back
DECC (IPC 025), Royal Meteorological Society (IPC 029), Grantham
Institute for Climate Change (IPC 032), Myles Allen (IPC 037),
Professor Pierre Darriulat (IPC 049), WWF (IPC 054) Back
Q34 [Professor Allen, Professor Hoskins, Dr Stott], Q89 [Professor
Lindzen],Ian Strangeways (IPC 022), Robin Guenier (IPC 024), Royal
Meteorological Society (IPC 029), Grantham Institute for Climate
Change (IPC 032), Jonathan Cowie (IPC 033), Natural Environment
Research Council (IPC 036), Corinne Le Quéré and
Andrew Watkinson (IPC 050), Mark Richardson (IPC 077) Back
Q34 [Professor Hoskins] Back
IPCC, 'IPCC Process,' accessed 15 July 2014 Back
Richard Tol (IPC 040) Back
University of Reading (IPC 035), Myles Allen (IPC 037) Back
Natural Environment Research Council (IPC 036) Back
DECC, Future of the IPCC Review - UK Government 2014 Response to IPCC,
25 February 2014 Back
Q230 [Professor Walport] Back
Q117 [Dr Shuckburgh], Royal Meteorological Society (IPC 029),
University of Reading (IPC 035), Natural Environment Research
Council (IPC 036), Myles Allen (IPC 037) Back
Q61 [Professor Lindzen] Back
Q35 [Professor Hoskins] Back
Corinne Le Quéré and Andrew Watkinson (IPC 050) Back
Q40 [Professor Hoskins] Back
Q101 [Mr Lewis] Back
Q41 [Professor Hoskins], Q117 [Dr Shuckburgh], Q289 [Mr Barker] Back
Q41 [Dr Stott], Q231 [Professor MacKay] Back
Qq287-290 [Mr Barker, Mr Warrilow] Back
Qq58-59 [Ms Laframboise, Professor Lindzen], Q63 [Mr Lewis], Christopher
Walter (IPC 005), John McLean (IPC 016), Madhav Khandekar (IPC 019),
Donna Laframboise (IPC 039), Professor Pierre Darriulat (IPC 049) Back
Qq58-59 [Ms Laframboise, Professor Lindzen], Donna Laframboise
(IPC 039) Back
Donna Laframboise (IPC 039) Back
Christopher Walter (IPC 005) Back
Qq63-69 [Mr Lewis] Back
Q142 [Sir Peter] Back
List of IPCC observers organisations (April 2014) Back
WWF (IPC 054) Back
Q44 [Professor Allen] Back
Q4 [Professor Hoskins, Dr Stott], Q11 [Dr Stott], Q35 [Dr Stott],
Q44 [Dr Stott], Q124 [Sir Peter], DECC (IPC 025) Back
Q4 [Professor Hoskins] Back