6 An EU development and cooperation results
framework
| (35735)
17709/13
SWD(13) 530
| Commission Staff Working Document: Paving the way for an EU Development and Cooperation Results Framework
|
| Legal base | —
|
| Document originated | 10 December 2013
|
| Deposited in Parliament | 22 January 2014
|
| Department | International Development
|
| Basis of consideration | EM of 3 February 2014
|
| Previous Committee Report | None; but see (35700) —: HC 83-xxxi (2013-14), chapter 10 (5 February 2014) and (35144) 11672/13 and (35334) 14081/13: HC 83-xxx (2013-14), chapters 16 and 17 (29 January 2014)
|
| Discussion in Council | May 2014 "Development" Foreign Affairs Council
|
| Committee's assessment | Politically important
|
| Committee's decision | Not cleared; further information requested
|
Background
6.1 The Commission begins thus:
"In an increasingly performance-oriented
society, metrics matter. What we measure affects what we do. If
we have the wrong metrics, we will strive for the wrong things."
— J.E. Stiglitz, A. Sen, J.P. Fitoussi, "Mis-Measuring
our Lives".
6.2 Recalling the line of travel towards a stronger
results focus — the setting of the Millennium Development
Goals (MDGs); the adoption of the 2005 Paris Declaration on Aid
Effectiveness, the 2008 Accra Agenda for Action and the 2011 Busan
High Level Forum; and, most recently, the Report of the UN High
Level Panel on Post-2015[35]
— the Commission notes that over the last decade
a consensus emerged among donors and partner countries around
the following principles:
— ? ownership of development priorities
by developing countries, with the use of country systems as the
default approach;
— ? focus on results through country-led
results frameworks; and
— ? the importance of transparency and
mutual accountability.
6.3 The Commission also recalls that, in the 2011
Communication Agenda for Change[36]
and the related Council Conclusions,[37]
the EU and its Member States committed to promote common results-based
approaches and strengthen their capacity for monitoring and evaluating
development results; and that the Communication A Decent Life
for All provides a further long term perspective of the EU
development agenda in the post-2015 Sustainable Development Framework.[38]
The Commission Staff Working Document
6.4 Against this background, the Commission Staff
Working Document sets out what the Commission is currently doing
to deliver against the commitment in Agenda for Change
to strengthen the Commission's ability to monitor and report operational
results achieved by EU funded development and cooperation projects
and programmes. The Commission describes the paper as presenting
a preliminary approach to the process of drafting an overall EU
development and cooperation results framework, and as describing
how, once finalized and implemented, this framework will bring
together information on results achieved by the EU's development
and cooperation assistance.
6.5 To that end the SWD:
— includes a reflection on results frameworks
that have been developed by some bilateral and multilateral donors,
describing important conceptual issues that have to be addressed
when designing the framework; and
— highlights how the introduction of a
results reporting system could lead to improved management practices,
accountability, transparency and visibility of EU aid, thus enhancing
its impact and demonstrating how funds spent contribute to the
objectives set out in the Agenda for Change and achieve
measurable results.
6.6 In her Explanatory Memorandum of 3 February 2014,
the Parliamentary Under-Secretary of State at the Department for
International Development (Lynne Featherstone) says that the results
framework will track results aggregated from EU funded development
and cooperation projects and programmes, thereby bringing the
EU in line with other development actors who already systematically
report results. The Minister describes results framework's two
main purpose as:
i. "an accountability tool to communicate
results to stakeholders; and
ii. "a management tool to provide
performance data to inform management decisions, ensuring resources
are allocated efficiently."
6.7 The Minister notes that the Commission draws
on the experience to date of four multilateral agencies[39]
and DFID, who already have results frameworks in place, noting
that the "four level" structure of the framework is
the same format as DFID's Results Framework — the levels
being broken down along "the results chain" thus:
— "level 1 describes the global
operating context;
— "level 2 contains the results
towards which the EU has directly contributed by means of EU financed
the projects and programmes;
— "level 3 measures operational
effectiveness; and
— "level 4 measures organisational
efficiency."
6.8 Having explored common problems and key issues
that have arisen from the results framework of other donors, the
proposal identifies the need for:
· "a clear logical link between the
various levels in the results chains, in particular between global
development progress and the contribution of the EU;
· "a balance between coverage and quality
i.e. the importance of keeping the number of indicators manageable
to safeguard that the data collected is of adequate quality; and
· "indicators that can be aggregated
in a robust way across projects and programmes. The Commission
will provide clearly defined methodological notes on acceptable
and comparable data sources in order to ensure this."
6.9 The Minister then turns to issues of attribution
versus contribution, which she defines as whether reported results
should be specifically attributable to the EU's efforts or instead
considered as contributing to the results obtained by partner
countries:
"The paper states that while both approaches
are acceptable from a technical perspective, from an aid effectiveness
perspective, with a focus on country ownership, the contribution
approach is more desirable. The Commission therefore suggests
a contribution approach and reporting results as 'country results
supported'. This is in line with the majority of Multilateral
Banks but not with DFID, which uses attribution. The document
states that the EU will consider attribution where it is possible
to identify results directly linked to their support."
6.10 The Minister further notes that the Commission
recognises the importance of setting baselines to allow results
to be assessed and reported:
"The paper states that setting targets helps
to establish the level of ambition but does not come to a conclusion
about whether setting targets is the right approach for the Commission.
If targets are set, then the EU will need to improve and strengthen
its data collection and measurement techniques in order to achieve
reliable estimates that are able to be meaningfully aggregated."
6.11 With regard to the frequency with which data
should be collected for reporting purposes, the Minister says:
"The five institutions studied use differing
approaches. The majority (World Bank, Inter-American Development
Bank, DFID and African Development Bank) report results from projects
and programmes during the implementation phase. The Asian Development
Bank reports results once projects and programmes have been completed.
The paper does not reach a conclusion on what the Commission's
approach should be and stresses the need for further analysis."
6.12 Finally, the Minister notes, cross-cutting issues,
periodic reviews of the framework and the criteria for the selection
of indicators are considered:
"Crosscutting issues such as gender and
climate change will be incorporated. The paper specifically highlights
gender and the EU's aspiration of sex disaggregated results reporting.
The paper states that any final results framework may need to
be subject to reviews in order to ensure adequate quality and
the inclusion of emerging policy priorities. Final indicators
have not yet been decided. The document states that the indicators
currently used by the five other donors are being examined and
those which are compatible with EU selection criteria will be
incorporated."
6.13 In sum, while the approach proposed by the Commission
draws from donor best practice and will enhance the quality and
scope of the information available to demonstrate EU development
results, while at the same time providing relevant information
for internal management decisions, the Minister highlights these
key issues that remain as yet unresolved:
— "attribution versus contribution",
i.e. whether reported results should be specifically attributable
to the EU's efforts or instead considered as contributing to the
results obtained by partner countries;
— whether targets should be set within
the results framework to help establish a level of ambition;
— the frequency with which project and
programme level data is collected and reported;
— cross-cutting issues, such as gender
and climate change e.g. sex disaggregated data; and
— future reviews of the results framework
e.g. a review to take into account the Post 2015 framework.
6.14 The Commission sets out a timetable for finalising
the results framework and proposes a deadline of September 2014.
The Government's view
6.15 The Minister comments as follows:
"Better, timelier, results data is vital
if we are to secure good value for money in our development programmes
and demonstrate this to UK taxpayers. This is something the UK
has been consistently calling for since DFID's Multilateral Aid
Review (MAR) was first published in 2011. In addition, the UK
is pleased to see that the Commission envisages using this tool
as both a communications product and also for internal management
purposes."
6.16 The Minister agrees that the proposed four level
results framework structure is appropriate, and that a link between
levels 1 (high level global development progress) and 2 (outputs
more directly linked to EU assistance) of the results framework
should be elaborated where possible.
6.17 She then notes four issues about which she is
concerned and which she believes have the potential to devalue
the framework as a management tool: attribution vs. contribution,
the setting of targets, the frequency of reporting and sex disaggregated
data:
"The document states that the Commission
aims to report according to a contribution approach but will consider
attribution where it is possible to identify results directly
linked to EU support. UK officials have encouraged the Commission,
at a working level, to use an attribution approach as much as
possible and to calculate results in shared projects by assigning
pro rata shares of results. The UK believes that not attributing
results implies the possibility of over- or under-selling the
results achieved by the Commission.
"The document does not commit to using targets
within the results framework. The UK believes they are an important
management tool for judging performance and should be set as part
of creating project logframes.
"The UK position is that in order to make
the framework effective as a management tool, reporting results
annually, from all projects, is essential. If results are not
reported annually, on a project by project basis, it is difficult
to identify projects which are under-delivering and to take remedial
action and therefore the rationale of the results framework as
a management tool is weakened substantially.
"The Commission aspires to disaggregate
by sex, where relevant and to a relevant level. UK officials have
asked the Commission for a clearer commitment to disaggregating
data by sex."
6.18 With regard to her own Department's contribution,
the Minister says:
"The UK has actively lobbied the Experts
Working Group on Results in order to reflect the UK position.
We gave presentations at the Working Group meetings on our own
results framework, explaining some of the issues we have faced.
DFID ministers and senior officials in DFID have raised the results
framework as a UK priority with their counterparts in the Commission.
DFID also has two Seconded National Experts working in the Commission
for the unit responsible for the development of the framework.
"In the negotiations of the Implementing
Regulations of the 11th EDF, the UK made significant improvements
to strengthen the focus on results. These include publishing an
annual rather than biennial report on progress, drawing from EuropeAid's
new results framework.
"We will continue to look for opportunities
to influence the design of the framework, including the periodic
reviews which may take place in the future. We will also continue
to press that the Commission delivers against its proposed timetable
for finalising the framework and rolling it out."
6.19 Finally, the Minister says that although implementation
of a results framework will incur costs, she believes these will
be more than offset in the long run by increased value for money
from Commission aid programmes.
6.20 Looking ahead, the Minister says that Council
Conclusions on this Staff Working Document will be prepared through
CODEV (the EU development working group) for likely adoption at
the "Development" Foreign Affairs Council in May 2014.
Conclusion
6.21 Recently, we considered a European Court
of Auditors' Special Report which examined the provision of climate
finance for developing countries by the EU. As we noted, tackling
the impact of climate change in the developing world is high on
the political agenda. Building resilience to it is increasingly
central to the EU's humanitarian work. It will have a higher
profile in the next EIB external lending mandate. The likelihood
is that expenditure on climate-related development assistance
will treble in the 2014-20 financial perspective (from €3.7
billion to €11.6 billion, in 2011 prices). The Court of
Auditors points up areas in which the EU and its Member States'
activity could be more efficient. But the report has nothing
to say on how effective it has been because — as both the
Court of Auditors and the Minister noted — it does not comment
on the outputs and outcomes achieved by the programmes examined.
This is because, as the Commission Staff Working Document demonstrates,
both here and more widely, the Commission/EEAS has only now committed
to putting in place a comprehensive results framework. With a
trebling of expenditure in this area, such a results framework
was, we said, more than ever vital (and all the more so in view
of the €58 billion of total external action funding in the
2014-20 financial perspective).[40]
6.22 The subject was also covered in a recent
European Committee debate concerning another Court of Auditors'
report, which examined €1.3 billion of EU support for governance
in the Democratic Republic of the Congo. We recommended this
debate because the common denominator between this Court of Auditors'
report and many other documents that the Committee has considered
over the years is the effectiveness with which the EU —
the Commission and the European External Action Service —
has spent EU taxpayers' money in this and other development and
cooperation work.
6.23 As the Minister suggests (c.f. paragraph
6.19), there is also a great deal of read-across to the European
Development Fund (EDF: the main instrument for delivering EU cooperation
under the Cotonou Agreement with ACP States and the OCTs). EDF
11 (2014-20) will total €31.6 billion. The UK's share is
14.68 %. Over the past six months, we too have considered the
updating of the EDF regulations to which she refers, which lay
down the detailed procedures for managing this expenditure.[41]
It was thus illuminating to hear the Minister say during the
debate:
"Centrally, we have been working closely
with the Commission over the past couple of years to improve management
by results, which was a major theme of DFID's multilateral aid
review of 2011 and the update, published only two months ago.
It committed us to working with the EU to improve: results; monitoring
and reporting; working in fragile contexts; and partnership behaviour.
The Secretary of State raised the matter at the highest level
in the Foreign Affairs Council in Brussels, and DFID has shared
best practice on our results framework. As a result, the Commission
is in the process of adopting a strong, new central results framework,
which will have a clear effect on monitoring, evaluation and reporting.
"In parallel, we are leading reform,
through negotiations, of the 11th European development fund for
African, Caribbean and Pacific countries, which will run from
2014 to 2020. According to the new regulations, each programme
in future will be conflict-sensitive, set out realistic and achievable
output and outcome targets, and be transparent about risks and
mitigation. We are focused on ensuring that programmes can demonstrate
tangible results, which are reported back to the EDF committee
and which link to reporting from the Commission's results framework.
The changes will, I hope, drive more active managing by results
across EDF programmes and give greater transparency, enabling
member states to hold the Commission to account when it is not
delivering the performance that we expect.
"All that effort should start translating
into improved programming from 2015, when the first EDF 11 programmes
are implemented. Member states should receive the first annual
report with results data by the beginning of 2016. That is not
to say that everything is going well; clearly, real and serious
issues are impacting on the value for money we seek from our contribution
to the EDF."[42]
6.24 This is all well and good. The Minister
refers to a commendable degree of UK involvement thus far. However,
beyond saying that she and her officials will "continue to
look for opportunities to influence the design of the framework,
including the periodic reviews which may take place in the future
... [and] ... to press that the Commission delivers against its
proposed timetable for finalising the framework and rolling it
out", and referring to prospective Council Conclusions in
the Spring, she is not clear as to how, and over what timescale,
the aspects of the Commission proposal that she believes would
undermine its effectiveness are to be definitively addressed.
6.25 We would accordingly like to hear from the
Minister, no later than 24 April, about what progress has been
made, what elements she is by then seeking to have included in
the Council Conclusions, and how she then expects any remaining
deficiencies to be put right.
- In the meantime, we shall retain the Commission
Staff Working Document under scrutiny.
35 A New Global Partnership: Eradicate poverty
and transform economies through sustainable development:
Report of the High Panel of Eminent Persons on the Post-2015 Development
Agenda (2013). Back
36
Available at http://ec.europa.eu/europeaid/infopoint/publications/europeaid/documents/257a_en.pdf. Back
37
Available at http://www.consilium.europa.eu/uedocs/cms_Data/docs/pressdata/EN/foraff/130243.pdf. Back
38
Available at http://ec.europa.eu/europeaid/documents/2013-02-22_communication_a_decent_life_for_all_post_2015_en.pdf. Back
39
Asian Development Bank, African Development Bank, World Bank Group,
Inter-American Development Bank. Back
40
See headnote: (35700) -: HC 83-xxxi (2013-14), chapter 10 (5
February 2014). Back
41
See headnote: (35144) 11672/13 and (35334) 14081/13: HC 83-xxx
(2013-14), chapters 16 and 17 (29 January 2014). Back
42
The record of the debate is available at: http://www.publications.parliament.uk/pa/cm201314/cmgeneral/euro/140205/140205s01.htm.
(Gen Co Deb, European Committee B, 5 February 2014, cols.
3-12). Back
| 
