EVALUATING IMPACT
70. Some of the evidence received in the inquiry
highlights the importance of impact evaluation—assessment
of how effective the aid delivered was in achieving development
outcomes—for encouraging new and better coordinated ways
of working. The submission from the ODI says that evaluation is
central to the Paris Declaration principles and that "efforts
should continue on an evaluation framework to support lesson learning
and a stronger evidence base and to test more thoroughly the underlying
logic in the Paris Declaration".[99]
ActionAid's view is that the current systems for evaluation and
monitoring are not sufficiently credible:
"There is a need for monitoring and evaluation
within the aid system—which is currently largely done by
donors about themselves or each other—to become more independent
and southern-led, for example via a jointly-funded international
evaluation body or recipient-led in-country bodies."[100]
71. There are many new independent impact evaluation
initiatives. There is the OECD's Network of Networks on Impact
Evaluation (NONIE) which aims to establish international standards.
DFID has established an Independent Advisory Committee on Development
Impact to assure the independence of its own evaluation function.
The International Initiative for Impact Evaluation (3ie) is a
leading initiative set up in the last year and from whom we heard
in this inquiry. In his evidence, Howard White of 3ie suggested
that one 'quick win' in terms of more credible evaluations would
be to move from OECD DAC Peer Reviews—in which donors assess
other donors—to reviews of donors by recipient countries.[101]
In a similar vein, UK Aid Network said:
"I think what we need to make sure that DFID
and other donors are doing is supporting country-led evaluation
processes, so this should not be a donor effort to measure development
outcomes in Tanzania—it should be a Tanzanian effort […].
The second thing I would say is […] one of the things we
would like to see come out of these international conferences
this year is a commitment from all the donors […] to actually
establish that kind of independent process internationally because
we do need to be able to hold individual donors to account."[102]
72. Credibility is not drawn simply from the
independence of the evaluation but also from the robustness of
the methodology. We met Professor Finn Tarp of the University
of Copenhagen during our visit there. He and his staff noted that
donor agencies tended to conduct informal self-assessments which
measured outcomes against objectives and could at best prove association
between donor intervention and outcome, rather than more rigorous
counterfactual evaluations which could establish a causal relationship.[103]
We discussed this with the Minister who said:
"I do think it is an evolving area and there
is a recognition that enough has not been done and much more needs
to be done, and in that sense there is a need to invest more in
research to get the right kind of methodologies […] I do
think there is a need to develop better methodology."[104]
73. Credible monitoring and
evaluation of development impact is needed to show objectively
that aid can make a difference. Such assessments both increase
public confidence in the development effort and enable agencies
such as DFID to tailor their programmes to use budgets most effectively.
The credibility of these assessments stems in part from their
independence. Self-assessment and donors assessing other donors,
as happens under the Development Assistance Committee's Peer Review
system, have a role to play but should be complemented by other
initiatives. We recommend that DFID actively support such initiatives,
especially those led by developing countries, and submit to—and
encourage other OECD donors to submit to—reviews conducted
other than by peers.
74. The credibility of impact
evaluations is also drawn in part from their methodology. We recommend
that DFID consider funding more robust impact assessments, such
as counterfactual ones carried out by independent development
economists, to bolster its evidence base.
81 
