Annual letter from David Peretz, Chair,
Independent Advisory Committee on Development Impact (IACDI) to
the Secretary of State for International Development concerning
DFID's Evaluation Policy
2 December 2009
THE RT
HON DOUGLAS
ALEXANDER MP
Secretary of State for International Development
DEAR DOUGLAS,
Independent Advisory Committee on Development
Impact (IACDI)
Under the Committee's terms of reference, I
am required to send you an open letter every year. More detailed
information about the Committee's work, including minutes of our
meetings, is available on the IACDI website.[106]
In the past year IACDI has carried out an in
depth review of the quality of DFID evaluations, based on a study
led by Roger Riddell, a member of IACDI.[107]
It confirms the provisional judgment we made last year that the
quality of DFID evaluation is similar to that of other leading
bilateral agencies, and our view that DFID can and should aim
for a higher standard. Our recommendations, grouped under 11 headings,
have been shared with DFID management and posted on IACDI's website.
Action on many of them is already underway, and we know that implementing
them all will take time. I am attaching a list of five key recommendations
that we believe should be given priority attention.
In last year's letter[108],
I focused on proposals designed to reinforce the independence
and effectiveness of the central evaluation function in DFID.
They were limited to changes that can be implemented within the
existing institutional framework, without major structural change
or legislation. We welcome actions taken in response, and in particular
the adoption of DFID's new Evaluation Policy[109]
(Building the Evidence to Reduce Poverty). It includes
a new mandate for DFID's Evaluation Department (EvD), and commits
DFID to implementing most of our proposals. We are conscious that
independence should not mean isolation, and therefore welcome
the proposals to give EvD a new role in seeking to enhance the
quality of decentralised evaluations carried out in DFID, and
to ensure that DFID draws on the lessons of evaluation to improve
the impact of its development interventions. But there remain
two areas of concern.
— First, to match good evaluation practice,
to reinforce independence and to give greater visibility and influence,
the Head of EvD should have a more senior grade and status and
report directly to the Permanent Secretary. We welcome your commitment
to review this issue next year and look forward to contributing
to that review.
— Second, while we welcome the decision
to increase the administrative resources available to EvD to reflect
the additional tasks it is taking on, we remain concerned to ensure
that sufficient resources are available for evaluation and intend
over the next year to carry out a more detailed review of the
level of resources needed for an effective central evaluation
function. We also believe there should be more flexibility than
hitherto in the use of resources to finance externally-contracted
work both for centralised and decentralised evaluations. Good
quality evaluation helps to increase the value for money and impact
of DFID programmes and recipient countries clearly benefit from
lessons learned about how DFID interventions can be made more
effective, and it we believe it wrong that the quantity and effectiveness
of this work should be constrained by DFID's declining administration
costs budget, while DFID's programme expenditure is increasing
so significantly. We will continue to press strongly for increased
flexibility in this respect.
Action on these two outstanding issues and implementation
of the other proposals made last year as well as the recommendations
of our quality review should lead to a major improvement in the
quality and effectiveness of DFID's evaluation work and its use
in improving DFID's development impact.
Turning briefly to other matters:
— Earlier this year the Committee approved
both a programme of central evaluations[110]
to be carried out in 2009/10, and a list of evaluations we expect
to be carried out over the next three years. We are keeping this
list and evaluation priorities under review.
— This year's annual report by the Head
of EvD (will be available on IACDI website, Friday 4 Dec), draws
together lessons from evaluations completed over the last year.
It highlights lessons learned in fragile states which will be
highly relevant to important elements of DFID's future work, and
more generally it stresses the need to do more to ensure that
lessons from evaluation are integrated into how DFID works in
a real way.
— In addition to the resource issue, we
intend over the next year to give priority to considering how
DFID can better ensure that the lessons of evaluations are used
to inform decisions on new interventions and aid allocations,
and effectively communicated to external partners including recipient
countries. In this context, we also plan to examine DFID's internal
systems for assessing the success and impact of development interventions.
In conclusion I should express my appreciation
for the time you and your senior officials have taken over the
past year to meet with me and with the Committee to hear our views.
I also welcome the commitment to high quality and independent
evaluation to underpin policy decisions and accountability for
the impact of UK aid expressed in your reply to my letter last
year. I look forward to our next meeting.
I am sending copies of this letter and attachments
to members of the International Development Committee of the House
of Commons.
David Peretz
Chair, Independent Advisory Committee on Development
Impact
Annex
Review of the Quality of DFID's Evaluation
Reports and Assurance Systems
SOME KEY FINDINGS AND RECOMMENDATIONS FOR
PRIORITY ACTION
1. DFID top management should take a lead,
as it has committed to, in reshaping defensive attitudes of the
past into a culture where independent evaluation is welcomed and
championed as an essential contribution to lesson learning and
accountability. This will provide the necessary context for successful
implementation of many of the review's detailed and more specific
recommendations, and equally many of those detailed changes in
systems and practices will help support the needed change in culture.
2. Decisions about what to evaluate and
when need to reflect DFID policy directions at the corporate and
country level, so that lessons learned from evaluations can contribute
effectively to decision taking in a timely way. We ask DFID management
to work with IACDI to help make this happen, while maintaining
a balanced programme containing evaluations that also contribute
to accountability for past actions and are of relevance to DFID
staff and others at the operational level.
3. As a matter of good practice, all major
new development interventions by DFID should be "evaluable",
with performance frameworks set at the planning stage that provide
clarity about what success is expected to "look like",
and that identify and specify the quantitative and/or qualitative
evidence that will be monitored and used to judge performance,
including value for money, and that address the issue of attribution.
4. The management response to evaluations
needs to be made clearer and more transparent, both to provide
higher quality initial responses to evaluation findings and to
ensure that where lessons for the future are identified and agreed
these are followed up and applied wherever relevant in the Department's
work. To ensure that this happens the review recommends that a
senior management committee be made responsible for considering
and assuring the quality of the Department's interactions with,
response to, and lesson learning from all evaluations managed
by the central Evaluation Department and major decentralised evaluations.
5. Existing efforts should be strengthened
further to ensure that evaluations supported by DFID promote the
goals of the Paris Declaration and Accra Agenda for Action for
aid alignment and mutual accountability between donor and partner
countries. In particular DFID should give greater priority to
working together with other donors to help build and use evaluation
capacity in recipient countries to carry out evaluations and to
lead joint evaluations with donors. DFID should also promote better
cooperative practices among development partners for identifying
issues for evaluation, avoiding overlap, and for managing and
assuring the independence and quality of joint evaluations when
they take place.
106 http://iacdi.independent.gov.uk/ Back
107
http://iacdi.independent.gov.uk/wp-content/uploads/evaluation-quality-review-synthesis4.doc Back
108
http://iacdi.independent.gov.uk/wp-content/uploads/annual-letter-and-press-release1.pdf Back
109
http://www.dfid.gov.uk/Documents/publications/evaluation/evaluation-policy.pdf Back
110
http://www.dfid.gov.uk/documents/aboutdfid/performance/evdworkprogramme-0809.pdf Back
| 
