Written evidence submitted by the Institute
of Development Studies
ABOUT THE
INSTITUTE OF
DEVELOPMENT STUDIES
1. The Institute of Development Studies (IDS)
is a leading global charity for international development research,
teaching and communications. Founded in 1966, the Institute enjoys
an international reputation based on the quality of its work and
its commitment to applying academic skills to real world challenges.
IDS aims to challenge convention and to generate fresh ideas that
foster new approaches to development policy and practice. Such
problem-focused thinking requires a commitment to multidisciplinarity,
not just within social sciences, but across research, teaching
and communications.
EXECUTIVE SUMMARY
2. IDS welcomes the opportunity to provide evidence
to the International Development Committee. We are broadly supportive
of DFID's renewed emphasis on the assessment of effectiveness
and the proposal to establish a new Aid Watchdog. Our own recent
research bears out that aid effectiveness is also a priority for
the UK public.
3. However, we have some concerns and reservations
about DFID's approach to assessing the effectiveness of aid: in
particular, the potential for development priorities to become
distorted, accountabilities to be blurred and the focus on cost
cutting to put the vital work with the very poorest people in
peril.
4. We organise our comments around the four issues
highlighted in the call for evidence.
DFID'S APPROACH
TO ASSESSING
THE EFFECTIVENESS
OF AID
AND THE
ROLE OF
DFID'S NEW
AID WATCHDOG
5. IDS's recent work with the UK Public Opinion
Monitor[156]
shows that the UK public say development aid is morally right
and that the UK should play a leadership role in efforts to alleviate
global poverty. However, amid widespread concerns that most aid
is wasted, they also say it should be cut to deal with the UK
budget deficit. This supports the Government's renewed emphasis
on assessment of effectiveness, particularly the focus on impacts.
6. The emphasis has the potential to help reassure
the UK public and to improve the effectiveness of UK Aid. However,
we have a number of concerns:
7. Too great an emphasis on outcomes can
actually reduce accountability. Consider the MDGs. They
are mostly outcome focused, but it is very hard to hold any given
actor or process to account as none on their own has control of
outcome delivery. We still need to track inputs and outputs, but
relate them to outcomes. Specifying all three allows the testing
of theories of change as well as the assumptions and risks inherent
in them. A failure to get the balance right runs the risk of having
a perverse effect. DFID and its partners might well become ever
less responsible for checking on how and why they are having an
impact. Reluctant compliance will surely be accompanied by a tendency
to resistance.
8. The investments for which impacts are
easiest to assess will be the ones prioritised. Investments
in processes, participation, governance, health systems and women's
empowerment, for example, do not yield ready indicators and do
not submit to randomised impact studies as easily as health, social
protection and education interventions do. For example, it is
possible but less easy to relate investments in health system
governance to certain types of health outcomes. Or to relate investments
in strengthening citizen participation to development outcomes.
9. Will increased accountability to UK
citizens crowd out accountability to citizens in aid recipient
countries? Aid involves many sets of actors with different
ideas about what is success and how to achieve it; this needs
to be reflected in methodologies for defining and assessing the
impact of aid. Evaluations have multiple objectives: to improve
interventions in real time, to build relationships with intended
beneficiaries, to meet donor needs, to demonstrate impact, and
to generate global public goods. Convergence and overlap is far
from automatic. In an environment where resources for monitoring
and evaluation are unlikely to grow (they are usually the first
thing cut in downturns) the more powerful will get their needs
met first, and donors are the most powerful. Accountable states
depend on empowered citizens. When selecting the aid instruments
they use, what they fund and how they deploy staff, DFID should
always be mindful of the implications for strengthening active
citizenship.
10. Does DFID have the capacity to use
the learning and information generated? This one could
go either way. One could argue that, by commissioning more focused
DFID-demand led evaluations, the appetite of DFID staff to use
results will increase. And the DFID research fellows are, we imagine,
a useful way of brokering knowledge for DFID policymakers. But
in the light of further staff cuts, our concerns persist. Using
information intelligently is a time consuming task.
11. Will non-DFID ODA be subject to the
same poverty reducing tests as DFID ODA? The Aid
Watchdog must be an ODA Watchdog, not just a DFID Watchdog. It
should report directly to the IDC. As more ODA is used in the
MoD, FCO, DECC, DEFRA, HO and others, it must be shown - and be
seen - to have an impact on poverty. If not, DFID's work will
lose credibility.
12. The Aid Watchdog needs to be pluralistic
in its approach. Using experts to conduct randomized control
trials (RCT) fills one gap. But if you only have a hammer in your
toolbox, soon enough everything begins to look like a nail. The
issue has to drive the methods. And it is not only tools that
need to be pluralistic, it is the defining and framing of the
issues. Different groups have different priorities and different
definitions of success. The IDC has an exciting opportunity to
shape a pluralistic vision for the Watchdog, one that combines
multiple methods and perspectives to serve multiple accountabilities.
13. Impact needs to be communicated more
effectively. Much of the cynicism towards aid stems
from the way aid effectiveness is communicated. This needs to
be completely refreshed. We are not talking about promoting DFID,
but about creating an enabling environment for $100bn a year of
ODA to make a bigger difference. The improved balance between
inputs (spend) and outcomes (impacts) will help. But instead of
relying on DFID partners to tell the good stories, let's also
hear from the intended beneficiaries directly. Instead of being
exposed to an implausibly unbroken string of successes, let's
hear about the occasional failure and what DFID has learned about
what to do differently next time. Attention also needs to be given
to the most effective ways in which to communicate aid successes
(and failures) to the public, taking account of prevailing attitudes
and the most appropriate ways in which to frame messages. This
suggests that past efforts to monitor and understand public attitudes
towards development and to engage with the public on development
issues should be maintained.
14. Value for money (VFM) is not the same
as low cost. As DFID notes VFM=economy x efficiency x
effectiveness. It is about outcome per pound. But sometimes it
is necessary to spend more to reach the poorest, because the increased
benefits of doing so more than outweigh the increased costs (see
UNICEF's new Equity Strategy under Tony Lake's leadership). If
costs were the main driver, efforts to reach the poorest would
be put in peril. Efforts to reach those in future generations
should also be given due consideration. For example, the DFID
Annual Report details global performance on the MDGs and the specific
performance of 22 DFID priority countries but does not discuss
sustainability of these results. We don't have any information
about the progress made in paying for these results from
a country's own resources. Less efficiency, measured in pounds
spent per service provided, may be defensible when spending is
directed towards domestic policies and institutions that can ensure
sustainable service provision.
15. Decisions about what to cut in future should
be guided by evidence of effectiveness, not what is most likely
to be "noticed" by the public. It is easy to take the
recently leaked DFID letter out of context. Nevertheless it did
suggest that the strength of public opinion mattered as much (or
even more) than "the things that aid can best deliver and
that can make a real long term difference" in determining
DFID priorities. If DFID is going to be driven by value for money,
it must show how public opinion strengthens rather than undermines
that approach.
REDUCTION IN
DFID'S ADMINISTRATION
COSTS
16. It is hard to argue with reductions in costs
that do not undermine a stated purpose. The key question is what
is given up as a result of cost reductions.
17. Any reductions in administration costs should
protect DFID's ability to deliver aid and to act on the aid effectiveness
evidence generated. Being more effective, with a bigger spend,
in more fragile contexts, sets up a situation where cutting certain
types of staff will be counterproductive.
18. Increasingly, and especially in fragile states,
there is an awareness that "cookie cutter" approaches
need to be supplanted by context-specific "will and capacity"
assessments lending direction to locally-identified priorities
and ways of working. Well-informed staff that are capable of such
policy assessments are critical to aid effectiveness.
19. In conclusion, DFID needs to provide convincing
formative evidence that spending more money with less staff
will deliver better results, and it should outline exactly how
it will test this hypothesis.
DFID'S USE
OF TECHNICAL
ASSISTANCE AND
CONSULTANTS AND
WHETHER REDUCING
THIS EXPENSE
COULD BE
AN APPROPRIATE
TARGET FOR
THE NEW
GOVERNMENT
20. The use of technical assistance and consultants
is a way of deepening and broadening DFID capacity in a flexible
way. But it demands a certain type of management capacity within
DFID: developing TORs, identifying consultants, evaluating bids,
communicating goals, managing relationships, engaging in joint
learning, assessing the quality of the work, and using the work.
21. We would like to see:
- · DFID
evaluate its past use of TA and consultancy;
- · Greater
use of in-country consultants to enhance the relevance of DFID's
work;
- · Consultants
from the countries concerned and with local knowledge can play
an important role in identifying innovative solutions to complex
problems;
- · Greater
use of consultants that can help DFID bridge effectively with
other sectors outside of "development".
THE WORK
OF CDC, A
DFID OWNED FUND
MANAGEMENT BUSINESS,
WHICH INVESTS
IN THE
PRIVATE SECTORS
22. We strongly support an expansion of DFID's
ability to work with small and medium scale businesses in the
countries in which they work. This is where sustainable and equitable
growth will originate from. We suspect that CDC investments, along
with other private-sector growth initiatives such as the PIDG,
support to the IFC, and, beyond DFID, the ECGD, are justified
because they "crowd-in" private finance or because they
take investment risks that the private sector shun but which are
important for development results. How well in practice do the
portfolios perform against these two important aims? Specifically,
how compelling is the evidence that they contribute to poverty
reduction? DFID should support more evaluation work in this area.
CONCLUSIONS AND
RECOMMENDATIONS
23. The Committee should ensure that in the renewed
focus on effectiveness, valuable investments that are less easily
measurable are maintained.
24. The Committee should push for the new Aid
Watchdog to be an ODA Watchdog.
25. The Committee should call for the new Aid
Watchdog to develop a pluralistic approach, combining multiple
methods and perspectives to serve multiple accountabilities.
26. We would call for careful consideration to
be given as to how to best to communicate the results of these
assessments to the public in a credible and convincing way that
supports the ring-fencing of the aid budget.
27. The Committee should ensure that DFID and
the Aid Watchdog remember that value for money is not the same
as low cost - this is particularly relevant when designing work
with the very poorest people.
156 Aid to Developing Countries: Where does the
UK Public Stand?, the first report of the UK Public Opinion
Monitor (UKPOM), 8 September 2010 Back
| 
