Using performance indicators
15. It is not easy to measure the results of aid
given the difficulties in attributing change to particular donors,
the impact of external events and lack of robust data. DFID uses
logical frameworks for each project or partnership agreement to
define key performance indicators, to track progress within the
project itself, then its main outputs and ultimately the outcomes
achieved against overall project goals.[39]
DFID has tried to set tighter objectives and improve monitoring
arrangements in recent years. But the indicators for recording
achievements under both project and strategic funding to CSOs
have significant weaknesses, preventing DFID from identifying
the development benefits achieved through working with CSOs.
16. The Partnership Programme Agreement indicators
are less robust than those for projects, due to the more strategic
focus and greater flexibility of these Agreements. Indicators
often include improving organisational capacity, encouraging pro-poor
policies in developing country governments and improving partnerships
as well as better outcomes for poorer people. DFID has consciously
chosen to rely on a lower level of detail in monitoring these
activities. But the NAO found that only a quarter of the Agreements
had indicators which were specific and measurable.[40]
DFID faces a dilemma in finding sufficiently measurable
objectives which are not so tightly specified that it loses the
benefits of encouraging more strategic outcomes. Current indicators
are often process oriented so it is hard to identify development
outcomes.[41] And although
some partners identify desired outcomes such as improving market
access, agreements seldom have baseline information to enable
DFID to assess any subsequent progress made against these objectives.
17. In almost half of the projects reviewed by the
NAO the indicators were not sufficiently robust to track progress.[42]
DFID has to balance the requirement for sufficient performance
data to provide assurance with the need to avoid onerous reporting
requirements for small CSOs. Achieving this balance requires careful
planning and sometimes results in delays to projects.[43]
Nevertheless, there are few indicators which assess the achievements
of CSOs working in areas such as advocacy and capacity building,
despite DFID's increased expenditure on these activities. Measuring
results in these areas requires a different type of assessment
than for service delivery projects. Service delivery project results
are often tangible, such as increased number of water pumps, whereas
advocacy and capacity building projects may require more qualitative
assessments of institutional change, using available models of
organisational capacity.[44]
DFID does not consistently use such models. Governance indicators
such as participation levels in community meetings or rights awareness
are another under-used source of information, particularly when
assessing results of advocacy projects. The lack of robust indicators
not only hinders project monitoring and evaluation, but also makes
any fraud and corruption harder to identify.[45]
Results measurement of individual
schemes
18. In addition to specific indicators of individual
projects and Agreements, results measurement is important in monitoring
the performance of the major funding schemes. DFID measures the
results of each project under the Challenge Fund scheme but does
not attempt to evaluate how the scheme as a whole is performing
against its wider objectives such as facilitating dialogue with
smaller CSOs.[46] Nonetheless
DFID increased the funding for this scheme by 40% in 2005-06.
DFID explained that the increase was due to broadening the objectives
of the scheme to cover service delivery in difficult environments.
It accepted that further work was needed to evaluate the impact
of the Challenge Fund as a whole.[47]
19. DFID did not establish performance indicators
for the Partnership Programme Agreements scheme. The first ten
Agreements were awarded to CSOs which already had long standing
relationships with DFID and funding levels were set at broadly
similar levels to previous years' funding under other schemes.[48]
DFID recently paid consultants £30,000 to propose a new performance
measurement system for the Agreements. They recommended the use
of criteria such as organisational effectiveness, influence and
innovation, with implementation from 2005.[49]
But no such performance framework has yet been adopted.
DFID now intends to put a system in place from 2008.[50]
20. Funding received by partners from DFID varies
significantly. For example, Voluntary Service Overseas received
£96.38 million between 2002 and 2004, while WaterAid received
just over £2 million, both through partnership agreements.
These differences do not relate to the performance of the organisations,
but reflect their ability to secure funds from other sources.
[51]
39 Q 4 Back
40
C&AG's Report, para 2.12; Q 71 Back
41
Q 71, 73 Back
42
Q 2 Back
43
Q 17 Back
44
C&AG's Report, para 2.26 Back
45
Q 9 Back
46
Q 3 Back
47
Q 3 Back
48
Qq 36-37 Back
49
Qq 41-42 Back
50
Qq 45-46 Back
51
Qq 47-49 Back
