Select Committee on International Development Eighth Report


2 Performance: managing, measuring and reporting

The 2001-2004 Public Service Agreement

5. Annex 2 of the Departmental Report 2003 provides a record of progress against each of the targets contained within the Department's 2001-2004 Public Service Agreement. Performance is stated as being 'Below target' for six of the 26 targets. But whilst explanations have been provided as to why underperformance has occurred there is insufficient information about how the Department is addressing this underperformance. We looked at two of these missed targets in more detail during our examination: debt relief and primary school enrolment.

6. DFID shares a target with the Treasury on debt relief (target 2(a)) but the annual report does not tell the reader which part of this DFID is responsible for achieving. The Department told us that it was working on producing joint delivery plans for all joint targets[4] and then outlined how work on the debt relief target was shared with the Treasury.[5] This, plus a separate memorandum,[6] gave a much clearer sense of how the Department is working with others and dealing with underperformance than the Departmental report itself does. For instance, DFID told us that there is a joint team of DFID and Treasury officials in the World Bank and IMF, and there is an ongoing dialogue between the DFID team, other donors, including the international institutions, and the specific country concerned.[7] Target 4(a) is to increase primary school enrolment. Underperformance is attributed to the use of an artificially high baseline of school enrolment rates.[8] Within the report the Department has commented mainly on how data problems have affected performance measurement, and has not provided sufficient information about the steps it is taking to address the underperformance and increase primary school enrolment rates in the countries concerned. In response to our written questions we were informed that steps are being taken in three countries, Zambia, Tanzania and Ghana, in particular to improve enrolment rates, including through significant education spending commitments.[9]

7. We recommend that the Department aim to include more information within future annual reports about how it is pursuing its objectives. In particular there should be more information about what steps it is taking towards the achievement of joint targets, and then what specific contribution it has made to performance against these. Explanations for underperformance against targets are essential but the Department must do more to ensure that these do not become a substitute for adequate descriptions of actual performance. Equally, it must be clear what specific steps are being taken to remedy underperformance. In particular explanations must be included, as addenda to each relevant PSA target, of how problems with data have affected the measurement of performance and how these problems are being addressed.

8. Finally, the Department notes that from 1 April 2003 it will only report progress against the new Public Service Agreement targets.[10] We do not regard this as a sensible or helpful move. It is essential that the Department remains accountable for any outstanding targets from previous Public Service Agreements, rather than just dropping them with each Spending Review. Continued reporting against outstanding targets is also a requirement of the Treasury[11]. There will come a point where old PSAs need to be relinquished. However, for the present we recommend that DFID continues to report progress against any targets or sub-targets which are outstanding from previous Public Service Agreements until they have been met or until the target deadline has expired.

The 2003-2006 Public Service Agreement

9. DFID's new Public Service Agreement (PSA) contains no targets for Objective 3: Reduce Poverty in Europe, Central Asia, Latin America, the Caribbean, the Middle East and North Africa. In addition, the PSA targets supporting each objective do not cover the full range of DFID's activities under that objective. This is consistent with a move across government to pare down the number of PSA targets, and a move within DFID to focus on how to demonstrate its own performance in moving towards the 2015 Millennium Development Goals.[12]

10. The Department assured us that in areas not covered by specific PSA targets there were the same internal reporting and accountability arrangements. This is through Directors' Delivery Plans which underpin each objective and include 'lower-level' performance targets.[13] Progress is reported annually against these to the management board,[14] which should provide them with a balanced picture of performance. It is important that in next year's annual report the Department presents a balanced picture of performance which covers not only the PSA targets but also the other areas of activity which underpin each objective. We welcome the suggestion by the Department that the 2004 Departmental Report be structured around DFID's objectives. Next year's report should present a comprehensive account of activity and performance under each objective including the reporting of progress against internal targets in areas not covered by specific PSA targets. As a matter of good practice this further performance reporting should follow the same format as that concerning the PSA targets.

Resources and Budget

11. In the 2002 Spending Review the Government announced that DFID's budget will increase to nearly £4.6 billion by 2005-06, compared to £3.4 billion in 2003-04.[15] The Spending Review makes the commitment that DFID's spending should result in demonstrable improvements to the lives of the poorest people[16] and the Department considers that the new PSA has added greater transparency to the budget allocation process.[17] But neither the annual report nor the Spending Review makes explicit the linkages between the additional funding secured and the assessment of the increase in key deliverables. We would suggest, in future reports, 'joined up' reporting between the Spending Review and the Departmental Report to demonstrate the links between funds secured and commitments made in the Spending Review and the performance recorded in the Departmental Report.

12. In response to our recommendation in last year's report DFID have simplified their tables of resource allocation for 2002-03, which now show information on resources allocated by objective. [18] Although improved from previous years, the financial tables in the Departmental Report remain confusing and do not offer a sound basis for readers to track financial developments. In particular, it was only after additional questioning of the Department that we learnt more about the basis for the figures and cleared up significant confusion with reference to expenditure figures for specific countries. For example, we queried the swings in spend on the Ethiopia programme line[19] and found out that the 2002-03 country programme figure includes £24.8 million for humanitarian aid but that the future allocations are not divided between forms of aid.[20] It was clear that spending on humanitarian assistance, for a lot of the countries listed, was included in either the country's programme line or a separate 'humanitarian assistance' line,[21] causing significant fluctuations in past and planned expenditure on the country programme lines. The Department told us that significant restructuring has also affected the figures shown,[22] causing similar discontinuities elsewhere in the tables. For example, programmes appearing under the heading 'Innovative Approaches to Development' up to 2002-03 have now been transferred to the heading 'Improving the Effectiveness of Multilateral Aid'. The end result is that it is virtually impossible to identify the reasons for the large changes in spending; often the Department did not seek to explain these.

13. Finally, the financial tables did not fully reconcile to one another. Up until 2002-03 spending is recorded in cash terms but from 2003-04 onwards it is in resource terms. This adds further difficulties to tracking progress and also gives the Department the excuse of 'resource adjustments' rather than properly explaining errant numbers or significant changes. We would like to see clear, consistent and transparent reporting of DFID's past spending and future plans in a way which facilitates tracking progress and demonstrating changes in policies and priorities. We recommend that the Department ensure that in future years financial tables are stated in equivalent terms and that more complete and fuller explanations are provided for year-on-year variations.

14. Because DFID make three year, or longer, financial commitments to many countries we wanted to know the extent to which DFID's future spending is already committed to medium-term development assistance. However, we could not assess from the financial tables the levels of 'free' resources available to the Department. Because of this lack of information we were unable to assess fully their ability to respond flexibly to unexpected changes in direction or events. It would assist understanding of the report if the Department identified the extent to which expenditure plans represent funds already committed to country programmes and projects, as well as providing a full breakdown of direct budget support expenditure and plans.

Flexibility

15. Financial flexibility is key to the Department's strategy of rewarding good performers and sanctioning countries who are not committed to reducing poverty. A DFID paper reviewing their resource allocation process[23] suggests that, whilst the Department is adept at using this flexibility to finance unexpected humanitarian events and respond to economic shocks, there was potential to expand the use of financial flexibility as a means of responding to changes in performance.[24] This is supported by comments in DFID's Development Effectiveness Report that "there is scope for DFID to be more consistent in balancing need and likely performance in resource allocation" and that contributions to multilaterals should be more performance-orientated and "based on a more objective and explicit measure of the organisational and development effectiveness of the multilateral institutions themselves".[25] Nevertheless the Department already has examples of how it has responded to changes in performance and commitment levels. Within a three year budgeting framework there is discretion to change investment levels quickly.[26] For example, in Bangladesh the country team were able to move swiftly to support the government's sudden reform programme in the area of jute mills without coming back to headquarters for approval.[27] At a national level this flexibility can also be seen. For instance, in Tanzania budget support was cut after the government decided to purchase a $40 million air traffic support system,[28] and the Department also provided other examples.[29] We were pleased to learn that the Department is committed to making use of financial flexibility in order to link funds with performance. However we would welcome more information (and acknowledgement) in the annual report about how this process works. There is still work to do in order to make contributions to multilaterals more performance-oriented. We attach a high priority to the Department securing good value for money for these contributions, and demonstrating that it has done so.

16. Flexibility is also essential in an environment where risks as well as opportunities arise which the Department must respond to, but is not generally in a position to control or even influence, such as when dealing with humanitarian crises. The Department realises this and has increased the unallocated portion of financial resources. The resource allocation process now involves looking explicitly at the opportunity costs of allocating money to one area or reallocating it elsewhere; the implications of these decisions are communicated to Ministers.[30] We welcome the Department's explicit appreciation of the difficult aid choices which must be made and the fact that this is communicated internally. There is a continuing challenge for the Department in both handling risk and communicating how it does so. We consider that the annual report could be more explicit about the choices that the Department has to make and how it addresses new risks and opportunities, by providing examples related to specific countries and programmes. Although there is much information on DFID's risk management processes available elsewhere, there is very little within the report itself. Communication of what risks the Department faces and how it is handling them is central to managing the expectations of the public and Parliament. We would also like therefore to see a brief overview of the Department's risk management processes within this report. Finally, we would welcome more information on staff flexibility, and in particular what happens to staff when responding to sudden crises or changes in policy priorities.

Validation and evaluation

17. The continued movement towards budget support will require new methods of evaluating aid outcomes. Often it will no longer be just a particular project which is assessed, but rather a whole series of measures which underpin the Poverty Reduction Strategy, or conditions placed upon direct budget support. This has profound financial implications, as poor performance by an aid recipient carries with it the consequence that its government budget could be slashed if donors withdraw their support. The shift to budget support gives donors greater leverage for influencing policies within a country, and places much greater responsibilities upon donors to monitor aid outcomes. Currently, DFID often acts in concert with other donors and shares the responsibility for assessing aid outcomes and performance. Some of these issues are discussed in more detail in chapter three.

18. There are several processes in place for evaluating the effectiveness of direct budget support and Poverty Reduction Strategies (PRSs). We were told that once a country has developed a full PRS, it is then expected to produce an annual progress report. The Boards of the World Bank and IMF discuss these, "as the basis for continued World Bank and IMF assistance".[31] More general reviews are taking place. For instance, the Operations Evaluation Department of the World Bank is undertaking an "independent review of overall progress in PRS implementation".[32] The impact of funds spent through budget support is currently evaluated in a similar way—joint review work by the World Bank and IMF, within each major country.[33] DFID has also commissioned a study aimed at evaluating direct budget support. This is examined in greater detail in the next chapter. We welcome the work being undertaken to evaluate the success of these new methods of disbursing aid. It would have been relatively easy for the Department to commit funds and then sit back for three years, but instead they have chosen to engage fully and monitor the effectiveness of the aid and commitment of the recipient government to poverty reduction. We will continue to follow this closely and will be looking for evidence that DFID, even when acting in concert with other donors, continues to take hard decisions when UK aid objectives are not being met. Other donors' own objectives may not necessarily fit with those of the UK.

19. The obvious problem, acknowledged by the Department, with this shift in how aid is disbursed is the fact that many developing countries do not have robust statistical bases. This places limitations on effective performance measurement. DFID noted that "poor quality data is characteristic of poor countries generally" but DFID is working hard to strengthen data collection in a number of countrieswithin the framework of a commitment to locally defined indicators and data wherever possible.[34] This was a issue we addressed last year as well. We believe that the Department should continue to attach a high priority to building statistical capacity in the countries in which it operates. Obviously this process will take time, therefore in the interim we would expect a frank summary of the limitations of the performance measurement system as it relates to the Department's Public Service Agreement, and what validation has taken place. This should include any findings from forthcoming NAO work to validate the PSA data systems.

20. Throughout the annual report DFID claims credit for making key impacts upon global development policy and programmes. For instance:

  • "DFID has played an important role in moving forward the global agenda on untying and harmonisation."[35]
  • "In 2003/03 DFID successfully campaigned for five countries with particularly large numbers of children out of school … to be eligible for fast track support."[36]
  • "DFID worked to secure international commitment at Monterrey to the principal that developing countries should have a greater say in shaping the policies of bilateral and multilateral donors."[37]

However, whilst there is a robust framework for reporting progress against policy targets which are PSA targets this is not the case for these internal policy targets. We consider that improvements could be made in how these policy achievements are reported upon, in order to satisfy a sceptical reader who may not necessarily accept that these positive policy outcomes are as a direct result of the Department's work. One way would be to include more details about the original policy target, the timeframe to achieve this, actions taken and other partners.

EC Development Funding

21. The Department have provided details of DFID's resource allocation by objective (Table 2), a significant development from the previous year. However, EC Development Programmes funding, estimated at £865 million or 23% of the resource budget in 2003-04, appears as a separate line with no objective or targets indicated. For such a significant sum it is important for the Department and taxpayers to be clear whether and how this money delivers the government's development objectives. With more volatile exchange rates the £Sterling equivalent of the €Euro amount attributed to the UK can vary significantly year-on-year. Because the total DFID budget is set inclusive of the EU element, this means that adverse exchange rate movements have to be funded out of the remainder of the Department's budget. This is essentially "top-slicing" and reduces the resources the Department can direct to priority areas. We would welcome DFID's view on how it protects its priority budgets against the risk of currency fluctuations.

22. The Department told us that whilst poverty reduction is now at the centre of EC policy, "question marks remain as to how widely the policy has been applied" and that only 44% of EC funding had gone to low income countries in 2001.[38] The EU's record in terms of the share of aid reserved for poor countries remains substantially worse than that of individual member states. DFID also told us that they are "working to produce a framework to measure the effectiveness of multilaterals" and that a key objective is to increase the poverty focus of this EC development expenditure.[39] The enlargement of the EU may pose a new challenge for achieving DFID's objective. The accession countries have small aid programmes and also have understandable interests in their own immediate regions.[40] DFID will need to find ways of ensuring that the accession of the "ten" does not reinforce the tendency for the EU to focus on the "near abroad".

23. The relative weight of the EU in member states' aid programmes has changed significantly in the last decade. The Overseas Development Institute (ODI), in analysing DAC data to show the share of member states' aid channelled through the EU in 1992 and 2001, has identified an increase in the number of countries which provide more than 20% of their aid through the EU.[41] Whatever the reasons behind this shift it provides for the possibility of more countries showing greater interest in the quality of the aid provided thought the EU. As the ODI pointed out, the constituency for reform may be larger and more determined than has been the case to date. The European Commission produces an annual report on EC development policy and the implementation of external assistance. We reviewed the 2001 report. This was a year of reform for EC external assistance and the annual report was detailed and comprehensive. This annual report does not, however, provide a succinct assessment of how the EU external assistance meets the UK's development objectives. That is what is required in DFID's annual report. In order for DFID to meet its own objectives we consider it essential that the issue of reforming EC development issues and developing systems to monitor the effectiveness of this multilateral aid continues to be given a high priority within the Department. In addition, we would like to see more information in the departmental report about how these funds are used, the framework for distributing EC development funds, current shortcomings and limitations in this, including in measuring aid outcomes and DFID's own efforts in this area.


4   Q3 Back

5   Q3 Back

6   Ev 45: response to written Q12 Back

7   Q3 Back

8   Department for International Development, Departmental Report 2003, Cm5914, May 2003, paragraph 1, page 135. See also Q21 Back

9   Ev 45: response to written Q12  Back

10   Department for International Development, Departmental Report 2003, Cm5914, May 2003, paragraph 1, page 139 Back

11   HM Treasury, Public Expenditure System paper, PES (2002) 29, deposited paper 03/722, November 2002, paragraph 14 Back

12   Department for International Development, Departmental Report 2003, Cm5914, May 2003, paragraph 1.15, page 15 Back

13   Ev 38: response to written Q8 Back

14   Ibid. Back

15   HM Treasury, Spending Review 2002, Cm5507, July 2002,paragraph 14.1 Back

16   Ibid. Paragraph 14.2 Back

17   Ev 38: response to written Q7 Back

18   HC (2001-02) 964 Back

19   Department for International Development, Departmental Report 2003, Cm5914, May 2003, table 4, page 126 Back

20   Ev 37: response to written Q2 Back

21   Ibid. Back

22   Ev 37: response to written Q5 Back

23   Department for International Development, Strategic Review of Resource Allocation Priorities, January 2003 Back

24   Ibid. Annex 1 Back

25   Ibid. Paragraph 2.2.2 Back

26   Q6 Back

27   Q7 Back

28   Q30, See also Defence, Foreign Affairs, International Development and Trade and Industry Committees First Joint Report, Session 2001-02, Strategic Export Controls: Annual Report for 2000, Licensing Policy and Prior Parliamentary Scrutiny, for further information on issues surrounding the decision of the Tanzanian Government to purchase a new air traffic control system.  Back

29   Q30 Back

30   Q4 Back

31   Ev 38: response to written Q6 Back

32   Ibid. Back

33   Q28 Back

34   Q21 and Department for International Development, Departmental Report 2003, Cm5914, May 2003, paragraph 2.54 Back

35   Department for International Development, Departmental Report 2003, Cm5914, May 2003. Paragraph 2.63 Back

36   Ibid, paragraph 3.20 Back

37   Ibid, paragraph 2.58 Back

38   Ev 37: response to written Q4 Back

39   Ibid. Back

40   European Development Cooperation to 2010, Overseas Development Institute Working Paper 219, Overseas Development Institute, May 2003 Back

41   Ibid. Back


 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2003
Prepared 30 October 2003