Select Committee on Public Accounts Thirtieth Report


3. Achieving a sustainable impact from projects

19. Achieving a lasting beneficial impact is one of the key objectives of development assistance. Between 1997 and 2002 the Department completed 193 dedicated water and sanitation projects, which represented a commitment of £61 million. The Department's evaluations of these projects showed that, where assessments were available, three quarters had completely or largely achieved their objectives (Figure 4).[65] But there was a lack of available evidence to assess the extent to which these projects were achieving a lasting beneficial impact. Two thirds of project completion reports that included an assessment of sustainability raised doubts and risks as to whether a sustainable impact would be achieved.[66] There are risks inherent in providing development assistance in developing countries and the Department's achievements must be considered within this context. The Department has also tried innovative approaches in risky places in order to achieve development outcomes.[67]

Figure 4: Achievement of objectives for water projects completed between 1997 and 2002


Note: Output level rating is used to assess the project's planned results. Purpose level rating is used to reflect whether the intended changes have occurred in the host country as a result of those outputs.

Source: NAO analysis of Project Scoring Information

20. The Department has responded to the need to achieve a sustainable impact in a number of ways.[68] The Department has recognised that unless the broader policy framework in developing countries is strong, then individual projects are not going to be sustainable.[69] Improving governance has been a major focus of the Department's bilateral aid programmes for a number of years as it has recognised that building effective institutions is an important pre-condition of achieving sustainable development.[70]

21. The Department also seeks to address sustainability issues in project design. The National Audit Office found that some recognised good practice criteria received less attention than others. Most commonly, projects did not consider the availability of replacement parts, the use of cost-effective technology and hygiene awareness issues.[71] The Department told us that it is possible to break down the components of sustainability into five key factors (Figure 5).[72] In the past, too many projects have been designed without addressing all of these issues.[73] The Department has published a series of strategy papers to inform staff thinking on the priorities and methodologies that should be considered when developing country programmes. These papers include "Addressing the Water Crisis" and "Achieving sustainability, poverty elimination and the environment". However, little attention has been given to knowledge management within the Department or the best approach to disseminating to country teams the good practice and lessons learned from project evaluations.[74]

Figure 5: The five key components of sustainability
Political support: ensuring that there is host government to addressing the issues in that sector.

Economic sustainability: ensuring that the appropriate charging structure is in place to ensure that revenue is raised for operation and maintenance issues.

Social sustainability: addressing the community's requirements are understood and addressed in the project.

Technical issues: ensuring technology is appropriate to the community's needs.

Policy and institutional issues: ensuring the institutional infrastructure is in place to manage the projects and deliver services.

Source: Ev 1, Q 8

22. The inability of local partners to manage donor-funded interventions can lead to problems with sustainability. In the Two Regions Project in Ghana, the partner government did not have the capacity to address issues that emerged during project implementation, thus leading to a reduction in the effectiveness of the Department's funding.[75] The Department accepts that, in the past, too many projects have given insufficient attention to what happened after funding had ended and expertise had been withdrawn. Greater attention is now being given to the skills and institutional structure that are needed to manage the project after the Department's involvement ends.[76]

23. Increasingly, the Department's water projects have focused on strengthening the capacity of those bodies responsible for water delivery and transferring knowledge from its advisers to developing country administrations.[77] The aim is to secure a longer-term impact, and the Department believes that it has a comparative advantage over other donors in providing this type of assistance.[78] This has led to successful projects which have had a lasting beneficial impact. For example, a Departmental water project in Maharashtra in India led to the introduction of a new charging structure which enabled the government to charge those who could afford to pay and raise funds to maintain systems.[79]

24. The Department has a well-established approach to evaluating its projects. Country teams are required to carry out project completion reports for all bilateral projects with a commitment in excess of £1 million.[80] However, the Department's performance in completing such reports has been poor. Project completion reports should have been carried out for 32 of the 193 water projects completed between 1997 and 2002, but only 17 reports had been produced. In value terms, this represented coverage of 56% of eligible water projects.[81] The lack of available information means that it is not possible to determine fully the value for money of the Department's assistance[82] or undertake a comprehensive assessment to identify those factors leading to project success. The Department has devoted considerable efforts to improve compliance with its project evaluation procedures and told us that twice as many projects were reviewed last year compared to previous years.[83]

25. The main purpose of project completion reports is to assess the extent to which the project goal and planned outputs have been achieved. But there is less emphasis on assessing sustainability or the wider impacts of projects, often because they are completed too early to judge the longer-term impacts.[84] The evaluation of impacts needs to be considered at the outset of the project and, as necessary, appropriate baseline information needs to be collected to assess outcomes. The Department has not always given sufficient attention to the need to establish robust monitoring arrangements at the project design stage.[85]

26. The Department also carries out a rolling programme of ex-post evaluations to assess the long-term sustainability and wider impacts of projects. However, due to the wide range of its activities, the Department is only able to achieve partial coverage in any one year.[86] The Department is increasing the resources devoted to evaluations by 25% in each of the next two years and is establishing a direct link to the management board.[87] Efforts are also being made to improve the quality of evaluations[88] and make them more timely and relevant.[89] As part of the increased evaluation effort, there is also scope to revisit projects some years after funding has stopped to assess impacts and learn lessons.[90]


65   C&AG's Report, para 7 Back

66   ibid, para 8 Back

67   Q 49 Back

68   C&AG's Report, para 1.13 Back

69   Q 19  Back

70   C&AG's Report, para 1.13  Back

71   ibid, para 1.15 Back

72   Q 8 Back

73   Q 8 Back

74   C&AG's Report, para 1.22 Back

75   ibid, para 1.10 Back

76   Qq 9, 76 Back

77   Q 9 Back

78   Qq 19, 53 Back

79   Q 32 Back

80   C&AG's Report, para 1.18 Back

81   ibid, para 1.8 Back

82   Q 11 Back

83   Q 11 Back

84   C&AG's Report, para 1.19 Back

85   ibid, para 1.20 Back

86   ibid, para 1.21 Back

87   Qq 11, 57 Back

88   C&AG's Report, para 1.18 Back

89   Q 11 Back

90   Qq 52, 57 Back


 
previous page contents next page

House of Commons home page Parliament home page House of Lords home page search page enquiries index

© Parliamentary copyright 2003
Prepared 3 July 2003