House of Commons - International Development

Select Committee on International Development Twelfth Report

8 Implementation, Monitoring and Evaluation

Implementing the Strategy

98. We very much welcome the substantial funding commitments which DFID has made, particularly the £6 billion allocation over seven years for health service strengthening. However, we agree with Interact Worldwide that the allocation of this funding is not well articulated in the Strategy and it is not yet clear how it will be broken down between different funding streams and different countries.[182] As ActionAid commented, "implementation is the real challenge".[183] Yet it is worth noting that the chapter in the Strategy on How we will turn our strategy into action is the shortest one in the document.[184] MSF accepted that £6 billion "is a substantial financial commitment" but added that "disaggregated data on the different budget lines would be a telling indication of the financial rather than the rhetorical priorities within DFID's overall HIV/AIDS response."[185]

99. The Minister told us that the role of DFID's country offices, and its dedicated country programmes, would be crucial in ensuring that the Strategy as a whole, and the sector support aspect in particular, was implemented effectively.[186] DFID described its approach to implementation in the following terms:

In countries with strong commitments to development, good governance and improving capability, we tend to focus on supporting the development and implementation of comprehensive country-led HIV and AIDS strategies, directly funding governments, as well as working with civil society and international donors and agencies. In more fragile states, where governments may be unable or unwilling to respond effectively, we tend to provide technical support to strengthen government capacity alongside direct support for service delivery via the UN or civil society.[187]

100. Witnesses emphasised to us the value of taking a country-specific approach. Dr Buse told us that this was particularly important in relation to health system strengthening where DFID should aim to identify the weaknesses affecting the health service in a particular country and then to seek to address them specifically, rather than adopting a broad-brush approach which treated every country as if the problems were identical. He highlighted that global targets, such as the one DFID has set of ensuring that there are 2.3 health professionals per 1,000 population, ignores "national specificity and differences" and were difficult to work with.[188] It was important to assess "from a country perspective what needs fixing"—whether that might be more health professionals, better diagnostics, or improved health surveillance—and then develop a plan to address that specific need: "that way you get the variety of stakeholders involved in owning whatever kinds of outcomes or targets you are trying to seek."[189] Dr Buse also emphasised that the form the epidemic was taking, and the required response, were country-specific.[190]

101. DFID makes clear in the Strategy that it understands how important it is to 'know your epidemic' because "when interventions are tailored to local circumstances, the money will have a greater impact."[191] Its written evidence explained that:

Working in line with the AIDS Strategy, country offices are responsible for the design and delivery of HIV and AIDS responses as agreed in negotiation with the host government and other key stakeholders, and taking into account the local context and the constraints of DFID's overall financial framework. […] Decisions on which aid instruments should be used are taken by DFID offices at country level, depending on what is more appropriate for the situation in that country.[192]

The Minister accepted that different countries faced very different circumstances but he believed that DFID was able to address this effectively because "one of the strengths that DFID has is its country offices and its country programmes where there is a very high level of devolution within this organisation compared to many other government bodies".[193]

102. The Strategy and DFID's written evidence have provided us with some examples of how its country offices are responding to the different challenges faced in implementing its HIV/AIDS strategies in partner countries. In Bangladesh DFID is supporting a major urban health programme which will address stigma and increase the focus on prevention of HIV in women and children, in line with the country's own AIDS strategy. In Lesotho, DFID's work has focused on supporting the establishment of the National AIDS Council, legislative reform and support for the People Living with HIV network.[194] In 2007, we saw for ourselves the valuable and innovative work DFID was undertaking in partnership with the Government of Vietnam to tackle HIV/AIDS amongst sex workers and intravenous drug-users.[195]

103. There are many excellent examples in the Strategy of HIV/AIDS work which DFID is undertaking with specific countries and specific groups. What is not clear to us, however, is the extent to which DFID intends to scale up or replicate these projects elsewhere.

104. The Strategy states that individual countries' Poverty Reduction Strategy Plans, which determine how development assistance will be used, and other national development plans "should reflect AIDS plans".[196]It also emphasises the importance of the International Health Partnership in this respect. As discussed earlier, this is a partnership of donors, developing countries and multilateral agencies launched in 2007 by the UK Prime Minister to reinvigorate progress towards the health Millennium Development Goals. One of the IHP's two guiding principles is that "developing countries should prepare robust national health strategies that reflect national AIDS plans that highlight the need for stronger health systems". The IHP is being piloted in eight countries.[197]

105. We agree with our witnesses that the significant funding commitments which DFID has made in the Strategy are impressive and that its analysis of the current situation is excellent. However, the challenge remains for DFID to turn the rhetoric into practical implementation and to demonstrate much more clearly how it will achieve the targets it has set and the commitments it has made.

IMPACT OF DFID STAFFING CONSTRAINTS

106. Witnesses expressed concern that implementation of the Strategy may be hindered by DFID having insufficient staff to take on the necessary tasks. Alvaro Bermejo believed that this was a factor in DFID reducing its direct engagement with civil society organisations working on HIV/AIDS.[198] Carol Bradford from the UK Consortium on AIDS pointed out that DFID staffing constraints also affected the ability to collect the data required for evaluations: "if you are already overworked, additional reporting requirements are always complicated".[199]

107. We have expressed concerns in the past that the headcount reductions which DFID is required to make to meet Government efficiency targets may be beginning to have an impact on its ability to deliver its objectives. In our Report on the DFID Annual Report 2007 we accepted that DFID could not be exempt from efficiency targets but highlighted our anxiety that this might affect the Department's ability to work in the poorest countries, often fragile states, which most need assistance, but where activity is most labour-intensive.[200] The significance of this issue for DFID was reinforced when we explored it in oral evidence with the Permanent Secretary as part of our work on this year's DFID Annual Report. Her view was "Our staff are very pressed, they are working very, very hard. […] we are coping but we are struggling".[201]

108. We welcome the Permanent Secretary's frankness and appreciate that this is probably as far as a senior Government official can go in highlighting the difficulties that departmental staff reductions are creating. We raised these concerns, as they relate to implementation of the HIV/AIDS Strategy, with the Minister. He paid tribute to the "sense of mission" amongst DFID staff and said "I have rarely come across a group of people so motivated and so passionate about what they do"—an assessment with which we would wholeheartedly agree. Nevertheless, he accepted that it was difficult, with fewer staff, to ensure that DFID was "achieving change and making a difference" and using effectively the "record levels of resources" it has been allocated. He believed the Department would have to "work smarter", use its country offices properly and ensure that it had the right skills mix.[202]

109. We will return to this subject in our forthcoming Report on the DFID Annual Report 2008 but we are keen to reiterate our concerns, in the specific context of the new HIV/AIDS Strategy, that staff reductions at DFID may have reached the point where they risk adversely affecting the Department's ability to deliver its objectives in vital fields such as health and social care.

Monitoring and Evaluation

110. A recurrent theme running through the evidence we have received is that DFID's new Strategy lacks quantifiable and verifiable targets and that there are very few indicators against which its effectiveness in tackling HIV/AIDS can be measured. Lucy Chesire, a TB/HIV advocate working in Kenya, told us:

Monitoring and Evaluation are critical components of country programmes, yet it is still very unclear in the current health system strengthening strategy how DFID will carry out its M&E […] DFID should come out very clearly and tell us what targets and indicators they are setting for health system strengthening and how different partners will be able to work collectively to achieve this.[203]

Tearfund echoed this view:

DFID needs to develop a robust monitoring and evaluation framework to accompany the new Strategy. It should set out clear targets and indicators to be reported on annually by DFID and FCO field offices. Data from these indicators must be made publicly available and clearly articulate the UK's contribution towards the achievement of international targets.[204]

111. Alvaro Bermejo believed that the Strategy was not sufficiently specific to assess what outcomes are expected from DFID's investment.[205] Carol Bradford of the UK Consortium on AIDS, who has been working with DFID to develop indicators, accepted that there were currently "many limitations to proper measurement" and that the absence of spending targets or budgets made the funding commitments very difficult to track. She acknowledged that "the health systems measures and indicators are not very good and they need further developing".[206]

112. In our 2006 Report on HIV/AIDS, we expressed concern that DFID's indicators of success in its 2004 Strategy Taking Action were linked primarily to funding targets rather than to outcomes—a weakness in DFID's overall approach which we have highlighted in other previous reports. We recommended the development of outcome indicators which "should set out DFID's contribution to achieving the international targets on HIV/AIDS treatment and care".[207] Despite DFID's claim in its written evidence that the new Strategy "focuses on outcomes and results", [208] there remains an emphasis on the amount of money which will be spent rather than the impact which will be measured.

113. Alvaro Bermejo told us that he believed that DFID had taken a backwards step in the new Strategy in terms of the lack of targets and specificity, which would hamper its implementation and the ability to monitor progress.[209] Stuart Kean of the UK Consortium on AIDS echoed this:

In relation to the indicators at this time, I think there really are questions to be asked about the targets and I think that, as with so much within the strategy, it is a matter of if you were sitting in Lusaka or in Nairobi what would you be doing and how would you interpret it. A high level of interpretation is going to be required. […] I recall DFID saying that they had 126 targets in the previous strategy and there was a concern to move the other way. I believe they have probably gone too far the other way and now it is very difficult to see what are the specific targets that individual civil servants are going to be trying to implement.[210]

114. We were struck by the contrast in the approach taken in the HIV/AIDS Strategy compared to the process which DFID told us it had undertaken in developing the Global Malaria Action Plan which was launched at the UN High Level Event on the Millennium Development Goals on 25 September. Andrew Steer, DFID's Director General Policy and Research, told us in our evidence session on the UN High Level Event:

I think the issue of money is what certainly gets the headlines but what we would rather do is start from the desired outcomes and increasingly focus on that. There was a very real effort here to ask the question "What is the problem and what are we trying to solve?" For example, there are 500 million people suffering acutely from malaria every year and one million deaths. What is reasonable to achieve? It is a halving of that number, and we can monitor those. […] While the money is absolutely essential […] it is the delivery on the ground that we need to monitor from now on as much as it is the money.

The Secretary of State emphasised that, in relation to the Malaria Action Plan, "the very specificity of the pledges" was one of the best guarantees that they would be delivered.[211]

115. There are obvious similarities in the global challenges of tackling HIV/AIDS and tackling malaria. We are impressed by the process which DFID followed in developing the Global Malaria Action Plan which focused on desired outcomes and used that information to determine decisions about inputs and mechanisms. However, it is not evident to us that DFID adopted a similarly rigorous procedure for developing its new AIDS Strategy We believe this was a missed opportunity and we regard the lack of specific budget allocations, targets and outcome indicators as a significant deficiency in the new HIV/AIDS Strategy, which we hope will be addressed in the next stage of the process.

116. DFID says that the Strategy will be independently reviewed in three years' time and that the social protection programmes will be reviewed after two years.[212] In addition, a Monitoring and Evaluation Framework is being developed which DFID originally stated would be finalised "by November 2008"[213] but which the Minister subsequently told us would be published on World AIDS Day on 1 December.[214] We asked the Minister why the Monitoring Framework had not been developed in parallel with the Strategy and published at the same time, which would have given all stakeholders a better understanding of what DFID was trying to achieve. Mr Lewis agreed that it would have been better for the Framework to have been published at the same time but this had not been possible because DFID had been involved in "protracted negotiations" with the Treasury on whether it would be a "three-year or seven-year Strategy".[215] Carol Bradford reassured us that "real progress" was being made with developing the Framework. She pointed out that DFID's last HIV/AIDS Strategy did not have a monitoring and evaluation system in place at all—"so this is a definite improvement".[216]

117. We regret that DFID was not able to publish the Monitoring and Evaluation Framework at the same time as the Strategy was launched in June. All stakeholders, including ourselves, need to understand the specific outcomes that DFID is seeking to achieve through the funding commitments it has announced and how it intends to measure progress towards them. We hope that, when it is published, the Framework will provide the answers to the important questions about implementation and monitoring and evaluation which the Strategy itself has left open.

182 Ev 60 Back

183 Ev 46 Back

184 See Achieving Universal Access, Chapter 5 Back

185 Ev 81 Back

186 See for example Qq 61, 64, 80 Back

187 Achieving Universal Access, p 58 Back

188 Q 24 Back

189 Q 24 Back

190 Q 33 Back

191 Achieving Universal Access, p 44 Back

192 Ev 37 Back

193 Q 64 Back