The Work of the Independent Commission for Aid Impact - June 2012

Written evidence submitted by Graham Ward CBE, Chief Commissioner for the Independent Commission for Aid Impact

In addition, please find below our response to your additional questions following our hearing.

1. There has been some criticism that ICAI chooses its study subjects in an ad-hoc fashion, with little sense of how these fit into a broader evaluation strategy. What is your response?

ICAI’s overall aim, as set out in our May 2011 work plan was and remains to "do justice to the breadth, depth and complexity of UK aid expenditure". Our intention is to provide Parliament and the taxpayer with a balanced picture of that expenditure, identifying both good and poor practice.

The ICAI work plan was developed based on input from a range of sources, including your Committee, responses to our public consultation and discussions with DFID, other Government Departments and stakeholders.

To establish what we would look at in our first three years we used our four criteria of materiality, risk, coverage and interest, as set out in our work plan. In developing a more detailed proposal for our Year 2 work plan we continued to use these four criteria and overlay them with inputs from your Committee and other stakeholders.

In addition, in our first year we have published reports that have helped us to learn about DFID’s business and to understand some of the wider challenges the UK aid budget faces. Subjects covered in this way have included multilateral expenditure, fragile states, corruption, budget support and programmes designed in support of the MDGs.

Overall, our first year has covered reports which loosely fit into three categories: ‘thematic’ – e.g. Anti-Corruption, Budget Support; ‘key relationships’ e.g. World Bank, UNDP; and ‘specific programmes in country’ e.g. Bangladesh, Zimbabwe and India.

In our second and subsequent years we will continue to provide a balanced picture with coverage of new subjects. We expect to focus more on (a) formal follow-up of early reports and (b) some of the thematic issues we consider important. These may include, for example, corruption, fragile states, engagement with multilaterals and delivery channels, as well as ODA expenditure by other Government Departments. By doing this we plan to get a broader and more in-depth picture of UK ODA spending.

When considering our work plan we consider a number of issues, including the breadth of coverage and the overall programme. Whilst we have limited resources and cannot scrutinise all ODA spending, it is our intention that ICAI’s body of reports will add up to a comprehensive review of the UK aid programme. Our Year 2 Annual Report will draw out some wider key themes from the first two years and make more in depth comment on the broader issues.

2. How do you respond to the criticism that you have chosen some subjects that are too early for evaluation work?

Our snapshot approach to independent evaluation can look at the success of a programme or policy area at any stage of its delivery.

We have made the decision to look at certain programmes or areas of the aid budget using our four criteria. Our criteria explicitly set out the possibility that we might look at something because it is new or innovative. We believe that it is important to review some programmes at an early stage, in order to ensure that they are effective and on a trajectory to ensuring maximum impact for intended beneficiaries. If a project is not on the right trajectory, or if the right processes are not in place from the beginning, the long-term sustainable impact may be adversely affected.

For example, we took the decision to review Girl Hub when it had been in operation for two years and had spent £2.9m of its £15.6m budget. Whilst still in its early stages, we chose to undertake this review because there was a large amount of money still to be spent and we wanted to ensure that the early course of action was appropriate, so that future spending would have the maximum impact. As the report shows, there was a lack of clarity about what Girl Hub should address. We, therefore, made some specific recommendations that, if implemented at this stage, will support Girl Hub to deliver improved impact for adolescent girls.

The ICAI report on DFID’s Support to the Health Sector in Zimbabwe, however, looked at the programme after it had spent £100m in support of the health sector over the previous eight years. This enabled us to examine a programme with a large spend in one sector and to look at the impact over a number of years. As a further illustration, we plan to carry out a post implementation evaluation of the DFID-funded Western Orissa Livelihoods Project. This will enable us to look at whether the previously observed benefits have proved sustainable and whether the lessons from previous impact assessments and evaluations have been used to influence other activities and evaluations in Orissa.

Your final question was on whether ICAI could provide the Committee with our analysis of DFID’s work on anti-corruption once we have undertaken this exercise. We are currently planning to deliver this work as a full ICAI report in our third year and will ensure that we consult with the Committee as we formulate the scope of this work.

July 2012

Prepared 23rd July 2012