The Newton Fund review: report of the Sub-Committee on the work of ICAI Contents

5Measuring performance

Delivery partners and ODA compliance

40.ICAI’s review also highlighted weaknesses around the delivery partners’ ability to award grants to projects that are ODA-compliant and BEIS’s willingness to hold them to account on the latter. Both the Department and UKRI seemed confident in delivery partners’ ability to make the right grant allocation decisions.45 In UKRI’s view, their ODA-compliance processes ensure that:

Programmes are predominately scoped in collaboration between UKRI and research and innovation funders in the partner country. These are designed to address challenges identified as relevant and important by the partner country and ODA requirements are incorporated into these scoping discussions.46

41.However, ICAI found that there was no effective oversight and management of those country partnerships and felt, as a result, that they lacked transparency and accountability. The absence of an overall strategy for the Fund also meant that development impact and ODA eligibility were treated at times as “merely a threshold”, an “add-on, or even in conflict with what delivery partners perceived as the primary aim” of the Fund instead of being integral to delivering poverty reduction and development gains in line with the UK Aid Strategy.47

42.Worryingly, the 2018 mid-term review of the Fund found that “funding partners in some countries are becoming increasingly assertive and pushing the boundaries for what constitutes ODA.”48 In light of the seriousness of these concerns, it is unclear to us that a simple review of country strategies, as is proposed by the BEIS, will be enough to ensure that all future grant awards are ODA-compliant and delivering development objectives.

43.As highlighted by our predecessor committee, departments in receipt of ODA funding are responsible for administering ODA programmes effectively and efficiently, including programme management and reporting, and highlighting when required administration standards are not being met.49 The latter cannot be outsourced to delivery partners on grounds of “world-leading expertise on research and innovation’’50 or lack of feasibility51 especially when evidence is subsequently produced that some programmes are failing to meet those standards.

44.The current arrangements for ODA compliance which are left at the delivery partners’ appreciation and with no explicit link to an overall strategy create risks of gaps in coherence and elements of ambiguity to arise. The training and support available to delivery partners so far has not seemed to have had the desired impact; instead it has left them and BEIS under the erroneous impression that their projects were already delivering substantial development impact. BEIS needs to take a bigger role in coordinating and monitoring delivery partners’ ODA eligibility processes and go much further in implementing ICAI’s Recommendation 1, starting by publishing as soon as possible its much-delayed revised ODA compliance guidance for delivery partners and country reviews.

45.We feel that without also publishing an overall strategy for the Fund that is in line with the UK Aid Strategy and clearly states the responsibility and accountability mechanisms for delivery partners in delivering it, BEIS will continue to fail at enforcing ODA compliance and development impact to the required standard. We recommend that the Department publish a strategy with clear performance metrics for the Fund as soon as possible. BEIS should update us on progress towards publishing its revised ODA compliance guidance and a strategy in its response to our Report.

Defining capacity building

46.The Department did not comment on the mid-term reviews’ findings and simply argued:

Our own independent evaluations found no evidence of ICAI’s suggestion that delivery partners had tested the boundaries of ODA eligibility, and the BEIS ODA Ministerial Board, with cross governmental representation, continues to endorse the delivery model of the Newton Fund.52

47.This response does not address the core of ICAI’s concerns and seems to confirm that the Department has no intention of changing its allocation system in any substantive way. In ICAI’s view, the Fund is particularly weak on delivering development impact under its ‘People’ pillar in terms of capacity building:

This involves academies providing fellowships to individual researchers from partner countries. Under rules set by BEIS, fellowship awards can be used to support research on any topic, whether related to development or not. The basis of their claim to ODA eligibility is therefore that they are helping to build research capacity in developing countries. However, we found that the Newton Fund’s approach to capacity building focuses predominantly on creating opportunities for individuals to build their skills.53

48.UKRI provided us with a detailed overview of its allocation process. We welcome this information, but we feel the process it described is mostly designed to identify research excellence and does not have development impact at its core. It also requires very little oversight and involvement from BEIS which unavoidably has led some delivery partners to make some questionable allocation decisions.54

49.When pressed to justify some of the more controversial grant awards made by the British Academy, which used capacity building to meet the ODA eligibility criterion, the Department argued that it was for delivery partners “to use their expertise to identify and support specific activities that will have the greatest impact.”55 Although not responsible for these specific allocations, we also asked UKRI for its view. The witness argued that he “firmly believe that arts and humanities research makes a vital contribution in the ODA space across the full range of development challenges and the UN Sustainable Development Goals” and that “capacity building is central to UKRI’s investment under the Newton Fund and is much wider than training and exchanges hosted by UK institutions.”56 Although we agree with both statements, we feel they do not engage with the core of ICAI’s claim which is that the bar for what constitutes an ODA-eligible, capacity building project is too low under the Newton Fund and unlikely to deliver satisfactory development impact.

Measuring impact and value for money

50.We also cannot support UKRI’s other argument which is that “research and innovation projects can take a long time to achieve measurable impact and the first Newton awards were made in 2014, so we would not necessarily expect measurable impacts yet.”57 As shown by the National Audit Office, departments in charge of administering ODA use a range of different metrics to measure impact and face a number of challenges in doing so:

Departments often work in fragile states, and in dangerous environments with unpredictable and sensitive political and social backdrops. Programmes need to be flexible enough to respond to changes while allowing enough time to achieve long-term impacts.58

51.We acknowledge UKRI’s point that some “outputs and outcomes are easier to measure than others”59 but the choice made by BEIS to measure impact solely via regular updates from delivery partners on a project’s financial position and activities does not convince us any more than it persuaded the NAO, who said the approach “does not allow BEIS to consider the effectiveness of its expenditure in these areas.”60 DFID, itself, is consistently rated as ‘Very Good’ amongst other UK government departments on the Aid Transparency Index61 and part of this is due to DFID’s consistent approach impact measurement: despite operating in fragile and challenging contexts, “DFID expects each programme to have a business case and a performance framework at the outset, and for a programme’s performance to be reviewed annually and evaluated on completion.”62

52.DFID’s approach to performance evaluation should be replicated by other departments like BEIS which are administering increasingly large amounts of ODA and must therefore demonstrate, prior to being given this responsibility, their ability to measure impact.

53.If it is to be renewed at the next Spending Review and continues to be entirely ODA-funded, the Fund’s grant allocation system should be radically redesigned to ensure that developmental impact becomes its genuine driving force (and not just a constraint around its secondary purpose). BEIS needs to impose stronger ODA-eligibility criteria and encourage delivery partners to switch from allocation processes designed solely for research excellence projects to ones that prioritise development impact. Fellowship awards should not be used to support research on any topic, but related to development if they are to be funded using ODA. BEIS should update us on steps taken towards completing these measures in its response to our Report.

Transparency

54.In January 2020, BEIS was awarded a ‘Good’ score for its first assessment and ranked third amongst the ten governments departments reviewed by Publish What You Fund.63 As the second biggest spender of ODA, this is a significant achievement for BEIS and one that we warmly congratulate them on. However, we note that Publish What You Fund’s recommendations for the Department are in line with Public Accounts Committee’s, and our own, view that it is crucial for BEIS to start measuring and reporting on performance of ODA programmes. We also note, from research conducted by the Centre for Global Development, that there is still room for progress on the extent to which the UK’s ODA spending on researching is reported in a way that allows public scrutiny of where and how it is spent, and the extent to which it is tied aid.64 The Centre makes two recommendations:

[…][T]here should be sufficient transparency to permit the level of scrutiny that ensures this. For all agencies spending aid, including those other than DFID, any spending classed as ODA should be on the Devtracker site and should adhere to International Aid Transparency Initiative (IATI) standards, including showing the full business case and the transactions for the full project life. If UK research aid is tied to a UK institution it should be correctly reported as such to CRS and IATI.65

55.In a subsequent review, ICAI has also demonstrated the importance of publishing data on ODA expenditure that is in line with the International Aid Transparency Index (DFID’s manages this via its DevTracker platform).66 In ICAI’s view, BEIS is still far from having a comparable system and, problematically, it may not fully comprehend the importance of having strong governance arrangements because it has outsourced of its learning process to an evaluation firm:

As found in previous ICAI reviews, both Funds still lack a comprehensive and consistent approach to gathering evidence on their outputs and outcomes […]. There is no Fund-level overview of the GCRF and NF’s projects and spending, except for an Excel-based activity tracker. […] BEIS is outsourcing a large amount of its learning processes, such as development and implementation of its KPIs and evaluations to a contracted evaluation firm. While this is a common practice across government, BEIS does not appear to be fully integrating learning from this approach in a timely manner.67

The reporting and transparency requirements should be increased for both BEIS and delivery partners to allow public scrutiny of the Fund’s spending, impact and of delivery partners’ performance on ODA compliance. BEIS should follow DFID’s example and publish on a single platform all relevant information on a) delivery partners’ ODA-compliance processes, b) the ODA compliance guidance it is currently developing for delivery partners and how it will enforce it, c) details and statistics of Newton Fund grant applications that are accepted or rejected depending on ODA eligibility, d) business cases, performance and transaction data and reports (e.g. programme completion and annual reports) for all ODA projects it is responsible for. In its response to our Report, BEIS should outline progress towards achieving these recommendations.


45 BEIS (INF003) Q7-a; UKRI (INF001) Q4

46 UKRI (INF001) Q1

47 ICAI, The Newton Fund, a performance review, para. 4.33–4.34, p. 22

48 ICAI, The Newton Fund, a performance review, para. 4.30, p. 21

49 International Development Committee, Definition and administration of ODA, para. 8–24

50 BEIS (INF003) Q6-b

51 UKRI (INF001) Q1-a

52 BEIS (INF003), Q7-a

54 ICAI, The Newton Fund, a performance review, Table 2, pp. 24–25

55 BEIS (INF003), Q6-a

56 UKRI (INF001) Q4

57 UKRI (INF001) Q4-a

62 National Audit Office, The effectiveness of Official Development Assistance expenditure, para. 2.5, p. 36

64 Centre for Global Development’s ‘UK Research Aid: Tied, Opaque and Off-Topic?’

65 Centre for Global Development’s UK Research Aid: Tied, Opaque and Off-Topic?’, p. 16

66 ICAI, How UK Aid Learns, September 2019, para. 4.42, p. 20

67 ICAI, How UK Aid Learns, para. 4.42, p. 30




Published: 15 June 2020