Annex B
DATA SYSTEMS ASSURANCE
SUMMARY OF
LEVELS OF
DATA SYSTEMS
ASSURANCE
|
| Level of assurance
|
|
| PBAs | Substantial |
| Procurement | Full |
| Portfolio quality | Substantial
|
| EC aid | Full |
| IDA | Full |
| Admin costs | Substantial |
| Headcount | Substantial |
| Relocation | Full |
|
| |
RISK OF
MISSTATEMENT OF
GAINS
| | |
|
| Likelihood | Impact
| Overall |
| | |
|
| PBAs | Low/Med | Medium
| Low/Med |
| Procurement | Low | Low
| Low |
| Portfolio quality | Low |
High | Medium |
| EC aid | Medium | Medium
| Medium |
| IDA | Low | High
| Medium |
| Admin costs | Low | Low
| Low |
| | |
|
| | |
|
SWITCHING FROM
PROJECT ASSISTANCE
TO PROGRAMME
BASED APPROACHES
(PBAS)
1. A new process has been set up from 1 April 2006 to
collect data on programme based approaches (PBAs) via the project
header sheet (PHS). Detailed guidance has been issued to spending
departments and there is no evidence to suggest that they are
not fully complying with this. However quality assurance (QA)
of the data is limited and it has to be decided whether an increased
level of QA is needed and where this responsibility should sit.
2. Our Statistical Reporting and Support Group (SRSG)
staff check the PHS data to ensure activities which clearly are
PBAs, eg Poverty Reduction Budget Support (PRBS) and Sector Wide
Approaches (SWAps), are correctly included as PBAs. If SRSG spot
any coding errors they will raise them with the person/department
involved and ensure the PHS is correctly completed. However they
are unable to check on whether projects/programmes other than
PRBS and SWAps are correctly coded. There was some discussion
on how more detailed QA could be carried out during the preparation
of the PBA guidance but this was not brought to a conclusion.
3. The risk is that spenders do not categorise their
programme activities correctly as PBAs which could result in the
amount of funding being over or under estimated. It is difficult
to say which is more likely as we only have three months of data
(under the new system) on which to base judgements. A higher level
of QA is probably required at least for the first 12 months so
that we can identify any errors and as a result raise awareness
or clarify issues with the departments involved.
4. Responsibility for the quality of data clearly rests
with the spending department.
5. A full audit trail is available.
6. Following the introduction of ARIES, programme officers
will have to respond positively to a series of questions on qualifying
conditions before an activity can be classified as a PBA. This
will ensure that each of the conditions is separately considered
and should ensure more consistency throughout the department.
7. We judge that the overall level of assurance for this
data is SUBSTANTIAL.
SAVINGS ON
PROCUREMENT
8. Procurement efficiency gains are derived from data
within Procurement Group (PrG)'s contract management system (CMS).
These data are checked and ratified by Contract Team Leaders and
Policy, Training and Management Service (PTMS) Team staff. They
are included in an annual procurement Value for Money return which
is reviewed by our Internal Audit Department prior to sign off
by our Finance Director, Richard Calvert.
9. Regular checks are also made throughout the year by
both Contract Branch Teams and PTMS staff of the appropriateness
of specific entries and where there are concerns these figures
are investigated and, if necessary, removed.
10. PTMS maintain oversight of CMS and run the reports
from which the efficiency information is generated. There is a
full audit trail to support these reports which include original
bid documentation, bid evaluation reports, and finalised contracts.
11. The procurement function sits at the heart of the
new ARIES system which has been designed to provide DFID with
much clearer data on the overall value and range of procurement
being undertaken by DFID throughout the world. Despite the significant
long-term benefits of ARIES there will be challenges in the short-
to medium-term as ARIES is gradually rolled out throughout DFID.
This will require PrG to run parallel systems to cope with parts
of the office operating ARIES, and those using old systems. The
efficiency data derived from two systems will need to be carefully
managed to ensure that the use of parallel systems in the short/medium-term
does not adversely affect the overall quality of the information.
12. We judge that the overall level of assurance for
this data is FULL.
IMPROVEMENT IN
THE PERFORMANCE
OF DFID'S
BILATERAL PORTFOLIO
13. Internal control systems are basically sound but
there are some areas where guidance is being refined to ensure
consistency and consideration is being given to whether greater
quality control would be useful. Internal controls are described
in the table below.
| Criterion | What this means for data
| What this means for data recording & processing systems
| What this means for reported gains |
| Accurate | Project review scores are approved by Head of Department (HOD) or a person delegated on the HOD's behalf in DFID's Performance Reporting Information System for Management (PRISM) system. Project officers/managers are responsible for ensuring there are no typos when the figures are input. It is considered that typos at this stage will be very rare and would usually be picked up if they did occur. Their impact would be unlikely to affect the aggregated figures presented. Consideration shall be given to whether any additional process would be useful and proportionate here.
The approved commitment value apportioned to the activity is entered in DFID's financial systems from the project document and so should be accurate to the same degree. This document will retain a history of changes and the financial system also stores any changes made to the commitment value. DFID's new ARIES system will have additional controls to record any changes to the financial commitment as they take place.
| Data values are aggregated across DFID every quarter. An extract file from the live PRISM database in saved at this time to enable figures to be checked and gone back to if necessary in future. Data checks can take place at any time, and reports can be run to confirm good quality information is stored in PRISM.
The same programme is used to run the data analysis each time. Figures are double checked against the live system to ensure accuracy.
The value for money percentages taken from this system are applied to a different set of financial information: DFID's total bilateral expenditure for the year in question. This calculation is undertaken by the efficiency reporting team and fully documented and cross-checked.
| The PRISM team holds a summary chart in DFID's filing system recording analysis across quarters and this matches data in the efficiency reporting team's spreadsheets.
|
Complete | DFID has strict controls to ensure annual reviews and project completion reports are undertaken on time. Our compliance rate is in the high 90%s each quarter. Where reviews have not been entered on time, a spending block is attached to the project/programme until the review is input and approved and senior managers are informed. The reporting system includes the previous review if a new review is outstanding (unless it is the first review that is due).
| The only data removed in the processing stage are cases where a project has been scored "x" which means "too early to say". DFID has been driving down the instances of use of this score so that it now accounts for only around 2% of projects scored.
| Reporting includes all results and figures are shown in tables and graphs. DFID has made a commitment to enhance our existing Autumn Performance Report/ Departmental Report annex on data limitations to capture more fully value for money data limitations.
|
Valid | Data are a valid representation of DFID's bilateral portfolio as they claim to be.
| Data retain their validity during processing.
| The validity of the data is explained in reporting; that the portfolio includes all eligible DFID bilateral projects/ programmes one year and over with a value of £1 million or over.
|
Appropriate | Data collected are generally fit for purpose although it is recognised that we are defining "value for money" as meaning that project outputs/purposes as set out in the project "log frame" (or equivalent performance measurement framework) have been achieved. This does pre-suppose that DFID is selecting appropriate projects/ programmes and setting sufficiently realistic and yet stretching objectives for these. Processes around country and divisional planning attempt to ensure that this is indeed the case. Consideration is being given to whether greater quality control of log frames and/or support for log frame development would be useful in DFID.
| Data are to generate information that enables:
— effective management/delivery of the gain; and
— a clear statement of progress to be made.
Data can be disaggregated beyond the level reported in the efficiency reports so that individual divisions and country programmes can see how their figures impact on the total and the impact of undertaking work classified as high, medium and low risk can be analysed.
| Reporting is appropriate for the intended audiences.
|
Consistent | Data are produced by a number of users across DFID according to common guidelines. DFID is in the process of revising the guidelines to give more detailed information to users and try and ensure consistent application. Many users already use external evaluators as part of the review team and record this in PRISM or to undertake the full review but the extent of this external scrutiny is not fully recorded; this will be captured in future.
| There has been one change to the data system over the efficiency period; annual reviews are now required after a project has been operational for one year whereas this was not previously required until two years. This change was implemented at the start of 2006 and is designed to improve project/programme management and increase the number of projects/ programmes in the dataset. It is possible that this change might mean some projects not previously reviewed, eg short term humanitarian projects, are captured to a greater extent. Analysis will be undertaken to look at the balance of the portfolio and try and determine whether this has altered, with potential impacts on the performance scores. It should be noted that this will not be able to fully quantify the extent of any changes since there is always fluctuation in the portfolio over time.
| Reporting is consistent with the measures and methodologies set out in the published revised Efficiency Technical Note (including any supplementary notes on methodology published) and with the available guidance.
|
Timely | Data are compiled continuously on a live system and quarterly figures are drawn from this showing the situation as of the appropriate date at the end of each quarter.
| The time between project review, approval and loading to the internal system is generally about a month. There is an "early warning" system that advises project officers of annual reviews due in the next three months to ensure timely completion of reviews. A separate list shows when project completion reports are due/ overdue. A calendar will be made available in July showing a complete range of due dates for all reviews types due in the 12 months period.
| Data are reported as soon as possible after the end of each quarter; generally about one to two weeks to allow for full checking.
|
Owned | Responsibility for data quality is clearly identified at the level of the project officer and their departmental head.
| Responsibility for data processing and the maintenance of data quality during processing is clearly allocated to the PRISM team and Information Systems Department.
| Responsibility for reporting robust gains is clearly allocated to the Efficiency reporting officer who is within the same overall team as the PRISM team.
|
| | |
|
14. Components of the system for identifying and managing
risks and issues relating to data quality are detailed in the
table above. In addition it is important to note that DFID has
a good system for training users of the PRISM system which should
reduce the chance of user error.
15. There are not believed to be significant risks which
could threaten material misstatement of reported gains. Further
mitigation activities may be introduced following our current
project to assess whether quality control systems and external
scrutiny at all stages of DFID's project cycle management are
appropriate. Work to improve the existing guidance on project
scoring is designed to mitigate the potential risk of inconsistent
scoring as it is recognised that there is room for an element
of subjectivity here.
16. Accountability for the quality of reported gains
is clearly documented and attracts significant management attention
at all levels of DFID.
17. A governance structure exists to oversee the design
and operation of data systems and to provide challenge and steer
in response to issues relating to the quality of reported data.
18. A full audit trail is available. Auditable records
are held by the PRISM team covering both data analysis and documentation
of review findings. Detailed information on why individual project
scoring decisions were reached would require detailed scrutiny
of records.
19. ARIES will replicate the current project/programme
scoring system in PRISM. Input will still be manual. However,
to improve quality we have developed a workflow system which will
send the Annual Review or Project Completion Report for approval
via a Quality Assurer. The Quality Assurer (possibly the Project
Manager) will be responsible for checking the document and the
input of scores on ARIES for accuracy. If he/she is not content
with the input, he/she can reject the workflow task which will
send a message back to the Project Officer advising the reason
for rejection. If he/she approves the workflow, the task will
then be forwarded to the Head of Department (HoD) for approval
or rejection. If the HoD approves the Annual Review (via a tick
box) the scores are accepted on ARIES. If rejected, a message
will be sent to the Quality Assurer and the Project Officer giving
details of the reason for rejection and asking for re-submission.
20. We judge that the overall level of assurance for
this data is SUBSTANTIAL.
INCREASED CONTRIBUTIONS
TO LOW
INCOME COUNTRIES
(LICS) VIA
EC AID
21. Raw data are published annually by the OECD Development
Assistance Committee (DAC), so the EC LIC focus figure is only
updated annually by our European Union Department (EUD). When
the DAC publish the annual official development assistance (oda)
disbursement data, EUD use their online database to select the
disbursements from the EC. EUD then select the oda recipients
who are classed as least developed countries (LDCs) and other
LICs, ensuring that the LDCs that are classed as middle income
countries are not selected. This total is used to calculate the
percentage of oda to LICs. The method is well documented, as are
spreadsheets of previous years' calculations to ensure consistency.
22. The calculation method is documented and passed on
to EUD's statistics adviser. On an annual basis the statistics
adviser ensures the list of LICs is up to date according to DAC
definitions. The LIC focus figure is compared to previous years'
and apparent inconsistencies would be followed up with the DAC.
The final figure is signed off by the Head of EUD.
23. A risk to the quality of the data is if the DAC publish
incorrect or inconsistent data. In past years the DAC included
all European Investment Bank lending as oda, which is incorrect.
This was questioned by DFID, and because of this the DAC revised
their figures for 2002 and 2003. The DAC has assured DFID that
this will not happen again, and EUD do not foresee any other inconsistencies
or errors occurring.
24. A full audit trail is available for this target.
25. The introduction of ARIES will have no effect on
the EC data, nor on the processes to maintain that data.
26. We judge that the overall level of assurance for
this data is FULL.
INCREASED SUPPORT
FOR THE
INTERNATIONAL DEVELOPMENT
ASSOCIATION (IDA)
27. IDA data refers to various types of payments made
under the IDA 14 agreement. These are explained below as World
Bank, Private Sector Infrastructure and Debt.
World Bank
28. Requests for deposit of promissory notes and/or encashments
against them are received in DFID's International Financial Institutions
Department (IFID) from the World Bank. These are cross-checked
against IFID's own records of commitments. An administrator processes
any payments and this is reviewed by a senior administrator. Any
discrepancies between the Bank data and our records are queried
directly with the Bank. DFID's Accounts Group has final sight
of each payment.
29. IFID maintains a spreadsheet listing promissory notes
deposited with the Bank of England and encashments against them.
Accounts Group and the World Bank also maintain a similar spreadsheet
which provides a useful cross-check function at the time of each
transaction. The administrator's data input and spreadsheet maintenance
work is checked by the senior administrator.
30. Data quality lapses are managed by a standard process
of double checking figures at the time of each transaction. Further
assurance is provided by more than one unit keeping records (as
described above) and being involved in the transaction process.
31. Any errors noticed would be immediately investigated
and corrected. The senior administrator who checks all recorded
payments and encashments is responsible for investigating any
errors and implementing appropriate changes.
32. A full audit trail is available. Paperwork received
from the World Bank is saved in a specific folder in DFID's electronic
filing system. All transaction activity is also saved in this
system. The administrator has an explicit work objective relating
to accurate maintenance of the relevant spreadsheet. As the spreadsheet
is saved in our system, the "check in check out" function
provides a record of earlier versions and changes by various users,
thus providing an audit trail.
Global Funds and DFIs Department
33. The narrative for World Bank data above covers equally
the generic work of DFID's Global Funds and DFIs Department (GFDD)
on international private sector infrastructure. In addition to
providing inputs to reporting achievements against DFID's corporate
strategy we keep records of a balanced scorecard that we have
produced for the Private Sector Infrastructure Programme and the
list of performance indicators agreed for the programme with partner
donors. Reported data is validated by the respective programme
managers and confirmed by independent evaluation. Internal checks
are undertaken within GFDD to ensure the accuracy of reporting.
Full audit trails are maintained including information held on
the websites of programme facilities.
Multilateral Debt Relief Initiative (MDRI) IDA contribution
34. Funding was approved by Parliament in May 2006 under
a statutory Instrument of Commitment which includes a detailed
funding schedule/timing of payments over the life of the commitment.
Funding is additional to IDA Replenishments. The first payment
of £26.85 million was made as scheduled in January 2007 although
requests for payments from the World Bank will be required as
part of the payment process. The next payment is scheduled for
January 2008. Funding requests will be cross-checked against IFID's
own records of commitments/expenditure and the agreed funding
schedule to ensure we avoid paying twice, or paying in advance
of need. Payments will be processed within IFID by an administrator
and reviewed by a senior administrator. Any discrepancies in the
World Bank data will be queried directly with the Bank. Further
assurances will also be provided by Accounts Group who will confirm
or query any payments processed by IFID.
35. IFID will maintain a spreadsheet detailing the amount
and timing of payments in accordance with the agreed funding schedule.
It will also show expenditure and will be cross-checked with the
DFID's expenditure system as necessary. Accounts Group and the
World Bank will maintain a similar spreadsheet providing a useful
cross-check function. The administrator's data input and spreadsheet
maintenance work is overseen by the senior administrator. There
is the standard process of payment input and review in accordance
with ADAMANT principles.
36. Data quality lapses are managed by a standard process
of double-checking figures at the time of each transaction. Further
assurance is provided by more than one DFID section keeping records
(as described above) and being involved in the transaction process.
37. A full audit trail is available. Scanned copies of
the signed Instrument of Commitment, approval by the Secretary
of State and DFID's Director of Finance and Corporate Performance
and World Bank funding requests (when received) are saved in DFID's
electronic filing system and readily accessible All transaction
activity will also be saved in the system. The team has a work
objective to ensure that all relevant documents are stored in
an appropriate area of the filing system and can be easily retrieved.
38. We anticipate that ARIES will automate processes
for the collection of IDA data, but we do not yet know what the
impact might be on the surety of IFID data systems. As assurance
for IDA is already "full", we expect that Aries will
further strengthen the data.
39. We judge that the overall level of assurance for
IDA data is FULL.
ADMINISTRATION COSTS
40. Data is supplied by DFID departments to our Resource
Management Group (RMG) in our forecasting (FINSTATS) and Resource
Allocation Round (RAR). RMG set internal controls on the FINSTATS
returns to ensure they are returned on deadline and full and appropriate
information is provided. Control over the data provided is largely
devolved to Departments. Departmental Finance Officers manually
input data and assess forecast expenditure. Departments take responsibility
for returns and should investigate any unexpected figures or variances.
All FINSTATS returns have to be signed off by the Head of Department.
41. The FINSTATS data is susceptible to human error.
If forecasts are significantly inaccurate this can have a substantial
impact on our administration budget and efficiency programme due
to the risk of unforeseen overspend or underspend. However the
data has a high level of quality assurance by RMG who challenge
departments on suspect data. RMG check the variance between quarterly
(or tri-annual) forecast data and actual expenditure recorded
in our financial systems and provided by Accounts throughout the
year. They also re-visit forecasts as part of the year-end exercise,
during which RMG compare figures with Accounts Group, who have
their own internal controls in place to ensure data quality. RMG
raise any significant difference with Departments to ensure they
see the complete picture and data is as accurate as possible.
RMG also provide training and guidance on FINSTATS to support
Departments and improve the quality of data provided. This process
requires a significant level of staff input to ensure substantial
quality assurance. With these checks in place substantial misstatement
is possible, but unlikely.
42. Our level of assurance over the data supplied by
DFID's financial systems is only partial due to concerns over
the validity and appropriateness of the data provided. In terms
of actual expenditure RMG rely on teams managing the financial
systems to ensure internal controls are in place. RMG are confident
that the information is accurate (based on payments going through
the system), timely and complete but it is not necessarily appropriate
or consistent with our needs. Actual expenditure can look very
different from forecasts provided by Departments, and year-end
figures (eg accruals) rely on the accuracy of the data supplied
by Departments. There is no definite system for challenging the
data present in our financial systems, as it is not clear which
Department should take responsibility (see next paragraph). This
can make it difficult to trust the quality of the data, or to
source and resolve inaccuracies. However, steps are being taken
to improve the quality of DFID financial systems through the ARIES
project.
43. Accountability for the quality of the data supplied
through FINSTATS lies clearly with Departments and they take ownership
for completing returns. Responsibility for maintaining and keeping
the system up-to-date lies with RMG, and all Departments are aware
of this. The quality of data appearing in our financial systems
is less accountable as responsibility lies across departments:
RMG supply allocation data; Accounts maintain the system and load
the figures and Departments make the payments.
44. A full audit trail for the FINSTATS exercise and
any significant discussions regarding administration expenditure
is available on RMG's intranet site, on paper file (in minutes
and submissions) and in our electronic filing system. These documents
clearly set out RMG's calculations, conclusions and recommendations.
However, RMG relies on Departments to keep a full audit trail
of Departmental negotiations and calculations before they submit
their results to RMG. RMG rely on our Information Systems Department
to archive data from the financial systems, and Accounts for final
outturn data through the annual Resource Accounts.
45. We anticipate that ARIES will have a positive impact
on the collection of financial data. It will provide: data in
resource terms; a single data source which will improve accuracy
and consistency of information; the ability to access data in
real time which will give greater accuracy and efficiency when
reporting; a single integrated system to replace local accounting
and budgeting systems; and improved audit trails. All this will
strengthen the data we receive and process.
46. We judge that the overall level of assurance for
this data is SUBSTANTIAL.
HEADCOUNT
47. Our Human Resources Division (HRD) rolled out on
23 November 2006 a reporting tool which enables Divisions to run
headcount reports from a weekly snapshot of the HR database. This
allows them to see for themselves the staff for whom they are
responsible, and provides an ongoing opportunity for them to advise
HR of any necessary changes, for example as a result of a staff
move.
48. Although HRD has no control over the quality or timeliness
of such notifications, which then require input to the database
by HR staff, previous checks with Divisions have confirmed that
this process works well, with any discrepancies invariably the
result of the inevitable time lag between a change taking place,
the notification being received, and the necessary action being
taken. The new reporting tool will allow Divisions to run more
regular checks, and take corrective action sooner.
49. Accountability lies with individuals in maintaining
their own records on DFID's online HR system (Yourself); with
Departments for notifying HR of staff moves, and with HRD for
inputting those changes to the database accurately and as quickly
as possible.
50. The HR Management Information Team liaises closely
with the operational HR teams when any inconsistencies are discovered,
and monitors that the necessary changes are made.
51. An audit trail is available. The snapshot reports
produced from the database are retained as a historical record,
and to provide further information or detail if required.
52. The introduction of ARIES will have no effect on
the quality or range of headcount information available, nor on
the processes to maintain that data.
53. We judge that the overall level of assurance for
this data is SUBSTANTIAL.
RELOCATION
54. Information on posts planned to be and actually relocated
was supplied to our Knowledge Division Director's team by Divisions.
Corporate Strategy Group monitored the data at regular intervals,
queries were referred back to Divisions and the data revised accordingly.
55. Data could become inaccurate if Divisions failed
to provide timely information on progress of the relocation exercise.
Divisions were requested for updates and for confirmation of data
at regular intervals by the Knowledge Division Director's team.
56. A full audit trail is available, listing posts to
be relocated, names of staff who took up the posts in East Kilbride,
and dates of relocations.
57. DFID's relocation programme is now complete. We judge
that the overall level of assurance for this data is FULL.
Goal/purpose of efficiency programme: to achieve £420
million of sustainable, credible, evidence-based efficiencies
in 2007-08
| Ref No | RISK
| Prob
Score | Impact
Score
| REMEDIAL
ACTIONS |
| 1. Service quality |
| | |
1.1 | Service/programme quality is not maintained.
| M | M | — Review quality measures for PQ, procurement and admin on a quarterly basis.
— Produce quality measures for PBAs (on robustness of countries' financial management systems), IDA and EC.
— Consider whether we need to measure capital costs of ARIES against admin savings (see box 10 in NAO report).
|
2. Data assurance |
| | |
2.1 | Gains may not be measured accurately or may be misstated, ie data quality/systems risks.
Eg baseline/quality risks—current IDA and PBA targets don't show what quality of aid would have been if not switched to IDA/PBAs.
Eg existing systems are used to measure gains in order to avoid additional costs of new data collection and reporting systems—these aren't designed for this purpose.
Eg DFID's management information systems fail to provide accurate and timely information
| M | H | — Keep data systems assurance return up to date.
— ETN should highlight risks involved and any strategies to mitigate these risks.
— Independent validation, checks on audit trails. IAD review of data systems assurance.
— Incorporate requirements in ARIES design.
|
3. Methodology |
| | |
3.1 | Internal judgement of level of efficiencies/internal assessment of performance/lack of robust evidence may lead to misstatement of gains (IDA 25% and PBAs 20%).
| H | H | — Peer review or independent validation (on IDA NAO said "a proxy efficiency factor of 25% would be more robust if the Dept's ongoing evaluation of WB practices was accompanied by independent research verifying this factor).
— Review data systems assurance rating for IDA and PBAs. Review again for IDA and PBAs once external review of 25%/20% estimates has been undertaken.
— Commission work from external consultants on our quality of scoring.
— Review data systems assurance rating for PQ. NAO said data assurance would be enhanced if external review used more extensively. Review again after consultants' work on our quality of scoring.
|
4. Other |
| | |
4.1 | Efficiencies are not sustainable/no cultural change on efficiency/insufficient senior management attention.
| L | H | — Senior mgt involvement in programme, awareness campaign to integrate efficiency into day-to-day thinking and systems (efficiency not economy drive, apply to all we do not just areas with eff targets), compare to other donors (SIDA).
|
| 4.2 | Not reaching overall target and no contingency plan.
| L | H | — Review progress on each target quarterly, investigate reasons for any shortfall and take action accordingly.
|
| | |
| |
SIGN-OFF CHECKLIST QUARTER 4 2006-07
MEASUREMENT METHODOLOGY
Required
— Has the efficiency measurement methodology been
agreed with OGC Efficiency Team? YES.
— Has the efficiency baseline been agreed and recorded
for the start of measurement period? YES.
Methodology and baselines agreed with HMT and OGC in our
efficiency technical note last year. Revisions to some methodologies
and baselines have been approved by HMT and OGC.
DATA MATURITY
Required
— How mature are reported Efficiency Gains according
to the following categories?:
|
| Classification | Definition
|
|
| Likely to change | Figure is an estimate which is highly likely to change.
|
| May change | Figure provides a good indication of total gain but is still considered subject to change, eg because year-end adjustments are anticipated.
|
| Will not change | Figure is not expected to change and will be post any year-end adjustments. (NB Figure may be based on estimates if this is set down in the methodology).
|
|
| |
Note that data lags are considered separately.
Completed in the spreadsheets and narrative report.
DEMONSTRATE NO
REDUCTION IN
QUALITY OF
SERVICE DELIVERED
AN ANNEX
ON QUALITY
MEASURES HAS
BEEN PROVIDED
Required
— Has the quality measurement/assessment methodology
been agreed with HMT/OGC? QUALITY MEASURES PROPOSED IN ANNEX FOR
RELEVANT TARGETS.
— Has the quality baseline been agreed and recorded
for the start of the measurement period? N/A—QUALITY MEASURES
ARE NARRATIVE
— Do recorded measures/assessments of service delivery
demonstrate that there is no reduction in service quality? YES.
DATA ASSURANCE
AN ANNEX
ON DATA
SYSTEMS ASSURANCE
HAS BEEN
PROVIDED
Assessing risk of misstatement
— Has the risk of misstatement of gains been considered
on an initiative basis (in terms of both impact and likelihood)?
YES.
— Have appropriate sources of assurance been identified
for each data system? YES.
Internal controls
— Are controls in place and operating as intending
to ensure the completeness, accuracy and validity of data during
collection, processing and reporting? YES.
— Where sample data is used, has assurance been
gained that the data are representative? SAMPLE DATA NOT USED.
— Where data have come from external sources have
clear guidelines been issued? ONLY EXAMPLE IS EC AID, WHERE DATA
COMES FROM OECD DAC.
— Where data have come from an external provider,
what steps have been taken to confirm the provider's data is fit
for purpose? N/A.
— Are processes and controls clearly documented?
YES.
— Are staff adequately trained in the operation
of systems? YES.
— Are responsibilities for operating controls clearly
allocated? YES.
Managing data quality risks and issues
— Are risks to data quality during collection, processing
and reporting identified and mitigated? YES.
— Are the controls in place to manage risks to data
quality proportionate? YES.
— Are there any significant risks that cannot be
managed cost-effectively which will require disclosure in Efficiency
Technical Notes or departmental reports? NO.
Accountabilities
— Are responsibilities for the quality of data and
reporting lines clear to everyone (at all levels of the delivery
chain)? YES.
Overseeing data systems
— Are senior management satisfied that controls
over data quality are operating as intended? YES.
— If there are control failures, are they reported
promptly to an appropriate level of management and responsibility
for addressing failures allocated? NO CONTROL FAILURES IDENTIFIED.
— Are data quality issues addressed promptly as
they arise? YES.
— Are overall reported gains consistent with your
knowledge of the department's business? YES.
— Are any systems limitations identified and disclosed?
YES.
Audit trail
— Do all staff understand the need for a full audit
trail? YES.
— Do all staff understand the requirements of an
audit trail? YES.
— Has a full audit trail been maintained for each
quarter? YES.
— Are satisfactory arrangements in place to ensure
the safe retention of the audit trail? YES.
| 
