Further memorandum submitted by Iosis
Associates (RPA Sub 13b)
RURAL PAYMENTS AGENCY: REASONS FOR FAILURE
INTRODUCTION
1. This submission is primarily a response
to a statement made on the BBC Radio 4 programme "File on
Four" on 20 June. Towards the end of the programme, Michael
Jack MP, Chair of the EFRA Committee, stated that he "can't
easily identify where the body is buried that tells me fundamentally
why the RPA weren't able to make their payments on time as particularly
advertised by Lord Bach in January this year".
2. This response particularly takes information
from the Uncorrected Oral Evidence (HC 1071-iv) of the 22 May
session of the EFRA Committee, at which representatives of Accenture
were questioned in depth.
3. In this note I point to one key failure
which was particularly responsible for the very late realisation
that the Accenture systems could not produce those all important
lists of payments for authorisation by an RPA operative so that
the data could be sent to the cheque writing process: end-to-end
tests of the complete business process (including the technology)
were not carried out.
4. Then I go back to a key failure in the
management, one that is probably common to numerous government
projects: the failure by RPA to properly assess and manage risk
and report risk to the core of Defra.
5. Also another mystery is mentioned below—but
I don't have a complete analysis of that one.
THE KEY
FACTOR
6. Q561 (Mr Drew) produced an answer from
Mr Naish that gives the key to the "buried body":
"The way the system works when it establishes
entitlement is that it needs to take the claims across the whole
of England in order to make those entitlements. It is difficult
to run those end pieces of the programme on a small part of the
population."
7. It ought to have been possible, during
system tests, and then at any stage once some claims had been
verified, to advise the system that it has a complete set of data,
give it a notional sum of money in the pot, and then run "those
end pieces of the programme" ("the last stages of the
process") as a test. This would probably be done by taking
a copy of the dataset of verified claims to date, using a separate
computer system loaded with the necessary software, and running
the last stages on the partial data accumulated to date at the
time of the test.
8. Thus it would have been possible to see
if the entire process could be run to completion:
— if it had any problems, or even a
fundamental problem, with the data; and
— if (as repeated tests were run during
the build up of data to the full set of claims) the growing size
of the data was going to cause any problem as the process was
scaled up to the full 120,000 claims.
9. An extension to this would have been
to generate test data with 120,000 records, but the evidence from
the 22 May session suggests that that would have been a significantly
large sub-project in its own right.
METHODS TO
AVOID SUCH
PROBLEMS
10. The primary way in which the customer
should seek to avoid the problems found with the RPA SPS is to
have in place and make use of the necessary expertise to assess
and manage risk, and propose mitigation methods. Then the reporting
structure from Programme Manager to Chief Executive (and in this
case all the way to Defra Chief Executive) has to be through personnel
who themselves are skilled in risk management, operate an environment
in which the full facts are clearly seen as essential to decision
making, and are accountable for failure. That requires an organisation
in which personnel have a regular cycle of assessment of capability
and contracts of employment that include provision for best practice
methods for resolving under-performance.
11. At a recent meeting (5 May) of a group
of specialist ICT[24]
suppliers with Join Wailing, a Director in Cabinet Office e-Government
Unit, I introduced the use of a contracted technical audit team
as one way for a customer organisation to monitor the progress
of a large project. Such teams were used successfully in the mid
1980s when the Stock Market moved to screen-based trading, using
completely new computer systems. The audit team has to have access
to the main delivery contractors, and the resources to undertake
system tests as necessary. Such teams are used in, for example,
the auditing of safety-critical industrial processes, and one
was engaged to audit and test a massive update of the ticketing
system of Singapore's rapid transit public transport system (the
move from simple processes using magnetic stripe tickets to more
complex processes using smart cards). This approach was clearly
news to eGU.
12. If the statements by RPA to me in their
letter of 16 May[25]
are true, clearly the OGC Gateway reviews fail to identify serious
risks of failure—on the one hand core documentation was
not present; on the other hand the programme is managed in a professional
manner. However, the File on Four radio programme stated that
the SPS project had become stuck at one of the OGC gates, which
merits further investigation.
A NOTE ABOUT
THE POSITION
OF CONTRACTORS
SUCH AS
ACCENTURE
13. Within the 22 May session there is a
sub-theme suggesting that Accenture had what is eventually described
as a "moral and... professional obligation" (Q443) and
later a "moral responsibility if not a contractual responsibility"
(Q631) to guide the client towards a better solution, warn that
the business process outcome may not be delivered, etc Assuming
that Accenture's answer is confirmed by a study of their contract,
Accenture was right to say that they do not have such a responsibility.
14. Certainly it is possible for the relationship
between contractor and customer to include giving advice on best
practice, but in the end the customer makes the decision. With
a contract that does not include the requirement to give such
advice, the customer is liable to react very negatively if the
contractor says that they should do the job differently: the contractor
may then be labelled "difficult" and excluded from further
work. (Personally I'm currently in that position with an organisation
that is constituted as a Membership Company without shares: I
was a contractor to the Company at a time when I was also a Director
of the Company—unpaid as a Director, paid as a contractor.)
These are business relationships, governed by business law, and
the customer, in this case the public sector, has to be organised
and managed in such a way that it can discharge the moral responsibility
to the public. From the supplier's point of view, the professional
obligation within a supplier company may well cause personal anguish
to its employees and Directors, but it is only those organisations
where there is legal obligation and/or contractual obligation
to warn their clients (eg advisers such as lawyers, accountants,
even technical consultants) who can be taken to task if they do
not issue the necessary warnings. Personally, it causes me grief
if my tax monies are not spent wisely by the public sector.
ANOTHER REPEATED
THEME IN
MEDIA REPORTS
15. Throughout the spring of this year,
media reports were indicating that throughput of the SPS team
operating the claim verification stage was very low. If those
reports were correct, it was easy to demonstrate that the promised
payment dates would not be met, yet apparently RPA management
did not take any remedial action. As reported widely, and addressed
in the 22 May session of the Committee (eg Q405), a change, under
the new Chief Executive, from a task-based approach to a claims-based
method, including bringing mapping system technical support staff
into the processing centre, dramatically improved throughput.
This again suggests that RPA management was not operating its
risk management and mitigation processes in an adequate manner.
Peter Tomlinson
Proprietor
Iosis Associates
July 2006
24 Information and Communications Technology. Back
25
See my submission to the Committee on 19 May. Back
| 
