Supplementary memorandum submitted by
the Department for Environment, Food and Rural Affairs (RPA Sub
16)
FURTHER DETAILS
ON THE
TESTING PROCESS
1. Testing schedule (Q.223)
The SPS claim processing functionality was introduced
in RITA releases R3A0, R3A1 and R3A2. These were implemented in
May, July and October 2005 respectively. The testing for these
three releases overlapped and ran from November 2004 until December
2005. The agreed Test Approach for these releases was as follows:
(i) Component Testing (conducted offshore
by the India Delivery Centre in Bangalore).
(ii) Assembly Testing (conducted offshore
by the IDC in Bangalore).
(iii) Application Integration Testing (conducted
onshore by the Accenture led joint testing team in the UK).
(iv) Solution Integration Testing (conducted
onshore by the Accenture led joint testing team in the UK).
(v) Regression Testing (conducted onshore
by the Accenture led joint testing team in the UK).
(vi) Technical Testing (conducted onshore
by Accenture).
(vii) User Acceptance Testing/Deployment
Testing (managed by Accenture, testing performed by RPA staff).
(viii) Operational Acceptance Testing (managed
and performed by Accenture).
2. Testing objectives (Q.224 & 256)
The main objectives of the testing were to prove
that RITA behaved in accordance with the system design and supported
RPA's business processes.
Each test phase is designed to meet a set of
agreed lower level objectives. In summary these were as follows:
(i) Component Testing—lowest level
of testing. The objective is to confirm that each component or
unit of the system functions is in accordance with its technical
design.
(ii) Assembly Testing—concerned with
bringing related components together to test them as a whole.
The objective is to ensure that the internal interfaces and interaction
between components has been correctly designed and implemented.
(iii) Application Integration Testing—functional
testing. The objective is to ensure that the system functions
are in accordance with the functional design and that the business
processes are supported. This will include confirmation of interface
data mapping and "disconnected" testing of interfaces.
(iv) Solution Integration Testing—concerned
with bringing related systems together to test them as a whole.
The objective is to ensure that the interfaces and interaction
between systems has been correctly implemented and that the business
processes are supported.
(v) Regression Testing—to ensure that
any changes introduced do not have an adverse impact on existing
functionality.
(vi) Technical Testing—to verify that
the system is capable of operating at realistic and peak workloads
in a fully integrated live environment.
(vii) Deployment Testing—final stage
of system validation to enable the users of the system to determine
whether to accept the system. Tests are based on the business
activities performed by the users of the system.
(viii) Operational Acceptance Testing—to
ensure that the system can be correctly deployed and meet the
system availability, resilience, reliability and disaster recovery
Service Level Agreement's as defined in the Service Definition
Document.
RITA was also subject to "live data testing".
These tests were driven by a copy of live data and aimed to confirm
that accurate payments could be made with real data. They represented
a rehearsal of what was due to happen on the live system. As per
earlier submissions, this phase of testing was completed after
the system was put live.
There was limited "negative" testing
to ensure that the system did not do things that it was not designed
to do, ie explore what happens when customers/users do things
that are unexpected. The testing conducted was focused on ensuring
the system supported the business processes as documented and
behaved in accordance with the design as documented.
The "live data testing" went some
way to mitigate this in relation to customers completing Claim
Application forms in an unexpected way.
The people working on testing were mainly Accenture
staff from the UK and the IDC in Bangalore, India. There was a
small number of RPA staff involved in RITA testing. The RPA staff
involved were civil servants who had operational experience of
old schemes rather than temporary employees and were seconded
from various sites to work alongside the Accenture team.
Testing progress was reported by Accenture at
a number of joint meetings (Release Owners Meeting, Test Governance
Meeting, Programme Board & CAPRI). Ultimately, the RPA Business
Owner of the testing activities was Ian Hewett.
3. End to end testing (Q.238)
There are two types of End-to-End testing:
(i) End to End Process Testing.
(ii) End to End System Interface Testing.
Both types mentioned above were covered by the
Test Strategy which was in place at the time.
The Solution Integration Testing (SIT) activity
was concerned with testing the interfaces between RITA and the
Finance system via the Managed Gateway (a messaging hub).
Definition of SIT from the Test Strategy Summary
was sometimes called System Test, tests the business-related functional
and technical requirements. Testing is driven by the Business
Processes with an emphasis on the integration between the Applications.
The tested solution is then suitable for use in Training and User
Acceptance Test.
The Release schedule, ie delivering the functionality
required to support SPS 2005 processing in stages meant that functionality
was built and tested in stages which tied in to the dates the
business required the functionality to be deployed in the live
environment. In the testing performed, the aim was for the input
to one stage of testing (eg R3A2 at the highest level) to take
the output from the previous stage of testing (eg R3A1 at the
highest level).
Therefore, true end to end testing of the entire
SPS 2005 processing year from receipt of claims through to making
a payment was not tested in one single test phase. Testing was
performed in chunks, eg Data Capture was tested as part of R3A0,
Level 2 validation, Claim value calculation and making the final
payment was tested as part of R3A2.
4. Testing of Rural Land Register mapping
functions (Q.303)
The Rural Land Register (RLR) was introduced
by RITA release R1A. R1A was the first RITA release and it was
implemented in September 2004. All RLR mapping functions were
tested as part of R1A
RLR mapping functions have been regression tested
in each of the subsequent RITA releases (R1B, R3A1, R3A2, R3B1
and R3B2). Regression testing is where we test something which
hasn't been changed to prove that it is unaffected by changes
that been made to other system functions.
Since the RLR was implemented, processes and
working practices will have changed. Further testing to ensure
the system, processes and working practices work together effectively
has not been performed by the testing team.
5. Accenture contract (Q.296 & 301)
The contract for developing the RPA Change Programme
systems was awarded to Accenture in January 2003. The original
contract cost of acquisition was estimated at £34.1 million,
broken into £18.1 million revenue and £16.0 million
capital. The seven year contract, covered 20 months of development
and ongoing support and maintenances until 31 December 2009. The
current Contract extended the development period but not the support
and maintenance ie it still expires on 31 December 2009.
The Contract was awarded on a fixed price, fixed
scope basis and this premis has remained throughout. The scope
of the Contract was to undertake a Business Process Re engineering
exercise followed by the build and maintenance of a system to
support the findings.
The original contract did not include CAP Reform
costs. At the time of both the procurement tender the subsequent
award CAP Reform had not been negotiated. Therefore the nature
and timing, or indeed whether there would be any reform agreement
at all was far from certain and as such was deliberately excluded
from the contract (save an ability to remove certain aspects from
scope [Bovine schemes]).
All changes to the contract, and consequential
commercial agreements have been considered as successful in the
opinion of both RPA and OGC.
Current forecast for the Accenture contract
(circa £54 million) is not significantly above the original
Pre Tender Estimate (PTE) for RITA of £49.4 million. Indeed
the current forecast includes CAP Reform and SPS 2006, whereas
these were not covered in the PTE.
6. Analysis of other policy options (Q.275
& 331)
An analysis of the impacts of the Scheme to
be adopted in England was published on the Defra website in summer
2004.
http://www.defra.gov.uk/farm/capreform/background/pdf/webnoterev16.pdf
7. Governance papers (Q.206)
The RPA Ownership Board minutes from September
2004 onwards have been published on the Defra website.
http://defraweb/corporate/delivery/executive/rpaob/
8. Rate of which claims were validated with
Entitlements (Q.252)
RPA has been working on the assumption that
there are 120,000 claimants for 2005. After removing duplicate
claims, consolidating multiple claims into single business claims
and voluntary withdrawals, the actual number of claimants establishing
entitlement was over 118,000. Of these 116,474 is the maximum
number to receive a payment since some are not activating their
entitlements.
9. Dates of Ministerial meetings and advice
during 2005 and 2006 (Q.323) Advice to Ministers
2005
23-03-05
28-06-05
18-07-05
28-06-05
28-06-05
28-06-05
28-06-05
19-07-05
19-07-05
19-07-05
13-09-05
06-10-05
12-10-05
26-10-05
02-11-05
09-11-05
16-11-05
23-11-05
30-11-05
07-12-05
15-12-05
22-12-05
| 2006
06-01-06
13-01-06
19-01-06
26-01-06
01-02-06
09-02-06
15-02-06
23-02-06
07-03-06
17-03-06
21-03-06
28-03-06
6-04-06
18-04-06
28-04-06
3-05-06
12-05-06
5-06-06
19-06-06
11-07-06
|
| |
Ministerial meetings with officials
2005
17-05-05
30-06-05
19-07-05
26-07-05
27-07-05
13-09-05
29-09-05
4-10-05
5-10-05
13-10-05
20-10-05
8-11-05
11-11-05
1-12-05
16-12-05—Reading
| 2006
10-01-06
30-01-06
6-02-06
14-03-06
28-03-06
11-04-06
18-04-06
4-05-06
8-05-06
17-05-06
31-05-06
20-06-06
12-07-06
|
—Reading
|
| |
|
Ministerial meetings with stakeholders
Ministers continue to meet with various stakeholders on a regular
basis to discuss a wide number of issues including the Single
Payment Scheme. Following Margaret Beckett's statement on 16 March,
regular meetings have been held by Lord Bach and his successor,
Jeff Rooker, with the Presidents of the NFU, TFA and CLA exclusively
on the SPS.
Lord Bach meetings
22 March 2006
29 March 2006
5 April 2006
12 April 2006—Mark Addison Chair in LB's absence
19 April 2006
25 April 2006
4 May 2006—Mark Addison Chair in LB's absence
Jeff Rooker meetings
10 May 2006
24 May 2006
7 June 2006
21 June 2006
12 July 2006
next scheduled on 30 August
Department for Environment, Food and Rural Affairs
July 2006
| 
