Good Governance - Effective use of IT

Written evidence submitted by Charles Symons (IT 05)

Summary

· This submission addresses a critical aspect of Q7 of the PASC inquiry on how well do current IT procurement practices work. As noted by the PASC, "central government is notorious for large IT projects running over time, over budget and ultimately failing". One should add to this list that few parts of the public sector know how to control value for money on projects that are delivered.

· IT projects fail mostly because of the failure to manage the procurement of software. Repeated enquiries have recommended actions that, whilst all important, are insufficient to solve the problems.

· This submission recommends a set of practices to rectify the weaknesses, namely a) adopting ways of measuring unit costs and hence value for money of supplied software, b) sharing performance data across departments to establish norms for value for money and c) adopting best practice at using these data to control value for money and delivery of new projects to time and budget. At present some parts of the UK public sector use practice a); that is simply not enough.

· It is noted that software suppliers all measure their own performance (as recommended here) and make healthy profits whilst the taxpayer suffers all the cost overruns and delays. But it is not in the suppliers’ interests to educate their customers on how to manage them properly. Customers must learn how to be in charge.

· The methods recommended here have been pioneered partly in Australian government and partly by the COSMIC organization which has prepared this submission. Whilst the methods are not yet widely used by software customers, their take-up is increasing, with many reports from public and private sector organizations of excellent returns on investment. Some examples are given in this submission.

The scope of this submission

1. This submission concerns a very specific but critical aspect of Q7 posed by the PASC, namely: "How well do current procurement policies and practices work?"

2. The ‘specific aspect’ is that procurement practices in much of the public sector do not give any reliable indication of the value for money it obtains from software-intensive IT systems and generally result in poor control of the procurement of systems to time and budget.

3. This submission is also relevant to Q6 and Q8 on the skills and assets the Government needs to own, and to Q9 on adapting to the ‘age of austerity’. We also address Q12 on how the UK compares to other countries with regard to government procurement of IT systems.

The current situation

4. Procuring IT systems means buying hardware and software, telecommunications services and sometimes new physical premises. But the usual cause of IT projects failing or being delivered late and over budget is difficulties with procuring the software.

5. Few parts of the UK public sector keep track of the unit costs of bought-in software and are able to judge value for money. As far as we are aware, no part of the UK public sector uses best practice to control delivery of software to time and budget.

6. Overcoming the problems of projects failing or being delivered late requires many well-known actions including good project management, clear and stable lines of accountability, clear specification of requirements, etc. These have been repeatedly recommended over the years and they remain vital. The missing ingredients – measuring value for money and using the resulting data to control the delivery of new software – will not alone solve the problems. But without these ingredients, the problems will continue.

7. Just four years ago, it was reported [1] that "approximately 30% of government IT projects are delivered to time and budget". At that time, the CIO of the Department of Work and Pensions negotiated a pledge from 12 of the largest IT system suppliers that this figure would be raised to "over 90% within 3 – 4 years". We suspect this initiative has got nowhere (the PASC should ask for a progress report). A 2007 study [2] of 105 outsourced UK public sector IT projects costing £29.5 billion showed that 30% were terminated prematurely and that the cost of overruns for projects that ran to completion was £9.0 billion (30.5%).

The missing ingredients

8. Public sector customers for IT systems need to acquire three assets and the associated skills to exploit them.

a. Measurements of the amount of software required and delivered so that unit costs can be measured

b. A common repository of unit costs and other performance data from all public sector software-intensive IT projects which can be used to share experience and to support contract negotiations with IT suppliers

c. Processes by which customers can exploit the data to control and improve value for money and the delivery of new systems to time and budget.

9. It is important to note that the principal IT suppliers all have their own repositories of performance measurements for all their projects. However, it is not in their commercial interests either to share their data with customers or to propose any processes by which their customers could properly control their services. Lacking these data and processes, customers pay high unit prices and suffer the extra costs of overruns, whilst the suppliers continue to make healthy profits. There is without question a causal link between the inability of customers to measure and control their software suppliers’ performance and the poor service that customers receive [3].

A little history

10. In 1990, the Central Computer and Telecommunications Agency (CCTA) recommended that central government departments adopt a method of measuring software and using the measurements to determine value for money and for estimating future projects [4]. Several departments responded, notably the Inland Revenue which has used the method successfully to help control value for money since its systems were outsourced in the early 1990’s. A few other departments continue to use the method.

11. Unfortunately, there is no pooling of performance data across departments and there are no standard processes to exploit the data to control new projects. Attempts by departments to compare their price/performance against external benchmark data have been largely a waste of money.

12. About ten years ago the Government of the State of Victoria, Australia, developed the ‘Southern Scope’ process [5] to help control the scope, value for money and delivery to time and budget of software projects. In 2005 they published results showing that by using this process they had reduced the unit cost of externally-supplied software by a factor of three and that the average cost overrun had been reduced from 84% to less than 10%. This method was presented by representatives of the Government of Victoria to the UK Office of Government Commerce, but no action was taken.

13. Although the emphasis of the PASC inquiry is clearly on the procurement of administrative systems, the methods recommended here are equally applicable to the procurement of real-time and embedded software. This is highly relevant to systems procured by the MoD and by some other departments. A pilot study, paid for by the MoD Procurement Agency, was successfully completed in 2002 of using the measurement methods on the Eurofighter software. No further action was taken.

The experience in other countries

14. The following are countries that we know have adopted some or all of the three ingredients (measurements, pooling of price/performance data, and a process based on Southern Scope).

15. Australia. The approach of the Government of the State of Victoria has now been adopted by the Commonwealth Government in Canberra.

16. Italy. Several years ago, the Italian equivalent of the OGC (‘DigitPA’) published very comprehensive guidelines for the procurement of software covering all three ingredients. DigitPA maintains a central repository of price/performance data. There are anecdotal reports that they, too, have reduced the unit cost of software by a factor of three.

17. Finland. The Ministry of Justice has successfully piloted all three ingredients resulting in achieving a unit cost of software of €300, down from a range of €500 to €1000.

18. The EC Directorate of Taxation and Customs Union (TAXUD) has been using all three ingredients for some years. Since 2006, TAXUD has required that each software supplier "shall justify the effort he/she quotes in his/her proposals/offers for software development, maintenance, testing and related documentation...." (by using methods recommended in this submission) [6].

19. The Chinese, Japanese and South Koreans have established repositories of software project performance data, managed by national research institutes, to which public sector bodies contribute data.

20. Countries that in recent years have published studies by government auditors lamenting the problems of procuring software include Canada and the Netherlands.

21. There is no doubt that the public sector in most countries does not have proper controls on software procurement. The situation is not much better in the private sector, though these methods are being increasingly used, with some reports of an excellent ROI.

Recommendations

22. The UK Public Sector should

a. adopt the internationally-standardised COSMIC software sizing method [7], a more advanced and general method than that recommended by the CCTA

b. establish a repository of public sector software project performance data and use it to monitor and improve value for money in software procurement

c. adopt the ‘Southern Scope’ process for software procurement and require software suppliers to follow it to ensure delivery to time and budget

23. This work needs a small investment and will require a long-term commitment. Given the size of the prize, doing nothing is not an option, especially in this ‘age of austerity’.

(COSMIC is a voluntary, not-for-profit organization of software metrics experts from the Americas, Asia/Pacific and Europe, founded in London in 1998. It is dedicated to improving practices in software measurement and project estimating. For more see www.cosmicon.com. )

References

[1] ‘Suppliers agree to cut IT costs for Whitehall’, the Financial Times, 1st December 2006.

[2] ‘Cost overruns, delays and terminations: 105 outsourced public sector ICT projects’, Dexter Whitfield, European Services Strategy Unit, Research Report No. 3, December 2007.

[3] ‘Software industry performance – what you measure is what you get’, C. R Symons, IEEE Software, Nov/Dec 2010.

[4] ‘Estimating with MkII Function Point Analysis’, CCTA, 1991.

[5] ‘Southern SCOPE: avoiding software budget blowouts’, the Government of the State of Victoria, e-Government Resource Centre, www.egov.vic.gov.au .

[6] Invitation to Tender documents issued by the EC Directorate of Taxation and Customs Union, 2006 through 2010.

[7] ’Software Engineering – COSMIC-FFP – a functional size measurement method’, ISO/IEC 19761:2003.

January 2011