The big data dilemma Contents


1.The term ‘big data’ is used to describe the collection and analysis of data on a scale or of a complexity that makes the use of such data challenging. According to the Information Commissioner, big data is a way of analysing data that “uses massive datasets”, “often involves bringing together data from different sources” and may involve processing data “in real time”.1 Research Councils UK gives the following description:

What constitutes ‘big data’ varies between disciplines and sectors. It goes beyond the extremely large and complex datasets generated by, for example, the Large Hadron Collider, DNA sequencing, Earth observation, government records and transactions, commercial or online interactions, to include data from new technologies. Smaller scale data of high complexity and variability, for example from environmental monitoring and the Internet of Things, where sensors capture and process large amounts of fast-moving (and often personal) data, is such technology. Regardless of the source, big data is about gaining value and insights from extremely large, complex, fast moving or combined data, across a range of sectors in innovative and beneficial ways.2

2.Big data has huge potential value to the UK, both as a driver of productivity and as a way of offering better products and services to citizens. It can help businesses and entrepreneurs “to identify areas of opportunity for innovation in new products, processes and services; improve customer engagement; identify inefficiencies; improve productivity, identify market trends; and use the UK Government’s Open Data ( to innovate and to create new companies”.3 In the public sector, “intelligent data analytics can help public service organisations to increase the operational efficiency of public service delivery, reduce expenditure and costs whilst delivering increasingly personalised services to citizens”.4 The European Commission’s Digital Single Market Strategy for Europe states that:

The growth of data is exponential—90% of all data circulating on the Internet were created less than 2 years ago.

Only 1.7% of EU enterprises make full use of advanced digital technologies, while 41% do not use them at all. Digitisation also offers unprecedented opportunities to other economic sectors, such as transport (e.g. intelligent transport systems) or energy (e.g. smart grids, metering).5

3.In 2013, the Government identified big data as one of its ‘eight great technologies’, which it envisaged would contribute to future UK growth. The opportunities offered by big data, and the challenges involved in realising them, were highlighted, also in 2013, in the Government’s Information Economy Strategy: “Business sectors across the economy are being transformed by data, analytics, and modelling. Data is increasingly being produced at a rate that means that current techniques are insufficient to fully exploit it.”6 The ‘eight great technologies’ were given £600 million in the 2012 Autumn Statement, of which £189 million was assigned to big data technologies, particularly for bioinformatics and environmental monitoring.

4.Much coverage of big data focuses more on the risks than the opportunities, particularly in connection with the storage and processing of personal data. Our predecessor Committee examined the privacy issues arising from big data in the context of social media in their 2014 Responsible Use of Data report.7 It cautioned that: “The UK is already a leading player on the global stage in using social media data and we are keen for this status to be maintained, but only if that can be achieved while ensuring the personal privacy of UK citizens.”8

Our inquiry

5.Since our predecessor Committee’s earlier inquiry, £450 million has been allocated in the 2015 Spending Review for the Government Digital Service. The Data Protection Act 1998, which transposed the 1995 EU Data Protection Directive, will need to be overhauled within the next two years as a result of the agreement of an EU General Data Protection Regulation in December 2015.9 The Government is also developing a Digital Strategy, which it expects to publish soon.10

6.We undertook this inquiry to follow up our predecessor’s report in the light of these developments, and to begin what we aim to be a programme of inquiries looking at the ‘great technologies’. We sought written evidence on the following terms of reference:

(1) the opportunities for big data, and the risks

(2) whether the Government has set out an appropriate and up-to-date path for the continued evolution of big data and the technologies required to support it

(3) where gaps persist in the skills needed to take advantage of the opportunities, and be protected from the risks, and how these gaps can be filled

(4) how public understanding of the opportunities, implications and the skills required can be improved, and ‘informed consent’ secured

(5) any further support needed from Government to facilitate R&D on big data, including to secure the required capital investment in big data research facilities and for their ongoing operation.

7.We received over 80 written submissions. We held three evidence sessions covering the health, direct marketing and financial technology sectors; the Information Commissioner and others concerned with the ethics of consent for data use; organisations tasked with assisting industry and other researchers use big data; and ministers for the digital economy (Ed Vaizey MP) and internet safety and security (Baroness Shields) as well as departmental officials. We are grateful for the assistance in our inquiry provided by Heather Reeve-Black from the National Audit Office.

1 Information Commissioner’s Office, Big data and data protection (July 2014)

2 Research Councils UK (BIG0057)

3 Research Councils UK (BIG0057)

4 Tech UK (BIG0039)

5 European Commission, A Digital Single Market Strategy for Europe, COM(2015) 192 (May 2015)

6 HM Government, Information Economy Strategy (June 2013)

7 Science and Technology Committee, Responsible Use of Data, Fourth Report of Session 2014–15, HC 245

8 Ibid, paragraph 35

9 European Commission, Press release, Agreement on Commission’s EU data protection reform will boost Digital Single Market, 15 December 2015

© Parliamentary copyright 2015

Prepared 11 February 2016