The politics of polling Contents

Summary of conclusions and recommendations

Below is a summary of our conclusions and recommendations contained in this report. Conclusions are shown below in roman type; recommendations are shown in italic type.


1.We expect that polling organisations will continue to seek to innovate, in order to improve the methodologies used in polling and to improve their suitability for estimating voter preferences. It is therefore important that every opportunity is taken to learn the lessons from recent elections. It is also crucial that polling companies and others conduct critical inquiries in public so that the causes of inaccuracy can be better understood, as was done after the 2015 General Election. (Paragraph 93)

2.Analysis of political polls conducted since the 1940s does not show that polling has become more inaccurate over time. However, the three high-profile failures of polling in the UK in the last three years—covering two General Elections and the referendum on the UK’s membership of the EU—raises the possibility that things might have taken a turn for the worse. The internet has certainly made polling easier and cheaper to conduct. However, a combination of difficulties in persuading a representative range of members of the public to take part in polls, shifting demographic predictors of the vote, and an increasingly volatile electorate, have by common consent made it more difficult to estimate political opinion accurately. It is entirely possible that polling failures will become more common in the future. (Paragraph 94)

3.Amongst the methodological issues faced by polling companies, the changing utility of demographic variables for the weighting of samples, particularly the declining validity of weighting based on socio-economic class, is a significant challenge. Polling companies can no longer rely on traditional weighting variables, and so will need to continue to develop new ways to adapt their methodological approaches. Further work is needed to better understand the impact of newer variables such as voters’ educational level, age and attitudes to policy issues such as the NHS and (currently) views on austerity and the UK’s relationship with the European Union. (Paragraph 95)

Media reporting of voting intention polls

4.Recent high profile polling failures can be attributed to a range of methodological challenges, but this is not the whole picture. There are disturbing problems with the way in which voting intention polls are represented by the media. While British Polling Council members are now required to report whether a poll shows a statistically significant change since the previous poll, this information is not always included in media reports. The way in which voting intention polls are represented by the media is often misleading, with a particular tendency to over-emphasise small changes in party fortunes that are indistinguishable from sampling variability. This practice remains largely unchecked. (Paragraph 137)

5.Although the British Polling Council rules require that details of methodological approaches are published, this is insufficient to combat poor reporting practice. This is particularly true of election coverage, where dramatic headlines may not represent the full results of the poll, or may only represent the narrative preferred by a particular editor, which may be misleading. (Paragraph 138)

6.We welcome the efforts which the British Polling Council currently makes to inform journalists and others about polls, including its ‘Journalist’s Guide to Opinion Polls’ published on its website. We recommend that the Guide should be developed to include an authoritative definition of what constitutes a properly conducted poll (as opposed to a small unrepresentative survey), and a list of criteria which must be met for a survey to be recognised as a poll. We recognise that arriving at such a definition will be difficult, but believe that it is essential in order to deliver clarity to members of the public, journalists and others. Once developed, we hope that journalists will be able to use the definition when reporting on polls, and include in their reports a statement as to whether the particular survey met the BPC’s definition of a poll. (Paragraph 139)

7.We also recommend that the British Polling Council should develop its ‘Journalist’s Guide to Opinion Polls’ to include guidance on the types of information that should be included within articles that report on polls. This might include guidance on how to frame headlines to reflect accurately poll results, how to explain the margin of error, and possibly a health warning to remind readers that polls simply represent a snapshot in time, rather than necessarily being predictions of the future. When reporting on particular polls, journalists should be expected to note in their reports whether the organisation which conducted the poll is a member of the British Polling Council or not. To support transparency, journalists should also include in their articles a reference to the published poll. (Paragraph 140)

8.Where relevant, the British Polling Council should make public any examples they find of particularly poor practices of media reporting on polls. The polling companies themselves should also be encouraged to state publicly where they think their polls have been misused or misreported. (Paragraph 141)

9.The British Polling Council should also develop a programme of training opportunities for journalists on how to read, interpret and report on polling data. It would be helpful if this guidance could be produced as part of a collaborative approach in conjunction with the Market Research Society, IPSO, IMPRESS, the Society of Editors, Ofcom, the Royal Statistical Society and academics. (Paragraph 142)

The use of policy issues polls

10.Numerous polls are conducted every week which affect political discourse in the UK. In some cases, there is a failure by those who publicise such polls to communicate all of the relevant details about the selection and framing of questions to obtain a desired answer. We believe that most of these examples are deliberate attempts to manipulate polling findings, in order to distort evidence around public policy issues. We conclude that there is a case for the British Polling Council to play a greater role in proactively overseeing the conduct and reporting of polls. (Paragraph 181)

Oversight of polling

11.The different bodies involved in the oversight of polling need to respond to the challenges involved in the polling of the modern electorate, and to the misreporting and misrepresentation of polls. There are limitations in the current system of self-regulation for polling, and clear areas where the system could be strengthened. (Paragraph 233)

12.We recommend that the remit of the British Polling Council should be expanded to take on a more substantial oversight function. The British Polling Council should adopt a collaborative approach, involving both industry and independent representation. In taking on this expanded role, it will clearly need to work closely with other regulatory stakeholders in this area, including the MRS, IPSO, IMPRESS, the Society of Editors and Ofcom. Some of the functions of the newly-expanded British Polling Council should include:

13.It is often difficult for members of the public to recognise when polling results have been taken out of context or misreported, so it is not enough to simply wait for complaints to be made. We therefore recommend that the BPC and MRS should identify and report instances of bad reporting of polling (whether or not the polls have been conducted by their members) and draw them to the attention of IPSO, IMPRESS or Ofcom as appropriate. Such cases should then be investigated and dealt with quickly through the existing regulatory systems for the media. (Paragraph 235)

14.In order to ensure transparency around voting intention polling in the run-up to elections, we also recommend that the Electoral Commission should take on an enhanced role in monitoring voting intention polling conducted and published during the regulated periods which precede UK elections. In particular, there should be a requirement for the details of all published voting intention polls to be declared to the Electoral Commission, regardless of who the poll was commissioned by, what its purpose was, or how much it cost. The details of all the sources of funding for such polls should then be published by the Electoral Commission, although we recognise that, in order to protect commercial confidentiality, the actual sums of money involved need not be made public. We acknowledge that this will require an extension of the Electoral Commission’s existing remit and recommend that action is taken to achieve this. (Paragraph 236)

15.We are not convinced of the case for introducing a ban on the undertaking and publication of voting intention polls in the run-up to elections. In the future, if polls continue to be a poor predictor of the eventual outcomes of elections, and if the media reporting of such polls continues to influence public and political discourse in a misleading way, then arguments by supporters of a ban would be strengthened. (Paragraph 248)

Digital media

16.The evidence received by the Committee on the use of social media to influence political debate adversely was deeply concerning. We appreciate the complexities of considering a regulatory solution to these issues. We are, however, acutely aware of the urgency of the situation, as many witnesses highlighted that governments, regulators and the platforms themselves are on the ‘back foot’ on many of these issues and have been too slow to address the spread of misinformation and the manipulation of political information on social media platforms. We believe that these issues warrant serious and concerted investigation, and recommend that the Government urgently conducts further research into this issue. (Paragraph 318)

17.One way to combat the spread of misinformation online and to limit its potential impact on democratic debate is to ensure that people have the critical literacy skills to match digital skills to enable them to assess and analyse the information they read online. The Department for Education must ensure that such skills are taught to people of all ages, including children and young people at schools and colleges, as well as adults in further education. (Paragraph 319)

18.We were concerned to hear the issues raised by the Electoral Commission and support its calls for more transparency in online campaign material. The Electoral Commission has called for the Government to introduce secondary legislation to ensure that online campaign material must, like its printed equivalents, include an imprint stating who has published it. This will be crucial in helping to ensure that public confidence is maintained in the electoral system and we endorse this recommendation. However, we recognise that this will do little to address the challenges posed by international actors who try to operate below the radar. (Paragraph 320)

19.We have already recommended that the Electoral Commission should play a greater role in overseeing voting intention polling during election campaigns. In the light of the current challenges posed by digital media, and its ongoing work to ensure transparency relating to online campaign material, it is likely that the Electoral Commission will need to play an increasingly important role in helping to ensure that the democratic process in the UK is not subverted. (Paragraph 321)

20.We welcome the Government’s announcement of the Digital Charter, which will agree new standards for online behaviour. As identified in this report, digital technologies pose some very serious challenges and risks for democracy, which require urgent attention and decisive action. The Government should, without further delay, outline the specific actions it will take to address the Charter’s priorities, including around the legal liability of online platforms and on limiting the spread and impact of disinformation, and publish the likely timescales for its programme of work. (Paragraph 322)

21.The Government should also ensure that the Digital Charter’s work programme includes:

This work will clearly need to be conducted in close collaboration with, or even commissioned from, independent organisations including research bodies, businesses, civil society and other stakeholders. The challenges associated with digital media are fast-moving and the work outlined above should be pursued urgently. (Paragraph 323)

22.We also recommend that the Government should initiate talks within the Council of Europe, the Organisation for Security and Co-operation in Europe, the Commonwealth, the Group of Eight (G8) and other international bodies, to discuss international approaches to tackling the problems posed to the democratic process by the rise of digital and social media. (Paragraph 324)

© Parliamentary copyright 2018