Immersive and addictive technologies Contents

5Supporting responsible design and industry initiatives

147.Given the rapid growth and market dominance of the attention-maximising business models we have outlined in this report, Earthbound Games’s Colin Anderson told us that “we, as a society, need to understand what we expect our digital interactive media to deliver, what that environment is that we are trying to create.”278 This inquiry arises from the belief that we as legislators have a role to play in facilitating that public debate, setting those standards and getting ahead of the technology.

Online Harms legislation

148.The Online Harms White Paper articulates the Government’s vision for a “new statutory duty of care to make companies take more responsibility for the safety of their users and tackle harm caused by content or activity on their services.”279 It outlines the responsibilities that companies will have to address many of the challenges we have already outlined, including bullying, harassment and harmful content.

User-generated content and interaction

149.The White Paper envisages that this new regulatory framework will apply “to companies that provide services or tools that allow, enable or facilitate users to share or discover user-generated content, or interact with each other online.”280 When we asked how this would specifically apply to immersive technology companies, such as those developing games and virtual reality, the then Secretary of State, Rt Hon Jeremy Wright MP, responded that “when we are talking about the video games industry we are not talking about user-generated content.”281 However, he stated that “there will doubtless come a time where users can indeed generate that kind of content.”282 The evidence outlined in Chapter 2 demonstrates that this time has already arrived.

150.Not only do gaming platforms such as Roblox, popular among children, host user-generated games for others to play, but the chat features in internet-connected games and social VR are clear examples of services that allow people to interact online. Therefore, any immersive technology that facilitates player interaction will clearly come under the proposed regulatory framework, with the NSPCC stating that it “would consider gaming sites such as Twitch and Steam to be in the scope of any proposed social media regulator.”283 The former Secretary of State acknowledged this when he said:

I see absolutely no reason why the same principles that we have set out in the White Paper should not apply to them too. We should be saying to platforms, “It remains your responsibility to keep your users as safe as you reasonably can from harms”.284

Moreover, the then Minister for Digital and the Creative Industries articulated the broader responsibilities that companies have to mitigate against the potential impacts of their products:

It is not good enough just to create a game and wash their hands of the potential consequences. I must say […] that that is certainly not the attitude of many of the gaming companies that cross my path, but there is a problem with some of them.285

Excessive screen-time

151.A key area of contention around the Government’s forthcoming online harms legislation is likely to be how the regulator tackles “emerging challenges about designed addiction to some digital services and excessive screen time.”286 As we have explored, evidence on what constitutes “excessive screen time” is highly contested and companies are extremely reluctant to acknowledge their role in designing what might be considered “addictive” properties in games.

152.The 5Rights Foundation told us that although Government has shown leadership in questioning the impact of digital technologies on individuals and society:

all stakeholders, including government have shown a reluctance to tackle the relationship between design and compulsion, compulsion and addiction, and—crucially—the business model that drives the design of service.287

Although the 5Rights Foundation states that “compulsion or addiction by design” are not explicitly listed as harms in scope of the Government’s proposed online harms regulatory regime,288 the White Paper does identify “designed addiction” to be an emerging challenge that requires further research and that the regulator will need to consider. Moreover, the former Secretary of State told us that in his view:

we should be saying to the industry, “You have a responsibility to make sure you are not designing in a kind of addictive property to a game that should not be addictive”.289

153.This issue is also under consideration by the Information Commissioner’s Office in its development of the ‘age-appropriate design code’—a code of conduct for the design standards that providers of online services and apps used by children will be required to meet when they process their data. It will apply to any online services “likely to be accessed by children in the UK”, including games, social media and streaming services, and at its heart is the principle that “the best interests of the child should be a primary consideration” when designing and developing online services that will be used by them.290

154.The code states that technologies should not use “nudge techniques to lead or encourage children to […] extend their use”, arguing that:

Using techniques based upon the exploitation of human psychological bias […] goes against the ‘fairness’ and ‘transparency’ provisions of the GDPR as well as the child specific considerations set out in Recital 38. So does the use of reward loops or other techniques aimed at exploiting human susceptibility to reward seeking behaviours in order to keep users online. They may also run contrary to UNCRC right to be protected from economic exploitation.291

155.The Information Commissioner told us that the games industry:

is quite concerned about our code because it feels that it will undermine or impact the business model of those games through nudges and reward loops and the way that those techniques are built into games.292

However, Dr Richard Wilson from trade body TIGA explained what games companies might do in practice to demonstrate that they proactively prioritise user wellbeing:

game developers might want to consider ways to minimise the amount of time people were spending on games. They can build some of the features into the games, so you can have idle rest periods whereby a player would stop playing because his character cannot earn any points, for example.293

156.We believe that the ICO’s age-appropriate design code is a positive step in addressing the potential impact on children of design mechanics within digital technologies that are aimed at extending user engagement; however, it will not apply to technologies exclusively designed for, or age-gated to, adults. Yet we have heard that disordered technology use or spending can be experienced at any stage in life. We therefore welcome the Government’s intention for “excessive screen time” and “designed addiction” to be monitored by the future online harms regulator. However, we believe greater clarity about the Government’s intention in those areas, and a clear plan for understanding and dealing with those harms from the outset, are needed for the regulator to be immediately effective in this area.

Challenges for legislators and regulators

157.It is important to recognise that the games sector makes a significant economic and cultural contribution in the UK: in 2016, it directly employed 20,430 people and contributed £1.52bn in GVA.294 Yet it is both a young and an uneven industry: two-thirds of the UK’s 2,260 games companies were founded after 2010, and three quarters of those companies employ 50 people or fewer.295 The ICO told us that it is already thinking about how to work with smaller games companies, which will be important in managing the industry’s concerns and compliance.296 Deputy Commissioner Steve Wood went on to say:

There is particularly an opportunity for new businesses setting up to get this right the first time, to perhaps try to build in the innovation and even sell themselves on the basis of caring about data and putting the interests of privacy by design in at the same time.297

Indeed, as Professor Lorna Woods and William Perrin stated in written evidence, the size or age of a company does not impact on its responsibilities to its users:

Some groups are sufficiently vulnerable (e.g. children) that any business aiming a service at them should take an appropriate level of care no matter what its size or newness to market. Or, to put it another way, even the smallest sandwich shop has to comply with basic food hygiene rules from the day it opens for business.298

158.As legislators we also need to be sensitive to the potential unintended consequences of regulation. Outplay’s Keeley Bunting highlighted that their digital output means games companies “are not tied to physical resources” and can therefore be “quite mobile”, moving location to respond to external pressures such as tax or regulatory regimes.299 Furthermore, Dr Matthew McCaffrey from the University of Manchester argues that regulation of loot boxes, for example, “is likely to have the ironic effect of helping large firms in the industry like EA, while hurting smaller, independent developers.”300

159.To date, the technology industry has been innovating quicker than research or regulation can respond. As the British Esports Association states: “Fortnite—the biggest game on the planet right now—barely existed a year and a half ago. This industry moves at lightning pace and we all need to be on our toes to keep up with it.”301 Professor Przybylski told us that regulation of the online sphere has therefore been reactive rather than proactive:

There is a vacuum here and many of the companies that we have in mind—gaming companies and social media companies—have filled that vacuum with a notion of self-regulation. This has meant that from a policy perspective or a stakeholder perspective, from parents all the way up to lawmakers, fundamentally we are in a reactive mode; it is fundamentally a game of “Whac-a-Mole”.302

160.We have been told that the design mechanics our inquiry has considered will soon be replaced by others, and that the potential harms of these technologies will likewise evolve and new ones emerge. For example, Dr Zendle stressed that funding into research on loot boxes was needed urgently because games change “very quickly”, while Dr Jo Twist told us that loot box mechanics are already on the decline and that “next year there will be fewer games that have loot boxes, just because we innovate in our business model so quickly.”303 Given that loot boxes have been featured in games for many years—as have regulatory concerns about them—we would question to what extent that is true, especially in the light of the fact that the speculative spending they generate leads to such high revenues for companies. Yet the need for regulation to anticipate future trends in fast-paced immersive technologies is clear.

161.In advocating for a statutory ‘duty of care’ approach built on a principle of ‘safety by design’, Professor Lorna Woods and William Perrin state that “in considering structural regulatory options, weight should be giving to doing things quickly”.304 They recognise that this can impact the interplay between regulation and research, yet they argue that:

in innovative areas, there is often no long-term scientific research; or such evidence arrives too late to provide an effective measure against harms. Rapidly-propagating services, such as gaming platforms—which often combine live-streaming with user-interaction akin to that on a social media platform—are subject to waves of fashion amongst young people and are a particular challenge for long-term objective evidence.305

162.The former Secretary of State anticipated this problem when he called for companies, rather than regulators, to take a lead in proactively identifying potential harms associated with their platforms and exercising a duty of care towards their users. He told us that:

We should not be waiting for us, as legislators, or the regulator to spot the harm. We should be asking the online platform to spot it as it arises and do something about it.306

Esports

163.Esports—the competitive playing of video games for an audience of spectators who watch either online or in person—has been presented to us as an area of growth and innovation where the UK can play a leading role in promoting responsible design and practices. The Government told us it sees “significant potential for [esports] to develop as an area of real national strength” but recognises that more evidence is needed on its “potential social impacts, both positive and negative, and any implications for current or future regulation.”307 The ‘Audience of the Future’ project has seen £4 million of Government funding go to support the development of Weavr, a new esports broadcasting platform. UK Research and Innovation told us that Weavr’s:

use of AI & data learning to create highly personalised viewing experiences will allow for fully interactive and individualised insights for the audience, which will permit this project to demonstrate insight into how audiences of the future engage in immersive experiences and the pathways to future commercialisation.308

164.It is also interesting to consider the esports sector’s approach to promoting healthy levels of gaming among its professional and amateur players. The head of commercial for esports team Fnatic told us that the team’s focus on the physical and mental health of its professional players includes taking breaks from gaming:

we lost a big tournament at the end of last year that, with hindsight, we put down to the fact that we did not get the guys out and about as much as we might have done to break up the actual playing. Therefore, we focus on that very, very heavily. […] The lesson we have learned from that is we should [be] getting them out doing physical activity and worrying about their physical wellbeing more than we did at that time.309

Similarly, the British Esports Association, which runs esports competitions at a grassroots level in schools, told us that it is “working with schools and teachers and parents to educate what the benefits are when played as part of a healthy balanced lifestyle including physical exercise.”310

165.Yet there are no common standards for the duty of care that esports teams have to their players, as exist for other sports such as football. Chester King of the British Esports Association told us that while some publishers “have very clear codes of conduct” for players, the “huge variety” of games that comprise esports means that those codes are not necessarily consistent across the industry.311 When we asked ESL, which organises esports competitions globally, if it has a written code for what obligations teams that play in its leagues have towards players, Managing Director James Dean replied:

Not exactly. There is a lot of natural perception in terms of what is currently correct, but it is very much an evolving area.312

In its written evidence, ESL UK acknowledges that “as the industry grows, there will be a need for increasing professionalisation and regulation” and argues that the formation of an esports professional players association “could act as a mechanism to prevent exploitation in a young and growing industry.”313

166.We believe that with more and more young people aspiring to a career in professional esports, these industries must also acknowledge their duty of care to aspiring players. After he came second in the Fortnite World Cup, 15-year-old Jaden Ashman and his mother told the BBC’s Today that playing the game for eight to ten hours a day prior to the competition had caused tensions at home and situations where he had fallen asleep at school.314 While Jaden’s message to aspiring esports players was not to spend excessive amounts of time pursuing their goal, that is a message that the games and esports industries have a responsibility to take the lead on and create structures to prevent the next generation of esports players from engaging in excessive levels of play. Jaden’s story is consistent with other reports about the amount of time that leading esports players spend in training. Teams in South Korea, which is one of the leading esports nations, can spend up to 15 hours per day playing video games in order to prepare for competitions.315

167.Over the course of our inquiry we have heard about how esports is a rapidly maturing sector with clear Government support. There are significant opportunities for the development of esports in the UK to harness best practice in the use and monetisation of player data, which could serve as a model for other parts of the games industry. There is also scope for esports to go further in the promotion of player wellbeing and promotion of healthy gaming in schools. We ask the Department for Digital, Culture, Media and Sport to lay out within the next six months how a similar framework to the duty of care practices enshrined and enforced by the governing bodies of other sports can best be applied within esports.

Virtual Reality

168.Virtual reality is also an area of rapid growth and innovation that offers opportunities for the UK to lead the way on setting best practice in design standards. StoryFutures, which researches immersive storytelling as part of the Arts and Humanities Research Council’s Creative Industries Clusters Programme, told us in written evidence that:

By investing at the outset in VR/AR, the government can help ensure its development follows an informed, ethical path, helping to pre-empt and quickly respond to future issues.316

169.We have heard that there is currently a serious lack of understanding and guidance on best practice in the creation of immersive experiences using VR. Sarah Jones told us that in her experience there are “no guidelines” for “what you can do or should not do” with VR technology, and not enough data to “understand how it can impact people and what kind of length of experience there should be.”317 Although such judgments are likely to depend on the nature of the experience, and there is no desire to stifle creativity, she said that those who “are looking for different ways to trick the mind into feeling a different way” need to understand the ethical and wider implications of the immersive technologies they use.318

170.The call for improved standards was echoed by the British Standards Institution, which notes that improved standardisation in the VR and AR sectors “could include guidelines for a safe set-up of the technology, recommend safe immersion periods, user age limits, and where required give content warnings.”319 Similarly, immersive technology studio Inition told us that:

Advice should be given for the optimum length of a VR experience for the audience. This makes for more socially responsible output going forward.320

Yet we have also been told that before such standards are introduced, research will be needed on important ethical questions around the application of VR and its long-term effects.321 The Centre for Immersive Technologies at the University of Leeds states there is a pressing need for such research because:

Currently, immersive technology systems are becoming available to adults and children with no detailed understanding of their potential to create long term adverse consequences—despite unequivocal evidence showing that these systems can produce physiological change.322

VR/AR and ‘fake news’

171.There is growing concern about the application of VR and AR technologies to create ‘deepfake’ films of people supposedly making statements they never actually gave. From electronically recorded samples of someone’s speaking voice, any words can be computer generated to sound as if they were be delivered by that person. The Times newspaper demonstrated this by commissioning an Edinburgh company CereProc to create an audio version of the speech President John F Kennedy was due to give in Dallas on the day of his assassination. Sound engineers completing this using 116,777 sound units from 831 of his recorded speeches and radio addresses.323 Researchers at the University of Washington in Seattle have also created an AI tool which they used to digitally create audio-visual film of another President, Barack Obama, supposedly delivering a speech he never made.324 In May 2019, a fake video of the Speaker of the United States Congress, House of Representatives, Nancy Pelosi, was released, where her speech had been deliberately slowed to make her appear to be in some way incapacitated. This film was removed by YouTube from its platform, but Facebook refused to do the same, instead saying that they would down rank its distribution so that it didn’t appear at the top of the Newsfeed.325 ZAO, a recently launched free app from China, uses AI to allow users to swap faces with celebrities in video clips. This is a further demonstration of the rapidly growing application of deepfake technology to cheaply and easily create false films.326

172.The malicious creation and distribution of deepfake videos should be regarded as harmful content. The release of content like this could try to influence the outcome of elections and undermine people’s public reputation. Social media platforms should have clear policies in place for the removal of deepfakes. In the UK, the Government should include action against deepfakes as part of the duty of care social media companies should exercise in the interests of their users, as set out in the Online Harms White Paper.

Diversity in workforce and output

173.Across the technology industry, diversity is integral to ensuring immersive technologies are safe and relevant to all users. Flora Tasse from Selerio, a start-up working with augmented reality, told us that more diverse teams would help to tackle other problems of bias in product design. She said:

as a person of colour, if I was on that team and I tried that software and I realised that there is an issue and it does not work for me, that would become something that I might raise with the team and they might start thinking of maybe building in more diversity. If we get diverse teams we might get less biased software out there.327

174.However, we have heard about an enduring gender imbalance in the games industry’s workforce. Roughly the same number of women as men play games.328 Yet that contrasts with research by TIGA indicating that just 12% of game developers are female.329 Women also comprise 31% of the audience for esports.330 However, Fnatic’s Nick Fry told us that not one of the team’s 60 professional players is female because:

frankly, at the level we are at, we have to pick the best players to do the best job for the teams.331

175.Marie-Claire Isaaman from Women in Games attributes the current gender imbalance in the games sector to the way in which the industry has historically targeted men in the design and marketing of its products.332 Similarly, Timea Tabori acknowledged that this history “had a profound impact on the types of game that were developed, the types of talent that were allowed at the table to make the decisions for the types of story that games tell.”333 The British Esports Association’s written evidence suggests that the gender inequality in esports might be attributed to similar causes:

Some argue that marketing for competitive games isn’t as female-focused as it could be, others say we should have more female-only tournaments, and some say that harassment based on gender at a more casual online gaming level (separate to esports) is putting off females from progressing further in esports.334

Indeed, we were told by Isaaman that toxic environments can put people off engaging with games, especially in the light 2014’s Gamergate controversy discussed in Chapter 2.335

176.Furthermore, a 2015 survey identified that only 4% of the UK games workforce was from BAME backgrounds, compared with roughly 14% of the working population as a whole.336 Jodie Azhar from POC in Play told us of a perception among some that the games industry is not a viable career option.337 She said:

We are seeing young women filtering out. We are seeing people from different ethnic backgrounds not consider that video-game development is for them, so they are not pursuing it as a career.338

Yet Outplay’s Keeley Bunting articulated why such imbalances are harmful for the industry’s creativity and long-term health:

You would not want all of your books, all of your music, all of your movies to be made by a homogenous group. Video games are really important as entertainment and as something that speaks to society and informs society. It is important that the authorship be inclusive of people from a variety of backgrounds.339

Industry initiatives to promote a more diverse workforce

177.The games industry is clearly both aware of this issue and working to address it. Timea Tabori also told us that gender pay gap reporting has helped the industry to recognise the extent of the problem:

Even if before they would anecdotally believe that there is an issue and they would pay lip service that it is something that they are working on, I think actually seeing the numbers was a wake-up call for a lot of companies to go, “Okay, hang on, this is more than just a conceptual problem somewhere in the back of our heads. This is something that we can actually do something about. Let us sit down and think about what that is.” These little nudges to drive out the information and then inspire companies to action are a really good example of what Government can do.340

In design terms, we were told that more games are enabling players to choose the gender of their character, for example.341

178.Yet while efforts are under way to promote diversity across the sector, organisations working on this lack sufficient support. Kish Hirani, Chair of BAME in Games, which works to encourage more diverse talent in the industry, explained that the organisation is run on an entirely voluntary basis.342 Similarly, Women in Games has “very little resource so it has been a grassroots organisation.”343 This is disappointing when compared to diversity programmes in other sectors. For example, the Creative Diversity Network is funded by the UK’s major broadcasters, sharing best practice on diversity across the television industry and delivering ‘Diamond’—an online system used by the BBC, ITV, Channel 4, Channel 5 and Sky to obtain consistent workforce diversity data on programmes they commission. Similarly, in football, ‘Kick It Out’ is funded by the FA, Premier League, Professional Footballers’ Association, and English Football League and works to combat discrimination in the game.

179.We heard differing perspectives on what more the games industry itself could be doing to address this funding gap. Ukie’s Dr Jo Twist highlighted that the UK’s small companies do not necessarily have the resource to contribute more to such initiatives; however, TIGA’s Dr Richard Wilson told us that he has found it hard to encourage companies to fund research into this area, suggesting that “there is more that games companies can do, particularly the bigger ones”.344

180.What we have heard about the games industry’s attempts to tackle the gender imbalance in the workforce echoes its approach to understanding and tackling the potential harms of its games on players: while some take their responsibilities seriously, others could be doing much more.

278 Q805

279 HM Government, Online Harms White Paper, CP 57, April 2019, p 7

280 HM Government, Online Harms White Paper, CP 57, April 2019, p 49

281 Oral evidence taken on 8 May 2019, HC (2017–19) 361, Q403

282 Oral evidence taken on 8 May 2019, HC (2017–19) 361, Q403

283 NPSCC (IMM0012)

284 Oral evidence taken on 8 May 2019, HC (2017–19) 361, Q403

285 Q1535

286 HM Government, Online Harms White Paper, CP 57, April 2019, p 6

287 5Rights Foundation (IMM0126)

288 5Rights Foundation (IMM0126)

289 Oral evidence taken on 8 May 2019, HC (2017–19) 361, Q402