4The role of data, design and business models
99.The data practices, design strategies and monetisation models employed by companies play a significant role in exacerbating the potential harms that we have outlined. Colin Anderson from games company Earthbound Games recognised that a fundamental issue across digital media is that the business model “incentivises businesses to capture the maximum amount of attention”.185 He went on to say:
If there is a direct relationship between the amount of time that people spend in an application, be that a game or something else, and the rate at which they will monetise, then you do not have to be Adam Smith to figure out that that is going to incentivise behaviour within companies to create products that do that sort of thing.186
Likewise, the Information Commissioner’s Office observes that business models in the technology industries have:
led to the situation where keeping users ‘hooked’ on a given website underpins the entire business model of a great many technology companies, even when that process creates or fosters the harmful outcomes.187
100.In the digital economy, data, design and monetisation are inextricably tied. The 5Rights Foundation observed in written evidence that the design strategies of online platforms “are based on the science of persuasive and behavioural design, and nudge users to prolong their engagement or harvest more of their data.”188 UK Research and Innovation’s written evidence outlines the interplay of data, design and business models in the gaming industry specifically, stating:
The introduction of internet-connected games has significantly improved the design and gameplay of games. Developers can better understand their users and their behaviour when playing a game. However, it also allows designers to build better compulsion loops into their games to make them more addictive. If a publisher is working on a freemium model, then it directly benefits them to try and keep the player on the platform for as long as possible.189 They can also use user data to maximise and target in-game purchases, or vary gameplay to correspond to the likelihood of a consumer making a purchase. The issue is compounded if the user is playing on a mobile device, which can provide notifications that give the company more opportunities to pull the user back into a game.190
Collection and use of user data
101.Immersive technologies create and rely upon large volumes of user data, some of which will be highly particular to how the technology operates. For example, the Department’s written evidence describes how virtual reality devices can track eye movements and may also record “a person’s location and physical movements”.191 The Information Commissioner, Elizabeth Denham, described her organisation’s work on the rapidly evolving ways in which immersive technologies collect data as “an arms race” and said that:
We have been pulled into the modern world, but only recently, and we have to work really hard to stay up on these issues.192
She also told us that the online games industry specifically “has some maturation to do in understanding what their obligations are in data protection law.”193
Gaming telemetry
102.Internet-connected games enable companies to gather huge amounts of data on how players interact with, experience and behave in their products. This might include how long people play for, when they lose interest and when they spend money in the game.194 The remote collection of data by games companies, for the purpose of conducting analytics, is known as ‘telemetry’.
Use in game design
103.Games companies told us that data on how people behave in their games helps to refine their design and performance to improve the user experience. Kerry Hopkins explained that Electronic Arts “collects information on how features are used, how often they are used and whether users are facing frustration in the features.”195 For example, in EA’s game Battlefield:
People go to different maps of the game to play, and we had one map where folks just kept failing in the game. We were able to use the data we collected from the game to visualise what was happening. We found that a lot of people kept going down this one road in the map, clogging it up and failing. Through that data, we were able to say, “Let’s change the look of that. Let’s add some trees to this road. Let’s make some changes so that we don’t have thousands of players going down this road and failing at this mode.”196
Similarly, King’s Alex Dale explained that they use information about how people are playing games such as Candy Crush Saga, “to make them fun and challenging in the right mix”.197
104.Moreover, despite concerns around gaming disorder, some parts of the games industry use data collected about players to modify their experience and keep them playing for longer. Alex Dale told us that King does not use player data to change the experience for individual users—a practice known as ‘dynamic difficulty adjustment’—however, other companies do. For example, Electronic Arts has patented systems that mean “the difficulty level of the video game may be automatically adjusted” to “keep a user engaged for a longer period of time.”198
105.Other ways that games companies use data to shape the in-game experience include online multiplayer ‘matchmaking’—the process of connecting players to compete against each other in a player-versus-player online game. For example, if someone is identified of being at risk of quitting a game, an algorithm might pair them with a less-skilled player to boost their confidence and keep them playing, or match them with another player whose in-game resources will incentivise their opponent to buy a microtransaction.199 In written evidence, Dr Stephen Kaar and Dr Sachin Shah told us that it is:
critical that games companies behave in a responsible and transparent manner and are discouraged from exploiting information asymmetry i.e. using knowledge of a person’s personal or financial information or in-game habits and items to manipulate the player’s in-game experience to encourage spending, without the player’s prior knowledge. This issue has been raised in particular with matchmaking algorithms within certain on-line sports games, however the prevalence and effectiveness of such elements is not known.200
Use in marketing
106.The data that companies collect on players is also used to market to them. Neil McClarty from Jagex explained that the company uses game data to target players who may not have played for a while:
It may be that for someone who has not paid or played before, we may make them an offer if they wish to engage and make a payment, but no different to something where we would also prompt them by saying, “Try a piece of content” that they have not done in the past. We use our data responsibly but, of course, what we are ultimately trying to do is make them, like any medium, play and engage with the content.201
He also told us that it is “standard” for games companies to categorise players into “cohorts” for the purposes of design and marketing:
One of the key founding principles of game design is that you are almost creating for a particular type of player. We have cohorts of four or five archetypes, basically, of players in our minds when we are designing for games. It could be someone who loves quests or someone who is a bit of a free-for-all, they like to do everything, or it could be someone who is a bit of a social butterfly, who basically does not really engage in the game content but is very much chatting and online and talking to people and so on.202
He acknowledged that “you could cut the player base into genuinely hundreds of different, small cohorts based on their geolocation, their length of play,” and other characteristics.203 The company then targets its marketing messages at those cohorts, “so that we are engaging the right group of individuals with the right piece of content.”204
107.At King, gaming telemetry is combined with other sources of behavioural insight in the design and marketing of its games. Alex Dale told us that the company has about 50 or 60 data scientists analysing behavioural data on players, including phone and email surveys.205 The company told us that this work is to help “the designers imagine who they are building a game for”.206 However, as we have found through our work on companies such as Cambridge Analytica, the real value of such surveys is when they are combined with data on people’s online behaviours to gain psychological insights about them. Therefore, it would make sense for King to be combining that survey data with gameplay data. We asked whether the survey responses could be linked back to individual players, producing research on someone’s gaming habits alongside insights about them as a person, Alex Dale responded: “Yes, we can do that and we would do that for gaming habits in particular.”207 However, Adam Mitton later stated: “If you mean doing that on an individualised basis, I do not think we do that”208 and in supplementary written evidence King stated: “There is never any linking from an individual’s survey results to an individual’s gameplay.”209
Identifying normal and harmful patterns of play
108.Despite the vast amounts of granular data being collected, many of the companies we spoke to were not able to answer questions on average levels of engagement with their games. This frustrated our attempts to understand fully what normal and healthy engagement with games looks like, and to determine whether the experiences of people like James Good and Matus Mikuš are indeed the exception or are in fact more prevalent than the industry acknowledges.
109.Epic Games’s director of marketing was unable to give us any detail on what would be considered normal engagement for a “frequent player” of Fortnite.210 This is despite the company gathering data on how long people play for, which it told us it makes available to players, or their parents, through a “weekly play usage report”.211 EA’s Kerry Hopkins was unable to answer our question on length of play within FIFA, claiming “we don’t actually record play time.”212 Again, as the company told us it does record how long a player is connected to the game, and telemetry from in-game behaviour over that period could give an indication of play, we reject the assertion that the company does not know how long its users play for.
110.Data on how long people play games for is essential to understand what normal and healthy—and, conversely, abnormal and potentially unhealthy—engagement with gaming looks like. Games companies collect this information for their own marketing and design purposes; however, in evidence to us, representatives from the games industry were wilfully obtuse in answering our questions about typical patterns of play.
Data sharing and integration between games and social media
111.Most social media platforms provide a feature to allow users to login to other online services, including games, using their social media account. For the user, this delivers convenience, with one less set of login credentials to remember. It can also enhance game-play, by enabling a player to compare their score to those of their friends.213 For the social media platform and the provider of the other online service, it enables a free flow data exchange between both parties. By implementing a Facebook login, developers can access a range of data points on a Facebook user, and the principle of data reciprocity means those companies will also supply data back to Facebook in exchange.
112.All of the games companies we took evidence from allow people to login with their Facebook accounts, but again it proved difficult to get clear and upfront information about what data is shared between platforms.214 However, we do know that some games feature enhanced integration, such as the ability to play Candy Crush directly through the Facebook site, which in turn gives Facebook information about any spending behaviours.215
Effective age verification
113.The General Data Protection Regulation (GDPR) applies specific protections to children’s personal data and introduced new requirements for the online processing of it. Despite all companies that offer online services to individuals in the EU being subject to these laws, the evidence we received highlighted challenges with age verification and suggested that some companies are not enforcing age restrictions effectively.
114.There was consensus among the social media and games companies we spoke to that they are unable to operate robust systems to verify a user’s age. Snapchat’s Stephen Collins told us, “there is no foolproof verification system”, and Instagram’s Vishal Shah admitted “that a young person can get around” the registration systems the platform has in place to attempt to restrict use to those aged 13 or over.216 We challenged Instagram on why the platform did not ensure these systems were adequate before rolling out their products globally, to which Shah responded:
I would not characterise that as reactive versus proactive but more that in general when we let people sign up for Instagram we are trying to collect as little information upfront from them as possible.217
115.The games companies Electronic Arts and Epic Games similarly expressed a tension between operating robust age verification, for example by requesting hard identifiers of age, and adhering to the principle of data minimisation, where the least amount of personal data is collected from a user as possible.218 Canon Pence from Epic Games explained that the company does not request a date of birth from people signing up directly to play Fortnite: Battle Royale through the company’s website because:
At a high level, it is our view that we intend to collect the minimal amount. We don’t believe, in terms of servicing Epic accounts, that we need age in order to deliver what has been requested by the account holder.219
When we asked whether the company considers it necessary to comply with data laws by understanding the age of the people who play Fortnite, Pence replied: “We don’t.”220
116.Deputy Information Commissioner Steve Wood acknowledged some of these tensions and said the ICO “would be concerned if there was wide-spread age verification collecting hard identifiers from people, like scans of passports.”221 However, he pointed to more sophisticated technological solutions, such as age estimation, which uses “algorithms running behind the scenes using different types of data linked to the self-declaration of the age to work out whether this person is the age they say they are when they are on the platform.”222 Will Scougal from Snapchat told us that the platform is able to monitor user signals to ensure users are the appropriate age:
For example, it might be flagged that someone is connected to people who behave in a certain way, look at certain content and are in certain locations, which might infer that they are of a certain age. That would put them into a group of people who, for example, would not receive advertising at any point. It would put them into a group of people who would be flagged as potentially underage.223
117.There are consequences of the lack of effective age verification in the light of the potential harms we outlined in Chapter 2. For example, identity verification platform Yoti told us in written evidence that:
At present, companies such as Snap are not ensuring that very young people with smartphones are not exposed to inappropriate material, or grooming.224
Brad Enright also cited age verification in gaming and social media as one of the real divides between those industries and the gambling sector, which he described as “a sector that does this, has always done this and more recently is moving with technology to make it a slicker process.” He told us:
That just seems like such a major flaw in so many different things that this Committee and the Government are seeking to do in terms of online harms. That is the elephant in the room, that if you cannot verify that someone is or is not 18, you are relying on a tick box, how are you ever going to have robust controls?225
118.It is of serious concern that there is simply no effective system in place to keep children off age-restricted platforms and games. The reactive way in which platforms are dealing with this problem further highlights the problems of online industries rolling out products without considering, or mitigating against, their potential adverse effects on users.
Design mechanics
119.Many games, as well as social media platforms, are explicitly designed with the aim of maximising what the industry terms “engagement”, but which, as Tristan Harris outlined, translates as time or money spent.226 He told us that the reality of digital platforms is that:
behind the screen are 100 engineers who know a lot more about how your mind works than you do. They play all these different tricks every single day and update those tricks to keep people hooked.227
As there is a lack of evidence on the full effects of all design features, we have also heard the recommendation that:
The potentially addictive mechanics employed by some game and platform developers should be indexed and regulated by a governmental body or through some form of transparent scientific collaboration.228
120.The drive to make a compelling product that people want to keep playing is inherent to the creative process, as Tony Gowland of Edinburgh-based games company Ant Workshop told us:
You want people to love what you are making and I do not think there is anything inherently dirty or wrong about that. It is an artistic drive.229
Yet we believe that given the concerns of people who feel that they have lost control over how much time or money they spend on gaming, there may be a case for balancing that artistic and commercial ambition with a greater sense of duty of care in the way products are designed.
Random rewards
121.Evidence demonstrates how some games, as well as social media platforms, use psychologically powerful design principles similar to those used in the gambling industry. Dr Mark Griffiths’s written evidence highlights how the structural characteristics of video games—the features that induce someone to start or keep playing—resemble those employed to keep people using gambling products, such as “high event frequencies, near misses, variable ratio reinforcement schedules, and use of light, colour, and sound effects.”230 Match-three puzzle games such as King’s Candy Crush Saga demonstrate reward mechanics in action: for example, the game rewards players with incentives, such as pop-up motivational slogans or free ‘spins’ that offer another random chance to win ‘boosters’ to enhance game-play, at random intervals.
122.Similarly, design mechanics that encourage people to stay on, or return to, social media platforms include pop-up notifications delivered at random intervals; a lack of “stopping cues” to prevent people from reflecting on how long they have been using an application; and deliberately structuring menus or pages to nudge people into making choices that the platform favours.231 Dr Jacob Johanssen told us in written evidence that:
Social media have highly addictive tendencies because of the kind of ‘withhold—reward’ dynamics that are immanent to them. They constantly suggest that by quickly checking, taking a brief look, refreshing the app, I may be rewarded: a new message, interesting news, a new friend request, etc.232
123.The randomised nature by which players are psychologically rewarded in their use of games and social media mimics the design of slot machines. Tristan Harris told us:
A slot machine handles addictive qualities by playing to a specific kind of pattern in a human mind. It offers a reward when a person pulls a lever. There is a delay, which is a variable—it might be quick or long. The reward might be big or small. It is the randomness that creates the addiction.233
This principle was established by B. F. Skinner in the mid-20th century, who found that “random rewards reinforcements are stronger at maintaining behaviours” than continuous reinforcements. In short, as psychology student Pritpal Kalsi told us:
Skinner confirmed that we are neurologically hardwired to pull that metaphorical lever of chance much more when there’s only a chance of reward rather than guarantee of reward.234
Other design features in games
124.Although the evidence base is, again, limited, we have heard that certain types of games are particularly problematic for people with disordered use. Dr Daria Kuss told us, for example, that massively multiplayer online role-playing games:
appear to have a higher addictive potential in comparison to other games, including offline games. Players spend increasing amounts of time playing those games, up to 48 hours without an end. Those games are extremely immersive. They give the player all sorts of rewards in terms of social rewards, recognition and achievement-related rewards that they may not be able to get in a real-life offline setting.235
One of the key features of massively multiplayer online games is that the in-game environment is a ‘persistent world’, with things going on even when a player is not interacting with it. The Royal College of Psychiatrists identifies that:
Multiplayer online games can make it difficult for people to stop gaming as the recent games developed ensure the game continues when the player stops playing. In so doing, the pull towards continuing long sessions of play is maximised.236
125.The principle of “dark game design patterns” offers another framework to understanding the ethics of how game mechanics are deployed. These are defined as game design patterns that are “used intentionally by a game creator to cause negative experiences for players which are against their best interests and likely to happen without their consent.”237 For example, it is argued that requiring players to play at specific times or dates (referred to as “”playing by appointment”) is a “temporal dark pattern” because:
players are forced to orient their real-world activities to meet the obligations of the game, rather than the other way around.238
This was echoed by Matus Mikuš who told us that the time-limited reward structures in games kept him playing even when he did not feel he wanted to:
Some games, modern games specifically, tend to create additional layers of rewards and progression on top of the actual game. That is where the trouble really begins because you have rewards just for logging in that day and rewards for completing a game. It gets the ball rolling and gets people spending time when they do not want to. I am guilty of this myself. The game League of Legends has a system where if you win a game that day you get some extra points to buy a new character. There were days when I would come home from my practice, it would be 11 at night and I did not want to play a game, I wanted to go to bed but I was like, “There is this reward. If I play just one game and win it I get these extra points.” I would lose that game but, “I need those points now, I will play another game”. Sometimes you would be three or four games into that final one, it would be 2 in the morning and I am sitting there, “I didn’t want to do this”.239
126.Representatives of the games industry have emphasised that in such a diverse entertainment market, players have choice over what and how long they play. Dr Jo Twist, CEO of Ukie, told us:
I have choice in what games I can play. It is the same as I have a choice of what documentaries or what Netflix series I want to watch. I want to do something that is enjoyable. As soon as it stops being interesting, I will go to something else.240
Dr Richard Wilson the chief executive of TIGA, another trade body representing the games industry, told us:
we have seen from evidence to this Committee that there clearly are some people who play games excessively. I was particularly struck by the two individuals who turned up who said they contacted Game Quitters. They were clearly playing games to excess and were addicted. I am also struck by the WHO’s definition of gaming addiction, which is very tightly drawn. I think we have to recognise the WHO’s decision on gaming disorder. It is important to recognise it because it has been drawn in a very reasonable and conservative way. I think we need to take steps as an industry, and obviously working with Government, to make sure we can minimise the potential for gaming disorder.241
127.The games industry’s emphasis on player choice does not acknowledge the way in which many games use random reward mechanisms that have been scientifically proven to create repetitive behaviours, and the effect that this might have on the meaningful exercise of choice. Moreover, the reluctance to discuss engagement metrics or to acknowledge the psychological impact of core design principles in evidence to us suggests that highly-skilled designers either do not know the data and psychological studies and strategies that underpin their industry or, what is more likely, do not feel comfortable admitting it in a public forum. For an industry generating such high revenues from so many millions of players worldwide, that lack of transparency is unacceptable.
The free-to-play games market
128.The desire to keep players engaged in games also stems from the business models in the industry. As we have already explored, there are different ways in which games make money, including the ‘premium’ model where players pay a clear upfront price for a game; a subscription-based model, where players pay on a monthly or annual basis to access games; and the ‘freemium’ model, where a game is essentially free-to-play, but contains the facility for players to make in-game purchases.242
129.These models are becoming increasingly blurred, with more features from the free-to-play market being seen alongside premium pricing models. For example, “avid gamer” Cameron Graham explained that premium games can also contain in-game monetisation:
The typical game released by large gaming companies such as Electronic Arts or Activision will have a purchase cost of around £50, however once the game is bought, these companies attempt (and largely succeed) to sell in-game purchases for extra money.243
Such purchases may enable players to download extra content, or to purchase in-game items such as skins or loot boxes. James Good from Game Quitters told us that:
Every single game now seems to follow what is called a free-to-play model. In the past you would buy a game for £20 and get everything. Now you do not pay anything, you just download it and the only way you can progress is by buying loot boxes and downloadable content or DLC to buy new characters and maps. It does not seem like much but eventually you start paying dozens and dozens more pounds than you would have done if you just bought the game. Every single game now seems to be preying on this model that is really quite addictive.244
Colin Anderson of Earthbound Games likewise told us that “free-to-play is definitely where the market is trending towards”.245
130.The revenues generated by in-game monetisation demonstrate why it has become so popular. Dr Jo Twist told us that last year, 43% of revenue from games released in Europe came from in-game transactions.246 In 2017, US company Activision Blizzard reported that it made more than $4 billion, or more than half its annual income, from microtransactions in both paid-for console games, such as the Call of Duty series, and free-to-play games such as Candy Crush.247 As Colin Anderson explained, it is not hard to understand why there is a trend towards higher monetisation in this business model:
There is no upper limit to what you can purchase, so you are essentially enabling people to spend what they like. You are taking the cap off, whereas on a premium title where there is a fixed price, one set price has been paid. There is then little or no additional revenue.248
131.Despite the revenues generated, only a minority of players make in-game purchases. King’s Alex Dale told us that in the free-to-play mobile games market between 3% and 5% of all players will pay.249 Likewise, Outplay’s Senior Designer Keeley Bunting told us that in the games she works on:
less than 5% of our user base who choose to make purchases and even within them over the course of a month—and this is quite standard for the casual game genre—it may be less than $1 on average for players, depending on how long somebody plays your game for.250
Mobile games are just one part of the diverse free-to-play market in which spending patterns can vary greatly based on the nature of a game and its user base. For example, the makers of the free-to-play game RuneScape told us that “only about 10%” of the company’s players spend on microtransactions, and that all players spend on average “about £50 or £60” annually.251
132.The figures we were told evidence the widely held principle that the vast majority of revenue in the free-to-play games market is being derived from a small number of high-spending players. For example, Jagex also told us that it makes one-third of its revenue from that 10% who spend on microtransactions.252 The Gambling Commission acknowledges that this creates a situation of “extremes of behaviour at the margins accounting for a substantial proportion of revenues”.253 The potential for such a model to exploit a minority of potentially vulnerable players should be of concern, but again the games companies we spoke to emphasised that spending comes down to player choice. King’s Alex Dale told us, for example, that:
The level of spend they want to make in the game is very much player driven.254
Yet this position further downplays the role that marketing behaviours and game design plays in influencing player behaviour and incentivising spending.
How games incentivise spending in free-to-play
Design
133.There are a range of design features that incentivise spending. We were told that it is “very common” for games to use in-game currencies, such as gold bars, which players buy for real money and then use to buy additional in-game items.255 UK Research and Innovation explains that:
The argument for this methodology is that the player can also generate in-game currency through gameplay. However, one of the consequences of this is that it loosens the connection to real-world currency, and makes it harder for the consumer to recognise how much money each asset is worth. This is particularly significant for children who may not fully understand the value of money, let alone money converted into in-game currency.256
For example, in FIFA players can make purchases using two in-game currencies: ‘coins’ that are earned in the game, and ‘points’ that are purchased for real-world money. However, the currencies have different conversion rates—for example, a gold pack of players might cost either 5,000 Coins or 100 FIFA Points—making it a challenge to work out what the real-world value is.
134.King’s Alex Dale rejected the idea that in-game currencies are used to disguise how much players are spending and instead said that they give players “good value”.257 None the less, Dr David Zendle identified that it is yet another design mechanic borrowed from the gambling industry:
That is a very similar technology to financial access technologies in casinos, for instance, or forms of scrip, where the actual money you are spending disappears behind a middle currency.258
Furthermore, he argued that the similarities between game mechanics, including loot boxes, and practices in the gambling industry are too close to be coincidental. He told us:
I would be astonished if this was convergent evolution with no crossover from experts. I would find it very hard to believe that at some point someone has not consulted an expert on how to best monetise this random chance-based technology.259
Marketing
135.Similarly, marketing plays a role in encouraging people to spend. Alex Dale told us that it can be difficult to predict if, or when, a player might spend in a free-to-play game:
It could happen in the first week, after a year, after five years. It is very unpredictable.260
However, when it comes to player acquisition, games companies are able to target advertising at people who are identified as being likely to spend. He also said:
Facebook and Google and our other partners will have identified devices that they believe are highly engaged gamers, although it is not certain because it is statistically based. Obviously those are people in the casual space that are of interest to us.261
136.Building on the concept of cohorts, marketing can also be directed at players who are identified as likely to spend. Professor Andrew Przybylski explained how a hypothetical games company might target someone in a high-spending cohort (referred to by some as ‘whales’—a term taken from the gambling industry to mean an above-average spender)262 He told us:
They found that half of the revenue comes from 0.5% of players. You know what they do? They assign secretaries to manage the whales, so when you do not play for a little while the algorithm says, “Check in on this person” and then they have a human being e-mail and say, “You have not advanced to the next level”.263
Advertising-funded social media
137.The business incentives to keep someone using a platform apply as strongly to social media as they do to games. As we explored in our previous inquiry on disinformation and ‘fake news’, social media companies’ business models are founded on serving data-driven advertising to their users. The Information Commissioner’s Office notes:
For better or worse, the advertising model of revenue is now firmly embedded in the technology industry. This model is heavily reliant on personal data as both fuel and engine. There is a strong financial incentive for websites and apps to keep an individual engaged as long as possible and to serve personalised adverts to that individual throughout.264
138.There is a clear link between time spent and monetisation potential on digital platforms. YouTube’s Marketing Director, Rich Waterworth, told us that “the amount of money made is not linked to the time that you spent watching something” and that “ads being served is not linked to time spent on the site”.265 While this implies that there may be no direct causal link between the time someone spends viewing videos on YouTube and the platform serving adverts to them, it masks the underlying principle that the longer someone spends on YouTube, the more opportunities YouTube has to target them with an advert and make money in the process.
139.YouTube uses sophisticated algorithms and machine learning, combined with basic default user settings, to maximise the amount of time people spend watching videos on its platform. For example, it will monitor what a user is watching or searching for and combine that with machine learning to recommend videos it thinks the user will be interested in. This is then combined with a default auto-play function, which automatically queues up another video before the one you are currently watching has ended. The power of these systems to keep people viewing content is attested to by the fact, as we have already explored, that 70% of time spent on the platform is spent watching videos it recommends. Marco Pancini told us that a core design principle for YouTube’s parent company Google is “to make sure that technology is there when we need the technology, not to become a slave of technology”; however, it is unclear to us how this fits with a product that uses algorithms to determine how users spend the majority of their time.266
Usage monitoring tools
140.In recent months, social media platforms have followed smartphone providers in introducing tools to help people to monitor or control their usage. For example, YouTube enables people to track how long they have spent watching videos on the site in the previous seven days, set an automatic reminder to alert them after a defined period of viewing and reduce the number of notifications they get.267 Instagram’s Vishal Shah articulated that his platform’s similar wellbeing tools are intended to enable people to set their own limits for usage:
I do think there is a limit for how long people should be spending on the platform. I just do not think it is the same number for everybody, and I do think that having these controls gives people the tools to make those decisions for themselves.268
141.Such measures may result in decreased engagement. We heard that users who enable Android’s ‘Wind Down’ feature, setting a time period for sleep where they do not get notifications, have seen a 27% decrease in usage.269 However, it could also be argued that these tools have been introduced by platforms without clear research into their efficacy or strategies for monitoring their effectiveness. YouTube was not able to tell us how many users have enabled its wellbeing features.270 Vishal Shah said that as well as Instagram’s research into the tools being “anecdotal” the platform does not “have any specific goals against usage, nor do we have any specific numbers to share against them.”271
Role of distribution platforms and app stores
142.Throughout the inquiry we have heard about the role that large games distributors or app stores play as gatekeepers to these technologies and services. For example, representatives from Epic Games and Electronic Arts told us that if their games are played through Sony’s PlayStation, for example, Sony’s systems are responsible for providing initial age verification and therefore ultimately “own the relationship with users.”272 The digital storefronts may also offer device-level parental or spending controls, and are evidently trusted by consumers—one parent told us that a 4+ rating on Apple’s app store meant that “as a parent I should not have any cause for concern” about the game’s content.273
143.Such platforms are in a strong position to promote responsible design and distribution among the games and apps that they make available. For example, since late 2017, Apple’s developer guidelines for its app store has required that:
Apps offering “loot boxes” or other mechanisms that provide randomized virtual items for purchase must disclose the odds of receiving each type of item to customers prior to purchase.274
Likewise, Ian Rice from the Video Standards Council explained the leading console storefronts “mandate the use of PEGI ratings so you cannot upload a game to those storefronts if it has not been classified by PEGI.”275
144.Yet it is important to remember that these distribution platforms also benefit financially from the business models outlined earlier in this chapter, including by taking a cut of revenue from microtransactions. When Epic Games launched Fortnite for Android mobile devices, it chose to do so without using Android’s app store Google Play, and the company’s CEO admitted that this was to bypass the 30% cut that the app stores took from in-game revenue.276 We also heard that analytics tools provided by the platforms themselves facilitate granular data collection to maximise engagement. Michael Veale from University College London told us that Google’s PlayerStats API:
will provide you with predictions for, “What is the probability this person is in the highest 95th percentile of spenders? What is the probability this person is going to spend on your app within 24 hours?” They buy an add-on or something within your app, “What is the likelihood this person is going to spend this amount over a longer period of time?” This is being provided as a service by platforms, which are the co-ordinating platforms, to individual companies designing apps to market on these platforms.277
145.During this inquiry we have heard that online games and social media are both data-driven industries that use asymmetrical information and deliberate design practices to manipulate users into spending more time or money on their platforms. The argument that engagement is purely a user’s free choice is undermined by the amount of data collected about them and the use of that data alongside design features, such as random reward mechanics, that have been proven to have powerful psychological effects.
146.To provide clarity for policy-makers and the public, the Government should outline in its response to this report how it intends to support independent research into the application, extent and effect of design mechanics used in digital technologies to extend user engagement. Such research should then inform the development of a behavioural design code of practice for online services. This should be developed within an adequate timeframe to inform the future online harms regulator’s work around “designed addiction” and “excessive screen time”.
185 Q799
186 Q799
189 ‘Freemium’ refers to a business model where initial or basic access is provided free of charge, but additional features are paid for.
192 Q1490
193 Q1464
194 Q757
195 Q1058
196 Q1058
197 Q1265
198 Google Patents, “Dynamic difficulty adjustment”, accessed 11 July 2019
201 Q481
202 Q569
203 Q570
204 Q572
205 Q1347
206 Q1355
207 Q1354
208 Q1358
210 Qq1046, Q1117
211 Q1117
212 Q1057
213 Q1373
214 See Appendix 2 for categories of data that may be shared between Facebook and applications that use the login tool.
215 Q1381
216 Qq263, 965
217 Q1008
218 Q1103
219 Q1101
220 Q1102
221 Q1477
222 Q1477
223 Q328
225 Q1599
229 Q785
235 Q57
237 José P. Zagal, Staffan Björk, Chris Lewis, “Dark Patterns in the Design of Games”, Foundations of Digital Games Conference (14–17 May 2013)
238 Ibid
239 Q162
240 Q1406
241 Q1392
245 Q736
246 Q1438
247 Activision Blizzard, Inc. “Activision Blizzard announces fourth‐quarter and 2017 financial results”, (8 February 2018), accessed 20 February 2019
248 Q737
249 Q1257
251 Q434
252 Q451
253 Gambling Commission, “Virtual currencies, eSports and social casino gaming – position paper”, (March 2017), p 14
254 Q1321
255 Q1340
257 Q1340
258 Q83
259 Q89
260 Q1324
261 Q1325
262 Dr Charles Kriel, “Dictionary of Free-to-Play Game Design”, 2014, p 61
263 Q21
265 Q842 ff.
266 Q835
267 Q815
268 Q983
269 Q832
270 Q831
271 Q985
272 Q1174
274 Apple, App Store Review Guidelines, accessed 16 July 2019
275 Q1410
276 Tom’s Guide, Why Installing Fortnite on Android Will Be a Security Nightmare, (3 August 2018)
277 Q21
Published: 12 September 2019