What Online Games Can Learn from the Failures of Social Media

[Music] [Applause] hello my name is daniel kelly i'm the associate director of the center for technology and society at the anti-defamation league and today i'm going to talk to you a little bit about what online games as digital social spaces can learn from the failures of traditional social media over the last uh 10 years or so so let's dive in a little bit about me these are my identities that's me at sleep away role-playing slash computer game camp in 1999 and there's me uh right before covet 19 talking about online games and making them respectful inclusive communities for all people i don't work in the game industry uh but i'm a lifelong gamer shout out to dark sun shattered lands a classic and uh again i don't work in the game industry but i am a game

advocate and i do game advocacy um at an organization called adl and in the department of adl called cts so apl is the anti-defamation league which is a non-profit that was founded in 1913 with the mission to stop the defamation of the jewish people and secure justice and fair treatment to all the mission of cts the center for technology and society where i work is that same mission but focus specifically on digital spaces so traditionally uh we've worked with social media companies like facebook and twitter uh and google youtube on addressing hate on their platform and uh making sure that their communities are respectful and inclusive for all people and are made more so and sort of around the time that the center was established at the end of 2017 we started uh thinking and working on uh games and specifically online games as digital social spaces and thinking about what we can take from our work with social media and apply to online games as digital

social spaces so we all know that online games are digital social spaces i don't need to tell that this audience that um and they have been that for years this goes back to uh the games that i played when i was younger like everquest and ultima online that were social and even back before that however in recent years we've really seen an uptick in uh the degree to which games are social with the advance you know huge advancements but in mobile gaming with cross play allowing people to play across different platforms and in 2020 uh nuzu which is a data analytics firm that tracks uh data in the game industry uh listed as their top um their top sort of trend for the year uh and this is a quote from them is increasingly difficult to find someone between the ages of 12 and 30 who doesn't play games especially in western and asian markets for these younger generations games are

replacing social networks as the go-to digital destinations to meet and socialize with friends uh and so i've put on the slide two examples of uh something that's happened just in the last few months uh we saw animal crossing be used as a space for protests so this is a picture of a black lives matter protest in animal crossing we saw peta use it for animal rights protests we saw a sitting member of the u.s house of representatives use it to engage with supporters and constituents alexandria ocasio-cortez uh and on the flip side of the coin here we have uh fortnite being used as a concert venue uh so uh travis scott uh through his concert turned this sort of battle royale shooter into a concert venue for 12 million people so it's a really exciting time for all different uses of online games as social spaces but at the same time it's um at the same time that we're excited about the expansion of these spaces

um it's a time to be a little circumspect and ask hard questions uh about the expansion of online games as digital social spaces and what that will mean for those who want to use those spaces to harm others for example in animal crossing it's being used for a political protest but what is you know it may seem like a strange question but what is animal crossings policy about political disinformation right if uh bad actors enter into animal crossing and start spreading misinformation about when for example election day is how does animal crossing deal with that right um if uh users are organizing uh white supremacist events in fortnite uh what capacity does the team does the fortnite team have to detect that right what is fortnite's responsibility to intervene and educate young people who have started on the path to radicalization right i don't think these questions uh

are are answered yet uh and um and these and many others if game companies are not already thinking through and working on these questions uh they they will only become more urgent as time goes on and what we can see here this is a uh a survey that adl put out in 2019 and what we can see in the survey is that a lot of the uh the sort of phenomenon of digital social spaces that social media is getting a lot of flack in the press for now um exists in really meaningful and uh expansive ways in in online games already so here we have this was a survey a nationally representative survey of hate and harassment in american adults in online games and 74 percent of american adults experienced uh some form of harassment in online games and 65 experience severe harassment which is discrimination based on identity you can see the breakdown

of uh of which identities were targeted on the right there as well as sustained harassment uh sexual harassment stalking and doxing so these are really serious uh issues that uh i think it's pretty clear that you know facebook and the like are getting a lot of flack for not addressing these kinds of harms but they exist and they exist in in large amounts in online games as well similarly we measured the extent that extremism disinformation and conspiracy theories uh exist in online games so for example we asked players uh did they hear discussions about the superiority of whites and the inferiority of non-whites which is sort of textbook white supremacist ideology and 23 of american adults responded saying yes they had been exposed to the conversation about that can we we can't really extrapolate more from a from sort of a survey

but i you know in terms of like is this are they recruiting are they you know are they just hearing it is it just being normalized in the conversation we can't really know from the sort of broad uh uh the broad coverage of a survey um but it is something to be concerned about it's something that the game industry should take notice of that as we are seeing uh extremists on traditional social media so too there may be and we need more research and information and and work to determine the true extent but this is definitely an indicator that it's worth uh the game industry turning their attention to so i've sort of lumped together uh online games and social media so far but they are of course very very different right and so how do we determine sort of where in the life cycle of the digital social space online games are versus uh social media one way we can do that is to look at their terms of use their code of conduct so in theory

digital social spaces are ruled by these are governed by these sets of rules right a company puts forth their terms of use or code of conduct which a user agrees to do you know to abide by on their service uh and it is enforced you know in in the cases of many sort of very large platforms uh it can be very hard to enforce uh code of conduct or community guidelines uh consistently and at scale uh so it depends on the platform whether it these are in you know rules that govern the platform in theory or are they actually the ways in which the blackboard is governed but at the very least um these these rules and these codes are a public expression of that company's uh responsibility towards their players or users and their sort of philosophical approach to the idea of content moderation so if we look at league of legends and 2020 this is what's you know currently on their website we can see um you know a wide spectrum of uh of different kinds of

behaviors sort of incorporated in one rule right offensive uh harassment is included here sexual harassment uh hate speech right racially uh charged speech racial uh racism uh defamation all sorts of different kinds of of hate uh and harassment is sort of all bundled into one rule um and uh in order to sort of get a corollary kind of rule uh in social media you kind of have to look back at facebook in 2006 um which uh where where we see these rules you know of for all different kinds of content bundled into one rule and if we look at uh facebook in 2020 this is sort of uh an excerpt from one tier of facebook's hate speech rules uh it is a it is a portion

of tier one of their hate speech rules which is divided into three tiers and this is one form of behavior right and i'm not saying that facebook is the example here right there they have a lot of they have a lot of problems but i think if we look between uh their 2006 rules which lumps all of these things into one uh and we look at 2020 we can see the degree to which they've been faced with a variety of different kinds of behavior and have had to adapt their rules to uh address these kinds of behaviors on their service and we can see a similar kind of comparison if we look at fortnight in 2020 we have these sort of lumps of of different kinds of content all put in one rule and again if we look at twitter in 2006 right we have a very similar kind of lumping of rules uh and twitter has also sort of uh in the in the past 10 years or so uh 14 years or so

and you know made their policies and their rules more granular over time um and so if we think about facebook and twitter in 2006 and where they were right um that was the point at which uh facebook was going from being a service that was purely for college students to a more open market uh that was a point at which you know the press was was sort of marveling at twitter and this sort of new uh incarnation of microblog right um i don't think anyone writing or working there was thinking that 10 years from this point one of the main discussion discussions of the 2016 u.s presidential election is going to be focused on these two services right facebook in 2006 wasn't thinking 11 years from now we're going to be implicated in genocide in myanmar of the rohingya among this muslim community in myanmar uh they just weren't seeing that they didn't have the benefit of

hindsight and so that's what i'm hoping to provide a little bit here for the game industry and for online games to look at okay what is social media do what is the hindsight lessons that can be learned from how social media has failed over the last 10 years that can then be adapted to online games as we move forward so i'm going to divide this into three sort of big failures as we see them for social media one is that social media was designed for scale and engagement first without centering the safety of users especially for vulnerable and marginalized communities so the motto of facebook at its outset was move fast and break things uh one of the one of the big quotes from twitter over their lifetime i think this was from 2012 was that we are the free speech wing of the free speech party right so these are these are uh mantras and and ideologies that are centering uh the the

mass expansion of these platforms the engagement of users and is not really thinking about okay uh when we talk about free speech uh how are we thinking about the impact of speech right when we're talking about breaking things are we talking about uh just our systems are we talking about the experience of targets of uh of harassment do we do we feel comfortable breaking uh breaking the the the you know the you know the society that we are a part of right i don't think these sort of large questions were being asked so for the game industry i would uh we would recommend that they take responsibility for the experience of players and incorporate incorporate anti-hate by design principles when creating online games so this is just a small example when when you're designing a new game or when you're changing an existing uh game think about these couple of questions what are the speech implications of design and policy

choices in your in your game whose speech will be most impacted by the choices that you're making if you're trying to address a disruptive behavior in an online game is it best addressed through changing a product is there a form of community moderation empowering users to enact the norms of the community that you can you can work towards or is there or is there a need for a real global policy that will stay both for users and for the public that this kind of behavior or these kind of ideologies are not welcome in our on our service and how do you go about resourcing each of those different uh approaches and third i'll say you know what kind of research would be valuable as you are building these uh um your your game platform or these uh or changing your game who's doing that research how can you connect with them right um making sure that all of your uh design and policy and moderation

decisions are informed by research that either you are finding or that even potentially you know that you are initiating and conducting yourselves so uh failure number two of social media is change as as crisis pr so we have these platforms that were built for scale and engagement and when we see the real world impact of these of these systems uh despite you know calls from adl and other organizations over the years to change the times where we see real change on these platforms is in response to real life tragedies that cost uh human life so uh there's three examples here one is uh in charlottesville virginia in 2017 so adl and other organizations uh had been telling social media companies about the threat of white supremacy for many years and asking them to take it more seriously and take more steps but it was only when uh extremists

got together and rallied in uh in august of 2017 in charlottesville that uh and that and where we saw heather hair lose her life uh that we saw internet companies really um take you know significant steps to address white supremacy on their platform and to really rethink their policies uh the same was true of the pittsburgh tree of life shooting in october 2018. so um before that we were telling uh various internet service providers that gab is a bad actor platform that does not engage in content moderation that does not uh care about hate speech on its platform and it was only after the largest uh anti-semitic attack in american history uh that uh gab you know that various internet service providers cut off service to gab um and the same is true of of the christ church mosque shooting

uh in in christchurch new zealand in march of 2019 the muslim community in christchurch new zealand was attacked and and and the attack was live streamed on facebook live and it was only after the murder of many uh muslims in christchurch new zealand that facebook did this and that the global internet forum to counter terrorism uh enacted a uh a cross-platform collaboration that was more uh expansive than than previously um in all these cases uh social media responded uh to these uh you know was inactive and didn't listen to the recommendations of civil society before these tragedies and used the uh national spotlight of these tragedies to make the real substantive change after their uh platform had the uh the tragic effect that the tragic role in each of these and many other tragedies

um so what what should online games do to uh to get ahead of this to make sure that they're not put in the same position so they should be proactive they should be ambitious and they should be inclusive in making changes to their products so think about problems that may either be starting to appear on your service or may not even be on your servers but you've seen them over the past in social media how does your team identify and address extremism in your online game today how can your team support targets of repeated and campaign harassment in your online game and and build these build these capacities internally and uh and in hand in hand reach out to civil society organizations like the adl and others who work in this space and who have the expertise on how these things manifest in digital social spaces um we're we're here to help other other groups are here to help and and uh and represent the the

experience of various vulnerable marginalized communities and how they uh operate and uh and the problems that they face in online games uh and so uh failure number three i'll talk about uh is uh that that uh this may seem a little wonky but um there's no real transparency from the social media companies about the nature of hate on their platform and the success they're having in combating so um most social media companies at this point have produced something called a transparency report which reports on the actions the platform takes on various kinds of content in various kinds of categories whether that's hate speech or harassment or violent content but right now all of the data that is provided is unverified by any third party and it uses self-selected metrics so one of the stories that i tell is there's a major internet company who was talking to us at one point and said you know we did xyz and harassment is down seven percent

so seven percent of what right and what does seven percent mean in relative to the experience of targets on your platform relative to uh uh the the the scale of the problem on your platform how is this you know compared to other uh uh attempts to to reduce harassment so these kind of metrics and the data that that's driving them are are fairly meaningless to be to be frank no platform at this point can answer these three questions which is how much hate is there on your platform what communities are being targeted and is anything you're doing to address hate on your platform working are things getting better or worse you don't know for any social media company and so what we would recommend for online game companies is to create expansive public and meaningful metrics of the problem of hate on your platform and how you are addressing it and this is one sort of concrete example of how you could start that so most games or have or should have a reporting form

add to the reporting form uh uh a an option that allows users to select which of their identities were targeted when they are reporting uh harassment use this data in aggregate right over the course of time to inform how you build out and change your platform based on the needs and experiences of different communities of users and share that data with civil society and the public to say this is our no this is our platform this is the experience of the players on our platform and this is what we're doing to address this problem and here's what's working what hasn't worked right it's really clear that online games could really be a huge leader in this space if they provided meaningful transparency uh about the nature of hate in their different game environments right now uh there is there is no game company that i'm aware of that provide that releases a transparency report uh even at the level of social media i think uh in doing so uh and in following these

kinds of guidelines and others set forth by civil society and others um the game industry could really lead so in summary um be transparent make uh make your transparency reporting uh expansive uh public and meaningful um prioritize real real change and and don't get them caught in the cycle of crisis pr be proactive be ambitious in what you're trying to achieve in terms of uh uh changing and and incorporating into the design of your platform and include the perspectives of researchers and civil society and then design for safety and inclusion first take responsibility for uh what your players are experiencing in your game and seek input seek advice on how uh design and policy choices that you're making excuse me design and policy choices that you're making are going to uh impact your um your players

and especially those from vulnerable and marginalized communities so i talked a lot about how uh online games and social media are the same uh but there are of course uh unique opportunities and challenges for online games uh in and of themselves separate from social media so one of those is audio content moderation of course audio content moderation is not unique to online games but it is a a large uh and prominent part of how people communicate in online games and currently there aren't really so there are a lot of tools and techniques that are used for text chat moderation um that's uh moderating uh comments in a text chat like you would have on facebook or you would have on a chat box in an online game uh the tools and techniques to do that exist in audio content moderation they're virtually non-existent at this point uh and so there is a lot of work that the industry needs to do to address that

we know as a fact uh that that spaces which are not moderated uh end up becoming uh the disruptive behaviors in them end up becoming the norm uh and we really want to you know if the goal is to make online games respective and inclusive communities for all people then the industry researchers all of all of society really needs to step up and invest more in audio content moderation in another vein pro streamers and esports players are public figures with huge reach and are used primarily for marketing at this point but can really have an influence in terms of setting norms for the community i know many many great groups have done a lot of work on this especially the good folks at any key um but i think there's a there's a lot of room for uh thinking through how do we reach and influence uh pro streamers and esports players to help make online games

uh respectful inclusive for all people so in summary i guess i'll i'll say um or rather in in closing i'll say two things so what why action uh to address hate in games and to learn from the past 10 years of social media is necessary uh there are two factors so the reason that we do this work there's a lot of sort of knee-jerk reactions to games and wanting to blame games for the ills of all society um but i know you know from from personal experience i grew up as a gamer um i know that games can be powerful and good and one of the ways in which free frame the survey from last year was to look at hate and harassment but to also look at positive social experiences so 88 of players experience some form of positive social interaction in online games which include making friends finding community even finding a partner or learning about

themselves or others right the reason why we we fight hate in games and try to make them respectful inclusive for all people is not because games are terrible and games are toxic because games are great and we want them to be uh accessible and we want them to be places where people where people of all kinds where people can have these positive social experiences and not feel uh harassed out of them or silenced and it's super important for as games become larger and larger part of our social fabric that we that we move to uh create better spaces for all people and the other the less good side that i'll that i'll sort of close with is that the last 10 years we've seen social media build up platforms focusing on you know safety second engagement and scale first we've seen them you know disregard the requests of civil

society and use you know change only as a matter of public pressure or when tragedy uh and the loss of life implicated their platforms and third uh we you know we are totally in the dark about the nature of their platforms and whether anything is getting better and so when lawmakers hear those three things um they question the ability of uh of uh social media companies to govern themselves right we've seen legislation passed in europe we've seen you know we have a hearing coming up at the end of uh july here where tech ceos are going to be brought in front of congress i don't think that online games or the game industry is there but i think if we you know if the game industry doesn't learn from the past 10 years of social media then i think we will get there and it's and i think uh neither us nor the game industry nor game players want to see uh

game the game industry occupying the kind of uh space in the mind of of legislators uh and and others uh that we are seeing right now so we want we think games are good we want that goodness for all people and um there is a real threat which social media is experiencing now of uh control of your platforms being taken away by government who thinks that you're having a deleterious impact on society um so i'll i'll end there uh and say uh thank you very much my contact information is is there and i look forward to uh continuing the conversation with many of you and making sure that online games are respectful and inclusive spaces for all people and that we continue to work together to fight hate in games and learn from the failures of social media thank you [Applause]