How an Outbreak in WoW is Helping Epidemiologists Create Better Disease Models

[Music] welcome back this is our second session of the serious game summit and our games for health day our next presenter I first met about four years ago and had read about this wonderful piece of work where there had been a virus outbreak inside World of Warcraft due to a design flaw in some new content that the game was putting out and there had been an article published by an epidemiologist who had gone back to look at what it actually happened and tried to understand it from their their standpoint and you know I thought that would be a really interesting talk and I gave her health conference and that especially because it was dealing with World of Warcraft it's always good to for content-wise to associate associate yourself with the most popular game being played at the moment you can

almost never fail there but we got a lot more than we bargained for because not only did we get a great talk but we met too many of doing some really incredible work and it was really it was really apparent that the talk was connecting with people when like all of the senior scientists and people from all kinds of different groups and the federal government who deal with trying to protect us from epidemics and other kinds of similar episodes ran to the front of the room and we're like I'm getting in line first so I had promised Nina that we should have this talk at GDC because I think people should see what can come out of games even if they're not purposely built for a serious purpose and things can actually happen and the behaviors of players inside the games that are being made every day have potentially interesting opportunities for the rest of the world to learn from so you're going to get a chance to see some of that work and some

other work that's been built off of this so please welcome Nina pfefferman hey good thanks Ben every time then invites me somewhere it's it's somewhere I've never been and somewhere I have a great time so you guys look awesome thank you just so I know the lay of the land this is this is wonderful for me usually I get to ask the following question and there's maybe one or two hands that go up how many of us are gamers okay some question how many people are here from Blizzard all right we could say anything we want now no um okay so but how many people play Wow okay that's still that's still orders of magnitude above my normal response on this one I'm very psyched normally I give talks like this to epidemiologists and people from public health and the federal government and the Department of Homeland Security none of whom admit to

playing Wow even if they do so um how many how many this is gonna narrow it down a lot how many people were playing Wow in 2005 oh okay what are you guys doing here you lived this okay so I'm gonna talk to you today predominantly about the corrupted blood plague of 2005 in World of Warcraft and how we're using it and as Ben started to tell you I'm going to take serious games in a whole different way usually when when the the folks who start talking about serious games mean research into serious games what they mean is how do we make the user experience of the game such that there's a benefit to the user in and in the context of games for health in to their to their personal well-being and now I'm gonna step back from that and say actually I want the beneficiaries of the research that I do to be everybody not just the users of the game and I'm gonna use the game in order to gain insight into how to how I think Public Health

practice should happen and I'm gonna do it from the perspective of Epidemiology which is the study of disease epidemics and from my perspective usually it's infectious disease but at the end of the talk also to tell you about some stuff I'm doing with non infectious disease because of something surprising that came out of some ongoing work in a virtual world so I'm affiliated with way too many places but I'm professor at a medical school I'm a professor in a department of evolution and ecology I'm a professor of applied mathematics and all of them let me have students with me who play games which is fantastic okay so I want to make another disclaimer which is that I have shamelessly stolen lots of images from blizzards website and they are all beautiful and I own none of them and I hope they don't mind but yes all of the pictures of Wow come just from screen capture or from their website okay so in case you guys don't know how many of you know what an infectious stuff that demonologist does that's pretty good

actually you guys Rock in general first of all actually in the first section I don't know how many of you are also in the first session it sounded like the other room was having a lot more fun even though the talk in here was awesome and clearly better I don't know what was going on the other room but so I would encourage kind of yelling or screaming or throwing things if it stops being fun because we clearly need to have more fun than the people in the next room yes thank you boo okay so what does an infectious disease epidemiologist do well we analyze disease risks to a population we work on figuring out what the origin of a particular outbreak was we worry about public health strategies that can prevent outbreaks and make populations more robust to threats from the outside and from within and we investigate intervention strategies to end already ongoing outbreaks and we decide things like when do we make public health emergency announcements and what information do we give to the

media and how do we communicate with the public about health risks and each of these things have different levels of feeling of emergency and behaviors that go with them especially when you think in your mind just within the realm of infectious disease of the differences between communicating with people about the HIV epidemic and swine flu right those are very different timescales of affect their depth very different levels of concern and you expect very different behavioral responses depending on whether or not we as the epidemiologists grew up so we try very hard not to screw up but we also about well if we screw up how bad can it get okay so I'm a mathematical infectious disease epidemiologist I live in a smaller world than just all the things I told you about what infectious disease epidemiologists do I use math to try and answer all of those questions and so I try and predict if outbreaks happen where will they go what will they look like what will the relative outcomes be how bad will it get and there are a few different types of

models so I'm gonna I'm gonna quickly try and describe to you all of epidemiological mathematical modeling and about one and a half minutes we have basic models they are called the compartmental models where we've been people into health compartments you you are healthy but you could get infected you you are infected and you can infect others you you have recovered from being infected and are probably immune or you're dead or things like that and we think of each of those compartments and we thinks of rates of transfer we think of rates of transferring among those compartments and then we try and come up with measurements for those rates and then very much like physicists we say if there were you know of 13 billion people then we'd get this completely correct and as it scales down from 30 billion we're getting it slightly less and less correct but it should still apply and then unfortunately there's small-town America and then it just doesn't work and we all know it doesn't work in small town and then we've got network models I think in network models we draw literally a network of everyone who

meets everybody else and has the possibility for transmitting and infection and then we try and say well if we have those explicit contacts what do we know about the disease and then we get to make much more exact and a much more exact measurement about all of these disease transmission rates but we need to know a huge amount about the population and and now in small-town America this works great because we actually know a lot about contacts in a very limited range but then as soon as you scale up to reasonably sized town this breaks down entirely because you don't know how often someone goes to the grocery store and then we get to agent-based models and those are the simulations kind of like the AI controls for the NPC's in a bunch of games so these are the ones where we put some algorithms in back of Euler object and then we say here are the algorithms for decision-making for that object and they look around and they look at the other people who are making decisions around them and their environment and what happened to them in last five minutes and they go well given all of that I'm just going home and bolting the door and then based on that

we see who comes into contact with whom and then on all of that we we model the spread of disease based on the contact patterns and in the compartmental models we're assuming that random mixing happens all of the people from one compartment have an equal probability of meeting all the people in the other compartment and if we want to break it down a little bit we can we can make sub compartments but we really do treat everybody kind of as an average and in the network models everything is very explicit and then the agent base models the behavior isn't explicit but the algorithms for making those behavioral decisions are explicit every with me how many people are bored out of their wits already yeah all right I promise I'm getting off the EPI now okay so as a disclaimer you don't need to know this it could get all I just made everything seem very lovely and and easy it could all get very complicated this is just what this is actually like three-quarters of a bottle to describe transmission with behavior for one disease so the only crazy people like me

and my students do this at home for fun on the other hand here's the deal what it brings us to is that in all of the models that we're using and these really are the models that we use when when we when we as a group of epidemiologists decide what flu strains to put in the vaccine for next year when someone says what happens if God forbid there's a smallpox release in downtown San Francisco on Tuesday what will happen and what can we do to prevent it these are the models that we go to all of us and so this brings us to the basic fundamental problem with all these is that we're assuming the behavior of the people that were modeling and there's a lot of behavioral change that get that we have to assume is going to happen in the case especially of an epidemic outbreak when people are likely to be scared people are actually very difficult to predict we can use math to predict the disease really well once we know how people are going to behave these models act for as much as we can generalize them

and laugh about how simple they sound they actually do work really well we've got some great testing that says if we nailed the behavior part of it we've nailed the disease transmission part of it but then there's that behavior part and that part's really hard and right now we basically guess we basically go I don't know what would I do if tomorrow there was smallpox down the street I'd probably be scared what would I what would I think to do the problem is that people are actually very bad at making these kinds of hypothetical assumptions about themselves partially because they're not in the moment and there's a lot of simply emotion feeding into the decision-making and partially also because that that sort of scales up from the individual to not being able to anticipate what the actual environment around them is going to be like at the time and so there are some really deep ethical behavioural philosophical questions that come up you just can't ask people well what would you do if and get a good response and we can find measuring current behaviors but again

that's in a very different context and so when we try and figure out how do those current behaviors change if anything bad is happening we don't have a good way to know and the only thing that we do know is that the ways we've tried to scale that in the past haven't been that great so a lot of the times this comes into emergency evacuation modeling things like people getting out of the way of hurricanes the number of people who tell you they'll evacuate if someone tells them to evacuate is actually very different from the number of people who actually put their stuff in the car and leave so we know from from experience like figuring out even people are bad at determining their own heroism there's actually a gap in both directions you get false positives and false negatives of would you run into a burning building to save a child there are people who surprised themselves in both directions and it's not that they're not thinking really well about how to give you the right answer they just don't know the right answer because it hasn't happened yet okay so how do we begin to understand these things when you've got to make these really basic assumptions and so we just talked about this we can ask that what would you do

if questions but they get very complicated very quickly so just without answering out loud right now but think to yourselves at what point might you be willing to forego arriving at an emergency room if you knew that your chance of dying was 98% and your chance of infecting everyone in the emergency room was also around 90% so if you would you show up and kill people for a 2% chance of survival maybe but that's a really difficult question to start asking what if you didn't know with that level of certainty at what point are you willing to to stay and care for a sick child what happens if it's not your child so these have huge impacts on our outcomes of our behavioral epidemiological modeling and so now we get to the fun part now we get to see okay how does gaming help so in 2005 when some of the people in this room including me we're playing World of Warcraft there were six point five million players worldwide which is a number which I believe is now dwarfed by

the current game players but what makes it perfect from my perspective is that Wow had at the time a really unique demographic composition it wasn't just the stereotypical gamer it was working moms in the Midwest it was students it was professionals there was a large assault deployed soldier component in World of Warcraft there still is there were scholars working in it usually from the social sciences not so much from the epidemiology ideological sciences there were politicians there was I think a few years ago there was the first political world political rally in Wow anyone go pretty cool okay so a critical aspect another critical aspect of wow that made it a really good platform for this kind of research is that it's a highly social game by nature it's not just that there was an aggregate of a lot of people logged in to a massively multiplayer online game in parallel tasks it's that

you had to have social interaction in order to accomplish these tasks and that's not unique to Wow by a longshot but it was bringing all of these things together that helped so one of the things that was critical from our perspective is that there were years and years of social history behind the groups that were interacting if you had a party going out or if you had a guild that was a Gildan or a party that had been together for a long time before the break happened and they also have high levels of interaction among strangers people you're buying two things from people you're trading from people who are joining you on quests because you just can't get your power together and in 2005 Blizzard released something brilliant and they didn't mean to and no fault of their own this was not a design flaw you would have had to have been God to know that this was going to happen Blizzard occasionally opens up new releases to keep their high-level players interested and keep low-level players engaged without losing without creating a huge disparity between noobs and wizards essentially so they opened

up a new area in 2005 called soul girl and as part of the challenge for high-level players was a car a beautiful monster and as part of the challenge of playing her car he cast an area effect spell called corrupted blood and what it did was cast as an area effects felon if you were affected by it you wee part of it was to recast corrupted blood periodically and if you were a high enough level player to have fought your way to her car this was kind of an annoyance it was meant to add to the challenge of battle but it wasn't meant to by itself be a dangerous thing because of course this was a high level area to have gotten there in the first place you had to have been robust to most of these threats and that's a card and yeah he was gorgeous he's tall he's red it's beautiful he's there's a dramatic swishing noise when he turns around i i'm easily affected by these things i'm a gamer

okay so here's where things didn't go as planned when they opened up this area the higher level players didn't necessarily wait for the one of two options that Blizzard had in mind the two options being either you killed her car in which case no problem you healed from corrupted blood and everything was fine or her car killed you in which case when you resurrected no corrupted blood on the other hand there are some other options for this you can leave while still infected and run away or you can summon and then banish your and your pet could be infected and when banished your pet wasn't heal so when Yuri summoned your pet your pet came back with corrupted blood which is of course a cell-free casting area effect spell which is an infectious disease process so some of the people who went to fight her car left and went back to the cities carrying infection and this is where things didn't go so much as planned so here is where players started

to die and I don't know if you can see can you guys it's the resolution okay four okay awesome first of all there were skeletons this was actually a crucial aspect from my perspective of the gameplay because normally when you got player deaths in WoW no skeletons so wandering around with the ground littered with the bodies of literally player characters was actually kind of cool also corrupted blood was visually dramatic you exploded after a few seconds in a little cloud of blood and it went a big on the sounds the public gathering places were instantly shut down this is a guild hall or trading guild oh oh yeah that's Ironforge you can see not much normal anything is happening right now people are running around exploding in clouds of blood dropping dead and turning the skeletons and the plague spread completely out of control so really if you were logged in you were affected and it was amazing there was no normal

gameplay happening on the infected servers Blizzard tried to stop it by imposing a quarantine which is something we talk about in real world epidemiology all the time we talk about okay what happens if someone releases something bad and in Manhattan one of the easiest places in the world to quarantine because it you can only get to it by Bridge Tunnel or boat and usually we talk about that failing this was an example where within Blizzard the quarantine failed loser and finally had to do something we don't get to do in real life which is to reset the server's so this went on actually for four or five days before they did this and they were trying in real time to fix it and some of them were really nice people and took the time out from being really stressed to talk to me about it so I got to hear some of what they were trying to do and it was brilliant and none of it worked in the same ways that we would be if there was an outbreak of something horrible tomorrow a bunch of us would get phone calls from Homeland Security and we'd be scrambling on trying to figure out what to do and it would likely not work so actually even

the even the background perspective of the people in charge scrambling was actually a really good analogy for real-world events but did we as epidemiologists learn anything for this well how did the players behave during the plague was it close to how people behaved during the real world epidemic and the answer is actually somewhat surprisingly at the time is yes so we were I was looking at this I logged in because my student called me and said you know how we've been joking about running a disease in a game and using our output from that well we don't have to run the disease it's happening log your character in and take a look and do it for research purposes on the other hand we also had people doing that just for fun which was amazing so that's that's my little curiosity by I can't get my pointer but that's the curiosity part that we saw and that first was the critical thing we saw all of the things that we thought we would see we saw some courage we saw some compliance with imposed quarantine we saw fear we saw some suspicion of the quarantine and if blizzards motives and doing all of this

we saw some public concern the bulletin ports were going mad I don't know if you guys who are playing this we're reading through the bulletin boards there was everything from the equivalents of like aliens of landed conspiracy theories - oh it's a glitch though we said it tomorrow everyone calm down and just have fun and there was everything in between and there were people like oh my god if I if I you know what happens to my stuff if I can't go back and reclaim it there were looters there were people maliciously running in and infecting gills and some of the stuff that we so forgetting the grievers but the griefers are important forgetting the grievers the important thing to us was to see the diversity of response and the ones that we were looking at epidemiologically were the ones what are the things we hadn't thought to include in our models when we guess in our models when we say what should be happening what could happen and we guess what did we forget to include and that's where things like curiosity me helped us because we'd never thought of that what happens if people rush into a quarantine clearly now that it's been pointed out to us they're going to we have news organizations that are going

to want to send reporters we've got people who are just curious and don't worry about the consequences or I think it's not as severe we've got people who are actually there to monitor the situation from an oversight perspective we're gonna need to get in we'd never modeled that before so it's actually improving our models so what did we learn it pointed out these behaviors we hadn't considered which is very important but it's not everything we need it's a great place to start of course there are differences between Indian world choices and real world choices and one of the cool things about in-game world choices is that gamers are actually pretty good at dealing with probabilities because we we have certain inherently there's roll a d20 and know what happens engrained in our souls but we also know that only some people are gamers so the responses may be different we need to figure out what that is but so I like to make the analogy of this as being sort of clinical trials in mice for drug development we know mice aren't people but they're a damn fine place to start gamers are probably not exactly regular

people but it's a good place to start and then there's a question where are we going from here because that was a one moment in time brilliant thing that showed us something wonderful we needed the fact that it was an accident we actually the wrath of the lich king expansion later didn't really help because that was a planned event and everyone knew the effects were temporary and everyone knew that whatever they were experiencing was meant for this to happen corrupted blood by the fact that it wasn't meant to be there really gave us some great insight and I'm gonna tell you very quickly about some stuff that is actually because of Ben and games for health when I came and gave a talk on the corrupted blood plague I met the folks at whyville how many people have seen why Bell Oh I recommend whyville to you all especially for bringing out your inner child logging as an adult to play with the kids they're awesome it is a social and science virtual world for tweens eight to 14 year olds and what we're doing there's there exists with with why they'll developing this with new Metis in the company that makes it why pox which is an

you can get in rival and it's been studied really well from an educational side of what do kids learn about epidemics by a wonderful professor he has Minka fire at UPenn what we're doing now is studying risk perception in whyville and how do the different attributes of the disease and the likelihood of catching it on the prevalence observed and the symptoms that are experienced how does that all feed back to the risk and health decisions that they're making as people caring for their own health within that game this is what white box looks like you can see she's all speckly the rest of it I take no responsibility for with white box that's her fashion sense somehow to wear a poinsettia but you can see that a bunch of people are saying achoo at the beach and that was one of the symptoms of why pox at the time to interrupt what you were typing with the word I chew and one of the really cool things that we noticed and it's not infectious disease but it is affecting our research is that we noticed that some of the kids in whyville we're typing were typing achoo into their own

text it wasn't being auto inserted and we stopped for a minute we went why on earth are they doing this well they can't mimicked the facial splotches but lots of people with the facial splotches were covering up their faces anyway there were masks or pretty hats or things like that and then they would cover up their faces and say achoo so we thought a lot about faking they said okay well what what could this be well it could be that white box is actually perceived as cool or good right and that they were trying to do this well that didn't that wasn't in line with what we were reading in the chat or what they were saying to each other they were saying no Y pox is really annoying it's really bad it's interfering if nothing else it's interfering with typing commands so it's really annoying if you want to say something in the middle of the word say the word achoo comes up it didn't process we thought maybe they were just screwing with the studies because there are lots of kids running their own epidemiological studies of Y pox and why though maybe they're just screwing with the data but that didn't actually turn out to be what

we ended up thinking what we ended up thinking was that it's actually a form of empathy it's actually an idea of I'm feeling left out all of my friends are experiencing this negative thing I don't know how to correctly provide empathy to them and so this led us to an interesting hypothesis and find it up and we're winding it up our interesting hypothesis so this is a teaser for our coming research it's actually that in looking at kids and addiction behaviors we've been come we've been focusing as Public Health people on saying kids are not estimating their risks correctly they're trying to get high and they're not realizing they'll get addicted this led us to make the hypothesis no no what if kids are trying to get addicted because they realize that something bad their friends are going through and they don't know how to provide empathy appropriately to them and this may not be something that kids have to realize that they're doing in order to do it and which would drastically change and we're working on this study now drastically change how we would try and target anti addiction campaigns for youth so it's an

ongoing study hopefully I'll have results for you quite soon and that's pretty much the end of the talk so I don't think this'll ruin gameplay but if anyone would like to work with me and it has more than a hundred thousand players so yeah so part of what we're doing is now taking these insights and going back into our epidemiological models and gaming in the nation service haha so thank you very very much you