Biofeedback in Gameplay: How Valve Measures Physiology to Enhance Gaming Experience

[Music] thanks so much for coming I really appreciate it my name is Mike ammeter I'm an experimental psychologist at valve I get to do a ton of different things there but one thing I focus on is a hardware research or alternative Hardware research looking into kind of input devices that might be around in the future and so the purpose of the talk today is I'm gonna give you guys at least share our thoughts on how we think about feedback might be useful on going forward on this you know provide at least creating compelling new experiences home for players I'm gonna be coming in kind of hot on the stock I do apologize there's a lot of material I want to cover so I might be speaking a little bit quickly cuz I do want to get through it all but I promise to stay and answer any questions you guys have all right so let's get to it so we talked about both your back and game play and how valve was using physiology to hopefully enhance gaming experience goes to this presentation hopefully now give

you guys an overview of what quite a few back is I will walk through potential applications I'll show you some examples of things that we're doing at valve we'll discuss the the different physiological measurement techniques you can use in biofeedback testing walk through some of the pros and cons and then I guess interspersed throughout the talk well I guess you know try and put in a nugget so kind of where we think this technology can be taken in the future what we think is possible with it so let's start off by defining biofeedback this is my definition it's not as really grammatically correct but essentially I wanted to kind of I guess give you a clearer picture of what's possible what we mean by biofeedback and essentially it's the measurement display analysis modification manipulation and response to this virtual signals and what we're doing is recording biological signals from the body performing an analysis on them displaying them and presenting them back to the user and the hopes are kind of eliciting a response and the analysis part is actually a way of kind of quantifying player sentiment or emotion so it's a way of detecting signals from the body and saying you know this person's feeling frustrated or

happy or aroused or sad or angry and so forth one I guess one I guess thing to think about a time in turn when you're when you're doing this sort of research is that there is a feedback loop possible where signals you detect from people or from users from players have a feedback loop where you know if you're recording someone's heartbeat and you're displaying it to them and they see their heartbeat in real-time they see they're getting more aroused they might you know that might induce like you know a further increase in arousal so you get kind of a feedback loop where subsequent singles depend on prior States or your prior emotional states and one thing to keep in mind beyond that is that when we talk about emotion people are not stable emotions are not stable they're transient but volatile and there's something to manipulation so you know we can record a motion real-time but it doesn't necessarily mean that because someone's feeling aroused now they're gonna be aroused five seconds from now or a minute from now or an hour from now and so forth and so you know it gets a little tricky when you're kind of doing analysis at least you know post hoc unprofessional signals

um I guess it's important to keep in mind that you know you're looking at beat to beat measurements of emotion or arousal of his logical signals and it's not necessarily something that could be stable there could be a long term just a visualization of kind of you know the biofeedback leave where you start with a F is no more laser pointer you start if you start at the top of the ball believe you can see we start with a physical response it provides an input to a system on the system perform analysis on that creates a response outputs that back to the player which could induce a further physiological response so this is the feedback loop I was talking about in terms of biofeedback calm mechanics so why are we interested in biofeedback what's so great about it well if you think about current control schemes so you know you're mousing your keyboard your gamepad and so forth they provide one dimension for input but one dimension what I mean is that they're mapping player intent to on-screen action so you know the player wants to fire a gun he presses the B button the player wants to accelerate the car they hold down the right trigger things like

that you're mapping player intent to an on screen action what you don't get what current control scheme is at least at the moment you know are other aspects of cognition are cognition you know maybe long-term planning you might not know what the player is planning to do you know five minutes from now what the long-term strategy is and you're also you know most likely ignorant player sentiment how player feeling so their other aspects of game player at least the players experience that are not visible through current control schemes and so we'd like to attack that problem we'd like to see if you know investigating you know these other axes of input whether there's other aspects of cognition or player sentiment and see if we can create better gameplay experiences because of it so here's just a silly little metaphor where you know you mapping player intent to traditional control schemes and you get kind of a fuzzy picture of the players experience and you see either you know gamepad and mouse and a keyboard I don't have like Kinect and Wii mode in sixth sense and sunny move style devices up there I'm gonna ignore kind of the natural user interface devices in this talk I'm not because we don't think that potential

I'm not because they can't be used for about feedback type I guess measurements but right now they're commonly used for again wrapping player intent you know you're swinging a baseball bat you know you're rattling the Wiimote or whatever to you know you're matting player intent again gone screen action not they're not used as much for biofeedback and also because I wanted to make this more of a future facing talk I wanted to talk about less common technologies and not mass-market devices that are available now so it's kind of like well you know what's gonna be possible in the future so again I'm not ignoring them because we don't think they're valuable but just because we wanted to make this more future facing so again you know while our feedback I've been kind of touching on this but we think that adding player sentiment into the equation will create better gameplay experiences it's a new and it's been you know mostly ignored dimension of the players experience you know in any almost games and pretty much all games and so we think it's possible if you add this component into the gameplay experience you can create you know so you know kind of a stock sentence but it's true we think you can create more immersive dynamic incog great game experiences you can adapt the

gameplay or the game to how the player is feeling in a given point in time and hopefully create a better experience because of it um so we go back to my cell a little metaphor you can see I like using you know Windows clipart or PowerPoint clipart but we're wrapping player sentiment you know to traditional controllers you see a fuzzy picture the players experience you add in you know there's social signals and if it gets better right at least we hope it does you know if you know if it doesn't that's fine and hope we will learn something along the way but we think that we can create a better gameplay experience if we add in kind of know of arousal or emotion or whatnot so let's get to late let's actually walk through like you know how we kind of do this sort of research so we need to define emotion right if you want to measure player motion we need to define it again this is my definition you know it can be fuzzy that's fine but I view it as a subjective internal state induced by response to external events you know at least for the purposes of kind of this research and if you want to think about ways in which you could analyze emotion you can think of it as a vector so the vector has a magnitude and direction right in this case the

magnitude of the emotion vector is arousal so how strong is it and the direction of the emotion vector is the valence so the positivity or negativity is a positive emotion or negative emotion and just with those two components you can create you know you can get kind of a nice feel for how you can map different emotions on to this axis so you look at the the y axis you can see arousal coming from negative to positive x-axis is valence going from negative to positive and so you can see well if someone's you know kind of in a medium level of arousal and really really it's really really positive well they're happy right if they're kind of not really aroused at all are and you know it's somewhat negative it might be fatigued you know if they're really negatively aroused or sorry really highly aroused and it's a really negative emotion it might be angry so if you can if you can measure valence so positivity or negativity and you can measure emotion you can do quite an earth sorry you can measure arousal you can do quite a lot in terms of actually you know ascertaining what the player is feeling what emotions are feeling whether it's you know jubilance or enter you know being happy content relaxed

engaged bored fatigued and so forth so the physiological signals all walk through and again there are lots more that I'm not going to talk about today but here are five of the five common ones on heart rate SEL which is your skin conductance level essentially you know how much energy your ph content of your sweat changes and you can conduct more current through your skin at any point in time it's correlated with arousal it's the same thing as GSR galvanic skin response so those two terms are interchangeable facial expressions eye movements EEG s then I'll yeah I'll mention that you know there are like you could look at pupil dilation body temperature posture and so forth I'm only gonna really talk about the first five so let's start with heart rate it's a beat to beat interval of blood flow basically you know what you you you get a measure index of arousal here you can measure baseline rates and changes over time and there's some research that indicates you can actually deconvolve the components of the harpy waveform and actually get you know more complex emotions like maybe engagement or boredom and whatnot so like I have the

the diagram on the right is the the P Bo the QRS complex it's kind of the the main spike of the waveform and there are various ratios involved there that people think if you look at kind of the patterns of you know the patterns of the QRS complex in various folks that you can actually get more complex emotions beyond just arousal but it's a little bit hazy but I wanted to mention it because it is a potential possum what is the possibility for it for each metric I'll have kind of a pro's and con's slide just so you can get kind of like a quick summary of you know the what's good and what's bad about these things heart rate is you know it's get a great index of arousal very cheap to do very common people are familiar with it very easy to measure and you can you possibly you know do some you know advanced math to get valence with it problems with it is that's prone to movement artifacts X I cut a slide of a mouse we made that had a heart beat we actually had a harpy mouse apartment you could had a blood volume sensor and you could detect heartbeat just by like having your hand on the mouse but we get movement artifacts on because of that and so you kind of need to find a stable place on the body to measure heart rate otherwise

you're gonna add noise into the equation you also get a delayed onset to stimuli so you don't actually know what events are eliciting on the increases in arousal necessarily because heart rate increases that you know it increases gradually I mean you can get that pulse-pounding effect sometimes but I'm still you know heart beat especially compared to other metrics has kind of a longer lead time and then again it's you know I said you could possibly do the math to get valence right now it's really difficult to determine you know if it's a positive or negative emotion they did you just know but someone's aroused SCL again skin conductance level this is measuring the electrical resistance of the skin basically could be your sweating more bits actually the pH content of your sweat changes and you can conduct more current through your skin what you get is simply the more the more I guess current you can conduct is very highly correlated with arousal so you get a waveform of arousal over time which is actually pretty useful and probably the coolest aspect of SCL is that you get spikes that you know you have a lag time about 1 to 3 seconds maybe - five seconds depending on who you talk to in response to eliciting events so if

someone you know someone has having an sto measure I come up behind them and like startle them you'll see a spike like in arousal like when a few seconds later in the waveform which is pretty cool so you get kind of your almost instantaneous responses pros and cons of SEL so it's you know great index of arousal tonic and phasic responses so tonic just refers to the overall level of arousal over time it's like heart beated gradually you know you gradually you can tell generally where Peter folks are being are highly aroused or not when you get the physic responses which are the spikes so these just kind of transient little little little blips and arousal that you know we could actually make you know do cool things within gameplay so again minimal lack of stimuli I'm very cheap very robust the movement you know you can actually have a mouse that you know you can the sensors are located on the palm and it it works pretty well people can move around it's not a problem and you can measure in a lot of different places yeah we're focusing on easy-to-use devices that are kind of tied to the hands but you know you could you know measure up the mastoid process you actually do heart beat here actually if you had a headset and and how to clip

here but yeah it's pretty robust and pretty intuitive I guess or at least pretty easy to use one problem although again you you can get the spikes on because it is a waveform or because it can be noisy it can be difficult sometimes to associated listening events especially if you have multiple events in combination if you're playing an action game they can be difficult to determine which event I'm actually caused of spike if a lot is going on again difficult to determine valence you know requires interpretation which is not something that we want to do necessarily but it's something that we kind of have to because we don't really get valence when I get arousal and one other interesting I guess concern with SCL is the range is variable across subjects so you can get people who go from you know essentially it's one micro sim into like two you know they could be really highly aroused and that someone else could go from like two microsiemens at baseline to like 15 and so is that the same thing like you know key is that 2 to 15 range equivalent to a 1 to 2 range and how do you map you know the subjective feeling of arousal that someone is feeling I'm to those ranges so it's something to think about is that the range is a lot variable or is you know in heartbeat viewing people

typical resting heart beats you know fallen you know the 50 to 80 range and you know you can kind of compute maximum heartbeat based upon you know age and gender and so forth but with that CL it little bit trickier so facial expressions use just recording movement of the facial expressions or sorry facial muscles it really good at classifying emotion you can get both valence and arousal pretty well by recording someone's face you can do this remotely with cameras and have you know people code after the fact there are companies that are starting to try and automate this process which would be amazing I don't you know I don't wanna evaluate them here in this talk but it's simply something that people are thinking about you can also do it intrusive lis via EMG which is electromyography essentially it's just putting sensors on various facial muscles then you can measure how much they contract and you can actually you know you get you know really good measurements of what someone is feeling I'm gonna given point in time so again you know the work on facial expressions has been kind of around for a really

long time it's well established and so you know here this is an example of you know six I guess canonical humor emotions anger surprise disgust sadness happiness and fear and so you can detect all of these I'm pretty readily and you can check the magnitude of these pretty readily by recording facial expressions so the pros and cons you know it's great index surveillance getting experience and you get the full picture of emotion and you get this instantaneously you know people you know your faces will respond involuntarily I'm really bad at it I can't hide my emotions even as much as I try and so I will respond really quickly and it's you know it's it can be frustrating but again it you know at least in terms of like its potential applicability to you know as an input to gameplay you know having this instantaneous response is pretty cool people will respond almost automatically to various things the downsides you know definitely intrusive you know if you know people like having a camera on them or if you know you actually have a sensors on you it's you know crazy intrusive it's expensive at the moment I'm through its both in terms of time

and money again people are working on kind of you know bringing this down to you know automation through webcam but we're we're kind of not there yet and processing near across a lot you know if you have a human coder who's watching people and categorizing that can take a while and if that you notice human coder they can be subject to bias and then they also require training but if you're applying on some kind of are sure depending on some kind of automated system it's a box right like we'd like to kind of validate these techniques ourselves that's the heartbeat is a lot easier to do that with kind of you know software butts evaluating folks out of our hands we don't like that actually so that's kind of a a personal preference but we want to kind of validate these things in-house and so you do if they kind of depend on external hardware them to make determinations and so it's up to you is kind of how valid that ends up being but I mean just in terms of the potential applicability facial expressions or you know have a lot of promise fourth one is so I've Moomins basically you know we're just tracking where people are looking in real time so you know you can use remote or you know head mounted cameras to measure the reflectivity off of

people as you shine an infrared light up the people bounces off and you can kind of get an idea where someone is gazing so you get you know real-time record of where people are looking when we record both your eye movements and fixations eye movements are called Sur cards fixations you know when the eyes stop moving or its eyes always really moving but when that kind of centers on something further Evadne that's kind of I can use it's an index of processing your so that's see and I guess it's just a fun fact so when your eye is actually moving when your Shikata visual input to the brain is suppressed and so since we're making eye movements for about two and a half hours per day you're effectively blind about two and half hours per day and we don't realize it I'm just kind of something to keep in mind so if you think about displays where you update the display when the eyes are moving you could do whatever you wanted and people wouldn't detect it until they stopped moving and so it's just there's a lot of potential use there at least in terms of you know introducing surprising elements into the display when eyes are moving we're always away from that but again trying to think about what can be possible in the future so here's an actual I'll just show you guys a quick

video of you know eye tracking this is portal 2 so if you don't want any spoilers or see any levels in portal 2 you can look away I'll let you know when the video is over but yeah just a quick video kind of like you know someone playing portal to you and kind of visually consuming a map so the you know the the red circles are fixations the size of the circle is proportional to the length of time spent fixating so when the eyes are stopped moving and then the the lines canikin the circles are the the secod it's the eye movements and so we actually um again the focus of this talk is not really on playtesting but we do use these a lot in playtesting in terms of you know determining how people are consuming our levels you know you can actually do I had to cut these slides unfortunately you could actually do quantification of eye movements and determine kind of what cognitive mode people are in like there's there's different patterns by movements that correspond to you know maybe scanning the display just you know visually consuming it versus searching for a particular item versus problem-solving you can actually quantify these based on

like the trajectory in of the eye movements and the length I guess the length and number of fixations and so we you know we'd like to think about that in the future in terms of like when we're play testing our games is that okay we're like you know we can tell what people are doing in a given point in time so you get rudimentary in citing their cognition just from where people's attention is landing or at least where they're looking which we think is pretty powerful so pros and cons of eye movements it's an index of attention right so it's not you know as tied to emotion it is some of the other I guess some of the other physical signals but again you know it's still pretty useful um and as I mentioned it could be a rudimentary index of thought and if you couple this with people dilation which is you know a great correlate of arousal you can get arousal as well and so I'll pretty much any actual actor you buy will be able to measure pupil dilation you know it's a unique metric and very very reliable eye tracking technique is kind of returned now it's still very expensive but the technology is mature so you get pretty good and pretty robust measurements of visual attention the cons it's incredibly expensive the most expensive probably the most expensive of all the

signals at the moment you know talking I'm talking in the tens of thousands of dollars for an eye tracker there are companies that are trying to bring that down by an order of magnitude you know in the five grand range and then they're actually I know there's an open-source project where people are trying to do this with webcams I don't think it's there yet but again it's nice that people are trying to kind of bring this technology down to you know consumer price points so just something to think about you know other cons and you know different requires a lot of analysis it looks like analysis I mean you can automate stuff and you can write scripts but if you want to tag various features in the display can require a lot of analysis then to kind of do kind of at least the the quantification of scan pattern that I was talking about that requires a lot of work as well so you can get great data out of it you have to work for it and then just one you know facto it is that you know it can be intrusive as well like you know they've done studies where you tell people that you're tracking their eyes you tell end or you don't and the people who know their eyes are being tracked you know have different admins like they're they're more deliberate I'm in the right movement so you so again as with any kind of measurement technique

you know we're introducing bias into the equation unfortunately but you know the data for the most part is pretty clean but I didn't want to like what you guys think that it was completely biased free right so last one to talk about it is EEG s so this is measuring electrical potentials of the brain EGS are primarily time-based signals so finding out when an event occurs but you know you can put sensors all over the head and you get course matches the location which is pretty cool and I guess what the primary use for at least in terms of like their immediate applicability is you can look at the frequency spectra I think so the frequency kind of bands of electrical potential on the brain and the latency the spikes but if you deconvolve the frequency spectra you can get measurements of emotion like there are alpha waves that are kind of low frequency that are correlated with relaxation and there's beta waves are correlated with alertness and there are delta waves that are correlated with kind of like focused attention so you could get these measurements in the begin or you know you could measure engagement in your games for example their tech when a player was engaged or

tech when a player was relaxed and so forth just the upper left picture is you know potential scalp locations like eg studies and in labs you know to acquire hundreds of electrodes on there are consumer grade headsets that are a few hundred bucks that you know do a pretty good job of measuring emotion that you use far those sensors is a couple that only have two sensors on the forehead but the right graph is just cuz I like you know cool-looking graphs and it's just kind of showing you how you know different regions of the brain can light up at various points in time for different stimuli so he geez you know positive you know you can get a lot of them you can get valence again not a complete picture but you can do a pretty good job and you can get rudimentary insight into thought I do not want you thinking that you can get complex thought through with through consumer-grade EEG s at the moment you know we can do some stuff but the brain is very very noisy and so to actually you know extract signal from noise usually requires you know hundreds of trials with minimal differences between conditions and so forth and so you can get localization you can tell when people might thinking about a leftward

action where's a rightward action or maybe you're pushing or pulling action but you know getting like you know we're not we're not close actually so you know playing Halo with your head yet maybe in the future but we're not there yet kong's you know it's expensive you know intrusive you know first of the data again it's not clean the can very enjoy seem very hard to kind of extract signals sometimes and then difficult to validate right again you're relying on someone else's product to kind of validate stuff for you and again we you know we're trying to kind of come up with our own our own solutions to this and so it's a you know I'll point them I guess I want to heart what I bring up I guess just because it's something to think about is that you know like there are disagreements about how to kind of quantify you know waveforms in the brain or EGS in the brain and so it's important to realize that the data you know the data are getting is dependent upon one person's interpretation of what's going on or you know one company's interpretation of what's going on other techniques so you know we have pupil dilation body tan body posture yeah I'm not looking to talk about them

I will mention that people have actually done studies and so if you couple um body posture and pupil dilation so you can do people dilation remotely with an eye tracker you can do body posture remotely with a deficit camera like connector or whatnot you can account for about 80 percent of self-reported frustration just to those two metrics to you know non-intrusive metrics and so not not too shabby in that regard there's also lots of stuff we haven't thought about some stuff we have that I guess I'm not talking about but you know this is you know a partial painting of the picture there are you know many more ways you could be extracting information from the body or from users and so you know these are some of the common ones some of the most useful ones but you know it shouldn't be you know shouldn't feel like I'm constraining kind of like the notion about feedback to the metrics I've mentioned all right so let's get to the good stuff so you know how can we use this data you know we could start with the most basic right like passive viewing of a cubic data you know I show you your arousal in game and I'll show you a video that later on which actually is pretty cool you can think about modifying game experience based upon you

know player's internal state what a player is feeling at any point in time you know I'll walk you through an experiment we did where we modified the the director and left for dead which it's an algorithm that kind of controls the gameplay experience and left 4 dead where we added biofeedback to that I'm and trying to create a better experience you can think about adapting difficulty in real time so you know if you could detect that a player was frustrated right or that a player was feeling challenged or player was not feeling challenged right you could modify the game in real-time to adapt to that you know as someone to think about you could think about you know detecting when a player was bored and you know figuring out a way to kind of reinvent these sorts of you know gameplay techniques are which are not possible with current control schemes you know could be possible you know I'm not gonna say that we could definitely do them but they could be possible with biofeedback devices we're really curious about optimal arousal patterns we wonder if they exist like you know is there a pattern of arousal air pattern of emotion in game you know walk someone's playing a game that will lead to the most enjoyable experience you know and again it probably is not the same for all folks but just you know thinking

about you know what would contribute to an optimal pattern of arousal and so I can do with these techniques you could start trying to like attack that question and say yeah you know it's it's um you know a step three you know silly example so what the pattern you know it's peaks and valleys or maybe it's steadily increasing or has a giant peak in a valley and then you know tiny Peaks or maybe it's steadily increasing peaks or valleys um who knows but just trying to figure out and see if it's possible to figure out you know if there is kind of a pattern you know at least in you know various levels or you know made across specific genres of you know that would lead to great you know to the most enjoyable experience we think it's an interesting question and worth attacking more applications I'm so you can think about actually inputting passagio that you're using physiological data as direct inputs to gameplay like you know just a couple you know quick examples like you know tying health to arousal like if you stay calm your health decreases you know gradually if you get how they arouse your health ticks down quicker you can think about in-game prompts that are tied to emotional state you know if someone's engaged you don't want to distract them you minimize pop ups for example if someone's kind of bored or you can tell

that they're wandering around and lost then you could you know kind of detect that in theory and add in prompts to kind of help them out if you think about NPCs responding dynamically it's like hey Mike why are you so sad I'm like you know they've detected that I was frowning right I mean you can think about experiences like that where you know characters or players in-game or responding to what you're experiencing an ending point in time you know could create a really experience you can think about you know requiring you know subjects to remain here people players remain calm right like you know having you know are getting really aroused to proceed in game you can imagine a lie-detection game where at a full year interrogator you had to maintain a calm or a low level of arousal and you know if you if you got aroused so you got nervous you know you lose right or they would detect your lie so things like that again beyond that you just lots of stuff they're thinking about so you think about actually doing matchmaking based on physiological profiling so you know maybe you want to pair like a passive player with a really emotional player right maybe that is a worse pairing then you know a really bad player with a

really good player maybe that leads to worse experience and so maybe you know if we could pair passive players together or emotional players together or easily frustrated players together we can do a better job with matchmaking you know we don't I'm not saying it's possible and just well I'm like it is possible I'm not saying we can prove it but I'm saying it's something we're thinking about we think it might be worthwhile along with that kind of the passive viewing of you know your own behavior in game do you think about you know competitive play you know people browsing through a list of matches and trying to figure out what match to watch you know and you know watching the arousal patterns of you know really skilled players and it could be interesting it could be a way to kind of filter you know how you expect at competitive matches or if you're just watching a competitive match in general and just knowing you know if the you know the competitors we're feeling aroused or scared or whatnot any given point in time could be pretty interesting and then you know beyond single-player stuff you know we could think about ways to use this in multiplayer like you know you can see a spike in your teammate you may detect when they're in trouble or you know maybe if you want to grief the other team and you can create a spike in arousal you can earn points for listening responses you know I guess in your opponents and then final

application again I'm not really gonna talk about play testing I think I have one slide on it it could be a whole nother talk and this is more about kind of via the gameplay side of things but you know we can definitely there's so much to be gained you know in terms of quantifying how players feel and like what you know and play this thing we want to know what they're doing why they're doing it and how they're doing or sorry and how they feel about it what they're doing is easy right the other two are subject to noise and so you know if you have kind of you know biofeedback stuff in there we can start making better attempts to quantifying emotion and getting a better understanding of how people are feeling as they're playing our games all right so I'll walk you through three actually main experiments we did are and they're they're all still ongoing but you know I'll kind of walk you through what they are and what you know why we think they're interesting in terms of using biofeedback so the first is it's modifying the director and left-footed to second is actually using physiological signals as direct inputs to gameplay in Alien Swarm and then I'll talk about playing portal 2 with your eyes it's actually using your eyes as

controllers and then I have a couple slides on kind of passive viewing and some multiplayer tests we ran where you could view you know physical signals of your opponents and your teammates and then I have one slide on play testing so left4dead so in left 4 dead and leopard it to you the player gets a different experience each time they play and the way this works is we have a you know a kind of a system called the director that creates a dynamic experience you know I guess throughout I guess throughout gameplay and what it does is it modifies enemy spawns where health and weapon are place which boss monsters occur and so forth and it does this based upon an estimated arousal level so the director has an estimated arousal level in mind and it says yeah they're highly aroused I'm gonna let them you know kind of drop down for a little bit and not bug them or they're not aroused based on my estimation of it I'm gonna give them you know a smoker to attack or a hunter attack or I'll have a mob spawn and go after them and so what we wanted to know

was if you replace this estimated arousal level with actual physiological arousal from the player can you create a more enjoyable experience and then beyond that going back to what I was talking about earlier is can we determine optimal arousal patterns in this regard so really quickly just going over how the director works you have survivor intensity as a single value it's increased in response to in-game trauma so you know you get attacked you lose health you kill an enemy that's close to you for example intensity increases intensity decays to zero over time I think it's I don't thirty Seconds to a minute depending on Vegas various factors and the goal the director was to create kind of peaks and valleys in this estimated arousal level so you can look at the graph here the middle line is the most where it says survive hi I was actually slowing though did you use experienced gameplay players for your tests yes so so it actually berries um so we did for the leopard a to study we have these are

all elm not valve employees but you know external play tennis with people who had played leopard at two before who had come into the lab so yeah but the question you know what you can do with novel game takes place and are those patterns of rails are going to be different yeah absolutely yeah yeah hi I actually work on a very similar problems with eye tracking with our play testing we've actually found that that when people have to focus on the screen for a long period of time they start to get headaches and we get problems with eye strain after a long like more than fifteen minutes of gameplay okay so I'm kind of curious if you tested beyond 15 minutes or yeah what kind of problems you may have run into so the question was I'm okay it's one out we can use eye trackers belong in the fifteen minutes because there is potential for high strain um yeah so we've tested people for an hour and a half you know people can the eye tracker we use is pretty robust so people can look away from the screen you know they can they can stand up and sit down

um if they wants we've had you know hour and a half long two-hour play testing sessions and haven't had really any complaints of eye strain you know we kind of you know we have I guess the you know that the bounding box in terms of what the eye tracker can detect let's you kind of move your head around you're gonna stay in the same position so we've been okay in that regard can you can you force arousal and the human player by giving them signals that they're in-game character is getting highly aroused like if I don't know is the characters left where dad's start breathing heavily or their vision gets blurry will that force arousal and the player yeah so the question was can you kind of induce arousal or various responses in the player by having in-game avatars mimic those same responses and so absolutely um you know we have their neurons in the brain called mirror neurons that kind of fire when you know you see an action being performed and so you actually it actually like is kind of analogous to you performing the action yourself if I see someone swinging a baseball bat the same neurons that would would fire when I was a swinging baseball bat fire when I actually see that process occurring and so you get

those kinds of triggers where if you see you know any gamer you know your in-game avatar you depending heavily for example absolutely when you see a change in respiration rate I mean it may not be you know may not be a large change but absolutely well players kind of be affected by what they see on screen Hey so the Alien Swarm testers how did that work out for them but which one side the Alien Swarm testers yeah they actually succeed yes so the question was to the Alien Swarm playtesters to actually succeed and yeah so I showed you a small snippet just cuz I was pressed for time but yeah the Alien Swarm yeah so I think about 60% of people made it through and again this is our algorithm right so we're tweaking it so we can make it harder or easier um but yeah about 60% of people made it through so far so did they have access did they see the time left and did they see the color change as well yeah so and so we've done it both ways we had they always see the time left they always see the color shift from green to yellow to red or back or whatever we've done studies where we show them the arousal and not when we showed them the arousal they got more frustrated because they

didn't understand why the timer was speeding up sometimes when they didn't see the arousal they were more likely to kind of be cool with it okay cool thanks I was wondering if you were able to track flow and mastery from players I mean if you master player it would be calm during the gameplay or any of you able to design for a flow you know experience so the question was and did we do we track flow and kind of you know mastery you know master players or players were really good or experts at the game you know did we look at you know kind of differences between them and I'm assuming kind of you know people who are who are not masters or novices or whatnot we've done some kind of comparisons of novices and experts we haven't checked flow you know that that can be a little tricky to operationalize sometimes but no we haven't done anything systematic in that regard we are really curious in nah bird expert differences and how that correlates to patterns of arousal patterns of histological signal but is stuff like I haven't I guess I didn't plan to talk about it can't really talk about at the present time

okay well yeah thank you guys very much Oh so I guess one more question I teach at Arizona State University I'm game design but I'm doing the collaboration with bio design and I was wondering if you would be interested in working perhaps with people who are doing like physical probes into the body and perhaps expand your research that way and work with academia are you all commercially oriented and you want to stay in that world oh so it was the question um you know what the Weisse is are we using and are we are we like averse to using devices that are actually attached to the body is that the question sorry yeah I can see how our research might compliment each other yeah but I come from academia and there's no like commercial application that we're going for other than well the bio design guys are going for designing for aesthetics right so I was wondering there's a intersection there would you

be interested in perhaps working with us oh sure I mean well in the state oh I mean yeah I mean well of course yeah we're definitely interested in kind of seeing what other people are doing I mean the whole point of this talk was to say hey we think this is really cool and we definitely want to encourage more people to be thinking about you know both kind of you know how to improve the techniques we have and then how they can be used in game play so both the commercial and kind of you know the basic and Applied you know concerns of this research so definitely we can talk and see if we can you know I think see if there's potential down the line for you know a collaboration of whatnot so I send you an email sure that's fine all right thanks guys appreciate [Applause]