Making Your Games Better with Psychology and Playtesting, the Uncharted Way

[Music] round one fight [Music] [Music] come back here let me stab you [Music]

[Music] [Music] [Applause] a few about poets Thank You Lana Luthor but every chance that you have insurance

[Music] [Music] [Music] let me kill you

[Music] I want to thank Robbie Cooper who is a British photographer and video artist for the use of his short film immersion to kind of open my talk with this year you might know Robbie Cooper for his work he made a series of photos a few years ago of people with their avatars mainly from mmo's but also from various other kinds of games that were really beautiful and revealing so good afternoon everyone and welcome to my

talk please silence your phones and other augmented reality devices and when you receive your electronic evaluations in the email please please please fill them out if you like the talk if you don't like it don't bother and thanks to the awesome conference associates who like every year are helping us out being incredible making GDC happen all the time so my talk today is the first kind of art II want to be intellectual game design talk that I've ever given at GDC so I'm pretty nervous in case you think it's a load of old bollocks but please bear with me it's also rather long and I might go over the hour but GDC have graciously given me an extra 10 minutes at the end so hopefully we'll have some time for questions my name is Richard Lamarr shinned and I'm lucky enough to be a lead game designer at a fantastic

studio called Naughty Dog I've been a professional game designer for a little over 20 years now working mainly in character action games and I've worked at Naughty Dog for nearly eight years I was either the lead or the co-lead game designer on all three games in the Uncharted series culminating of course in uncharted 3 drake's Deception which we released last November we've had oh gosh thanks we've had we've had terrific success with uncharted 3 and even though I'm not going to put too much focus on the Uncharted games today I'll tell you about some of the play testing techniques later on that we used to improve and polish this our latest game but as well as leaving you with just a little bit of practical takeaway today I have two other goals for my talk both of them to do with game design philosophy and both of which I hope will be useful to you in your game design practice one of them relates to my thoughts about the value and nature of what gets called experiential gameplay now we've used

this kind of gameplay to powerful effect in the Uncharted series for example in the peaceful village sequence in uncharted 2 but it's also being used in many different ways by many different kinds of games from the world of triple-a to indie and art games so we'll get back to this towards the end of my talk but my first and biggest goal is kind of given away by the title of my talk in the grand tradition of Jesse shells game design lenses where Jesse gave us a big set of language for talking clearly about many different aspects of the game experience I want to give us just one new lens that is the concept of player attention now attention in the way that psychologists talk about attention is very important when we're discussing the core of a good video game experience but it rarely gets discussed by designers here at GDC at least in my experience and I'd like to see if there's something that we can do about that part of the reason that I'm

interested in this is because I'm hungry for knowledge about the human mind and not just the kind of knowledge that science gives us but the insight into what you might call the lived human experience that we can get from art and what better art form to explore these questions with than other video games the other reason that I want us to start talking about attention is my personal dislike some might say my unreasonable dislike of a couple of innocent words here they are poor old immersion and immersive their relatives engaging and engagement are a little bit better but I'm afraid not that much now we use these words all the time when we're talking about what makes games great but do we really understand what they mean their literal sense seems confusing when we're immersed in a game what are we under the surface of this is some kind of game design soup are we inside the

gameplay or inside the graphics like James Woods making out with his creepy TV and Videodrome in their landmark 2003 book rules of play Katie salen and Eric Zimmerman talk about something that they call the immersive fallacy they say that there's an idea prevalent among game designers and media theorists that the pleasure of a media experience lies in its ability to sensually transport the participant into an illusory simulated reality one so complete that ideally the frame falls away so that the player truly believes that he or she is part of an imaginary world and they go on to claim that this conception is a mistake the use of the word immersion in this sense which refers to our going beyond the boundary between our day-to-day world of IRL and into some machine mediated consensual hallucination to

borrow a bit of poetry from William Gibson it comes from the world of virtual reality and it's an alluring idea certainly the level of identification that we have with the protagonists in a movie or the hero of a game would suggest that we get caught up in fantasies quite thoroughly and the amount of time that we at Naughty Dog's spend grounding the reality of Uncharted in lots of different ways is testament to the importance of helping an audience suspend their disbelief but the claim that you can make someone forget who they are and start to believe that there's someone else just by enveloping them in pictures and sounds of another place and time has never seemed quite right to me it doesn't do justice to the game artisans who craft experiences that get and hold our attention in many different ingenious ways and even if we could pull off this trick of making you believe that you are someone else would we really want to if I could reach into

your mind make you forget who you are while leaving all of your skills and emotions intact and have you literally believe that you and Nathan Drake hanging out the back of a cargo plane with the desert floor a quarter of a mile below you and a bunch of gun-wielding enemies up above you you probably wouldn't be excited and entertained in the way that everyone at Naughty Dog hopes for for the players of our games in fact you'd almost certainly be scared shitless no disrespect you understand I know you're all very courageous now I do think that games interactivity and what you might call overwhelming sensory inundation can change our identity in interesting ways that are worthy of a lot more exploration the interactive theatrical production Sleep No More and gas Bernoulli's film Enter the Void are two great entirely different demonstrations of that fact and here's some recent games that are played with this strange relationship between the

game's protagonist and its player Jaron stouts dinner date and swear he's deadly premonition now both of these games show us that things aren't quite as simple as having some kind of one-to-one identification with a game's hero as do just about all the games by belgium-based art game developer tale of tales they to mess around with this idea now I don't really want to be that semantics quibbling guy I actually think that the word immersion is fine when we use it casually or when we're sure what we're talking about virtual reality is very interesting and very important getting more important all the time and high fidelity wrap around simulations are immersive in the real sense of the word I think but in the context of TV based videogames the word immersion is a bit of a buzzword and it can point us in the wrong direction when we're trying to analyze the games that we love especially when we start talking about being immersed in gameplay so how do I

want us to talk instead well it's really pretty simple videogames entrace us by getting our attention and then they give us what we call a compelling experience by holding attention loosely defined as the cognitive process of paying attention to one aspect of the environment while ignoring others attention is one of the most widely studied and discussed subjects in the whole of psychology so let's take a closer look at it just quickly for many years scientists believe that human mind could only pay attention to one thing at a time and psychologists call this our attentional bottleneck the game designer calling North Way has talked interestingly about this on his blog in terms of attention splitting for example he talks about how Starcraft players call attention the third resource and expert players attempt to steal attention from their opponents with raids and harassment an

attentional bottleneck is important for game designers to think about in many games we have to keep track of lots of different pieces of information how much health we currently have how much ammo we have left where the enemies and other immediate threats in the level are and so on if the player only has so much attention to go around we as designers have to stay aware of this so that we don't overwhelm them now we now know that it's not quite true that we can only attend to one thing at a time our unconscious minds are always monitoring the world for the arrival of important new information we call this vigilance and it's been studied quite a lot since the 1940s because it's central to some very important jobs things like air traffic control military surveillance even lifeguarding it's totally relevant for game designers of course we're betting on the fact that our players are going to be able to be vigilant for hours on end whether you're keeping an eye out for incoming aircraft in flight control or staring eyeball drying leave it for

incoming enemies in action buttons awesome games ziggurat which you should check out on your iPhone but it's tough to be vigilant we all suffer from something called vigilance fatigue which means that after the first 15 minutes only 15 minutes of paying close attention to something we become much more likely to miss new relevant information if the sensory footprint of that information is how small or kind of weak now scientists used to think that the more boring a task the quicker vigilance fatigue kicks in but recent research shows that it's just as much of an issue for interesting vigilance based tasks like the ones in video games people focusing on the same task for a long time and remain vigilant eventually become distracted irritable and impatient think about your mental state at the end of a six hour online FPS jag or a protracted raid in an MMO

if you're anything like me it's probably somewhat agitated this interestingly switching to another activity especially one that is low stress or even more interestingly one that contains pastoral scenes of nature almost immediately restores our ability to be vigilant within a few minutes which I think probably explains the patterns of self-guided player activity that we see in games like Skyrim as we kind of cycle between exploration and more intense combat and then less intense and sometimes lengthy periods of just noodling around in our inventory now this need that we have to rotate between different tones and intensities of visual activity probably explains why movie makers talk about pacing here on the left is an intensity graph that the writer producer Warren Skarin made for the movie Top Gun as well as Freytag's famous triangular graph of dramatic structure and Kurt Vonnegut talking with graphs about the shapes of stories I definitely check out this

video on YouTube and we certainly think about pacing a lot when we're planning the Uncharted games and we space out intense periods of combat with less intensive exploration and problem-solving and with experiential sequences like Uncharted 3's lost in the desert montage so to wrap the section up I quickly want to tell you about the two different kinds of attention that we have the first kind is something that I've been exploiting every time I bring up a new slide using what's called your orienting reflex now this big change in your visual field has almost certainly succeeded in attracting your attention to the screen where I want it to be and it does so in a way that you don't have any real control over sudden loud sounds and motion in our visual field also have the same effect as do the mention of your name anything that threatens your survival and just plain novelty anything kind of new and different in your sensory environment has a tendency to attract your attention now this is

called reflexive attention and broadly speaking it happens at the back and at the sides of the brain the other kind of attention that we have is executive or voluntary attention this is the attention that we are in charge of which we choose to direct which we have what's known as executive control over now executive attention is one of a group of executive functions which also include things like problem solving decision making and the way that we monitor our own actions and this takes place primarily in the front of the brain but executive attention can be tricked as it struggles to organize the world and it has a tendency to assume that the world is continuous and predictable you might be familiar with the brilliant person swap stunts pulled by the British illusionist Derren Brown where a victim the woman in the white shirt ends up giving directions to two different people without even realizing that the person they're talking to has changed

it's uncanny she doesn't bat an eyelash now the whole spectrum of executive functioning where we make choices and take actions is right at the heart of our craft as video game developers this is what we're talking about whenever we talk about agency but it seems that we rarely talk about what the player is choosing to pay attention to I wonder if that's because our executive attention is kind of out of sight out of mind in any case I think we shouldn't forget that the players of the games that we design have a freedom and a capacity to express themselves simply by choosing what to pay attention to next before they even take an action and I don't know that the main that the main stream of video games often take advantage of this in a thoughtful way and I'll admit to often be guilty as a character action game designer of thinking of the player is pretty reflexive running from one shiny thing to the next that we've placed in

their path games like the contemplative dear Esther and the atmospheric Proteus take a quite different approach of course Proteus which was developed by Edie key with a reactive ambient score by David Kanaga is one of my favourite games of 2011 and if you enjoy meditative interactive experiences and you haven't yet played Proteus I implore you to seek it out so anyway that's a 10-cent tour of what cognitive scientists know about attention if you're interested there's a lots more good stuff to learn about this on Wikipedia so attention is probably important for game designers right like other kinds of entertainment designers we have to guide our audiences attention to the things that enrich their experience and we have to steer their attention away from the things in our game that don't or won't for example game designers specialising in level layout use the weenie technique that we borrowed from theme park design

so orient the audience's attention to these landmarks in the environment that poke up over the nearby things and then we provide kind of sequences of cues and revelations on the paths that guide the players where they choose to go having oriented themselves to these weenies so if we're going to talk about the ways that videogames get and hold our attention instead of talking about immersion we'd better take a quick look at some of the ways they do it when I first sat down to think about this I decided that most attention grabbers and keepers in video games fall into one of these three categories beauty story and gameplay now this list seemed like a pretty good starting point but it was going to change the more I thought about it so let's begin with Beauty the simplest thing that I could think of that gets my attention in a video game is great art you can talk about beauty

in terms of symmetry coherence harmony and ideal forms you can also express it with logic and number we know about the complementarity of color the golden ratio and the Fibonacci sequence but when I thought about it some more beauty didn't seem to be quite the right word not everything that gets our attention like this is traditionally beautiful think about the striking work of H R Giger or how much of the art we love is cute or a zany so I thought that maybe a better word than beauty was aesthetics that way we can include a wider range of tastes and philosophical ideas as well but then I started looking for a way to describe this in even more general terms by talking about something that draws attention to itself by virtue of its difference from those things around it so I decided to think like a graphic designer or an interface designer and call this contrast that way this category can

include anything in our sensory field that leaps out of us that grabs our reflexive attention in a way that is useful to our executive attention now things in this first category beauty aesthetics contrast easily get our attention I think they grab it quite powerfully in ways that we don't have that much resistance to however these kinds of things don't hold our attention for that long think about the way that people only spend an average of 30 seconds or so looking at even the most famous paintings in the world we easily tire of the contrast in the scene and our attention needs to move on to something else so my second category starts out a story now of course story is a hot-button topic for game developers as you'll know if you've been reading Gamasutra in the last few years my hope is is that if we talk about it in terms of attention then we can make it less loaded as a subject

story and games are almost certainly the oldest human cultural traditions and both of them demand and occupy our attention think about an early human shaman holding his village entranced around a fire with his tales of the gods and birth of the universe however unlike the linear story that's woven into the playful space of the Uncharted games not many video games have a story in a traditional sense and so I pretty quickly changed the section to narrative to encompass many different kinds of games from rockstars brilliant systemic narratives all the way over to Daniel Ben Mary's new game storyteller I think even the fact that a rook is shaped like a castle and a knight is shaped like a little horse is narrative thank you but to generalize even more broadly this is all to do with social stuff a 2007 study

at Florida State University showed that we exhibit something called attentional adhesion not only towards people that we are attracted to but also towards people that we perceive as rivals for those attractive people's affections now suddenly it seems scientifically comprehensible the interest that we all men and women gay and straight alike have in the attractive noble crafty cast of Downton Abbey so opening the door to the social in this way will also let us talk about how social games of all kinds hold our attention whether it's the split-screen co-op games of uncharted 3 or the social aspects of Facebook games and all of the other social games we've been playing recently the developmental psychologist Michael Tomasello has

written about something called the joint attentional scene and he describes how a shared experience of a third thing plays a fundamental role in our intellectual and social development and it might even be what gives us our essential humanity professor Janet Murray author of course of Hamlet on the holodeck has run with the idea of joint attention in the direction of games and you should check out her work you can find it online thinking about the social also made me think that this is why we've had so much success with the character-driven stories in the Uncharted games by putting a focus on the people in the game their fears and hopes and the moment-to-moment nuances in the relationships between them we've been able to get and hold people's attention much more easily than by trying to use kind of more abstract dry plot driven stories that many video games use and which focus on the power structures or

the history of a fictional world now I think it's averagely tough to get someone's attention with this story narrative social stuff screenwriting books like Blake Snyder's save the cat are full of advice about the very challenging task of quickly establishing characters that are likable or empathic so that will want to follow them along their path and I think that narrative also has an average ly strong hold on people's attention and in a way but in a way that's curious because it doesn't decay very quickly over time episodic fiction where we might have to wait days or even weeks between chunks of a story is testament to this weird fact there's an anecdote from about people in the 1800's lining the docks in New York City waiting for the latest episode of Charles Dickens serialized story the Old Curiosity Shop to come in on the boat

it was serialized in a pulpy magazine after he dropped a particularly spectacular cliffhanger on them in the previous episode so my last category is gameplay now games are made of several parts they have rules their mechanics that interact in a systemic way with players and resources to produce dynamics which in turn yield emotional responses aesthetics in a different sense than I used earlier for the players of the games now you'll recognize this as the MDA model proposed by Robin Hunicke mark LeBlanc and Robert sue Beck if you haven't read their paper about MDA you must look it up on the web it acts as a great concise explanation of how gameplay works to hold our attention and impact our emotions the philosopher Burnitz defines a game as the voluntary attempt to overcome unnecessary obstacles in pursuit of specific goals

now goals are what make a game different from a toy or a free play session we work towards long-term goals by means of attainable short-term goals that we put together strategically to get to the long-term goal things as simple as roll the dice or shoot the enemy even as simple as take cover now these build into compulsion loops by way of letting us reach these satisfying points of resolution the end of my turn the dropping of the enemy even the the or the collapsing of three clumps of grass into a bush in spry Fox's brilliant triple town we string these loops and goals together into a gameplay experience potentially of many many hours and I think this is also related to Jamie Gries m'as famous conception of the 30 seconds of fun in halo where an underlying pattern of player activity is repeated with continual variation but over and over again to create the long-term enjoyment of the game but

there's a problem with calling this category gameplay not all the video games we love even have goals and so by some of our best definitions they're not very game like strictly speaking minecraft at least before the single-player game came along and Sim City our two video games that are very open-ended but which I've played compulsively for dozens maybe hundreds of hours and they are clearly doing something right to hold my attention but quite how they're doing it has always been something of a mystery to me I'll come back to this later on so I almost changed this category to interactivity to make it more general but let's call it lewd ism instead like ludecke from the latin ludus as I'm sure you know which describes play of all kinds not just games and which acknowledges the playful attitude that I think is necessary and this scheme needed a bit of pretentious latin anyway so I think that

when the human mind finds a system it loves to study it if the moving parts of the system are easily visible to us this is Chris burdens art installation metropolis to which I recently saw at the LA County Museum of Art and it's a system whose dynamics are plain to see and it's fascinating isn't it these are all Hot Wheels cars by the way on an infinite loop they have a custodian to take care of them in the middle so we come for the regular structure and repetition in the system which gives our minds something to grasp quickly much like the way that we get fascinated by watching the waves lap into the shore and similarly like the self-similarity in this Romanesco broccoli but then we stay for the variability in the system like the ever-changing patterns in the flickering flames of a fire or the

variable schedule of reinforcement in the payout of a slot machine so to make things even more inclusive I'll also call this category systems now when a rule of rules resources interactions and goals are well designed gameplay is like mental catnip to us I think that this game play lewd ISM systems category is the hardest to get a players attention with for example it can be tough to teach a board game to someone who hasn't played it before and isn't that interested and we have to work very hard to design video game opening sequences like the beginning of uncharted 2 which grabbed new players attention very quickly simple mechanics probably get your attention more quickly since they're easier to understand and get behind but as Jason raw pointed out in his keynote at the Montreal International Game summit last year games are clearly the form that is best at keeping people's attention while a

film struggles to keep you for longer than about three hours tops a great video game can hold your attention for does hundreds even thousands of hours now this is also where the depth of a game becomes important game depth is a little hard to define but let's say that a deep game is one that produces a large number of interesting choices in a comprehensible way from a compact elegant set of rules the deeper the game the greater the opportunity it has to hold your attention for a long time with its gameplay alone I think it's probably true to say that a game holds your attention best when you feel that you haven't yet completely understood its complexity and I'd like to thank my colleague Caitlin Bernal for that insight please check out her talk on Friday so I hope you're at least somewhat agreeing with me by now that we

can cut through the confusion about what immersion and engagement might mean simply by saying good videogames get and hold our attention using a mix of the elements in these three categories also you can start to see how we could freely mix and match stuff from these categories to make different styles and genres of game drawing on them to serve each of our unique individual creative goals we can also keep them in mind to use them when we need to just when we're losing the player's attention with one category maybe when there's some succumbing to vigilance fatigue we could pull them back in by deploying another a very simple example of this would be the way that back in the days of 16-bit platformers we would vary the graphics and add new mechanics into the mix as the game progresses now we often get into difficult arguments like the story arguments when we try to compare video games to other forms like movies and when you work on a game like Uncharted

you end up hearing all about that stuff by using this concept of attention to look at other forms I hope we can get a clearer idea of what video games are and aren't like this kind of framework cuts us loose from subjective ideological evaluations of the value of diff forms and it leaves us free to evaluate what does and doesn't work both functionally and artistically so whether we're discussing games film sports or even just some drunkard falling over at a party the kinds of statements we can now make are simple and to the point and we can more easily draw out the ways in which each of these things are similar to and different from one another so as we learn from Collin North Way and the Starcraft players human attention is a limited resource there's only so much of it there's a cap on the rate at which it can be parceled out it diminishes and

recharges over time but you can improve your attentive capacity in fact the games we make are helping people do just that ongoing research from Daphne Bevelle EA's group at the University of Rochester whose explore of whom's experiments are shown here shows that people who play action video games have better faster visual attention skills than people who don't and the change takes place quite quickly after quite short play sessions so since attention is a resource you'd think that it would be great subject matter for video games but there haven't been that many games that are explicitly about attention you can find some very simple attention training games on the web but there are a couple of interesting ones that I'm interesting attention based games that I'm aware of Eric Zimmerman and the game the team at game lab made a game a few years ago called Arcadia where you had to play multiple action games at once it was very difficult and

of course Chris Hecker's game spy party is specifically about using the players attention as a resource that they have to manage throughout the game in fact I'd like to make the claim that attention is the basic currency in which video games trade and in which nearly every cultural form trades as well so I really wanted to talk about game addiction today which is a difficult and polarizing but I think interesting subject and I can't help but feel that we should talk about it more as a community of game designers I kept coming back to it over and over again while I was writing this talk because of the what seemed to me to be a clear relationship between game addiction and attention what we call addictive games pull your attention back to them over

and over again and then hold it for a really long time games that cause people addiction related problems in their lives are perhaps too good at getting and holding that particular person's attention so unfortunately it would take me a lot more than an hour to do this subject justice but I hope someone at another GDC can come back to this subject and run with it so the player has all this attention which they're bringing to bear on our games how is that practically relevant for game designers well I don't have time to speak to every kind of game so I'll just tell you about our experiences from the development of Uncharted often in our video games we want to direct the players visual attention to an aspect of the environment a ladder a door maybe an enemy or even a simple ledge now there are lots of things that impact the

visibility of things in the environment the silhouette of the object and the background that it's against its hue saturation and value all of which are affected by the lighting of the scene then there's the composition of the view of the scene and the lines and shapes within it and the density of detail and any animation that it contains now if even one of these things is out of whack then the player won't see what we need them to see and every uncharted game has presented literally thousands of these chat kinds of challenges for us to overcome as you can imagine but there are a couple of simple tips for you to know about here if you're working in character action games and maybe some other related kinds of games we found the players rarely look outside this central part of the screen unless it's too look at the meta information like the HUD and that causes them to often miss things that might be in plain sight but which are out in the corner also players

are more attentive to things in the lower half of the screen than in the upper half because of something called visual gravity now Bruce Bloch talks about these elements and many more in his excellent book the visual story which I think should be required reading for every video game designer and I also like this book on the right composition from the excellent DK eyewitness for just some simple tips about the way that lines of force in a composition drive your eye around the scene in useful ways to us the takeaway for game designers here is that subtle details in the art of your game don't just make it more or less pretty they make it possible for the player to play your game at all now my friend Robin Hunicke who works at that game company told me that working on journey their latest game they created a new term for this they called it attentional design and

you can see it at work in the opening moments of their game the way that the players attention is drawn first to the character that they will play at the very beginning of the game by the characters isolation in the scene and then the suggestion that the character is now controllable because of the way that it's risen to its feet the establishment first of the hill as a short-term goal for the player through the use of attention-grabbing detail and animation and then the establishment of the mountain as a long-term goal for the player by a change in camera position that frames the composition in such an impactful way all of these are subtle ways of designing to direct the players attention in gameplay towards the mechanics and the goals of the game now Robin told me that even these simple seeming things took many many hours of work to accomplish and finesse and I know from our work on the Uncharted series just how hard we have to work to direct the players visual attention and their auditory attention in subtle ways like these

now visual attention is a lot weirder than you might think we can only see a clear sharp image with a tiny tiny part of our retinas it's something called the fovea it's right in the middle of the retina it's about a millimeter across and it's responsible for a full half the signals that travel down your optic nerve if you hold your arm out at arm's length and look at your thumb that's about the size the width of your thumb is about the size of this tiny patch of visual acuity that you have so because of this we can't look at a scene in fixed steadiness the way that some other animals like birds for instance do instead we build up a 3d picture of the world by zooming our eyes around it with these quick simultaneous movements of the eyes called cicadas that's sa CC ADEs which are punctuated by periods of rest called fixations and sir cards are actually the fastest movements that the human body can make and we tend to fixate on clusters of

detail which explains a lot of the stuff that's in that composition book thanks by the way to jury for the use of his graduate thesis project foliation movie here I thought it was really a clear way of explaining how this works so knowing this you'd think that we would want to set up AI tracking equipment when we play tests to be able to see exactly what our players are fixating on and boy or boy what I'd like to I've always been fascinated by the kind of technology that can be used to see exactly where on screen someone is looking for instance web developers use it all the time and I'm itching to get my hands on some eye tracking kit but it's not something that we've done yet a naughty dog if there's any eye tracking companies out there who'd like to hook us up with a freebie please let me know valve though have had some experience using eye tracking as part of their play testing process as I think of people in

the Greater Sony organization Sony owned Naughty Dog of course in fact valve even rigged portal 2 so that you could play it using your gaze to aim the portal gun around the level now this kind of technology is an amazing boon for paralyzed two people of course and I think we'll be seeing a lot more of it in the future it's going to become cheaper and more usable and I think that eye tracking technology could well become standard equipment for game developers and maybe even game audiences over the course of the next ten years however right now I tracking technology is still very expensive around ten thousand dollars a seat I believe and even if you could afford I tracking it might not be desirable in his excellent talk at GDC in 2009 valves biometrics expert Mike Anne Binda mentioned a study that showed that when people know that their eyes are being tracked and you usually know because you have to like put your head in this weird equipment or put these special glasses on

then people move their eyes around somewhat differently they become kind of self-conscious about where their gazes so how else can we keep track of the player's visual attention to make sure that the attentional design techniques that we're trying to use are working well of course we study our players attention all the time when we're using one of our most important game design tools namely playtesting now I'm sure I'm quite sure everyone here knows about the importance of play testing although you'd be surprised to hear just how many game developers still make do without it now we play test our games all the time on each other of course as developers while we're making them but the kind of play tests I'm talking about now is formal play testing with groups of four US ten at a time kleenex play testers from the general public who have never seen our game before and who will only use to test it just once when I joined Naughty Dog to help finish jak 3 we ran

perhaps four or five play tests over the course of the project for uncharted 3 we ran 21 tests over the course of the last six months of development so you can tell how important to this to us and we run these tests in-house in a dedicated play test room each play testers station has a networked playstation 3 with a build of the game that we're testing they have headphones and there's a networked DVR box we'll capture video of the gameplay to our network as the players play screens separate the stations from each other so the players can't see each other's games even accidentally and we ask them not to talk to each other during the play test and because we want to be as scientific as possible in this we're ruthless about not giving them any help with what they're playing as the players play the game records certain information from about the gameplay session and posts it

to a database on our network we call this our metric data then at the end of the play test we get the play testers to fill out a questionnaire about their experience and we conduct an exit interview in quite small groups three people at a time although one person at a time would probably be even better and we video record these and these are really useful the questioners are really useful to help us track the improvement in the players perceptions of our game from test to test we've been lucky maybe to always see a slow gradual improvement and that is really valuable for helping us to stay sane in the knowledge that the game is gradually getting better especially if we've recently made a big change somewhere in the gameplay and we also get some interesting if kind of anecdotal game design perspectives that we can discuss from the exit interviews then immediately after the play test we take a look at the metric data which lets us visualize how playable the game

is one way we do this is by compiling a table of how many times the players died in each small chunk of the game and then I use the conditional formatting in Excel to see where the average and maximum deaths for each little bit of the game exceeds certain thresholds and this is super useful to immediately alerting us to the parts of the game that are still too hard because they just jump right out at you red and pink but back to attention one way that we can track the players attention without this expensive equipment is ridiculously simple it's just by watching them play or by watching their gameplay videos later in the game like ours it's to see when someone isn't seeing something or has forgotten their goal when players repeatedly run right past the object that they're meant to be interacting with it's a pretty sure sign that they can't see it if they go up to it from time to time and try and use it but then leave it alone for a long time you know they can see it but they don't

think it's important this is one of our play test gameplay videos by the way so we make notes about these situations as we observe them and then we make fixes between play tests and then we watch out for improvements in the next test and just by doing this you can make an amazing difference to the way that your player's attention moves over the sensory surface of your game and you can seriously improve the amount of fun that your players have with your game as a result but what about tougher problems where the players attention isn't so easy to observe well you can use your metric data in some clever ways to get a handle on these kinds of issues the environments in Uncharted are visually very dense thanks to our brilliant artists there's a lot going on in the picture of any random Uncharted screenshot and like we were just discussing it's very easy for things that are important to gameplay to get lost among all that visual information

so because of this we faced a nasty recurring problem in the development of the Uncharted games one that dated all the way back to the beginning of the series we found that players would often have a difficult time spotting the edge grabs in the environments now this was usually a disaster for us of course because it would stop them climbing onwards to the next part of the game there's a lot of climbing on edge crabs in uncharted and the problem was compounded by the fact that players would also get distracted by things that looked like they were climbing but which weren't so the solution to this problem was quite brilliant and it came from these three handsome men lead technical artist Tegan Morrison on the Left lead gameplay programmer Travis Mackintosh in the middle and gameplay programmer Jaros su Netsky in the the right what we did was to set up a system that we only used during our formal play tests that would record an

XYZ coordinate every time the player pressed the jump button and didn't end up jumping to a Ledge but instead jumped up and down on the spot and then we wrote those coordinates to the metrics database and then when the play test was complete we exported this data back into the game and we put a little red sphere where every thoughted jump had taken place for every single player in that week's test and we called this our bad jumps system you could immediately see the bad jumps clustered beneath objects that looked like edge grabs but weren't and they told us immediately what we needed to fix so in the days after each play test the environment artists would go through their levels with the bad jumps turned on we'd have discussions about what needed fixing when we needed to and and then the artists would make changes to the artwork of the things that our play testers thought had grabbable edges but in fact didn't to try and make them look less like handholds and we were able to

make a huge difference in the game by doing this I wouldn't care to say that we fixed this issue completely but definitely we were really satisfied by the results that we got just from setting up this simple system it only took a few hours to create and we invented some other similar attention tracking metrics for uncharted 3 how often did players pick up a new gun hold it for a moment and then drop it again in favor of their old gun that was really easy to record in our metric data so was the data you can see here the number of successful and failed attempts that players made to throw back grenades instead of ignoring the grenade throwback mechanic altogether and this technique seems like it could be applied to widely different styles of games I'm sure you are starting to be able to imagine them hopefully for whatever kind of game you're working on depending on its mechanics so before I wrap up this section I want to assure you about something that always bothers me whenever I hear a talk about the

psychological predictability of human beings and the way that we can use numerical measurements taken our players to make decisions about the creative path that we should take with our games these are tools just like any other and just like a knife they can be used to peel a delicious apple or commit a horrible murder it's often good to be scientific in a technical art like ours but you shouldn't go any further with these techniques than your gut instincts as a game designer and a creative person tell you that you should go what you're trying to do is to make sure that your game is landing with players in the way that you intend it to land you're trying to get rid of all the daft oversights that stop your game from being fun and interesting while leaving in enough chewy stuff the players want to sink their teeth into your game over and over again and this also applies to my whole invocation of this concept of attention

I'm not trying to be reductive and explain things away in terms of psychology and brain science I think that would be a terrible mistake this is just one more lens that we can use to look at what we're doing when we're designing a video game so now let's get to the topic I want to close on the subject of experiential video games what they have to do with the tension and why I think they're important I'm sure that some of you have been wondering why I haven't yet mentioned a concept that seems obviously connected to the concept of attention and the psychology of video games of course I'm talking about the concept of flow now flow is of course the state of optimal experience described by the psychology professor Mike Csikszentmihalyi and it's been much discussed here at GDC over the years for the strong ties that it has to games when we're in a flow state our attention is held and very highly focused whether we're doing needlepoint or speed typing

or playing triple town our attention is right where it needs to be on the subject matter that we're engaged with helping us handle the tasks at hand while our vigilance watches out for incoming tasks that we're going to have to deal with we lose some awareness of ourselves of our bodies in particular and we experience strange time distortion effects as hours seemingly pass in minutes but sometimes as moments stretch out allowing us to act more rapidly than we otherwise could when we're in a flow state we're riding the boundary between mastery where things are easy and challenge where our abilities are being tested if the systems too hard we'll get frustrated if it's too easy we lose interest in either case we drop out of the flow State and the system has lost our attention when we are in a flow State we're acting in a system as if it was a game manipulating the system

towards certain goals however we wouldn't always want to call this system a game surgeons are often used as example cases of people who go into a flow state as they work and we agreed earlier that video games like Minecraft don't even have explicit goals and yet millions of people myself included go straight into a flow state when they pick minecraft up so in trying to solve the puzzle of how we can go into a flow state in non game systems I finally understood how minecraft holds my attention so completely in the public debate about gamification we've talked a lot about extrinsic and intrinsic goals we know from educational and industrial psychology going all the way back to the 1930s that intrinsic goals are better motivators than extrinsic goals it's the playing of a game for its own sake that holds our attention not the

promise of some eventual reward so I realized that I create my own intrinsic goals in Minecraft and this is what puts me into a flow State I make a game of Minecraft by imagining a back story where I need to stay alive to unlock the mysteries of my uniquely generated world and in order to stay alive I have to engage with the system of the world around me exploring the environment and using the things I find in it to make the tools that aid me in my quest however the goals set for themselves by other people who play inside this self-same system that is minecraft might be quite different like building a scale model of the USS Enterprise or building a flippin working 16-bit processor but what's common for all of us is that we've had a profound experience of a flow state that was only possible because of the goal-setting attitude that we brought to an open system this

is what chicks and me hi calls the autotelic experience from the latin otto self and Tilos goal autotelic people he says are internally driven and they exhibit a sense of purpose and curiosity because of their nature they're more autonomous and independent and they're less easily manipulated with by threats or rewards and they love to explore systems just for the sake of exploring now I think all human beings are autotelic to some extent but our generations the ones that have grown up playing video games are very lucky because we've been trained to be more autotelic than any other generation of humanity before us the games that we've played have cultivated our curiosity and our independence and our ability to manipulate rich systems in goal-setting

ways I'm sure that you or people you know have had deep rich gameplay experiences in Red Dead Redemption or Far Cry 2 or Skyrim without following the main storyline of the game at all or maybe by running pacifist playthroughs this is testament to the power of the autotelic experience now as far as I know daniel been mayor he first coined the term experiential games on his blog all the way back in 2007 broadly speaking these are games that forego explicit goals and traditional game compulsion loops in favor of systematized open-ended explorative expiry answers often in pursuit of an artistic impact early attempts at these kinds of games and interactive art experiences frequently struggled to hold people's attention for very long compared to

traditional video games because they had maybe because they have a tendency to rely on aesthetics and narrative the categories that are the least effective at holding our attention however as game aesthetics and narrative elements have become more systemically interesting new opportunities have opened up for experiential games because of this autotelic drive that we all carry around with us in my indicate talk last year I described the way that tale of tales game the graveyard which you can see in the slide gave me the confidence to push through the experiential peaceful village sequence in uncharted 2 even when some of the people on our team didn't think this was going to be a success the calm beauty of the scene and the pleasure of just the simple systemic narrative elements that we scattered through it held people's attention for just long enough to create a powerful moment at an important point in the game players who were curious and

explorative found plenty to reward their curiosity and players oriented towards the ultimate goal of the game were able to move through the scene very quickly while soaking up just enough of the calm atmosphere to reset the intensity of the game in preparation for its final act and can find my indicate talk online if you're interested to hear more about this it's on YouTube and g4tv you'll hear my anecdote about people trying to punch the NPC's videogames are special in their opportunities for non-linearity of meaning but they're not unique in this regard philosophers have thought for years that so-called linear artistic forms offer similar opportunities in his 1962 book the open work the Italian author

scholar Umberto Eco most famous for his novel the name of the Rose argued and I'm quoting from Wikipedia here rather than from Ecco himself literary texts are fields of meaning rather than strings of meaning they are understood as open internally dynamic and psychologically engaged fields literature which limits one's potential understanding to a single unequivocal line the closed text remains the least rewarding while texts that are the most active between mind and society and life are the most lively and best in other words singularity of meaning is the enemy of art great art in all its forms is multiplicities systemic and can have many meanings even contradictory meanings which is why the same piece of art can speak to us in different ways at

different times in our lives if we expect to find new things each cut each time we come back to a work then that in itself is a draw for our attention Amy Hennig my dear friend and Naughty Dog's creative director and head writer on the Uncharted series talks about the very great value of leaving something to an audience's imagination in terms of the storytelling in our games so that the individual player fills in the details for themselves and the experience of the game truly becomes their own even before they've taken an action in our interactive system so to put it another way if linear games are well written they're not truly linear we're learning more all the time about how to devise gameplay that expresses a broad range of emotion that's crucially important for

pushing games forward as an artistic form as Jonathan blows braid has so ably demonstrated but I worry that we have a tendency to over focus on compulsion loops and explicit goals in the games that we make and we tend to look past the emotional value and hence the artistic value brought to a game by things like nuanced aesthetics the narrative and the social and the space for autotelic self-determination created by open systems in his GDC talk last year an apology for Roger Ebert Brian Moriarty gave us the definition sublime art is the still vacation of the inexpressible when we're trying to make games that succeed artistically our ultimate goal is to evoke ideas and emotion from the player about subjects that are difficult to express but as Brian suggested in his talk and as Csikszentmihalyi himself says the flow states that we enter when

we caught up in the formal parts of a game is almost emotionless as an essential part of its character our minds are just too busy when we're dealing with all of the rational tasks that the game that again presents to allow in the subtlety of feeling that were chasing as artists but I see room for both the flow of gameplay and reflective experience in our games if from time to time in the progression of a game we have the confidence to let the player just drop out of their flow state as the players attention begins to shift away from its laser focus on the goals of the game whether those goals are self created or imposed by the designer then we have an opportunity to catch the player in just the right way with an affecting piece of music or a view of something wonderful or just a look from

another character and in doing so we can create one of those powerful moments that we cherish in our favorite games have you ever stopped to watch the sunset in Minecraft or to have my review in Uncharted think about how you are in that moment as you're responsible logical goal-oriented mind gives way to the uncensored emotion of your innermost self for me this boundary where the cool clarity of focused rationality meets the chaos and immediacy of raw emotion creating in us a lived experience at a lived experience that is uniquely ours in both its intellectual and intimate qualities is the richest and most exciting aspect of videogames as an expressive medium game designers are often terrified to let gameplays iron grip on the player's attention drop for

fear of losing that attention forever I think what we can see from what we've looked at today that there's no need to panic if the player has come to trust you then they'll stay with you for a while at least and we have so many tools that we can use to get the players attention back again in a meaningful way that the biggest challenge we face will be deciding which one to choose artists from Shakespeare to David Lynch to 4 metre wider have understood how to shape emotion by guiding attention while leaving room for personal experience for years and in doing so they've impacted our lives in countless ways the emotional results that Mark Robin and Robert described in MDA are the point of games no matter how we get there and there's no one right path with this we developers are free to play in this

domain and make our own discoveries about what works for us we hope that we've shown with the Uncharted games that the techniques of cinema can make a great mix with video games but we mustn't lose sight of literary fiction of poetry theater and music as vital forms of self-expression that were only just beginning to synthesize effectively with the creative opportunities that video games present there's one piece of knowledge that I'm certain we all share it's that all these things new technologies and better development practices the mechanisms and mysteries the human mind and the great powers of curiosity and self-determination that we all have are in video games making a new kind of cultural form possible one that is active between mind and society and life and that's my talk thanks very much

[Applause]