AI Wish List: What Do Designers Want out of AI?

[Music] my name is Dave mark the thing is though I am going to represent for what it's worth all the AI devs in the world kill me now and I don't have all my slides disappeared now let me do this restart that and send it over and now it should show up because I am going to get bombarded with a wish list and hopefully no foul language from my five panelists up here who have many things to say on this particular subject so introducing really quick Noah falstein on the Left Robin Hunicke Raph Koster Lara Lynn limbs and Richard Lombardi Sharlee

Michonne I wish now it's Richard Lee Martian did it anyway the the back when we first started the summit we did something in 2010 very similar to this where we asked designers via email a whole bunch of questions what limitations are you sick of hearing about what what can film directors do with cast and crew that you can only dream up with your AI characters things like that and then we actually had a panel up here of some really really nifty out-of-the-box thinking AI devs so it's kind of the reverse of this and so we iterated through those answers that those designers gave us and kind of you know batted around well how would we handle that um know to some extent or another many of those issues have been solved yes it's in quotes because we have advanced quite a bit in the past you know nine years or so since we did

this but apparently not well enough or there's new issues because in 2012 Warren Spector made waves and amongst us AI folks in a Eurogamer interview where he sets up I've been actively trying to shame my fellow developers specifically John Carmack and Tim Sweeney can you imagine what games would look like if those two guys spent as much time working on non-combat AI as on rendering you imagine what they would look like if they were wanted to create a believable character as opposed to a believable gun I mean oh my god these guys are way smarter than I I don't know how to solve the problem but they could figure it out stop rendering start creating humans or mice I don't care which of course he was working at Disney at the time so defects I don't know what the state of mice AI is right now but he went on here at GDC down the hall from us and I got wind of this later that day Warren was thrashing on the AI summit down at the narrative summit where he

says we have to spend time on non-combat AI we figure out pathfinding and not having characters fall into holes in the ground and stuff like that how do we outflank the enemy we spend no time on going to the women's bathroom and having people notice which is interestingly topical people still come up to me and say wow that was so cool that was 13 years ago apparently was something he had done one of his prior games I was not aware of have you seen a game where a drink was spilled and a character responded appropriately I think not have you seen a game where there was a spurned lover that mattered and it wasn't just a movie masquerading as a game like a cutscene have you ever seen a game where there was a would-be lover that you could interact with if anybody knows how to create a virtual Game Master putting me in a box or you in a box I want to know good stories are constructed not found' emergence can only get us so far and if we can dynamically modify stories based on what players want to do a good deed d

campaign you know you want to screw her on with the dungeon master and he actually responds so this is what he had said down the hall and we chatted later that week about this and a couple of times since then we chained were you appropriately shamed by Warren yeah that I'm not John Carmack hey that's what you get from representing all AI Deb's everywhere I guess right so anyway Warren wanted to be on this panel and desperately but this is apparently the first GDC he's going to miss since the Civil War ah he's old he wanted to meet it watch this video Warren was one of my warns when my closest friends and mentor so I'm just not here so but he did answer a number of questions that I posed to him through email will see those sprinkled through but we have a number of different topics that we have talked about over the past couple of weeks that we want to cover

that subscribe to kind of their wish list and we're so we're just going to go through in a topical order and see who has what to say on these things we're not going to dive deep into technical solutions other than I'm a prod them okay wait a minute hold it what would that look like what do you mean by that well you know to get a little bit more information the goal of this is for us to sit here and go how would we solve that because that would make our games better so the first topic I want to talk about is is character AI just which is normally what we do whether it be combat or non-combat and just have you folks throw out a couple of things just in general what would make characters better that you're not seeing that you sit there and go off if we could only pull this off hit me what do you got well one of the things that I would love to see is recognition from characters that I am playing the game in a unique way yeah so for example I have a follower in

Skyrim or in Fallout and they comment on things in the world and they comment when we're attacked but they comment in a general way they don't comment at all about what I do they don't comment on how I solve problems on the fact that I had to take a healing potion on the fact that I tripped over something or I got stuck on a rock or changed weapons they don't seem to understand that I'm playing the game in a way that somebody else you know is unique versus somebody else and that's something that we can gather I mean we we know if we watch Pat of player behavior when players are doing behaving differently under certain circumstances and there's no reason that it means even as simple as recognizing when I go to turn in a quest in a game like destiny that my group are when I go to turn in after a strike that my fire team wiped three times but finished it anyway right we know that because I literally just did it there's no reason not to track it I'm sure they're tracking in a metrics too so if you have

it from metrics why not have the game reflect that back to me and let me feel special like the game recognizes me as a unique player yeah I think we focus we over focus on false plasticity with the character representation itself rather than actually looking at the data that we're constantly ignoring I think it's really it's really bad I mean you can just even just track what how often they press the controller and find out whether they're frustrated you know and journey a lot of the stuff that we did to edit down the relationship between two people involved paying attention to both people and then sort of smoothing over their differences and a lot of people thought that the other character was an AI like if it had been an AI would then been the best hey I programming in the whole world you know because it actually paid attention to you it was fake plastic right we do a lot of fake plastic in games and that kind of plasticity would be really fantastic III have a little bit though actually could around the platter the fake plasticity because I think that I don't think this is important is what you've been discussing but I do see a lot of low-hanging fruit around the way that NPCs whether they are enemies or

allies perform they're kind of individual identities and across the course of the Uncharted games you know we did a little bit more each game around sort of I think a lot of it has to do with head tracking gaze tracking whether people somebody gives somebody the side eye gate and the way the characters hold themselves in idle animations and these kinds of things I think a lot of it has to do with the spatial distances the sort of non hostile NPC's in particular maintain to the player character and the kind of surprised reactions that they do when the player character comes right up to them and like gets in their face and how they jump back and how quickly they jump back I know some of this is sort of processing time related and we were able to get more and more prompt with a sort of NPC reactions all the time I still think there's lots and lots of low-hanging fruit right there so I'm hearing is from you is reactivity to just immediate stimulus yes recent past

stimulus mm-hmm so yeah you know I really loved the example when we were chatting about the the way you play fallout yes told that that's great so when I play fallout I have a pretty reliable pattern that I always play sniper so I stay cited even when enemies are getting pretty close to take they have to get really up in my face before us which is shotgun or something except radscorpions because they freak me out and so when a rad scorpion is approaching me my play pattern radically changes I am shooting wildly I'm fumbling I'm backpedaling like a madwoman and that's when I why I say tripping that NPC should notice what I'm tripping this I'm constantly tripping over things because I'm backpedaling I'll get up on rocks to try to avoid it I'll go uncited much faster and switch to the shotgun much faster if a game paid any attention at all it would be totally clear that radscorpions freaked me out there's no reason that the NPC with me could not comment on that or if you want to be terrible if you're making a horror game that you couldn't take advantage of that because now you know what in the game scares me the most and

when you mentioned this learning is something that I learned a couple of years ago from the world of Improv Theater relating to high and low status characters that seem totally actionable in the world of AI for characters and games there at least it comes out of the work of Keith Johnston who identified that sort of high people who perceive themselves as high status I more still like their gazes a little higher people who perceive them think of themselves as low status are sort of a bit more turned in on themselves they're their toes point in they touch their face a lot and they act kind of quickly and more nervously and I think we could do a lot of that quite easily yeah hey that's the animators that's total that's like paying attention to how people actually are and then building systems that reflect how people actually are and like the the difference between AI that is plastic for real and AI that is

fake plastic is paying attention to what real is like get out of your head and the problem space of how do I respond in a you know a relatively short time loop how do I get this animation cycle to work and have the foot planting be perfect that kind of paying attention is is not it's not paying attention to the level where it's really going to make an emotional impact and you can actually fake a lot of that other stuff with art style the thing you can't fake is what Richards talking about which is the way that humans actually respond to one another and you know I think you know just to be contrary about this thinking about this panel a lot of my sense of these questions of how do we make you know computers do these things they suck at a lot of this stuff yeah and we have millions of years of evolution having us focus on the kind of stuff that Richard was just talking about and you're our primate ancestors there's all sorts of my sister used to work with chimps and being with a group of chimps and being with a group of game developers it can be really hard to tell the difference and what he made that

joke about me no no this is I you know I've realized my own kind of body language is completely compatible with what the chimps are doing but the point being make have the AI control you know scorpions or animals or have them control computers you know I I would love that we at Google we had a proposal for this game called BOTS or not and the idea was that you didn't know if the other characters were BOTS or humans but your your job as a human was to be the most convincing bot you could be so we're like working on what why make why make you know have the AI make the humans seem more realistic make it see do realistic robots and granted we would love to have it do a lot more than that but you know I look when did we start when do we really start looking I mean when do we really start trying to figure out the subtle thing that people already do with hand animation to make a character seem real when it's clearly a

bunch of lines moving on a flat screen you know we don't even dare waste babies yeah we can yeah we can cheat yeah we don't even do it with other players right like the amount of emotional bandwidth that comes through with another player in an online game is basically nil it's terrible we don't do the simple things like I track well we don't do the simple things like you know gaze and head moving to whoever spoke most recently we don't do body language for precedents or any of that but I think fundamentally though if I kind of summed up what what all of these comments add up to me is we are in a habit for a whole bunch of reasons of building our characters as props that are primarily reactive and if we want to exploit AI they need to have their own inner lives out of which this stuff arises organically they should be scared of you because they remember you and go oh crap this guy kills random civilians none of us ever do that yeah but that

requires it's not that we lack we have we have the CPU we have the RAM these days okay it's not a question of capability I think a lot of what we just said is actually trivially easy and not a challenge for you guys instead it's a matter of will we have to actually choose to prioritize characters over props and I think we have inspectors I want to read this cool because it's exactly what you were just saying this is I wish for better non-combat AI we do really well now in creating convincing combat behaviors individually in groups but our characters can't do the simplest real-world things in response to different kinds of inputs that games typically allow there's a reason why love stories and intimate dramas aren't getting made an AI is a big part of that it comes down to that subtlety of who are they is is that yeah so I mean I haven't made MMOs in a long time right but the big challenge in making mmo's is

the fact that you didn't make thousands of NPCs right and so that meant that the answers that single player gaming uses which is make you know props that are scripted you just can't do it right so either you end up with a thousand of the same NPC or you have are driven to have to do stuff that is procedural or AI driven whatever right and the the solutions to get some of that in your life are actually not hard at all a couple floating scalar variables for fear anger greed whatever yeah and and we you know it doesn't take very much but the pressure is towards what I would call a stage crafty narrative ISTA proach yeah and so you end up pre scripting the NPC's yeah to such a degree like you know and ultima online we did this we had that kind of behavior stuff and by the time we got to

Star Wars Galaxies the pressure on it was so much that we just said we don't need NPCs let's just have players do all the jobs and we turned what would have been into NPCs into terminals because we were like well you can't actually do better right now they might as well just actually be a physical prop with a button and this fence quest please beep I think your philosophical point is is well made rough at the other end of the spectrum so I think there's some simple hacky things that we can do by looking at the expressive affordances that the game gives player like even something like you know just crouching in I have crouched in front of so many dogs in breath of the while hoping for that hoping to be led to some treasure actually I'm afraid I'm hoping to get something from that but the way that Zelda breath of the wild rewards you with this little moment of interaction where my attempt to take a kind of a

spatial stance that is possibly socially expressive in some way is just met by the game and even though it's just simplest of mechanisms yeah I also want to say that games about 11 romance are being made they're just not being made with characters so I think you know when you think about games as you know mechanics is rules and aesthetics is the outcome the experience that you get and the middle is the dynamics and like what we're talking about is the player NPC dynamic it doesn't really have good juicy affordances for romance but you can build like a game like Florence which is all about falling itself and it could do really well and be really lovely and people can love if you if you turn the mechanics and the aesthetics into into the main part of the game and the the disappeared dynamics are in your mind right so we'll know how to do it as game designers it's just we don't know how to do it with you know first-person shooters and third-person role-playing well and and VR actually is giving us some of those affordances specifically we've been

talking about gaze and it's better you know particularly when I tracking comes in which is certainly getting close we're gonna have excellent sort of gaze not just on are they looking at me but I I saw I think was foe did this demo where you know you're being interrogated and I happen to look at the water are the cup on the table and the guy bashes it out of the way and says look at me and it was one of the scariest moments I've had you can trip that around because the same part of our brain that processes some of that fear and fight-or-flight stuff deals with arousal too and the intimacy of VR the the physical proximity that you're acutely aware when you're within arm's reach of somebody compared to somebody across the room and VR actually gives you that feeling so if the AI is feeding off of those things realizes that a character is right next to you and you know it's very different if the character looks at you and the character looks at you exactly well or and if it's another player and I think this is actually one of the things that we should always think about as developers and as

particularly if you're working in a hair right now this idea that gaze is going to be so easy to track like the machine is much better at actually catching the things that we're looking at and our micro-expressions than we are you know it's just like you can observe a date from a different table and tell whether one of the people is not into it like a computer can look at your face through a headset and tell whether you're scared or whether you're you know looking for a solution or whether you're like really intensely focusing and there's a huge potential to abuse that so like one of the things that it's I think it's very important for all of you to be doing is making work at that level because if you don't do it then someone who cares less about players well and I'd like to add one small thing which is at the heart of this though is the need to study and understand human behavior yeah because without it it's all stupid pet tricks and the example that I would give is I anyone who played oblivion especially I'm sure all the women felt this way but I bet a lot of men did too found it super creepy when you'd walk by a garden he would say you know how to move in

light armor stop it what are you doing right that's weird eye tracking you no matter what and commenting on you I just apropos of nothing right and then they do it again in Skyrim they still do the same thing right it's a stupid Tetrick it's trying to make the AI look smart and responsive but in fact not looking at what people actually do and making something really creepy as a result right so I think you have to study people and understand people for anything we're talking about here to work and then if the game doesn't give the player an opportunity to respond in context to the NPC to the sexual harassment then that game is taking a kind of philosophical stance I want an ethical stuff the moment cuz this next section selling the NPC here at the AI Summit last year our very own Alicia laid a cur during the rant session had a segment that almost got a standing ovation from from the room I do want to play that here real quick I love that

term heart official in television you'll find out in my rant tomorrow but what would that look like you just talked about the empathy of realizing okay wow maybe I just as an NPC said something that was okay a little bit off MIT I probably should not have done that but there's plenty of other examples well that that would require the writer not having done it I mean that's like that's the the fundamental point that was just making yeah it's like I mean it's not like it has the capability of knowing what it shouldn't say I mean example yeah there's thing I used one that if I walk into the tavern or or the shop or

something in in a game of like a Skyrim and I've got arrows poked through me I'm dragged I'm at 1% health I what can I sell you today no dude I'm bleeding I've lost an arm what's that Bloody Mary yes thank you dragons two of them please I'm trying to keep my volume up so yeah what would what would heart official intelligence is Alicia briefly described it last year in that clip what would that look like to you the empathy the understanding that you're caring for them because they're caring about something as well possibly even you well I mean the part of the writer's job is gonna be to identify what each character cares about because yeah he actually genuinely doesn't give a crap about you bleeding other than don't do it on my floor right and now that makes him a person yeah I believe in this guy because he's like take that out of here I don't want your

trouble yeah and won't sell to me until I've cleaned up right that those are the kinds of things and I guess the point for me is that in order to accomplish this it takes the team working in lockstep it takes animators and AI engineers and designers and writers all understanding together that we're trying to create something it's not it's not anyone jobs responsibility to magically make an npc who'll do what I just said right that's it that's a group of I'm not representing all the animators I know writers yeah so I I want to challenge that just a little bit the challenge I would be offering would be okay let's cut the writers for a minute just cut them all sorry right and think about okay what can we get across without the writers because I think that's the way we need to think about this we need to think what can I get across let's now let's take away the animations - do I have anything to get across like if I had a text output is there any there there I think often the answer is

no right like yeah it we put all the burden on the writers so just for a minute do the thought experiment of there is no writing right there just isn't any do the NPC's have any brain at all and that I guess is what I mean really when when I think of a heart official intelligence I go but none of them have hearts right how about let's use some AI to try building at least a rudimentary heart right and see if they do anything right I mean that would be the challenge I would want because even just here you're saying even if just in data someplace there was a number that was changing because it felt you know happy with you say it in a sort of way but on a deeper level right we can hook it up to writing an animation right so I think it's important that you've actually recognized that like one of the things that makes us care for others is that they're broken like one of the most compelling characters in your favorite book is probably someone who has a serious character flaw maybe you've seen

that flaw in yourself and then you identify with that character when we do write well we generally focus on specifications for Humanity that are slightly off in one of several vectors maybe even meeting DSM criteria right I mean like it's really it's really unusual people history on ik people people that are emotionally unstable people that are clinging and then cold those are the ones that we write about and care about and I think one of the things we always do as an industry as we try to build either friend or foe but like what about the middle space the person that you're in love with who suddenly gets angry and doesn't know how to not throw things or the parent that wants to raise the child but was abused and so therefore is just constantly difficult with a little kid you know what about an animal that's been beaten that you're trying to win back to trust you know that that kind of

a relationship is I think the thing where you really do have to do the juicy work of it's not that they don't have lines or that they don't aren't written it's that they're actually something interesting to us trying to solve the problem and be perfect you're trying to create a perversion of a human that isn't really real or isn't maybe is real somewhere I don't know any of them but I love all my friends I do but um you know we all have our difficulties so like I think it really is about actually really recognizing that when you're broken you're more interesting and it's the ways that you're broken that make people care for you I think that's absolutely right Robin and I think that the kind of intimacy and empathy that Alicia's talking about comes from when characters make themselves emotionally vulnerable to one another and just thinking out loud we've been talking about NPCs noticing what the player does do I think that vulnerability is often associated with avoidance and so maybe there's some mileage in noticing what the player

doesn't do or is it seems uncomfortable doing and that might open the door to the wonderful scorpions example of commenting wow you really don't like those do you you know I neither then cackling at you everything fully every time you run away from us or saying oh let let me help you avoid those or something to that effect yeah yep excellent I'm Warren I've left this up here for a little bit but he was actually touching on a lot of the same things his spilled milk thing how do characters react it's gonna be whether it was knocked over by mistake that the NPC might respond whether the player was playing the role of a child well that's okay honey you spilled the milk it's okay or whether the player just revealed that she was sleeping with the NPC's husband and so on you know there's all these different context to one physical event in the world and having an NPC understand and react accordingly would be would be monumental how do we define

that in such a way that we can even build that I guess that's my job isn't it our job well you know one thing I'd point out I think if if we polled the room about whether or not they felt emotional connection with let's say their Nintendogs or their sims the answer would start being yes okay and it's worth pointing out that those are simulation driven AI driven little entities with inner lives of their own their own needs their own desires and yeah we do let them down we forget to feed them and we set them on fire and trap them in swimming pools at that point that's not letting them down just I feel like I let them down I'm sorry it I guess my point is we have seen this work yeah and come on Sims is what eight little variables on a scale it's probably only need a bite each right I

mean come on we can afford to simplify every NPC in world of warcraft right like easily that's tweet-worthy do it but but I do think it's also it's also it's a problematic game I mean I Sims was the first game I worked on and my first job was building objects for the sims expansion packs on sims 2 and trying to design things to go into that system was incredibly difficult because what ends up happening is you you hack together you know a terrain basically of interest for all that all the AIS and then they're all running around on this terrain and basically what they're doing is they're listening to the objects which are echoing you know affordances back to them and then they're choosing those affordances based on a stack based ranking on the sliders but then when you try to build a new object and insert it into that system it has to get along with all the other objects in the system so every expansion pack is like it's like if you think of the Death Star you have to peel one panel off and then like wire in a thing and then put the panel back on and then it just breaks so the game like 90% of the design on the expansion team was idealize what an object would

do then have someone try to make it and shove it into the game and then fix all the ways that it breaks by tuning it and we used to call eateth the thing that you use to tune that game the Crippler because you just had to do all this like Ninian tweaking and constantly evaluating what what they were doing and a lot of times you were just guessing you were literally guessing what they were doing they were like creatures of their own will you know and like bugs that I had to deal with on that game included like time passing differently on different Lots broke the treadmill so that people just were always stinky when they were working out at a gym you know or working out at home and so like it seems so easy but there's a reason no one has done it because it is actually incredibly difficult to sort of deal with that mesh so that sounds like something to throw at the machine learning system we don't have to do it anymore they've go fixed yeah fix it I'm just saying designed better tools for your utility system but anyway I do want to move on because we've got a couple other topics that we want to cover a

narrative is something that we hear a lot about over the past few years lately of Korea and a lot of people in academia are working on this of how do we create you know stories how do we create dynamic dialogue contextual dialogue matter of fact we're going to be having a talk here tomorrow on on exactly that but dialogue and narrative are two different things career taking a pre-written story arc and saying well we're just going to fill in different words compared to creating a story arc or two different things what would you want to see give me a vignette of it would be cool if so I don't want to miss the middle spot right because on the one hand you have the hey the entire narrative is crafted and I'm just along for the ride on the other side you have procedural narrative right there is a middle point where with in Mass Effect I'm telling my own story along with playing the story that they created right so that's I think something that doesn't really get addressed enough I think everybody who

thinks you know we really want more player agency starts to lean toward procedural which is difficult and a lot of times can't right now achieve the same level of quality generally as you get from a more you know written story but that doesn't mean again that they we can't spend our AI dollars and time on having my path to a story have more meaning and more recognition from the other elements and again it's having the NPC's talk to me about the choices that I made right because I love mass effect but it's always super clear to me that oh I just triggered the threshold where he likes me now right and I imagine even like normal people who aren't designers recognize that too right oh I just finally won his love which is not how people really work right so how can you make that relationship more interesting and more dynamic and have you know Morden actually recognized more of the small things they do in the game and not just the obvious things that are set up to make more than either like me or dislike me I just want to draw a circle I want the one thing in the middle of

you said is every player can go off and play his own story in any game you I'm gonna screw with the game this way I'm gonna do this but it was the part about having the characters recognized that you have gone off the beaten path and adapt to that and react to that and and join you again going with the improv yes and and looking at things in the macro too right because a lot of times there's no recognition among multiple characters and multiple systems about what I'm doing in the game as a whole yeah you're trying to befriend everybody right and I think designers tend to look at it well like okay I'll do factions then or if you like this person this person hates you but again life doesn't really work that way so some sort of macro knowledge of an idea I guess the closest I can come as the director and left 4 dead right the director and left 4 dead has an idea about what a good experience is you're making your own experience and he's kind of rolling things out for you you could think of almost all games that way that as the players going through it what's looking at the macro of the

players behavior what sort of it's Orton is he creating what sort of experiences hey is he being an in his actions but talking nice to people yeah that's a play style someone should call him on that right or not necessarily leaning so hard on the characters who do that job but letting the environment do it for you so you know the simplest version of this would be a game in which your choices actually changed the way the world looks to you and like you know my next game is actually going to be procedural and procedural narrative and really large and I'm thinking a lot about this stuff right now and I think there's just a lot of good ways to clip back on the things like one of the things I keep thinking about like if you look at a game like destiny which is beautiful but like why model the buttons on the jacket of a body that you're going to shoot from 30 feet away and then loot like just why do it like that's someone's life like someone somewhere is modeling those buttons like don't do it just stop like just cut cut

with the detail at the level that it is just it has no impact on gameplay it's we're not making a fully rendered you know you know 60 frames per second Hobbit movie that needs to be badly edited or are we you know like why do that just don't do the useless stuff and then spend like Raph is sort of saying like just copy someone else's you know eight slider thing and put it on everybody and see what that does and skip the buttons but yes I briefly I'm narrative I do want to keep moving but so kind of riffing on what Harlan was saying I think that anyone working in AI who wants to look at screenwriting education would find a lot of good stuff there we've all heard of the hero's quest and that there's that kind of systematic systematization of story but there's things like sequence structure breaking you can break like feature

films down often into sort of eight or ten 12-minute sequences where and each sequence is kind of a mini story I think it would appeal to anyone with a mathematical bent because this is kind of fractal aspect to it stories within stories within stories right like the beat of a single emotional exchange that unfolds over ten seconds between two characters who want to press the save the cat button to generate so that's one of the books to look at save the cat and I just thrown one last thing that I you know back to what I was saying before that human-computer interactions are difficult but why not do computer computer interactions and you know some of the the most interesting proposals I've seen are having a eyes that can talk to each other and you know maybe there's some pre-scripted parts but the kinds of stuff Robin was talking about that those a eyes can be sensitive to the flaws that the other ones have because they're you know they can be tested within the rigid you know area of what they know

about each other and not have the the incredible difficulty of some random human that can come in and ask them any question at all and they can respond to each other in ways that make the environment around you including the NPCs feel more realistic without you having to actually interact with them and have a dialogue and all the flaws of the fact that it's you know the fourth time you asked them that question they say the same thing they never say why do you keep asking me the same question you know if you can do that well with a computer to computer and PCs then you can sort of get around that whole limitation or design in a world where the narrative is about you know you're all fragments of save data at the bottom of a giant you know post-apocalyptic server somewhere where everyone's been uploaded into a Metaverse and you're just all fragmenting and everyone you interact with is like fragmented in some really fucked-up way and then like you're trying to figure out like how to get something done in that space but like people are like glitching in a way that makes their you know flaws exactly stop realizing it already there's ways

to write around it but it really like what I'm saying like skip the buttons I'm not talking about like the actual buttons I'm talking about all the ways in which we imagine there's data where there isn't data like what you what you're talking about is there's three vectors in this person that I need to figure out in order to get something from them and then move on in my quest and maybe they're not a person maybe they're an AI maybe they're a ghost maybe they're like a random you know half a robot half person who knows you know like they could be anything they could be another species but the point would be to like figure out how to lower the the valence that I need to think about in order to get at the thing that really matters and I I do think that that's it's like systems writing it's not writing and it's not systems it does both which is one of the reasons why people don't buy AI tools which I think we're going to talk about in a little bit so you know yes a little conveniently you the fantastic Segway systems you know it's a broad term right and I'm talking like you've it brought

up the AI director from in left4dead it was the most famous one change the audio change the the coloring on the sides of the screen it changed how the content was being delivered and the pacing and everything so that's a classic example the world is there anything either that you would go beyond with something like systems or it could do better or something that we haven't thought of that if the game we've hit a few of them here so I don't want to rehash but is there something we haven't mentioned that if the game system under was changing something and how we were experiencing it well what I think made the left4dead director so special and that I haven't really seen used again disappointingly since that was what ten years right is that the whole point of the director is managing your emotional experience right it's all about your emotional experience I'm gonna take you up and then I'm gonna let you have a lull and then right at the right moment I'm gonna fill you with dread and then

nothing's gonna happen so you're gonna relax and then oh my god there it is right it was an entire AI system built around emotional emotional experience and I can't really think of other games that have done that sense and that's where the magic is because the magic is in the players emotional moment to moment experience and you understanding that as a developer whether you're a designer or an artist or an engineer is key to you making something that's really gonna stick with players and so when any of the AI we talk about here I think at the heart of that system has to be that it's about the players emotional experience and not just about mechanics or pushing buttons or dialogue or anything else I think it's easy to do with scary things I think you know that that pacing curve is easier to do with fear narratives and you know really high action games and then I think to RAF's point we've leaned on other players and we've just I mean in many ways like the the the sort of popularity of multiplayer games is reinforced by our realization that it's just easier to let

other people do it I mean it like I wasn't joking like we couldn't have written that would have been as good to play with for journey as another player there's just no way and like it would have been obviously glitchy and failed all the time and that like you know the fact that it's the people read it as another a as an AI was you know I think it was a kind of a deep commentary on what people expect from an entertainment experiences they expect to be you know sort of lauded and have everything be about them and it's all about them winning but I you know to RAF's point I think we've leaned really hard on other players and moved away from that kind of scripting of experiences and fear is one that is actually pretty easy to guess because you can you can do things to scare people really easily we have very easy for your triggers it's like at our lowest level of cognition it's much harder to hook us with romance or to hook us with mystery or to hook us with you know some sort of innuendo and like kind of guessing because that's when you start to see that oh it would be very difficult to to really execute at that level so I really wish somebody would

build even just a basic soap opera yeah I was thinking that so bring this narrative narrative like telenovelas type stuff yeah had it been on when I play that and it was about romance games and about the tropes that they rely on in making romance games and those tropes we map directly to the emotional experience in the ups and downs and valleys and the red herrings and all of that right mysteries have the same pattern so really understanding those very human patterns I think are the key to you know mapping out the emotional experience of the player in your game warren in his thing narrative somebody used the term game master and then again in his answers to me this morning he talked about wouldn't it be amazing if we could create a virtual dungeon master capable of dynamically changing the gameplay to meet the needs fulfill the desires and so in in the left4dead you rightly pointed out it was a horror thing I'm not fulfilling the desires I'm saying is it time for me to hurt you again yeah but I think also why make the player the center of the narrative in this way we are fulfilling their desires and like making them feel like a

rockstar and like like I think actually the that's how that's what our games already do like winner winner chicken dinner like you're the one and then just like oh I want another one of those oh and another one of those oh Jesus Christ I'm drunken it's 4:00 a.m. and I have to go to work tomorrow damn you pub gee but like we're thinking sip fire yeah but so we're so good at it you know we're so good at giving people that experience I think actually one of the harder things is to do the stuff that Alyssa's talking about you know like I'm gonna let's try and make it happen build something a little bit softer with a little less resolution in the graphics but where the system is really trying to tell that story yeah I think so I mean I'm totally on board with cutting the buttons and and and more than the buttons frankly again I think it's not an accident that Sims Nintendogs and many other games that achieve emotional connection use iconic communication use different levels of graphics resolution right we have chaste fidelity yeah and all that's done is create a large gap

between visual fidelity and emotional fidelity and you know today we have servers for these games that are literally thousands of times more powerful than what we had you know 20 years ago like insanely more powerful and we're actually using the server CPU cycles on rendering the buttons on the server so that we can do bullet collision against them which is like hey why because it's a hard problem people get like ooh that's a hard problem let's do it you know it's hot it's so not actually this most of these things aren't hard problems when we talk about recognizing you know emotional facts about human nature things like body language or you know in a minute we're going to talk about content generation and recognizing things like there are patterns in architecture or you know whatever the fact is most of these things if people are just willing to you

know venture a little bit outside of the field of game design have been studied by people for hundreds or thousands of years they actually already boiled down into convenient mathematical equations that could very easily be modeled and generated right the idea that oh well how do we make a pet react or how do we provide narrative or whatever okay you know you want to start studying procedural narrative gosh go back to Vladimir Propp and start there and work your way forward there is such an enormous body of this material out there it isn't that we can't go find this stuff to build systems around I agree it's that we are choosing to build other systems mostly rendering the ones that interest the developers personally right and there's a reason that the sims and then 10 dogs were so successful it's because they are a little beyond what you're saying it's because they are so accessible and the average person can't attach to them because they were made by diverse teams when I worked on the sims it was it was 5050 it was the first job I had in the

games industry 50% women in the studio I've never worked anywhere else that was 5050 not even my own studio is it that good like people from all over the world fans that had loved the sims they just came right into the pipeline and became creators like you know it I can't say it enough and like then the narrowness with which our schools train us in computer science and AI the narrowness with which we create the community of people in this room affect the breadth of the things that we make and like I think it would be interesting to think about the systems that you're all part of and how you could broaden those systems and make them more diverse and more inclusive so that you would get different perspectives on what was interesting yeah so Dave we need you to take all the a programmers and give them a liberal arts degree please you are here standing in for all the AI developers so yes on your Edgewood College III think actually by the way

before we get too much farther it's time to give Dave credit for you know how many people here at their first GDC okay so we knew there'd be a lot of you and you've been really attentive if we had to make up some you know presence who who really wasn't here but kept saying these saige things and he was kind of this ghostly thing and he was the narrative summit and wouldn't the narrative come up with him and call him a Spectre so you've all fallen forward really well I love it I probably got some more of this Warren Spector person but I'd love to hear heard of it I can't I can't wait until he watches don't break character I love it alright so moving on we've I do want to blast through this section on content generation there's been a lot over the past number of years even here in the AI summit about procedural content generation which of course means so many

different things whether meat you know art styles or texturing or creating buildings or you know even just it started by roguelikes we're gonna create random mazes or something like that but it's gone beyond that and a lot of the things we talk about it well how do we create you know these huge worlds without you know artists and designers sitting there and doing you know every single building or copy and pasting you know five thousand of the same building well I feel that there's something else that we could generate it what is it that you in your design set god if we could only just push a button and have this stuff appeared what would it be one of the coolest things I've seen with using AI for content generation was the the deep dream stuff that allows you to take you know one picture and combine it with usually you know some artists some abstract artists and you get some beautiful mashup of those two and imagine if the structure of a level was

you know the same for everyone but each player could pick their own art style and have that be reflected in how that level looked you know it's something that we can individualize right now and it's one of those things that computers do really well that you know doesn't get into the sort of squishy stuff that we're all you know having trouble doing yeah I mean just if you think about like a basic RPG where you pick like okay I'm gonna be the goth elite a costume person then just like everything in the world it's got candy cane stripes and ribbons on it you know like just make everything cute you know but dead for me a lot of this is about making a data rich environment right because again it feels to me like we basically settle for worlds made out of stage props and cardboard most of the time and to me one of the big advantages of AI is actually that it's basically its patient in that sense it can keep recursing and adding detail

that is actually real detail right so to me it's it's kind of obvious and trivial that we can procedurally generate better trees using AI or that we can even procedurally generate a better forest using AI sure fine but that's just kind of linear extrapolation off of what we already do you know it's like speed tree forest addition whatever and what's more interesting to me is can you do that but in a way that is data rich instead of just going for it no let's plot thousands of trees how about when I go look at a tree it's way more detailed now there's an ecosystem in that tree now there is a burrow in this one and you know the Gophers live under it and there's a woodpecker over in that one right that's the kind of detail that frankly we will never ever be manually able to do right we just can't afford it it's ridiculous and it's exactly the kind of intelligently constructed realistic responsive environment you want the woodpecker to know there's a

gopher right like I'm not talking just shallowly again it's not just splatting the textures it's can we actually create a data rich environment right because when we build world's today we build really big worlds that again if you strip out the graphics actually have very few objects in them yeah we actually do not make dense environments when our notion of a dense environment is even Skyrim or or Battlefield or something like that and you tally the objects you actually end up down and like what three digits then the objects that actually interact with each other is nothing it's like set pieces are made out of cardboard yeah testing lis rendered to cardboard oh yeah but what would a I could do even in a way that like because it's it's kind of a fractal or segmented thing like we don't actually need each tree to know about each tree but we could do something really interesting within the tree if that makes sense right each one of them could have variations and especially if

you know if you move away like I said how many of you have seen annihilation any of you okay so like amazing don't see it before it leaves the scene Solaris or stalker it has some similar themes so the old Tarkovsky movies but you know there's a fundamental notion of like humanity and like what we try to model and like what do we what do we hope for with contact like going into outer space and like Solaris is a perfect example you get to the planet the plan is sentient but the only way it can communicate with you is by making up like vague approximations of what your memories are and then making those physical for you and all that really reifies is the fact that you miss being on planet earth so like the whole point was like you left an egg wish you could go back but when you do while your family will be dead right like there's a thing that we do where we recreate the reality that we see right now if you the tree example is perfect just don't make it a real tree like make it that you go to a foreign planet and then you find an object and when you look inside that object there's more objects and you look inside those objects and then like come up with the design of mechanics that are like interesting from the exploratory

perspective that's very similar to when you're a child and you wander out your back into your backyard and then into the woods behind there and you find a stream then you look in the stream and you see the tadpoles and then you poke the tadpoles like think of it from a mechanical perspective not an aesthetic perspective and then you get the dynamics that are so interesting I think we over focus on the aesthetic qualities because that's it's it's short-circuits the kind of creativity that you have to do as a designer and like if you partner with a designer who has the kind of abstract imagination about systems and mechanics and how they create dynamics you can move away from an aesthetic that is really costly like the buttons and move towards the tree but you just have to abstract away from your natural sort of you know like when you first pick up a pen you're like I'm gonna draw a tree the tree that you imagine drawing on the paper is like like Raphael drew it or something you know like you imagine being able to draw like Leonardo da Vinci but the truth of the matter is is that you draw like a kid and like the tree is still a tree it's just a kid's version of a tree so like if we made a

kid's version of that experience of finding the tadpoles that journey is a really fun one but doesn't have to be here on planet earth it wasn't to be this stuff it can be abstractly related and I think that that's the content generation is about I think getting away from this notion of reality and where does the notion of experience we're down to about five minutes and I wanted to play this next clip because it actually does speak to the difference between the the set pieces the arts of the stuff that the rendering the world that we're creating and what would be the equivalent of that this comes from the critical what is it called a critical play project which is a whole bunch of video clips of a variety of designers I think some of you have actually been new and this is from Clint Hocking and he was talking about his experiences in Far Cry 2 and something he encountered in his own game and I'll go ahead and play the clip here

all right I love that because we've all had that experience in games where it's like oh because of the sound and the and the visuals and you know this it made us

realize it's the the suspension of disbelief that we all talk about in this my question is okay I didn't want to get wet wait I'm not gonna get wet that moment what would be an equivalent feeling that AI might provide give me just a little vignette of if I was in this situation I could not stop myself but from feeling this way I didn't want to kill it oh yeah I just didn't want it I couldn't put it down can I can I piggyback on that was nice to go first because mine's such a downer and it would be too and it highlights what I think is a versioning problem for us all as developers of action games and some other kinds of games which is that I think we're rapidly converging on the point where because of our representations both in terms of assets and animation and dialogue and visual fidelity where the violence in our games becomes intolerable to us and so my

answer to this question was going to be like that the characters react to the violence in our games in ways that feel more authentic because right now they either don't really react at all because they're tough guys and they've got to keep fighting you or they kind of do these kind of pantomimed over-the-top hysterical reactions we've all seen that that animation right character doing that and then running away but I think I'm fortunate to not have seen very much violence in my own life but through the representations of it that I've seen maybe in storytelling or in documentary evidence human beings reaction to violence is much more complex our reaction to trauma is often very unpredictable it either looks like no reaction or it looks like something unusual and unexpected and so I think there's a cluster of hard issues for us here it's not a complete downer because I think that it's going to inevitably mean that we have to steer in the directions that we've been talking about mainly for the last hour yeah towards other kinds of subject matter

other models of human experience right and that is the solution to build on what Robin said before about caring about you know damaged characters if you are responsible for creating post-traumatic stress disorder and some character you actually care about you know there's sort of a an eco style game where you're you're you're really responsible for someone else and things that you do maybe just to see Oh what would happen if I blew this up and all of a sudden this character doesn't trust you anymore because a terrible explosion happened when you you you know use your weapon that might be an interesting way of you know very simple ways to to make you care about some of these things I think maybe the difference between what you said originally I didn't want to kill it yeah compared to oh my god I didn't mean to kill it look what I have done and having the game sell that to you of you like what Noah was saying oh oh I have just affected well this is something yeah

this is I mean I think this is the beauty of games like Shadow of the Colossus and eco is that they do and and you know last Guardian they build a relationship between you and a character I mean yes the cameras suck and like a lot of there are a lot of problems of those problems but you know I mean there are definitely like control issues in those games which you know usually related to budget and timelines more than anything else but like you know those games do really focus on these these narratives were the surface of yeah your control and your power is it's a gambit between yeah conscious reaction from the system or an unconscious or unwanted consequence I do think that that's to good writing but if you could really build an animal that was an enemy that you also could create you know some kind of a relationship with through some sort of hardship at the end like it's going to eat you because it's hungry and you have to put it down but you don't want to like that would be a really into and then you have to make a decision at the end do you sacrifice yourself and let it

live or do you take its life like that's a that would be a really interesting game to play and I think it would create a lot of conversation about what you know what games are about and empowering users I don't think these things are impossible to imagine I just think that we're focused on the wrong thing a friendly robot named Floyd who sacrifices himself for you like we've been doing it the whole time you know Chris Crawford actually in the art of computer game design there is a like a direct quote about this he's just like the more fixed bits there are in your game the less of a good game it is there should not be fixed data in your game it should be dynamic it should always be changing the stories should be changing the characters should be changing the art should we change me he said this in 1982 you know kind of the corollary to two SIDS came as a series of interesting choices yes interesting ramifications or responses yeah responses so um orange he says the easy answer is just an npc would hesitate to go out in the rain - I imagine it whoa jeez it's raining and have that be unscripted uh but or

encourage the player to go out Oh would you just get out there I could see that but when it's another way than the AI would that just blow you away that oh wow that just happened well the thing the thing I don't like about Warren's answer is that it's so it it comes back down to the player at the bottom yeah and okay I want to D Center the player frankly right sure the reason I love games like the stuff done by David Reilly right our mountain and everything and so on is because they make you realize well you're not it okay inside the game there is an inner life inside the game there are systems there are parts that are moving there is clockwork and magic and soul and all this is going to go on whether you are here or not okay and so to a degree I almost go the game that makes you feel guilty over killing something or the game that's almost just that's that's the baby step to getting you to realize that there is an inner life back there yeah that's the step beyond would be I want to be proud of that thing I want to you know feel good

because that thing did something without me right that it's alive that it's doing things on its own right I wanted to apparently do them layer yeah the AI the systems there about conjuring a kind of life and if that's the case then the thing that would blow me away is actually not just the realization holy that's alive but it's doing things I didn't expect that weren't about me and as an individual with the point of yes kate and sergio earlier today in this very room had a really great example about navmesh that's flagged for keep off the grass signs NPCs will only go on the grass under certain circumstances no professor unless they know all the words - this is your land we are over time and I did I'm just not even gonna put this up but for a second Harvey Smith had jumped onto something

talking about relationships and everything but this bit right here is every time we start a new project we have to almost start from scratch we need to kind of I'm not talking about you know the coming up with the one way of doing everything but he touches on the fact that why are we starting over and over and over and over again and because of that becomes prohibitive because we have we don't have three years to iterate on this system so there could be a quite simple answer to this if I can dive straight in because we lack something that other creative industries have which is consortiums or consortium perhaps you know where the movie studios get together and they agree on technical standards around like what the cameras are going to do and then and then the hardware manufacturers kind of manufacture that and we have the beginnings of that with the kind of the stability of our hardware platforms I think we'd like it right now except you know in kind of part of his work accepting rendering world representations are all different so how do we plug into all that until we have

to establish just the AI right how do we represent stuff right right and it is happening right I mean in place it happens in places like this room where we agree on the name of something like a piece of navmesh and share our best practices but I wonder as we go forward we could be benefited by getting together on an industrial scale do more industrial organization real quick give me two sentences your final wishes if you had your AI team me all of the AI programmers in the world and you said I need this out of you because so I can make that thing that I want to get what would it be I'm here I'm you know thousands of AI does I'm putting I want you I want you to help me build a very intimate romance about two strong black women who fall in love okay well I'm embarrassed to say

that I just and maybe this is the most important thing I realized that all my experiences designer has led me to think don't solve the problem I'll just have to design around the limitations so maybe that's just a statement you know not to make enemies but the limitations of where we're at right now that's why we're here is we're trying to figure out what what are we missing so um I'll go I help me give me tools to help me illustrate in my games the way that human beings struggle and we're all just trying to get through I think there's a lot of good literary meaningful stuff it's going back to the empathy thing of the characters you being able to be valuable and connect to a lot of different things but yeah more human stuff okay so almost all games are online in some capacity now and gathering metrics but certainly and especially for all the games that actually are online and gathering metrics now please take some of that effort you're using toward

gathering metrics and looking at play patterns and please use them to help the player experience and not just to make money okay yeah absolutely I want it all I mean I want no who's this guy here now I want I want to have the the tools the machine learning the AI the rich data environment to be able to make a single probably online connected universe that we are in fact simulating down to the point where the little pink alien Gophers on the 13th planet around that particular green Sun which was entirely procedurally generated actually have a history and care about one another and I wanted to have all of that stuff and have all of it be alive specifically because I want to drop a player into that world and have them realize as they play that they are touching lives messing with things

that are alive they are trampling grass that struggled to grow goddammit and you're stepping on me again to realize that when they build their virtual cities when they conquer their virtual enemies that they are being colonialist about you know all of those things I want them to realize that in their daily lives they do the same thing in the real world because I want the AI and the machine learning and the the code and the systems out there to hold a mirror back up to us as humans I want them to use that space as practice for being better here so give me all of it so that people can wake up realize what they do day to day [Applause] I don't have an end slide to put up on the screen and I just made this up but tell me if this resonates not more of the same more of the different not just

better NPC AI but better AI NPCs and a better game AI that makes the whole game better slash s there s slash game slash world okay yeah okay right replace game with world and I'm on board with that lesson alright excellent again sorry we're over time but was this informative and inspirational and real quick who here said some point in the past hour and nine minutes said I have an idea of what I could do to possibly start solving some of these that's why we're here go forth and do it but be back here at ten o'clock in the morning from the second day of the AI summit thank you folks appreciate it