Adding Life to your Level

[Music] thank you hello my name is Seth frozen please you know shut off your cell phones try to squeeze towards the center so people are coming late can find a seat and do remember to fill out your GDC surveys that you'll get and in your email when after you've checked in alright so as I said my name's Steph I'm a systems designer so this is a level design workshop why is there a system designer up here well for a couple games that I've worked on my focus has been on AI behavior outside of combat on BioShock Infinite I was part of Lizz squad which is the team primarily responsible for Elizabeth was our companion character and on that project I was kind of more of a character designer I was using level design and systems designed to make Liz feel alive and right now on mafia 3 I'm working on what we're calling the living world it's the population and activity of our

version of New Orleans in 1968 alright let's dive in so games are interactive right we're not in the business of throwing something up on screen and just letting the player observe it we want to draw the player into this experience and make them believe in it believe that this space is part of a larger world an environment art is a really huge part of this but it's static and it mainly deals in history so how can we add some dynamism to this experience and one solution is the solution I'm personally most excited about is to add life to your level actors inhabiting the space who interact with their surroundings and react to changing conditions and you know just generally exhibit lifelike behaviors now the lead actor of course is the player and they're probably really the only actor that can actually be considered to be full of life but we can create the illusion of life for other characters in the world and ultimately that's what I'm talking about when I say life in your level or to put it another way a performance of life character is whether

they're humans in life or AI and games are performers the person sitting next to you no matter how well they know you has no idea what's going on in your head default or for that matter that anything's going on without performance were empty shells and gaming is definitely start out as empty shells it's up to us to film fill them up with that yoky goodness and create that illusion of life fortunately humans have a pretty good sense of how to how to perform life and on top of that there's actually only one person that we really need to convince that our AIS are alive it's the person holding the ctrl excuse me the controller and that's important actually game AI above all else should be player facing if the player doesn't see their performance they might as well not be on stage at all and that's where the bad news comes in if the performance ever falters the audience's suspension of disbelief just evaporates so right

out of the gate scripting an eyes behavior and reactions is pretty much a non-starter and believe me we did try on BioShock Infinite for a long long time Liz existed as an AI that was really only functional in the most basic sense of the word she would follow along and take cover and that was about it it was painfully apparent that we had a long way to go before she was a really believable and relatable human character so we picked an area of the game that would service her crucible it was the beach in battleship Bay and our first attempt consisted of a series of triggers that when the player hit them Liz would go to her next mark and at each mark she would play an animation the results were abysmal terrible the biggest issues were how few things she had to do in the space how poorly she reacted to any moderately irregular player movement and how much additional scripting we would have to do to make her respond to anything going on so after demonstrating to ourselves that yeah there's just too many rules to

being human for scripting to be a viable option we decided that the only path forward was systems right here I've got a video of Liz making use of some of these systems I'll get into the details of them a little bit later on but we could make suggestions to her the system's decide what she was going to do rather than being prescriptive about her behaviors and this is better for reproduction and robustness standpoint but it also maps pretty directly on to real-world examples of life and humanity our lives proceed according to a series of rules and patterns and humanity is born of these things it's a collection of tiny hints to others we can create a set of systems that act on a character and these systems describe these rules and patterns and then these systems continually offer options to the character for the options that they have to do their behaviors the result is a performance of infinite duration and that's kind of what we're going for because we never want that performance

to fall apart so the next step was to figure out exactly what patterns of behavior we wanted to give Liz and how we're gonna spotlight her performance and as we were figuring this out our main reference materials were theater Disney animation and our own everyday lives to organize my thinking when I'm kind of figuring how to give life to a character I break their behaviors down into five categories activity idle conversation locomotion and if it's in the game combat I break it down this way because there's not one single thing that we're gonna do to convince our players that our a is alive it's this wide array of behaviors and actions that all some together communicate life now when our goal is to breathe life and into it and into an entire city rather than a single character the activity and idle category is kind of blurred together a little bit but I think the categories are still useful for a kind of figuring out what your approach

should be all right let's talk about activity and the I's activity are the behaviors that demonstrate that they take an active interest in their world they belong in this world and they have desires and intentions of their own for Liz we call this system smart terrain on mafia 3 it's called world interactions but it's basically the same thing in both cases and it's probably something you may have encountered in your own work there are smart objects that are basically a big ball of data animation code art assets sound effects vo and visual effects they're game objects that have instructions for the character and conditions for use ok let's look at a few examples of Liz's activity let's do a show of hands for those of you have played bioshock infinite who remembers when after waking up on the beach you find Liz dancing on the pier and she invites you to join her most people so that was that was scripted 100% of players got that who got that far saw that moment now who

remembers Liz trying to pick up a medicine ball or skipping stones on the beach way fewer and that's actually what we want these were entirely systemic but they feel almost scripted they have vo and you probably only saw them once they're high-priority smart trains and they have long cooldowns and they're either placed on the critical path or have pickups near them to entice the player to walk over there okay last example who remembers Liz making shadow puppets in the lamplight and soldiers field even fewer great again this is entirely systemic but it's a little bit off the beaten path and lower in priority and these are all great little moments but ultimately they're fleeting so how can we keep Liz engaged over a longer period of time well we just cover the levels in these smart objects and then we let the system's decide how to strike a balance between active and idle this is a video that shows the smart

train placement and a level of BioShock Infinite all the green meshes and diamonds are smart trains so wait a second we're just gonna saturate our levels with smart train then let Liz decide what's gonna happen yes this is gonna work out at all you have to be comfortable with a little bit of uncertainty not everyone's gonna see everything and for each of these smart trains sometimes Liz will use it and sometimes she won't you just have to be okay with that a guaranteed scripted moment is the enemy of interactivity and variety and those are the things that are really gonna prop up in a eyes illusion of life that doesn't mean that every single action the AI makes has to be left up to the system but overall it will be a more convincing performance if the majority of it is and they think as a player you really do feel the difference between a scripted setup in something that happens opportunistically something that the character decides to do and decides to stop doing - Liz will dance for

eternity if you let her and after a few minutes you kind of stop leaving her and her as a character you remember that she's just an AI in this game that you're playing but if you stumble upon something like the skipping stones or Shadow Puppets it feels like a stolen moment it's private and special in some way and that subconscious notion that not every other player saw you just saw I think makes it even more personal and make the makes the character even more real fortunately whether or not a given player sees any of these systems riven moments isn't just dumb luck Liz is not picking things that random out of some weird and visible smart turning hat she's using systems but that doesn't mean that we're surrendering control in the name of robustness it just means it requires a different approach the system had rules for which smart trains to consider in each smart train has data on it the factors into that decision how desirable it is hues what direction it can be used from and how long it goes on cooldown combined with the actual

placement of these objects in the level the character is able to make the most natural decision possible in all instances for what activity to perform and by setting up all this data and tuning our selection criteria the system drives the characters performance across the entire game skewing it closer and closer to that ideal of a polished hand scripted sequence for Liz all of this decision-making was built into her brain but she's really only concerned with picking the single best option at any given time I'll mafia 3 we're dealing with a whole city so we need again a little bit of a different approach the interaction manager is our equivalent to Liz's AI it's a big brain in the sky that collects all of the world interactions in the city and based on similar types of data picks a bunch of world interactions to turn on and populate with people from that neighborhood once the manager chooses a world interaction it checks how far away it is from the player and assigns it to one of two buckets out of sight or in sight out of sight is for interactions

that are far enough away that we can spawn an AI directly into it and the player won't notice and insight is for ones that are closer to the player they'll try to capture people as they're walking by and often will restrict an interaction to be only available as one or the other if we say something's out of sight only then it allows us to bring in large groups of people or large props like a band or a second line in the street we can also use out of sight interactions to put a is in places they can't or shouldn't walk to like a balcony or a porch on the other hand when we limit something to being in sight only it gives the player a chance to actually see this moment before it's complete this is for things like buying drugs or well trying anyhow the manager load balances the relative density of out of sight and incite interactions making sure that there's always some of each happening in a given space the out of sight interactions ensure that when the player gets there there's already some level of activity

activity happening there you never get that moment of oh the players here quick everyone do something so then the insight interactions again give that player that opportunity to actually observe people acting with intentionality and behind the scenes the manager is running timers in the background periodically telling people to move along so that we see more intentionality and get new slots open for the interactions to fill new AI is doing new things because you know if you arrived at a location and nothing changed around you after standing there for 20 minutes all the people around you would start to feel like animatronics Disney rides deal with this by forcing you to move out of the scene and into the next one but with the player in the control of the camera and their movement we don't have that luxury so instead we have the manager systemically create a changing scene around you unless of course we need a static scene for gameplay reasons unfortunately we found

out that just having all of these activities available isn't quite sufficient if Liz reacts to seeing a songbird toy it doesn't do anything to make her feel alive the player doesn't see it staring at a wall it may as well not have happened our a eyes again are performing they need to be onstage and for games that means in the camp Rustom but we don't want to grab control of the camera just to make sure they see these moments so how can we encourage the player to notice these actions the character is taking well we make them noisy when Liz spots something exciting to her or for the player to pick up she'll call it out and when she sits on a bench she sighs with relief this can be applied to open worlds - like in Mafia 3 you hear a protest leaders vo ring out above the din of the city and when we hear a sound we just naturally want to turn and investigate the source but worst case if for some reason the player doesn't do that we're adding some life to the soundscape alright let's talk about idle idle behaviors are not

terribly sexy but oh my god they are important when an AI is idle is probably when they're at the highest risk of not feeling present in the world and if those quiet moments don't work there's just no sense in bothering with the splashy or stuff so we need our eyes to stay present by constantly implying that there's something going on inside of their head they can examine their surroundings find a place to rest fidget or just generally exhibit their internal state with their body language by saturating our levels with smart trains we are able to ensure that Liz never has to go too far to find something to occupy herself and when she does decide to go and like sit on a bench or lean on a railing she'll look at the thing that she's moving towards because when you're intent on something you look at it it's just a reflex for humans but we have to remember to build little things like this into our a eyes behavior because those are the things that make them feel alive and when she's not using smart train liz has a look tracker system that

kind of kicks in and keeps her present she will survey the space and pick an object to look at and the system boils down to a pretty simple guiding principle what do humans look at and why and for that matter what does Liz specifically look at and why like so many other human behaviors this principle can be described by a set of rules we tend to look at things close by and we won't look at things that are really really far away we don't look at things that are behind us or way above us we do look at other people especially if they're talking but we won't stare at them we look at things that make us feel something and unless it's totally fixating we tend to not keep looking at the same thing over and over again or for a very long time so with these rules in mind we decided that we would make these look at markers and place them all over the levels and attach them to characters and sound events as well each marker the eyeballs you see covering

this painting here of lady Comstock had two bounding boxes one that described what part of the object Liz could look at and one that described the area that she had to be inside of to be able to look at it in addition it had three important pieces of data how interesting it is so that that's both how higher priority it is to look at and how long it should hold her attention for how long the cooldown lasts and how she should feel while looking at it happy sad angry etc so by covering this painting with all these eyeballs we're basically telling Liz hey you should be pretty fixated on this thing so in addition to emoting when she's looking at stuff was actually had a like full-body emotion system that would allow us to change her animations and body language based on the narrative events as you progress through the game these are just a few examples but you can kind of see how they change her expression and her base pose and her fidgets and also how willing she is to

look at the player if she's pissed at you she's upset she's looking off to the side by making a motion variance as well of some of the really common Idol style smart trains were able to expand this reflection of her internal state to include some of her actions like if she goes instance on a bench in a huff she's pissed all right so what is this Idol category it look like for an open-world game like mafia 3 well it does in kind of let me let me explain I mentioned earlier that activity and idle blend together for this sort of game I just don't really find that it's worth drawing the distinction inactivity and idle for the characters in an open-world game it's not that we don't want characters doing idle behaviors it's that you're not spending enough time with these characters to really see the difference if you were to take a horizontal slice of Liz's behavior across the entire game of BioShock Infinite that's basically what we want our city to look like at any moment so active idle whatever we do

need the more mundane stuff so that the more interesting stuff shines but on an individual basis AI is basically need to show up do a micro performance and be on their way by the time you come back to them they're probably gone and our goal here is to make the city feel alive bustling in short now we have probably a hundred or more eyes in the space and that means that we can't exactly afford to have like a super intense AI brain helping each character decide if they should be doing an active or idle thing so we approach it from the other end by having objects seek out subjects billboards will broadcast and look at me event and nearby AIS will evaluate whether or not they can do that and if so turn their head world interactions upon people directly into the interaction or check passers-by to see if they can find the right person for the job and there are global factors that are affecting this search for a participant as well if it's raining for example a

reading newspaper interaction will just stop looking for someone because you don't read a newspaper in the rain we have global cooldowns as well to kind of keep the city from becoming a broken record where you have racist harassing who constantly or a busker every five feet on Bourbon Street so Isle gets a little bit blurry four characters in an open-world game but I think it's actually crystal clear on a neighborhood level a neighborhood idols by having an identity a specific population and type of activity that goes on there for each area the city we're setting up data to determine the demographics their race income age gender the density of pedestrians and what types of people they should be office workers tourists junkies hippies you name it the type and frequency of props that we give to pedestrians office workers should have a briefcase maybe the density and types of world interactions that can appear which is all the activity and sort of active idle stuff and last but certainly not least

whether or not pedestrians have a chance to spawn drunk as you're moving through this area droves of civilians are drifting past you and you're seeing enemies doing criminal activity or wiling away their day you just kind of gather a sense of what sorts of people are hanging out there and why and regardless of whether a given AI is using a world interaction they still need to be present and reactive re eyes are constantly sending messages back and forth to each other and evaluating whether or not they should run some reaction behavior these reactions rain from seeming saying hello to someone they're walking by in the street to freaking out because the players waving their gun around and a large portion of these reactions are aimed at the player and the actions they take because those are in currently higher visibility which again they're performing we want them to be on stage but that said a pedestrian a neighborhoods pedestrian population and their reactions they have especially amongst themselves is also important because it creates this murmur of life behind the activities of the

neighborhood and without this background noise we run the risk of the player feeling like Truman everything constructed for their benefit all right conversations when we're in a city conversations contribute to that background noise but there are also an important moment of performance for our characters firstly characters should focus naturally when they're in conversation and humans basically have two modes of attention when when were participe in a conversation one for when you're the speaker and one for when you're the listener when you're listening to someone speak you spend almost all of your time looking at their face you only let your eyes start away briefly and every so often so that the intensity of your gaze doesn't become awkward your intent focus communicates that you're engaged in listening they're words but when you're the speaker you actually spend most of your time not looking at the other person you look near them or past them but you'll only periodically actually look at their faces to kind of clue them in yeah I'm

still listening I'm still talking to you or sorry still where they're listening and that you're still speaking to them so besides getting their attention right we want to make sure that their movements are synced up with their vo and the best solution on both projects has been to give the characters the very neutral base pose and then tag the vo with animation it's faster and more flexible than full animation and these gesture animations are also opportunistic so they can drop away if a higher priority animation needs to play if the character needs to start moving the gestures fall away and the head tracking remains show goes on for open-world games like mafia 3 there's actually one more piece of conversations that we really need to be worried about and that's the content of those conversations variety here is key if you hear the same dialogue over and over and over again the whole city plunges into the uncanny valley and of course there's some hard limit on the number of lines that can be written and recorded and mastered and implemented so we mix and

match the players gonna be spending a lot of hours and in all likelihood they're gonna hear the same line twice we just can't beat those odds but if we structure our dialogue as a B style conversations then we have a pretty good chance of at least having a different response the second time they hear that line so by organizing our ambient conversation lines into these two buckets where a lines our statements and our categories by the types of responses that are reasonable to have for the other person positive negative neutral or laughter and then the B lines are played based on which bucket of a line we selected we're able to get a little bit better variety a little bit better bang for our buck locomotion locomotion is mainly an issue for characters they're going to be spending a lot of time with form or transient characters like pedestrians in an open game they mostly just need to not look

like idiots as long as they avoid other people and avoid obstacles and stop to wait before they cross the street we're pretty much golden however for companion characters like Liz where you're going to be spending upwards of 10 hours with them it's a little bit more complicated alright so for for that kind of character we need to get them on on stage right locomotion provides the backbone backbone of getting them onto onto the screen initially we had was following the player everywhere they went but in early play test we were finding that the player would lose track of her basically immediately and then spin in circles trying to find her and we didn't want to just teleport her around to keep her reliably with the player that creepy Watson effect kills her as a living physical thing immediately we would still teleport her if she got super behind but it's kind of a panic button so instead we figured out how to get her out in front of the player and you'll

see that system here in this video our solution was to calculate a golden path between the players current position and their goal because Liz had this advanced knowledge of the rough path that she would have to take she was less tightly bound to the player she could be a little bit more tolerant of changes to direction or speed and she would always know where to go if she needed to walk out in front of the player and wait for them we called this approach her goal side behavior and found that in addition to keeping her on screen really effectively it turned her into an implicit quest arrow which was a nice side effect but more than that the goal side behavior allowed us to pick really good candidates smart trains for her performances and as you can see in this video if a smart train is close to Liz or it falls inside of that cone then we can be reasonably sure that the player is going to see her use it now smart train is outside of that Konkan still be used especially if they're high priority but we're trying not to distract her too much from her efforts to stay goal side because that's priority one because the

player is facing forward they're generally going to be moving towards their goal and at worst if the players speeding along and Liz can't stop and use this smart train she'll stay alive by looking around at the various look at Parker's in this all right this final section is probably also the most familiar to a lot of you combat there's a ton to talk about with combat AI but I'm short on time and there are definitely AI programmers who could do a better job so I'll just talk briefly about non-combatants like Liz at first we were also trying to keep her on screen during combat she would run through the line of fire just to take exposed cover she seemed completely unaware of what was going on around her quickly though we realized that during the fight the player doesn't really care about where liz is that are way more interested in putting bullets into their enemies so we changed her behavior to just kind of get her safely out of the way above all else that doesn't mean she's off the hook though the danger is

real for the player and she should reflect that the way that we got her that kind of reflect this sense of danger was she would sort of peek out from cover periodically flinch if gunshots landed nearby and if her position got compromised she would leave it and run with her hands covering her head trying to protect herself she also had a few functional combat behaviors she would call out dangerous enemies and toss ammo and health to the player at critical moments she had your back and better yet we were actually finding that players would invent stories about what Liz was doing when she was off-screen she was scouring the battlefield and finding resources just to help him out never mind that she wasn't doing any of that her performance was overall convincing enough that people were augmenting it of their own volition exactly what we want and the final piece of Liz's combat performance actually comes after combat is over we were finding during development that the

combats were lasting long enough that all of her smart trained cooldowns finished and she would just pop out of her cover pose and saunter over to some world in sorry smart crane totally not fazed by the carnage she just experienced so to address this we created a special motion we called after combat emotion it would make her a little bit somber and prevent her from using smart trains for a little while much much better all right quick recap if if a world with it out AI is a dead world then so too is a world with AI is that don't convince you of their humanity and humanity is not a declaration it's a collection of hints taking an aggregate that make it apparent to others that were alive to figure out what these hints should be observe your daily life notice the ways that your attention Falls is you're in conversation or looking around your room notice the way that you move with purpose towards your goal that you always find something to do even when you're doing nothing at all

and each of these individual behaviors whether they're in action and idle and conversation and locomotion or in combat can be described by systems and these systems communicate to the player through body language vocalizations and a strict adherence to the laws of physics the player develops a sense that this rounded by living things these are the systems that are the heart of your life in your level and they never stop if an eye eyes performance ever falters they die in that moment always be performing alright that's it thanks everyone for coming if you do want to see some other really great talks that are about similar topics check these three out really really they're all about in our lungs are really solid and thanks to John Abercrombie and Sean Robertson to the guys up there for sending me their old Liz videos have a great ADC

[Applause]