Stop Fighting! Systems for Non-Combat AI

[Music] hi everyone how's it going um before i kind of get into this thing just to get take a pull from the room how many of you are video game specific ai developers like game ai okay a significant percentage which i would expect now how many of you who are game ai developers spend the majority of your time in something that you would consider to be combat oriented ai something that's murdering the player basically basically the same percentage of people almost the same number of people um cool i consider myself kind of an outlier i've spent the vast majority of my time in the industry working on non-combat systems uh simulation systems background characters the backup dancers you know that kind of stuff that's where i've spent most of my career and most of my time so that's kind of why i was super excited to do this type of talk the the desire for this and really the

catalyst of this came from a couple of different places uh the first time was in anger when i did a rant maybe would help with them on the right thing nope success um when i did a rant this was in 2012 many years ago where i said why don't we have more ambient ai characters in games right why don't why are we doing these like sort of looping idols why are we doing these random walks like we can do even just a little bit better so movie time i like movies so i'm going to show this little movie this is from pillars of eternity and we're just going to watch this briefly so here i'm getting jumped by some people in a town and i promise this is about non-combat stuff notice those people at the bottom right they're just hanging out like lightning is raining from the sky is it not playing oh no sorry let's try this again

yay oh no great that seemed to work well okay that's because i'm on double monitor here let's try this again there we go okay now those people at the bottom right so they're basically doing nothing right they're hanging out they're just doing their idle like lightning is raining from the sky fire people are exploding right it's like a whole thing and they're just like it's cool whatever this happens all the time it's fine this is like this is my pet peeve in games i see this stuff constantly all right let's see if i can somehow recover this thing no it moved this guy oh he's over there come back okay we're back so let's let's dig into this just a little bit right so pillars of eternity actually has a whole really interesting reputation system you can do all sorts of cool stuff with reputation i'm benevolent for example so that means

that i can choose more benevolent type of things how awesome would it be if me as a benevolent character those two people turned around and said this super awesome benevolent character is being jumped maybe i'll do something about that so that's the ambient ai and for all of those those of you who rose your hand about the combat side of things that that series of slides was for you um second motivation came from alicia alicia liedecker gave a rant two years ago about what she called artificial intelligence which i love and it's about why aren't we doing more of this kind of stuff and i'll very briefly speak to that and answer that because i was asked the very same question a senior producer came and said why on the sims you know working on the sims 4 why are the sims ai characters so much more expensive from a man-hour point of view than something like battlefield 4 well because my characters are on screen for a really really long time and have to look good because they're the focus of the of the character their focus the camera the player is constantly looking at them whereas something like

battlefield characters on screen for how many seconds that entire fight that you saw in that last video was 20 seconds or something your average fight is less than a minute whereas these deeper richer ai characters need to last for longer so that's your first kind of lesson about if you want to build deeper richer background characters be prepared to pay that cost because it's you know you can kind of fake some stuff but you know we are all sort of wizards in the background and it does take a little extra time so this talk is going to be t-shaped that's the way that i like to think about it i'm going to talk about a myriad of different techniques things that are in some ways understood and talk about how i piece them together to form these types of background type of characters and then we're going to talk about one particular system which i have not seen a lot of talks or literature on which is a scheduling system for scheduling uh ai characters in which would actually work for either of those types of things i've used it for deeper simulations and i've used it for small like background type stuff

the important thing to realize is that we are illusionists we're not going to actually simulate human behavior at least that's not really our goal right it has to run 60 frames a second so it's not about realism and that's not what we're trying to do with trying to simulate these background characters and trying to just simulate non-combat type stuff it's not about creating something that is real it's about creating something that is immersive things that are interesting okay so when i try and create this kind of stuff there's one giant button that i hit and anyone who's seen any of my talks or know who the the crowd that i associate with is probably going to guess exactly what's on the next slide but the big shiny button that i hit at the very core of my ai when it comes to anything related to this kind of nuanced behavior is the utility button um i use utility theory and utility behavior type things and utility curves and all that fun stuff you know there's tons of talks here i'm not gonna this talk is not gonna be about utility theory but that is the essence of it because it's really good at giving you really nuanced behaviors

um it can be i should say and really good at giving you emergent behavior which is i think what we really want to see that's kind of the magic that's like the the holy grail i think of a lot of this kind of stuff so let's think about that so how many of you uh out there use a utility based system in your ai a hell of a lot more than i thought to be honest um how many of you in that utility system uh do any kind of randomness whatsoever is it just deterministic as in max of whatever or is is there some randomness how many of you do randomness in your ai okay about less than half of that same group if you're going to try and get the kind of nuanced behavior i think you have to have some kind of randomness in there i think it's the only way that it works because if you choose max of whatever the decision is then at that point what you're choosing is a truly deterministic thing so uh there's many ways that you can do it the simplest way is a weighted random across the top scoring actions which is what we did on the sims 4. richard evans on the sims 3 did something very similar he just built a

different curve to push lower scoring actions lower and higher scoring actions higher to get a more sharp sort of uh choice but even just a simple weighted random across the final scoring actions is going to really is going to add that bit of nuance and i'll prove it to you i'll prove it to you with this example this happened to me i sort of recreated this but it happened to me just totally accidentally while i was just working on the sims so i'm like we want to test sort of invite in behaviors and you know one of the wonderful things about working in ai is you spend a significant percentage of the game just watching what happens so i went out into the street found a random person and said hey you know let's be friends enough and then come on in so what happened is they stayed out there and spoke about books uh for a short amount of time and that makes sense he's a bookworm okay then he went in and started reading and she did the same thing and it's like oh that's super cool right they went and they read together they spoke about a book that series that they love and they went and read together

of course none of that actually happened right that was just the emergence coming through that's a story that i as the player told myself about what was happening what actually happened is he got sick of socializing decided that fun was a that he wanted to score fun fun won and he's like okay here are all the things that solve for fun and he had a better than 50 chance to choose reading she had about a 25 chance to choose reading and the gods of the random number generator decided to smile upon me that day and there they are they're reading together so a big part of this is trying to make uh fit all of these things together that's gonna be a running theme of this sort of part of the talk all these little moving pieces fitting together so that was kind of magical for that i'm going to talk not super long about this part because i've already spoken about this stuff but stats are another huge thing right so all the different stats that you can attach to your game that are little things that i can hook into um skills relationships uh wants and needs the bottom one is from uh is like city

health and things like that right so any of these types of things that you want to be able to attach really they're basically any value that you can attach and they can sort of decay over time or do any of those types of things they can be attached to people or they can be attached to objects or they can be attached to concepts food on the sims 4 for instance was one of the things that um decayed so the quality of the food went down okay tags are the other thing i've spoken about this stuff as well um so are we lovers are we best friends are we soul mates these are just little bits of information you can attach to people and they can drive behavior and this is where the subtlety really comes into play for example if i'm a glutton or if i'm a bookworm i can generally score those types of actions a bit higher and it doesn't mean it's always going to trump those things right it doesn't mean that he's always going to choose to read it means he's going to tend to choose that it's those little tendencies it's the gentle push in that direction that's the thing that you're going for

and again for a nod for some of the people who are working on more combat oriented stuff over on the right uh i have the those are the actual buffs on my particular character when we were fighting those guys so any of these types of things these are all just tags tags that you can tag people with and i could talk forever on this i did last year so if you have vault access daniel brewer and i did a talk on knowledge's power and that's where i spoke in depth about this kind of stuff okay the next thing i want to talk about is relationships so if we're having all of these people what i think we're really trying to do is creating an ecosystem of different uh entities and different people that exist in this world and those people should have their own relationships how i feel about you may not necessarily be how you feel about me if you are kind of don't really like me that much but i super like you i might follow you around like a puppy um if you if it's if we like each other in the same amount then we may converge and and really want to talk to each other you can feel different ways

about objects this is my chair this is my bed this is my guitar or whatever so relationships can do quite a bit of again it's all subtle hits i'll give you another example no matter what that that sim chose to do if i went to go read if she was able to she probably would have sat next to me on the couch because we were friends so the posture scoring that says where do i want to go that feeds into the pathfinder that says where exactly what is the actual location that i want to be at we weighed higher for those types of interactions it gets even more fun when you talk about negative a negative relationship doesn't necessarily mean that i don't want to talk to you on the sims medieval we had a depth that we used so if i really liked you or really hated you i wanted to talk to you which meant that we got brawls and things happening in the tavern which was really cool so

that's the kind of thing that you're going for um if you're if you're either crazy or you already have a game that supports it influence maps are a really good way of being able to have more reason about the environment and all of that kind of stuff as well this has also been spoken about quite a bit by alex and by dave mark and various other people so you can check out those resources as well we're trying to do is we're trying to take all of these little pieces and fit them all together so that we can create something that's more than the sum of its parts all of these things working together will create this machine that allows for these types of emergent behaviors to come into play we're creating a sort of fertile ground so that emergence can come through and i think that's the secret of creating not believable but immersive non-combat ai and simulations and that kind of thing that's what we're really trying to do at the end of the day okay um we have to talk about lod a little bit i

think um if you're trying to create a big simulation one of the big things you have to deal with is a level of detail and there's tons of ways that you can handle all of that stuff the one that i want to focus on a little bit is basically statistical simulations and statistical planning and that kind of stuff if you have this whole ecosystem of um you know utility curves and all these things all working and measuring together you're going to create this this general trend of how of how sims or whoever you're doing like your character's behave your simulation behaves if this city that's over here is starting to decline and the player goes way over here so now i'm in low lod mode then that city is probably going to tend to trend lower and i can get updates from that city and i don't have to obviously simulate that so those types of um basically you're building a curve that you're just walking along as you go through and that can be useful for tuning as well uh an example is on the sims if a sim is in low lod mode then we're not

simulating their motives at all when they come back into high lod mode we figure out what time it is and then look on a curve that a designer built somewhere and figure out okay this is what the curve says they should be at and we'll tweak it a little bit either way those types of things can really help okay i'm going to this is just a few tools that i use for this kind of stuff um this is this is desmos but any graphing calculator can really help with these types of things laying utility curves on top of each other is the only way that you're going to really be able to understand how all of those curves line up together it's also a way for me to say okay so let's say that i'm a bookworm for example and i want to score these actions times i don't know 1.1 that seems good and you can play with and iterate without having to like dick around in the actual code itself here it is mathematical formulas and these formulas mean nothing here i literally just threw them in there but i'm sure yours will mean a lot moving on to the next thing sort of the next level up from that and

this is an often overlooked thing i think is excel i've done entire simulations literally in excel which is really cool it can be a little painful sometimes but i mean excel has a random number generator how many of you knew that it actually has a random number generator oh like half of you okay that's a lot more than i thought so you can actually just keep refreshing it refreshing that line and it will just regenerate your simulation i had an amazing thing i couldn't get a hold of it to show but i had this amazing one it was my favorite that i ever built on the sims 4 was i called it commander tamalox throws a party and uh which is a star trek joke for those of you who know and love star trek like i do commander tomlok was just for all intents and purposes was a sim and i had a bunch of other sims that were along in there and the entire purpose of it was to simulate what happens when i throw a party so we need a caterer we need a dj we need all these different roles to be filled so how do we choose a an appropriate sim to fulfill those roles

and so i created a system that did it and i tweaked it for a while said okay this feels right all in excel and then i handed it over to design and said here's what i'm thinking and so they got to play with the different skill values and different like mode of values and just whatever stats that they wanted to play with keep regenerating and saying yes these sims are appropriate kick it back to me and say okay great now i have the model now i can do what i want so it's overlooked this one's more for like nuke this next one is more for newcomers sort of less experienced devs i think overlook things like prototypes so that's why i mention it here these prototypes um sometimes i feel like i write more prototype code these days than i do real code because this is answering whatever question that i need to answer so like on a starship simulation game i'm working on on the left that's how all of the crew are generated there on on the left that you see you know i have a story simulation and story generator based loosely on propositional logic on the top right and

a whole star system generator on the bottom and they're just silly little like c-sharp or the one on the left is python like just little apps that i did you know it's in a thousand lines of code or 500 lines of code i can prototype this thing before i have to like okay let me architect it into this thing that i have to care about um so never underestimate the power of things like prototypes um also probably my biggest piece of advice for prototypes throw it and uh yeah throw it the away really like throw it away once it's done please how many of you have taken a prototype built a prototype and then taken that forward to production be honest how many of you was that a positive experience one person so one person out of the what 200 people in this room so half a percent um i worked on a game called brain quest which has nothing to do with ai and uh we built the prototype and before i got there the prototype was built in flash uh so we're using actionscript and stuff

like that like okay this looks good and they're like well we don't want to throw this away the game was for the original ds we built a flash interpreter for the ds that was the solution it ran as quickly as you thought and it's the only project i've ever been on where like depth of function calls was a real performance hindrance and literally unrolling loops hand unrolling loops was a thing it was amazing so throw your prototypes away please good god okay let's go a little deeper um now we're going to talk about a scheduling system um so this scheduling system uh you know i've spoken about this a little bit in in other things um humans follow a schedule like that's just sort of reality um you will generally throughout the day you know and this is definitely on my schedule i'm not awake at 8 30 in the morning um you'll definitely follow some kind of schedule and you go to work around the same time and so on on the sims we followed mostly a weekly schedule on you know the starship sim that i mentioned earlier uh they follow a daily schedule because it's a military ship

so you know this allows you to constrain or whatever the opposite of constraint is like expand your your horizons in terms of what you can score okay um the basic architecture works like this you have an agent which has some number of schedules and the and those schedules have some number of entries which in turn wraps some kind of time that's basically the idea you can have any number of schedules you can have any number of entries within those schedules here's an example this is a schedule you can have overlapping entries as i said so these entries can overlap this is one this row represents one single schedule and so here you see that there's a split so there's two entries that are overlapping you can have white space here which is a totally blank space and you have to consider how that works entries themselves have uh the start time and in time as i said and they have a set of tasks tasks i can't tell you how to do that's totally up to your game right whatever your tasks are they're interactions on the sims for example um but it's whatever whatever it is sometimes it's as simple as i'm gonna

walk over here and run some animation like it could be that easy or it could be super complex uh generally tasks can be chosen in if it's like if you're trying to do this as part of just like background ai just a basic weighted or even uniform random is probably fine um it's not gonna hurt me too much in my soul but i tend to go more the utility curve route as you've probably guessed the sims medieval heavily used a scheduling system and we just used a utility curve kind of standard sim scoring to figure out what we wanted to do once we hit that tasks are and forgive this because i enjoy math tasks are basically the union of all of the different task sets from all of the different schedules and all of the different entries that you're in so if i'm in three different entries then i will uh basically have the union of all of the tasks in those entries those are the ones that i'm looking at so let's look at another example um so we have this thing here we're going to

um i can't even see what it's doing on the screen yes so we add a new schedule um then what uh over here we have these kind of overlapping sets so you know these are the overlapping ones in this particular case so if we're at 100 hours for example then we're going to take all of the tasks from all those things 1300 hours is all the tasks from those guys right schedules can be prioritized so this might be a high priority schedule for instance this high priority schedule is going to change the way that we're going to process so for example just as a simple example we're at 8 30 basically um so if we're at 8 30 then we're going to look at our high priority schedule we're going to ignore it because this is a blank entry right and we're going to go to the next set and then we're going to basically choose something from within this set if we take a different example we're here then we're going to look at our high priority set we're going to completely ignore anything from our lower priority set this allows us to have schedules that overlap uh and

overwrite other schedules so we can say here's your general default schedule this is how you act every day but you are a traitor to the rebellion and so here you are going like during this particular time you're going to go and do traitorous things it allows us to do those types of like interesting things [Music] this is an example of what a ta like a priority might look like the only reason why i'm showing this is because please uh this is a big gotcha this this hit us and in other projects i've been on you want to have like a very small number of priorities this is actually maybe too many i don't think i'm even using critical so like two or three priorities not like the sims which had like 30. that gets nuts okay so let's talk a little bit about uh about how these blank entries work that's up to you what do you do on the sims we said that was your free time it was part of the design we said you're just not concerned you can do whatever you want for other projects it could be an error state so it's a conscious decision you

have to make of what happens when i'm out of a schedule because even if you design your characters to always be in a schedule there will always be some bug where they're not oh and generally add schedules through tags and things like that okay updates so the way that updating we have to talk about how updates work because we don't want to do like for each character in the world for each schedule in the world for each entry in the world update right that sounds terrible so you know we can we can do some like space some kind of like time slicing we can do some kind of like threading um but i think the more interesting thing to realize is that schedules have discrete moments where i actually care about anything that's happening in the schedule whatsoever beyond my task set changing right the task set that i'm scoring against only changes at these very specific cadences or these very specific moments so we have to define a new system um i'm going to define it kind of generically because that's how i think so i've created this little system it's like delayed callback you know it's just

it's basically a function wrapped in a time call you have this manager that has this heap of callbacks so you know priority queues are all the rage um you can add and remove callbacks right it's just an api for doing this stuff and you have an update so the way that it works is it's global to all of your schedule to the entire schedule system or maybe even your entire game why not and you can add so it can for all the schedules for all the characters across everything you have this sort of priority queue of all of the different updates and when they're going to happen so the magic here is that you're not doing this order and like update across everything most of the time you're doing constant time let's look at the first one and i don't need to update it because it's sorted based on time um the one gotcha that you have to deal with is you know if if you're making a like that was great on the sims medieval where they had very different schedules but in a military sim you know you still have to do something some kind of time slicing so just be aware of that so new schedule is added here it is that's our new that's our new moment that we have to care about we

throw into our little callbacks heap and we're done that's it that's basically the idea of how this core system works now if you if you take a step back and just take a like squint your eyes at this at this scheduling system like really just squint your eyes at it for a minute yeah it kind of looks like a state machine somewhere except it's you know where the transitions are based on like the constant march of time the ideas are that the states gen sort of define what different actions that i can choose from whether constrained or expanded change the set of tasks that i have the ability to choose from so you can think of those as different states so even if your game is not directly tied to time the idea is to create these small little discrete like states really that they're in what if you're in a heist or something like that and there's like different events that happen through there so it could even be event based so that's a thing that you can kind of

squint your eyes and and see that a bit so that's the idea um i spoke really fast but i think i have what three minutes yeah sweet that's actually only two minutes over than my practice time so thank you very much if there's any questions there's microphones [Applause] great i'll stand here and stare awkwardly at the audience for three minutes then thanks for talking um yeah i was wondering if you could speak to i guess the nature of time inside the sims and just like how you do it relative to like wall time and real time and like and then how you actually like the mechanics of the click i guess like the demand the mechanics of like

actual simulation time yeah and and how you accelerate it do you have your own calendar system like like i'm just kind of curious how you yeah move forward i mean like internally literally it is a double of you know ticks essentially um there's a date and time object and a time span object and that kind of stuff so you could kind of get different uh dates and times and that kind of stuff um yes we had our own basically internal calendar system that's right um so it's like how many i'm trying to remember what it was it was uh yeah minute seconds minutes hours days and then i think weeks was in there and then months certainly so those are all the different ones it was approximately one real time second per simulation minute in single speed and then double speed was actually times i don't remember what they got tuned to but something like times three and then uh three times c was something like times ten i don't remember exactly what the values were specifically tuned to um but that's basically how the ticks

worked does that answer your question yeah yeah i was just curious because i can't use the system necessarily no yeah we can't use the system yeah we had our we had a whole time system basically that worked cool great well that's it i'll be around so feel free to come up and chat with me about stuff if you want to know about any of the other stuff that i spoke about more in depth i'm happy to go more in depth on that kind of stuff but yeah that's it for me thank you so much thanks for coming you