Modular, Reusable Social Behavior In Video Game AI

[Music] all right um so uh bill and I are both engaged in a fairly large project to try to uh develop new approaches for reusable sort of a cross scenario and a cross character social behavior um so the first thing is what do we mean by social behavior um and what we mean by social behavior um is the kind of behavior uh that we found in facade where um the characters are engaged in activity which is primarily about um modifying uh the social State um they're not uh necessarily worrying about you know uh in this case you know weapon combos or combat Behavior or navigation but really about doing things that Express um Rich social State change like you know I don't like you but I'm hiding I don't like you and I'm trying to Ally with this person over here this is the kind of behavior that we really want to make um

radically reusable across characters so the vision we have is um reusable social behavior reusable means it's applicable to multiple characters in multiple game worlds um that it's easily overridable um for custom content so the idea would be that you've got these you know a big library of social behaviors you can sort of drop them quickly on characters and sort of get like to this 80% point in say a particular social situation you're creating and then that 20% % um there's a nice kind of layered system for going in and doing overrides and inserts um it should be highly mixable uh combinatorial Behavior Blends and behavior Contracting where one Behavior kind of uh will tell another Behavior to do something for it in its service um but somewhat generative to avoid commentator authoring if we have comment tutorial Blends but you have to author how everything combines with everything well then you're just back to this like giant authoring problem we want it to be real time where behav play out

continuously and are continuously reactive player input and we want it to be symmetric that every role in a particular social behavior can be performed by an NPC or a player that they can all perform and respond to the same actions so if we score facade in this case well um real time it was doing okay there's continuous performance and it's fairly interruptible you know there's never there's never really you know kind of um in engine cutscene like activity where they're sort of blabbing on for a long time or perform some action and you can interrupt them they're pretty continuously interruptible and they're performing all the time they never just stop and go into you know kind of a pearing noise Loop to uh establish their liveness uh while they wait for the next thing um they're moderately mixable uh so for example um you know the characters can make drinks while arguing about any topic um there's an eightball gag that can sort of mix in anywhere on top of things there's a lot of complicated staging that's going on that can mix in on top of any argument Behavior so fair amount of mixing I'd like them to be

even more mixable complete fail on reusable and symmetric um the majority of the behaviors are very specific to trip and Grace in this scenario so even if you wanted to make another story about trip and Grace there would be a bunch more Behavior authoring you'd have to do and if you were trying to make uh uh a sort of AI based interactive story about uh completely different characters well um you're going to have to almost start from scratch so uh not reusable um and not symmetric trip and Grace can do a lot of things with each other like be snarky and understand they being snarky and respond to the snark that the player can't do right the player can't sort of say something in a snarky way and know that they'll get the snark right um and there's a lot of cases like that where the characters are engaging in sort of richer social interaction with each other than the player can with them so um in prom week uh which we talked about a couple of years ago here um our goal was to say well can we push reusability at the dialogue level you know wanted to

have um uh a larger cast of characters where we could sort of generally write social exchanges between characters and have those social exchanges automatically retarget onto specific behaviors uh I'm sorry onto specific characters in specific social contexts um so social behaviors in prom week were more reusable we could dynamically retarget dialogue within a scenario but had very little physical Behavior effectively kind of dialog exchanges were sort of illustrated by tagged animations that went with them so I guess you could call that sort of physical Behavior reuse but it's sort of like you know zero author physical behavior um it doesn't provide overrides and inserts for cross scenario authoring uh but but was definitely far more reusable than facad uh social behavior um and it was symmetric um anything the characters could do the player can do in this case since it's sort of a Godlike game the player is sort of in a sense determining which of the affordances characters of Compu did they want to do

next they take um but you could imagine versions of using the underlying social engine where the players have a menu of exactly the same kind of affordances that the characters have this is very much the approach also that Richard Evans took with um Praxis uh the engine behind versu was this kind of symmetry at this kind of like discrete Choice Point level but um like in versu we achiev that by simplifying affordances the affordances are discrete choices that can sort of be computed and put into a menu um it's not real time at all um strict turn-based interaction not interruptable during a social exchange um and it's not mixable there's only one social exchange occurring at a time so the current project we're engaged in is to try to say hey can we get the best of both worlds here can we get uh the ReUse reuse and symmetry of prom week and the real ESS and mixability of facade in sort of one architecture and approach um we're doing this in the context of a DARPA funded project in non-specific cultural training um and what this nonspecific specific means is um uh what

happens if you as the player are sort of dumped into situations where you don't speak the language you don't understand the sort of you know cultural values and cultural norms um something's kind of escalating and what do you do to try to deescalate the situation to quickly sus out what's going on uh and keep it from turning into something horrible that's what's meant by non-specific cultural training um it's this is happening in the context of a sort of whole body interaction system so um if I just sort of play a snippet of this you know here's one of our scenarios called The Lost interpreter um where the player um has lost their interpreter and is trying to interact with uh people who doesn't share a language with other than a few words um and is trying to find out hey have you seen my interpreter from this uh photo and so you know there's a lot of uh you know real- time gesture recognition going on sort of pose and stance recognition facial expression recognition and so on and so those are the signals that are kind of coming into

the AI as opposed to say you know the text that was being used in facade you know text and mouse movement and clicking that was in facade um so this gives some sense of um what the whole body interaction looks like that back uh so you can imagine the full architecture for this is pretty complicated and that there's you know there's a sort of a lot of machine learning based uh gesture stuff we're doing um there's the sort of social pragmatics piece that has to kind of turn that take that and turn it into kind of higher level social moves that have occurred uh there's a uh uh drama manager that's trying to sort of manage for pedagogy and kind of dynamically um update the simulator to kind of make specific pedagogical points and then within the social simulator itself is this new NPC architecture and approach that we've been building for the purposes of this talk we're focusing specifically on the kind of social behavior uh um NPC system within the social simulator and not uh the rest of the architecture and this is uh being done by you know a fair large

distributed team um split across four organizations um so uh and the UC Santa Cruz team along with BBN um are responsible specifically for the autonomous characters um so what lessons are we borrowing from prom week what are we bringing from prom week uh um that we're then going to adapt and enlarge on the first is the idea of declarative modeling for the characters um you know we do a lot of declarative modeling for other aspects of NPC interaction right I mean we keep track of positions we keep track of like how much ammo you know there's a lot of sort of declarative State about the state of the world that we um do you know say put on a uh a Blackboard um for our characters to respond to but we don't do it so much for social interaction and facade didn't you know so a lot of the uh character State uh between characters um between trip and Grace was implicit in the behaviors it wasn't explicitly being modeled as values that are being changed by moves the AI is making um this was one of the first things that prom we did

is to say hey let's make all of that explicit traits social statuses um uh the the current social State existing between all pairwise combinations of characters make that declarative and explicit and now system can start reasoning about it and explicitly taking social action to change it the other big thing we did there that uh in prom week that we're adapting is this kind of utilities meet symbolic reasoning this kind of soft rule-based reasoning uh approach so we all know from hearing talks from say Dave and Kevin over the years uh the benefits of um and we may recognize this diagram uh the benefits of utility-based uh methods um that they keep you out of Corners in the state space you're constantly able to compute a value um that uh uh yeah that the the brittleness basically that symbolic techniques tend to have go away uh but symbolic techniques are able to express sort of uh much more complicated situations like you know if x uh feels High Romance towards Y and Z is does

something mean to X then um and then y does something uh uh mean to Z then there's going to be this in this uh uh tendency for X to want to date y because um y basically came in as a night and Shining Armor and like s you know saved X from being picked on by Z right that's uh this kind of temporal rule-based reasoning is really hard to capture as sort of algebraic combinations of sigmoids and EXP ential and so on is very easy to capture in a Sy uh in a symbolic system but then we lose then we're in brittle land right and we lose this kind of common currency and no edges uh no brittle Corners property that utilities have so what we do is on the right hand side of rules instead of sort of computing like do this action for example if something's true we just say add three to your tendency to want a date add two to your tendency to do something mean and so forth so implicitly you're actually Computing

multiple really complicated utility surfaces that are um implicitly being computed by a bunch of rule-based reasoning and so we're trying to get sort of the uh common currency and no corners aspect of utilities with a rich expressiveness of uh symbols so um that's uh those are the sort of two big lessons from prom week that we're then saying well how do we do this with real-time reactive 3D characters um so well the first thing we need to do is take the notion of dialogue Exchange generalize it so in prom week you can imagine that you have templated dialogue exchanges like this and I I won't go into the sort of natural language system but you can imagine that you wrote maybe a thousand of these and they're not specific to any character they're just floating in space right and you can imagine that all the AI of prom week is basically just trying to say in this specific situation in the game which of these many dialogue exchanges that are sort of just floating in space do I want to zoom zap zap down onto these particular characters at this specific time so that it's social State and

personality appropriate like that's the that's the sort of retargeting that's going on well we need to turn this now into behaviors right that what was what was before kind of a you know nlg based template now becomes um a fairly complex Behavior tree that's controlling sort of what you know initiators responders and onlookers are kind of all doing reactively in real time during the carrying out of that um of that uh uh uh social exchange um so the specific idiom we've developed to capture this is this notion of social games which we talked about social games and facade and um but in facade they were sort of like a design idea but weren't reified in the architecture now we're reifying them in the architecture so two examples of social games that we've implemented are Alliance and Authority Alliance is the general idea of um uh NPCs in the player and sort of any number of them having kind of a you know bire Alliance values between them how much I feel like you're currently uh

could be one of my allies um and then this binary allies thing which can be accomplished through social exchange if I feel high Alliance towards you and you feel high Alliance towards me we're going to have a tendency to want to take social action that actually establishes the the binary or Boolean social fact of currently being allies and then allies may break up and so forth so all of that states modeled The Authority game um is really about um Authority uh Authority that uh NPC and or characters generally players and NPCs have over each other and also authority over objects who owns objects who can do what with what objects um so uh okay so so a social game sort of like uh bottles up the social State associated with a particular General domain of social interaction then also associated with the social uh with social games are these kind of utility-based volition rules uh that we talked about um uh with with prom week that then uh um use that

State plus sort of inferred intermediate state to assign utilities to performing particular actions so you know here the bottom uh the bottom rule uh in um Alliance says that if you have a certain valtion to do an allies make move between X and Y and gifting is appropriate between X and Y and that would be computed by some other rules and there's an object in the world that's tagged as a gift then um you have a certain uh utility that you're going to sort of uh transitively move over from your volition to allies make to object transfer to perform the specific move object transfer um and object transfer is now going to be sort of one of these Behavior tree things that handles all of the sort of detailed momentto moment interruptible activity around transferring objects similarly in the authority game uh manipulate object is uh is something that's computed at the bottom Rule and you can see sort of intermediate rules that compute utilities on doing what we call Authority up moves okay so um once

you've sort of snapped one of these social games into your scenario you now have a bunch of utilities being computed for performing actions um you have the social State necessary to reason over those and you get a bunch of social interaction units that come in which are like the atomic actions now so here's the social interaction unit for object transfer which is actually a sequential one sequential and hierarchical it says perform these three lower level social interactions in C an an optional request of object an offer object and an accept object within each of those you have parallel behaviors for the initiator for the respond in the audience you have to have accept and reject behaviors because just because someone decides they want to initiate a social interaction doesn't mean that the responder wants to do it right so that's computed by valtion rules but then how do you actually perform accept and reject you have to have um the ability for the player to perform any role and the NPC to perform any role and so that means that you have a lot of behaviors that are also kind of listening in a context specific way to

these signals coming from the whole body interface and interpreting them according to um that the context um you have to have appropriate monitoring behaviors for player action reactive staging behaviors emotion side effects um and recursively an SIU may make sub decisions about performance using Behavior scoped poition rules uh and this is one of the key kind of architectural uh modifications we've made is to take this volition approach which was you know in prom week only used at the top level to make a high Lev decision about what um social action to do next and we want to say hey that would be really cool and Powerful to have that kind of uh decision-making capability all the way down right all the way down to like lowlevel uh animation control performance behaviors if if needed um so to describe a teeny bit about what we're doing there I um won't belabor um the uh the able language there have been there was a talk at GDC 2004 about it um it is a reactive planner that has has a library of behaviors a working memory that's

like a Blackboard system a sort of sensor and action abstraction for interacting with the world and then this active Behavior tree that the main thing that's different between this and many Behavior tree architectures is it's not a static Behavior tree you don't sort of author a behavior tree you just author bags of behaviors and the behavior tree is dynamically computed um as the character is executing and is changing moment to moment and this was a class of uh of architectures that sort of came from the late ' 80s um called reactive planners and in fact Doan um was working with Bruce blumberg at the media lab in a similar kind of architecture which is where he got the idea for the kind of behavior tree uh implementation he did in Halo and so this is there is like a historical connection between reactive planners and behavior trees um but the big difference is there is no static Behavior tree it's a dynamically computed data structure basically that changes moment to moment all right so um in the able execution there's sort of three places where AEL has choices it

has to make the conflict set which is the leaves of the tree these are all the things I could do next and this is where you get parallel inter or mixing going on um and AEL comes with its own sort of default algorithm that's based on priorities and anti-th thrashing heris and the notion of Step conflicts uh to uh to resolve um that uh there's Behavior choice for particular subgoal preconditioning Behavior specificity and there's conditional steps the new version of AEL uses rule-based utilities at any of these Choice points so the idea is that this whole sort of of you know uh C social simulation engine like reasoning can now be recursively inserted anywhere and able that it would make one of these other choices so you've got the default way you would do it and then you can write rules and do this sort of rule-based utility way and so that's how we're getting this kind of recursive decision-making all the way down working with u Michael has been a privilege in a trip uh he gets to pretend he has money and I get to pretend I'm interesting and I also have to talk very very quickly here to keep us on any kind of schedule so um what

what this this slide is just summarizing how we deal with the the question of character individuation in in this kind of framework characters differentiate are differentiated from each other by the the list of things on the left uh this is old hat and apple pie uh those things man those differences manifest in the way they do things how they perform what they look like how they move they're pacing and and at a higher level by the choices they make about what they will do we produce those differences then with the declarative representation of the character that we've been talking about so far and Rule sets that map those differences in representation into different Behavior choices and then we have the ability to override things specifically so that if somebody instead of getting louder and faster when they get angry gets slower and more threatening we can mix that in if that's the right thing for a particular character finally uh we can put in custom behaviors we can do it the old-fashioned way if that's what's necessary to make things go uh then the big question is how do we

manage all this combined content without having to deal with the N squared authoring combination that's going on well um we uh we we do Performance Management uh which I'll describe in a minute to sequence the low-level behaviors we interpret physical behavior of the characters part and this is to get that symmetry that Michael was talking about so that it gets translated into the same social signals that the NPCs are passing amongst each other we use something called wapons which are ways to wrap attitude into a behavior or cross cutting behaviors into another Behavior that's gold directed and finally then we do support author defined overrides to do with particular problems when they come up um Performance Management is a simple process well it's it's a straight rather straightforward process of prioritizing different activities uh amongst the many competing things that an NPC might do and even that the various body parts of an NPC might do at any particular time and then going with the highest prior priority for a particular set of body

parts so that you can talk while you're eating and your hands can remain eating because they not going to do the gesticulating that they do ordinarily while you were talking uh so that's that that takes care a lot of interaction at a at a low level when we're dealing with this problem with symmetry with the uh be the the player because remember the player standing in front of all these sensors actually doing things we get a signal from the sensing system that's a physical Behavior like ex the player extended their arm or they shouted uh we interpret that with a set of rules that that are seeded by expectations from the social simulator so if if an NPC has just offered something to the player the expectation that the player will either accept it or decline it will get passed over to the interpretation rule so the player extends their hand in that situation they must be trying to accept the object if they shouted instead that wouldn't be interpretable in the context it would fall into a set of initiation rules that would try to say well maybe the player's trying to do something new that's not going on right now and we need to start a new activity to accommodate what the player's doing

finally we're doing these things called wapons which are simply crosscutting behaviors that Express attitudes of what's going on or which yeah right or or which uh are um uh deal with common behaviors like thanking people or uh dealing with confusion uh this is to achieve this dream of a substrate of real social interaction that's going on and and and basically what are we trying to do with all this stuff well we would like to create a social engine uh we we saw an example of one of those things in re's talk yesterday morning about Sims Medieval so that all the characters have this kind of Baseline repertoire of social behaviors that they can draw on this enables new games with a purpose potentially like we're using it to train social skills or to accomplish the social process of teaching finally if the player is socially engaged with the game they'll become socially uncomfortable socially rewarded by smiles and all of these things can be used to enrich the kind of behavior that they've been having where we're going next we've got to deal with we've got all these games that have separate representations of State we have to

figure out how to share across them we're dealing with blending animations to go with our Blended behavior and hitting a bunch of problems that you guys are probably familiar with we're we'd like to go farther with these cross cutting behaviors and we would like to apply more of the social State information to the interpretation of the user's Behavior to deal with highly ambiguous or complex situations and thank you very much [Applause]