Taking Back What's Ours: The AI of Dishonored 2

Good morning everyone. It's a real pleasure to be here and I must say I'm totally terrified but I am Xavier Sadoulet and with my colleague Laurent Couvidou we flew all the way from France to talk about the AI in Dishonored 2. But before getting started we think a few thanks are in order. First we'd like to thank this community be it the member of the guild or the people presenting over the years at places like here at GDC. We what we will discuss today was inspired by what you guys shared and so we're happy to give some back today. Then of course we want to mention all the guys that worked on the AI for Dishonored 2 and supported us in the preparation of the talk. Um but lastly a big shout out to all the team in Lyon. It is those guys work we're representing today. So they deserve

proper credit. There. We're done being being emotional so let's get to it. So who already played the first Dishonored? Oh not bad. Okay. Who played Dishonored 2? That's good. But during rehearsals at the studio everybody everybody was raising their hands so um I guess we need to explain what the game is about. Um Dishonored 2 is an action adventure game played in first person. Like all Arkane games it belongs to the immersive sim genre among titles like System Shock, Deus Ex or the Thief series. In this game you play an assassin that can use either stealth or combat approach or both. Uh and he is given supernatural powers as tool to achieve his goals. But essentially the game is really about how the player chooses to approach each mission each situation.

Here is a short video showing you a glimpse of what the game looks like.

Now that I Now that you seen Now that you saw sorry what the game is about let's talk about some challenges we had to tackle during the production. The first one was scope. Not like the sniper scope more like game scope. Let's say the AI had quite a few things to support. Stealth combat verticality dynamic relationships teleporting player teleporting NPCs ambush behavior NPCs interacting with bloodflies time travel moving walls and floors and roof player clone time stop player taking control of NPCs fully interactable scripted scenes and what about fully interactable scripted scenes while the player takes control of a participating NPC.

So basically we have all the problems of action adventure games and much much more. Thus we had to rely very heavily on simulation because in every feature review there is always some guy in the team to ask something like this. Xavier Xavier what happens in that situation if I stop time while swapping position with my clone in front of a teleporting witch and then possess a bloodfly that's carrying a stun mine? Answer. Now. Yeah. Every time. And the thing is that it's totally possible to do that kind of crazy stuff with our game. So the bottom line is that there are some corner we simply cannot afford to cut. There are many many edge cases and we had to do things the hard way to ensure all systems work together.

And in order to implement those systems we were starting from nothing. Beginning Dishonored 2 we chose to build a new engine starting from id Tech 5 and reworking it to fit our kind of game. We rewrote the AI part from scratch. We also planned to take full advantage of the new generation hardware Dishonored 1 being initially last gen only. And as the Arkane Austin guys went to another project which is a pretty cool game called Prey there was only one AI programmer remaining from Dishonored 1 so we basically had to rebuild an AI team. In that context we decided to always try simple thing first. Avoid over complicate sorry over complication at all costs. We chose tried and true techniques standard way of organizing our AI systems known stuff with ample literature available.

We tried to step away from unneeded new shiny things. And very importantly we did our homework spent time writing technical documentation thinking on paper writing unit tests for our systems. Speaking about systems and standard stuff we've got it all. Sensory system knowledge system behavior tree smart object navmesh everything. But guess what? Those are not within the scope of this talk. What we are going to discuss today are three systems where we did things a little differently. Let's see one of those. Thank you Xavier. So hi everyone. I'm going to first present a few things about our rule system. So why rules? Very trivially making a next gen Dishonored with such a large scope meant more assets to present. And we're very design driven studio so

something our designers expressed early was their desire to have more control over how voice recordings and animations are used by the game. Having to make a code change every time an asset needs to be integrated is very bad and as we didn't want to turn our designers into programmers either we had to make sure we come up with something user friendly. Also last but not least we needed to find a solution with a performance cost that fits the hardware of the current generation of consoles. So how do we track all the simulation state to present these assets in a varied and meaningful way? This was an open question during pre-production and because we started our AI tech from scratch we had the complete freedom to try a new technique. This is where this GDC talk by Ilan Ruskin came to help. He presented a rule system used notably in Left 4 Dead 1 and 2. It inspired us and seemed like a good

candidate to help us overcome the challenge I just mentioned. So do yourself favor and go check it someday. Just to make things clear from the beginning we basically stole all of his ideas. So rules were something something that met our requirements. We went on and implemented a similar system in our new engine the Void engine. So yeah that's it for rules. Thanks thanks for listening. I'm I'm just kidding. I said a similar system not identical. Like every team we have our own requirements so we did a few things differently. For those that attended yesterday you saw actually two other use cases of similar systems so this is a common thing. My goal here is to give you details on on our specifics. So let's talk about this. First I need to go over the terminology and basic principles. We define a rule as the combination of two collections. A list of conditions and a list of

actions. Conditions are predicates. We combine them in what is just a a glorified if and if conditional. They pull the state of the simulation in other words the current game context. So concretely a condition is the comparison between a named variable and a constant value. Here's an example. We can have a rule matching when an NPC barks because they just busted the player. So we're testing the bark trigger variable that tells us why we're trying to bark now. And let's test a few more variables. So is the current HP a bit low? Is the world in high or very high chaos? And let's match this five times out of six. Note that we only have logical ends. This removes the need to handle operator precedences. It also makes evaluating the result trivial. A rule matches when every condition matches. So, it just

makes things simpler. Now to actions. They can be virtually anything. As you saw earlier, for us mostly playing lines of speech or animations, but could be anything else. Uh when there are several actions, they are picked one at a time. Every time a rule matches, only one single action is executed. When there's a number of assets, it's a great tool to provide variety in an explicit way. Finally, if you take a bunch of these rules and stick them together, you get what we call a rule handler. So, this is just a set of rules, our editable resource. So, let's talk about editing. One thing we made is an editor. Ilan, if you hear me sometime, we followed your advice and listened to our designers. Their wish was a tool integrated into our in-house editor. And I already told you about designers, right? Well, they want everything. So,

it had to come up with all the handy features that make such a tool usable, copy-paste, undo-redo, etc. So, we created just that. Let's see the rules editor in action quickly. So, this is me uh creating a couple of NPC bark rules. Let's say I want to play some lines when an aristocrat notices something uh for instance, a flower pot or a teapot. So, I won't give you a full tutorial here. Uh don't worry if it's going too fast. Uh what I want to show here is the type of editor we have. You can see that we have two rules here and here. And also two sets of conditions and two sets of actions. So, a respectable editor. Maybe not totally mature, but this is the actual tool we used to develop Dishonored 2. Now, what did we create with this? I won't detail everything, but as I

said, we have uh well, there are rules for NPCs and also rules for the player and mostly speech and animations. So, this proved to be quite a versatile system. One interesting use case is achievements over there. The rule system was a very good fit for things like check that X coins were looted to grant achievement Y. And since we all love numbers, this is how many rules were created for each category. So, the grand total is about 6,000 rules, 6,000. At run time, rule handlers are instantiated and will be used for evaluation. This means iterating all rules it contains one by one in order and executing the first one that gets selected. So, very trivially, this one doesn't match, so we go to the second one that doesn't match either. So, we try yet another rule and this

time every condition matches, so it's picked for execution. We never needed to match more rules than only one, so the evaluation stops here. That's some CPU time saved for other stuff. So, what do I mean by execution? Every time a rule is selected, one action is executed. There are several ways you can handle this. One way is to go over the actions sequentially. So, the first time a rule is selected, we pick the first action and execute it. The second time, we execute the second action until we run out of actions. So, things have been pretty straightforward until now, but please bear with me. This is where uh things start to be a little interesting. At this point, we can either start again from the first action or we can simply say these actions are depleted and never select this rule again. We allow both and we call these two options execution policies.

Sequential execution is handy, but sometimes what you really want is random variations. Thanks. For instance, for ambient barks, you don't want NPCs to say the same lines in the same order all the time. This gives a robot effect pretty fast and is distracting for players. The naive approach to randomizing things is to pick a random outcome every time a rule is selected. The problem with this kind of approach is that it doesn't prevent repetitions. So, this is an another annoying robot effect. So, we don't do this. What we do instead is that we shuffle the action list during initialization and then shuffle again every time we reach the end. Just like you would shuffle a deck of cards before playing a game of poker. So, this is called a shuffle bag.

Given a list of possible options, it's the most natural way to use them all while avoiding common pitfalls. Beware though, when shuffling, you should never put back the last action in the first spot. Otherwise, you'll get a nasty repetition, which we're trying to avoid. So, one workaround is to simply send it in any other random spot but the first one. He's going to point that at your face. He's going to use that as a backup. Okay. So, that's a total of four execution policies that we use in the final game. Sequential or shuffled execution of actions work great to prevent repetition for one entity. But what about several entities? We still wanted to prevent the same bark or attack from being played twice in a row by different NPCs. So, we obtain this by putting recently executed actions in a global list, forbidding

other NPCs to execute it until a cooldown depletes. Actually, once we had this option to filter actions at run time, we found other uses for it. We also ended up implementing it for conditions. So, we didn't plan for this initially and this evolved a bit organically, but it's probably something interesting to explore further. Earlier, I mentioned briefly that we instantiate our rule handlers during initialization. We actually say that we compile them. That's because we perform some significant data transforms at this stage, mostly for performance reasons. But one interesting thing that we do is that we combine rule handlers. This allows to add a custom set of rules for specific NPCs. For instance, members of the Howlers gang have a bunch of specific attacks. We can take these attacks and combine them with the generic attacks that every

NPC has into one single rule handler instance. Since we combine rule handlers, we need to determine how to sort the resultant set of rules. So, we call this a compile a compilation policy. One solution is to simply not sort and just concatenate everything. This is actually a very sensible solution for small rule sets. What you see in editor is what you get in game. So, clear and simple. We use this for most rule types. But for us, barks were the exception. We have too many of them and really needed more levers to get to the desired results. So, we first tried to sort rules by condition count as Ilan Ruskin advises in his talk, but it didn't cut it for us. Our lead narrative designer, Sashka, often found herself adding new empty conditions just to get to the outcome she wanted.

So, we changed our approach near the end of the production. The solution we shipped with is that we give every bark a priority and we use this as the first criteria for sorting, keeping the condition count only as a second criteria. Something else we do during compilation is that we share identical action lists. So, let me explain this with another example. Our fighting NPCs can kick or punch under different sets of conditions. So, offline, we have two independent rules, one to trigger close combat attacks when back to the wall literally and another one for player blocking too much. At run time, these two rules point to the same action list. So, the sequential execution shares the same counter. In other words, you won't observe the same punch animation twice in a row just because it was triggered under different conditions. And yes, in case you wonder, that does

give us a logical or for the few cases where we actually need it. Now, let me come back to variables for a moment and give you some more juicy details. These variables have a unique path to keep things simple and order in the editor. Their role is to tell us how to get to the actual data. So, let me list some data bindings we have. A variable can give the value of a C++ class member or be computed by a method. We call these providers. Or it can point directly at an NPC knowledge record or at a variable created in data by level designers. And we also have a few more bindings. So, when do we resolve variables? We resolve these variables on demand. Most of the time, we only need a few variables to be actually resolved during one evaluation. For instance, let's say that the first

rule of the set matches. We won't evaluate the remainder, so we don't have to pay the cost of resolving the variables we will never check. Still, resolving a variable every time would be very damageable for performance. We have variables that are actually method pointers and that do perform quite a bit of computations before they return a value. So, since resolving twice or more is useless in practice, we don't want to pay that cost each time a variable is tested during one evaluation. So, we cache the resolved values for the time of an evaluation. And having this cache does shave about 2/3 of the time spent evaluating rules. So, note that these figures are for 1,000 rule handlers, but we have only just a few of them per frame. So, on our performance metrics are okay for this generation of this generation of consoles and on walls.

So, time to close this chapter with what we envision for the future. One thing that is a bit clumsy with our current system is that some variables are only available to some type of rule for for no particular reason. For instance, a variable exposed for bars can be missing for addicts. If you are going to use rules in many different contexts, just like we do, make sure you take this into account and share your variables as much as possible. And of course, you always need better debugging tools and we make no exception. Something we miss is a visualization tool for the combined and sorted rules. Also, at run time, all we have to debug an evaluation is a good old dump. So, we'll try to improve this in the future. Uh finally, uh we've talked about our cache system, but we know there's more we can do on the performance side. So, if we expand our usage of rules even more, we know there's room for it.

So, to sum up, what makes rules so sexy? Firstly, they're very appropriate for simple and short-term contextual decisions. Secondly, they're a data-driven tool. In the end, they made our designers happy. I believe, at least, mildly satisfied. It should be obvious to everybody. If you have to make a code change for each and every new asset in a AAA game, well, you're just never going to ship it. And finally, you saw the metrics, we now have more than enough hardware resources to afford such a system, so there's no reason not to do it. So, your turn to rule, Xavier. Thanks, Lorne. You rule. Okay. Now, let's pick DSM. I do realize that I'm not making things easy on myself by starting a section of this talk with

some obscure acronym. So, let's try to explain a bit. Originally, DSM meant dynamic space management. Or maybe dynamic space mapping, but you could take any of of those other meanings I came up with while writing this part of the talk. Um yes, there is a pun. Um the point here is that it's not about what it means, it's about what it does. And DSM is our solution for spatial reasoning and influence mapping. First, let me let me show you what we use spatial reasoning for in our game. There you can see witches teleporting themselves and summoning a vine using dynamic position. Then, we have the always funny case of civilians running away panic and choosing destination dynamically. And lastly, we have the player losing NPC in combat and having them decide where they should go to chase him.

Missed. So, now to summarize to summarize all this, this is what spatial reasoning is about for us. Scoring position in the world. And in order to do that, you have to have a finite number of position and then to evaluate them. DSM helps us for both of those. So, let's start with the first part. Let me introduce the main element of DSM, the layout. The whole point of the layout is discretizing the geometry of the map. Or more precisely, discretizing the nav mesh, which depends on the map geometry. We do computation offline, going from each nav mesh face to a bunch of cells forming a layered grid. And voilà. The grid resolution is 1 m by 1 m by 2 m. And its origin is the maximum point of the nav mesh bounding box.

There is ample literature around on how to build a grid from a nav mesh. We can recommend this talk among other. Just I'm not making the I'm not going to detail those techniques today. The data representation of the DSM grid is a big array of cells. We encode the array index on two bytes, which allows more than 60,000 cells in a map. This is sufficient for us as in average, we have 20,000 cells. Now, let me talk real quick about the order of the cells in the array. They are naturally grouped by nav mesh faces. This is an interesting property. First, it enhance data locality. Cells close in the world tend to be close in memory. It's also interesting when the nav mesh is getting cut. We'll touch on that a little bit later. And lastly, it's basically free because it's just a consequence of the

way we construct the layout, iterating over all nav mesh faces. Each cell is a is a data struct containing the grid coordinate XY level, the nav mesh face ID it was created from, and the vertical offset from their center to this nav mesh face. Nav mesh offset is important because without it, we only have a bunch of flat grid layouts to play with, which is not an ideal representation of the world unless you're making game on Apple II. As you can see on the right, when taking nav mesh offset into account, our cells are far closer from what the world actually looks like. And as this is all about picking location in the world, it's better if those location make sense, right? So, using this offset, we can go from a cell to real world location. But what about doing it the other way around? Let's say you have a location in the world and wants to find which cell it

belongs to. Knowing the origin and the resolution of the DSM grid, we know we can translate from world coordinates to grid coordinates. And then, from those coordinates, we can construct a key that we then feed to a hash table we call access table and get the resulting cell index. Easy. Sometimes, the access table is not fast enough when you need to access a lot of cell at once. Thus, we also add a cell index stored in what we call access tree, which is in fact factor KD-tree. With this, we can formulate queries like give me the list of cell that are within a 5-m radius from this point in the world. Pretty handy. Okay. With those two ways of connecting cells, we pretty we pretty much covered. If we go back to our statement earlier, we saw that the world discretization part is supported by the layout. Now, let's talk about the evaluation part.

That is supported by what we call layers. A layer is simply an array of value. The array is allocated in one block and is the same size as the layout in the cell array. Layers are the dynamic part of the DSM system. They are equivalent to the classical influence map layers for those familiar with this concept. Unlike the layout, layer data is allowed to change at run time. It can store any data type and those value are typically information on the game world like danger rating of a cell, proximity to the player, etc. Their main function is to support scoring of each cell. The nice thing is that you can add as many layer as you need. And each layer Sorry. Okay. And each layer memory cost is mainly dependent on the type of data you choose to store in to store in it. Once we you have one cell index, you can instantly access any piece of data about

that cell in any layer constant time. Hence, we can combine layer values easily for our scoring need. Let's call this vertical traversal. But another interesting thing you can do with layer is to is traversing them sequentially until finding a cell that has the correct value. Once we found one, then we have a cell index and thus we can directly access data about that very cell, be it in the layout or in other layer. Then, we can carry on. Let's call this horizontal traversal. And please do note, good friends, that this technique was awarded the totally unofficial cache friendliness seal of quality. Traversing layers that way is really really fast, provided you put reasonably sized data in your in them, of course. Okay. Now, we can pick location in the game world and score them. Great. We're

ready to take a look at a concrete example, but before that we need to discuss a particular problem we had to solve. Remember the moving floors and walls of Clockwork Mansion? We obviously had many cases of nav mesh being cut dynamically. How do we change that? When the nav mesh gets cut by any kind of object like a door or a piece of furniture, the DSM gets notified with a list of the nav mesh faces affected. Remember how cells are grouped by face ID in the layout array? This allows us to know which cell belongs to the list of cut nav mesh faces very rapidly. And from here, it's easy to test which cells are actually cut and disable them. Disable disabling a cell is merely

another information to it. The access table and the access tree do not return disabled cell by default. In the end, it is really up to the client code to decide if the disabled status is taken into account or not. The the disabled status itself is stored in a bit field containing as many bits as the layout has cells. We use vertical traversal to check if a cell is enabled or not. All right. Let's talk about an actual example of DSM as a use usage in the game, which is search destination selection. The basic principle principle of search is that NPC choose one destination, go to it, and then choose another one, rinse and repeat until the end of the time. We use the DSM to decide which destination is selected. We decided potential search destination would be a nav mesh position of a DSM cell.

Thus, we just need to score each cell and which is the best one. Simple, right? To drive the scoring, we use a specific data layer we called search influence layer. Its value is used as a main factor for search destination scoring. Cells without influence are not even considered. We also use other factors to ponder this one, so that for instance, NPCs are spread out enough or choose a cell that are not too close or too far from them, etc. Whenever something is perceived by NPC, influence is seeded on the layer and propagated on among neighboring cells in all directions. The propagation speed is roughly the same as the player speed. And in order to support propagation, we keep connectivity information between cells in the layout. The influence value itself is stored on eight bits using seven bits for the value and one for propagation. Please note that when manipulating

influence, it is converted into a floating point number between zero and one. It's more convenient for things like scoring. We choose to have one search layer for everyone, so that the data is actually shared. And when the search is done, all the influence value are reset to invalid. Um NPC clean influence from cell once inspected, so that it becomes less desirable. Cleaning means setting the cell influence to zero. However, it regenerates over time up to a very low value. This is to avoid the search being ever finished before the end of the timer or before finding the player, obviously. NPC NPCs clean influence in two ways, in a cube of cells we construct around the NPC location, and within the inner field of view, we rasterize the triangle on the DSM grid.

We use the access table to retrieve the cells in each case. And to be sure the NPC actually sees the cell, we check there is a straight line on nav mesh from NPC to the cell center. So, this is expensive. We so we made sure to multi-thread all this properly and used time slice as we don't need instant results in that case. So, to conclude the search on search, there is something I'd like to confess. Yeah. Some among you probably noticed this bug in the previous video. The influence is not clean properly on this cell. That's clearly clearly a bug, right? No, obviously I'm just kidding. In fact, we have smart objects affecting search. We call them search bots and each one is linked to the a cell. On those, we forbid search influence cleaning until smart the smart object usage actually takes place. That way, NPC will eventually go to the smart

object and use it. Now, we are really done with the search part. Let's talk about code hard facts. During runtime, counting the layout or dozen layer, access table, access tree, etc., what would the memory cost for the whole DSM system be? So, who thinks it it's less than one meg? Who thinks it's more than five meg? Okay. So, all you guys that did not raise your hand, you win. DSM memory consumption was profiled around four megabytes. Some years ago, there was an interview from Naughty Dog folks stating that they were allowed three megs on Uncharted 2 for the whole game state, but with current gen console, guess what? Four megs is perfectly reasonable. Before concluding, let's talk about some things you want to watch out when using

DSM. First thing, the big performance consumer for us has been cell accesses. This mean you may need to watch out for the performance of your action function, make sure your KD tree implementation generates as few cache misses as possible, and really really think about the proper way to collect cell for each use case balancing between hash table and KD tree usage. Second things, we can't stress we can't stress enough that you don't want to store a four-byte float in a layer when all you really need is one byte. I guess they would agree. Page 328 in your book. Uh to finish on this part, uh let me tell you why DSM is sexy. For us, the system has been a breakthrough as far as spatial reasoning is concerned. It unlocks selecting position dynamically when all we had was pre-placed points before, which we had in this another one

from free destination or blood vine segment position. The second things is that there are lots of use cases for the system, far more than what we originally envisioned. And finally, try it. It was a simple to implement and provided you're reasonable, it's affordable. Now, let's talk about another affordable system coordinated by Laurent. Thank you. Thank you, Xavier. Uh so, now let's talk about crews. Uh it's our way of handling NPC coordination. So, where does this come from? During pre-production, we knew that we were going to have standard tech fitting for individual NPC behavior, um rules, DSM, etc., um behavior tree, knowledge system. But, our system designers

also had a strong desire to push the envelope and implement some ambitious group behaviors. Hey, designers, right? Hey, we knew from previous experiences is that this one desire was going to be hard to satisfy. You can get some basic things working with inter-NPC communication, but it's hard to get a fully coordinated group behavior. There's a point where you need an over approach to avoid getting trapped in a monstrous plate of spaghetti code. And even if there's already lots of literature about NPC coordination, we didn't find a perfect match for our desire, so we had to venture into uncharted But, one small delight came from a GDC talk, again by Mika Vehkala, so thank you, Mika. Among other things, he introduced the concept of situations, a technique used

in Hitman: Absolution. So again, go check this if you have any interest in this topic. Um our AI crews are largely inspired by these ideas, so uh again, we feel that we did things interesting on our own, so let's get to this. Um so, what's a crew? It's a group of people working together towards a common objective. Take this crew or that crew. And quite literally, they're in the same boat. Clearly, they will have to coordinate if they want to go anywhere. They have to row together. Although they can work together, these people are still individuals. Free them up, let the evening come, add some music and beverages, and soon you end up with this absolute mess. Agents making their own decisions, sometimes interacting, sometimes not.

So, in practice, what does this mean? Let me illustrate this with two videos of what we call the regular combat situation. Don't be afraid, that's our jargon for a bunch of guards directly fighting the player. So, here you can see that the NPCs are fighting, but things might seem a little disorganized. They're struggling, sometimes bumping into each other, all trying to reach the player at the same time. You see friendly fire going on. So, overall, it's it's not very pretty. Now, this time the crew is enabled. So, you might notice that the guards are keeping their distances a bit more. The elite guard stays in the background and is actually shouting orders. He also uses his gun a bit more effectively. He used it once, and he's going to stay in the background

and use it another time. So, things look a little more organized. So, what makes things different in the second video? There's a combat crew working behind the scene to coordinate NPCs. It assigns them a certain role in combat, for instance, melee fighter or ranged fighter. For NPCs in melee, we have a bunch of slots that pull the nav mesh around the player. It's the crew's function to find a proper distribution across these slots. The crew also controls the attack rate through shared attack cooldowns. So, we don't want the player to be completely overwhelmed. Uh etc. So, let's step back a bit. I might be stating the obvious here, but I want to drive this home. Two opposite paradigms exist, two approaches to group behavior, top-down or bottom-up. The top-down approach is to have a central coordinating entity that takes decisions

and gives directions, a a ruling mastermind. This allows for predetermined group behavior. The bottom-up approach lets all individuals take their decisions, and the occasional interactions create an emergent group behavior. See, the outcome is unpredictable or non-predicted as Roxane put it yesterday. So, I won't tell you that one approach is better than the other. There are pros and cons on both sides. So, my point is you have to find a balance between the two that fits the game you're making. So, what about us in this how not to? In our case, we had early designs such as this one. So, this is the kind of group behavior our designers envisioned. This one's from our combat designer, Jonathan. Don't worry about the details. Just know that this represents evolving situations with NPCs dynamically taking different roles depending on the environment and

player actions. My point is that it's very difficult to implement something along those lines when all you have is this absolute mess. That is just impossible. So, we needed a top-down approach to AI. We first tried a prototype of a situation system inspired by the Hitman talk, but things were very hard-coded and too rigid. So, since what we needed was a mastermind taking its own decisions, it became obvious that what we needed was an actual AI. Instead of reinventing the wheel, we figured we could simply reuse our standard AI tech. Our mastermind was going to have sensors, a knowledge blackboard, and a behavior tree. So, this is what we call a crew. So, what's the job of a crew? It works like any AI, but its main interest is the members that compose it. So, just to give you an idea, typically we're talking about two to 10 members.

When we update a crew, it first senses, which means that it starts by collecting the knowledge of each member. It then thinks, which means updating a behavior tree that decides which role assign to assign to which member. And finally, it acts. This mostly consists in writing directions in the NPCs' knowledge blackboards, so they'll play their role correctly when it's their turn to update. So, that's another important point. Individual behavior still exist. NPCs use the knowledge coming from the crew as any other source of knowledge, for example, their own sensors. So, if you disable the crews, NPCs just act as if they were alone, but they're not completely shut down. So, this is what I meant by finding a balance. If something good came out of the early 2000s craze of object-oriented programming, it's the single responsibility the single responsibility

principle. On one side of the balance, crews should be responsible for the coordination only. As soon as the coordination logic starts to smell, as soon as the coordination logic starts to drive the NPCs directly, things start to smell. Sorry about that. On the other side of the balance, the individual behavior logic should always work on a single NPC. As soon as it tries to perform coordination, it becomes a mess. So, our rule of thumb is that we try to avoid using any crew logic when a behavior involves only one and only one NPC. Now, you might wonder, how do you group NPCs together? We use a very basic request system. NPCs can emit a request to join a crew directly from their behavior tree. We also have crews controlled by the level designers, so they can create crew requests from our in-house visual scripting tool. This request is made of an objective and

a target, for instance, fight the player. So, just to show you what it looks like, this is a part of our individual combat behavior tree. The highlighted node at the tree root is what posts a request to join a crew, a combat crew in this case. At any given time, we sort requests by priority, and all NPCs sharing the same objective and target simply become members of the same crew. So, we allow an NPC to be part of only one crew. This is both simple and dynamic and avoids putting too much complexity at such a very high level. Now that we group NPCs together, let's see an illustration of group behavior. This is the crew behavior tree for regular combat. This is what drives the coordinated combat video I've shown earlier. It's nothing fancy. You see two branches, one for the roles assigned to NPCs currently engaging the player, and

another one for NPCs currently chasing the player. So, this is just a definition of what roles available for this part of combat. But that's only one part of the crew combat behavior. Here's what you get if you zoom out a bit, the big picture. As you can see, there's more than regular combat. Under certain conditions, the crew behavior tree changes and roles are assigned differently. So, now let me focus on the part labeled far target. This is activated when one member of the crew notices that their path to the player is way longer than the actual distance. So, that means that there are some piece of level that needs to be bypassed to reach them. So, what the crew does in this situation is that it splits the combat in two. A few members are sent to flank the players the player trying to reach the melee, while the others simply stay put taunting and using ranged attacks.

Note that we have two end conditions here depending on what happens to the flanking NPCs. As soon as any of them manages to reach the player, the far target tree is canceled, and the crew switches back to regular combat, which means that more NPCs will join the melee. But if too much time passes or if any of the flank any one of the flankers get killed, the crew marks the area around the target as unreachable. So, in this case, it's deemed too dangerous to try flanking the player, so we we'll just stick to ranged combat. So, let me show you these two outcomes in video. Here we have a few guards. I'm setting up a trap, and I'll try to lure them into it. They busted me, combat starts, and I get to a point a little bit out of reach. So, two of them are trying to flank me.

Only one of them gets killed, so the second one succeeds in reaching me, and everybody comes to melee. So, I didn't set enough traps. Let me try another thing. This time I'm putting two traps. Again, trying to lure them into it, and I get two guards killed. So, this time the remaining guards just decide it's safer to pick me from a distance. So, that's it for combat crews, but wait, there's more. We do use crews for more stuff. Everything I've shown so far is completely systemic, level independent. It can trigger anywhere in the game, but we also wanted to try you to use crews for ambushes and to let level designers set them up. There were a few cases in the first Dishonored where the scripted ambushes were messing

up with the systemic AI. So, this was an attempt to fix it. In Dishonored 2 level designers create an ambush crew from scratch, start and stop it as they want. On top of this, they manage the triggers and volumes that define the ambush area. They also place spots that NPCs use for positioning. Yet another case of coordinated behavior is search. We use the crew to determine number of NPCs to just search actively. In some cases, we even allow only one of them to search. There are also more of of coordinated behavior that may trigger when guards discover a dead body. They first gather around the corpse. One of them will shout a few orders and they'll split up in search for the culprit. If they fail in finding anything, they'll gather around again to debrief the situation before falling back to patrol. So, these phases are also handled by the crew behavior tree. So, before I close this, I I just want

to show you two pitfalls of this system. The first one is how to handle behavior trees interactions. I'm talking about how to control the execution flow between the individual and the crew behavior trees. For instance, you often want to know if some part of an individual behavior is done executing before you switch to a new stage of group behavior. We need some kind of notification for that. So, we implemented a simple dictionary that is readable and writable from both sides and is stored in crew knowledge. So, this is the trick. The second pitfall is how do you organize the updates in the game loop? Crews depend on NPCs and NPCs depend on their crew. So, that's a bit of a chicken and egg problem. It's important to keep a clean separation between the individual level and the collective level in the game loop. So, our solution isn't very interesting

since it's implementation dependent and changed a lot over over time, but please note that this is something that requires some thoughts to get right. So, clearly, this is just baby steps and we need to improve our implementation, but one thing is I wish we exposed more things in the crew behavior tree. For instance, the conditions for assigning a role to a given NPC could be defined in it. But for now, they're all written directly in C++ code. Same for the execution of roles. Something else we didn't implement is the gathering of crew members using some kind of psychic {slash} radio attraction. Could be used notably for coordinated distraction in patrol, but this remain at the idea level. And last but not least, this almost goes without saying, but again, NPC coordination is something very abstract and requires good tools for debugging. Ours were were pretty rough, so that's something we'll have to improve in the future.

So, here are my last words. Crews are sexy. We use them as a machete to clear the uncharted territory of NPC coordination. They allow us to separate the coordination logic from the individual behavior. Using behavior trees gives a visual representation of the flow of group behaviors, which is something very precious during development. So, I wish I will find more we will find more new use cases for all this in the future. Time to wrap up, Xavier. Thank you. Thanks, Laurent. So, here are some parting thoughts before we dissolve our crew. Um implementing Dishonored 2 AI was quite the ride. Even while being very conservative and prudent, sometime the good old keep it simple principle just does not cut it anymore in front of the horde of systems interacting together. That's why we had to commit to put

muscular systems behind simulation. It is interesting to observe that the systems we did the most research on were the ones that needed fewer changes over the course of the production. Whereas systems that grew more organically tend to be the one that really need need refactoring. Obviously, this is not possible to write technical document for everything. Sometime you just don't have enough visibility up front, but when you get the chance, take the opportunity to stop coding and start writing. AI is really about pushing proper content toward the player when it makes the most sense. Given the amount of assets modern games have to manage, a data-driven way to select those moments and the content is absolutely crucial to maintain a decent iteration rate. That's That's exactly what the rule system does for us. And coordination, it's a it's a hard topic. Hard to design and implement, hard to

debug and maintain. So, more often than not, games choose to fake it. Since the Left 4 Dead AI director, there has been a general trend of using some kind of master AI to drive something else than just individual NPC behaviors. Our experience with crews convince us that a top-level AI making suggestions to individual NPCs strike the correct balance in order to achieve elaborate coordinator behaviors. And lastly, we used what would seem an astronomic amount of memory in GSM. Likewise, creating a domain-specific language like the rule system can be scary in term of hardware and resource management. However, cost that were prohibity prohibitive on the last generation console can now be can now be totally reasonable. But people habits tend to change lower than hardware, which mean that if you are working on

current gen, you really really should try and have your lead allow you to use those extra resources. Please, use those megs, use the CPU cycle. Just use them so we can play cooler games with even cooler AI in it. Thank you. Thank you for listening. So, it is my duty to tell you that we are hiring. Rob is here. He's in the background. Yeah, Rob is here. He's our a global hiring guy at Zenimax. So, if you have some questions about this, just go see him after the talk. No, I think we have 3 minutes for question. That's right? So, over there. Good morning. I I was wondering, are the crews dynamic at all? Do you add and remove NPCs or kind of spread them around? You You're talking about the crews, right? Yes. How do we dynamically remove NPCs? Or just are they dynamic at all? Do NPCs kind of like They They are dynamic. They

are completely dynamic. As I said, it's the individual behavior that posts the request. So, basically, I'm currently searching. Find me another NPC currently searching and if there's somebody, let's let's group them. So, it's completely dynamic. They just they enter and and exit the crew as they need. So. Awesome. I see. Uh hi. I'm Sergey Mohov from Remedy. I have a question about the rule system. I'm wondering because you guys are using behavior trees and rule system on top of that. So, how do you make the rule system communicate with the behavior trees and how do you decide which parts of the behavior is going to be in the rule system, which part is going to be in the behavior tree because they're interchangeable essentially? Uh it's uh good question. Uh What? I can The thing is that the rule system does not really coordinate with the behavior trees. What behavior tree does is like saying, "Okay, it is time to play a bark." And then that would fire an evaluation and using rules

to select the bark that is played. Um likewise, um the behavior tree could say, "It is time to play an attack." And then rules are evaluated giving some context and say, "Okay, this is the attack that that that that is selected." So, So, so basically, rule system is just for the barks essentially, nothing else. You don't use it for anything else. It's So, I think you you mentioned all the use cases, but I didn't get that. You said the rule system was Was it like only for the barks or Oh, no, no, it's for more than that. Much more than that. We use it for triggering animations like attacks in combat or idle breaks, but we call idle breaks or idle animations, greetings. You can see NPCs greeting each other. All this is triggered by by rules. So, Okay. Thank you. One more question. One more question on the right? Okay, great talk, by the way. Loved it. Um how did you sort your rules cuz you showed some rules that were evaluating in order? How did we How dynamic was the designers control that the order of the rules were evaluated?

How do we sort the rules? Did I sort all of them? Sort them. What order did you process them in? Sort. Yeah, well, we sort them at comp No, sorry. Sort. As in which order how which order did you go through the rules trying to evaluate the conditions? Uh we just go from top to bottom. And that's designer driven, they sort it. design The designers write them in. Yeah. But what with lots of complex sort of conditions in there? Wouldn't they sometimes the one that they'd written last was much more relevant? I don't get that. Excuse me? Um if you've written three rules, yeah, for example, they might all be just written sort of in isolation to say if the conditions are this, do this. If the conditions are this, do this. But it strikes me that as the game is playing, a third option, for example, might be more relevant to that to the MPC who's trying to get priority. We're We're not dynamically sorting rules if if that's your question. Yeah, it is, yeah. So, it's it's the conditions that should be written in a way that the context may

gives us the the right rule at the right moment. We're never sorting rule after and time, just during initialization. As I said, we have some kind of cool down system. So, we sometimes filter rules. If they were previously selected, they won't be selected for a short amount of time, like for attacks. We We We don't want the same attack to be triggered twice in a row. So, I get that.