Authored vs. Systemic: Finding a Balance for Combat AI in Uncharted 4

[Music] all right good afternoon and welcome a quick reminder as usual please silence your cellphone's if you could please fill out the evaluation emails at the end I very much appreciate that thank you all for coming I realize it's you know before in the afternoon on Thursday hope you've all had your coffee I'll try to keep things moving along at a brisk pace so my name is Matthew gallant and I work on combat design at Naughty Dog if you a little background about me I'm from Montreal I've worked in the game industry for about seven years and these are some of the games I've helped chip I've worked on the combat design for the campaign mode in Starhawk I did the combat design for the infected encounters in the last of us and most recently I was in charge of Corps combat design for uncharted 4 a thief's end so my talk today is going to go over what we learned during this games development in particular I'm going to give you try to give you a sense of ideas roughly in the order that

like we tried them with all the intermediate steps like warts and all to give you a sense of our own learning curve as we develop combat for the game today I'm going to be discussing my own work but also the hard work of everyone on the combat design and AI engineering teams so we're gonna give a quick shout out to my teammates I'm really glad to have an opportunity to shine the light on all of their hard work shout out as well to our effects artist Nikki who took this cool group photo so this talk is going to be in two parts in the first part I'm gonna start by talking about combat design in the first three Uncharted games then I'm going to move on to our design goals French art at four what did we want to try that was you know new and different I'm going to talk about our fundamental AI technology what did we already have and what we were kind of building off to try to achieve those goals and then I'm going to talk about some things we tried early in development that didn't work then in part 2 I'm going to talk about the solutions we actually shipped with I'm gonna talk about some evergreen AI techniques some things we've done for a few games now that have just always been

really useful for us some limitations of our current system things we'd like to improve on in the future and then I'm going to close with some conclusions and takeaways so uncharted 4 is fourth game in a series obviously cement we started development with certain assumptions regarding approaches to combat design that had worked well in the past we inherited the engine from the last of us which has just been overhauled to take advantage of the power of the ps4 so in the first three Uncharted games the macro combat flow was very tightly authored designers relied heavily on zoning NPCs limiting the area that they're allowed to move and fight from so that green cylinder you see on screen that's the debug display for an NPC zone and the NPC will only fight from that area it was there for up the designer to identify key positions and strong areas in the layout and to make sure that the NPC's could cover the players progression through a fight zones also helped NPCs to spread out evenly across the entire combat

front some NPCs can be zoned to the left some to the middle and some to the right this helped prevent the NPC's from clumping up and it gave the appearance that they were using the space intelligently so here's an example of a fight from uncharted 2 I'm gonna have the debug on so you'll see the NPC's zones show up as those giant cylinders in the world I should actually plays okay so in this fight there are several NPCs assigned to a zone that is centred on the player and the zone actually moves around the player as they move this type of zoning forces the enemies to charge the player and keeps the pressure up no matter where the player moves in this big room further back we have two NPCs that are zoned to a large region around the staircase they're guarding the exit so this zoning does two things firstly it helps establish a front for combat kind of a consistent direction of threat and secondly it aids the player navigation if the last NPC they kill has them facing the exit to

the next area finally we have one sniper NPC assigned to a tiny zone at the top of the staircase he's only allowed to fight from that specific position the designer knew that this was the best sniping position you know I can see most of the map so effectively this zoning is baking the level designers intention and knowledge into the combat scripting zone assignment doesn't always have to be static designers can also set an NPC zone dynamically in script allowing them to change the zone based on the player's movement so in this fight the players pinned down by a turret and can choose to flank it by advancing left or right if the player goes left then the NPC with the purple zone stays on the balcony and this sets up the player to shoot him from cover however if the player goes right then that same purple NBC is assigned to a different zone he jumps down and takes cover on that

electrical box and this sets up the player for a cool propane tank kill so this is a really great fight but it shows that the combat designer had to micromanage exactly where the NPC's would move based on the players movement that's a lot of manual scripting so we can also assign new zones to NPCs as a fight progresses in this a fight the NPC's in the back are scripted to move up and advance on the player when the armored shotgunner dies this improves the pacing of the fight and keeps the pressure on the player so these tightly authored approaches to combat design worked really well for those previous uncharted games however we knew we wanted to try some new things in uncharted 4 and that might make us want to reevaluate some of our approaches to combat design whereas the previous games were fairly linear at fairly linear in general we wanted to explore much larger layouts in uncharted 4 this is a

screenshot from Madagascar which had these massive vehicle scale spaces that you could approach from any angle it's not quite an open-world game we referred to it as wide linear and our scripted approach to combat design would struggle to account for all the ways that the player could approach this fight Nathan Drake's climbing ability is one of the things that makes the Uncharted series unique we really wanted to push that aspect further in uncharted 4 and give the player lots of opportunities to climb during combat of course that meant having these vertical combat spaces requires NPCs that can reason well in 3d mentions another new feature was skippable combat which means that you can leave some combat spaces in stealth without killing any or all of the enemies behind you this isn't a new feature in games but it was kind of new to the Uncharted series from a design perspective this meant that we had to come up with all sorts of interesting mechanics so the player couldn't escape these encounters trivially and from the AI perspective this made it even more

important to identify important positions like the exit we also want to support a full stealth loop in previous Uncharted games this usually only went one way combat started once the player broke stealth and the player could never go back to fully sneaking in uncharted 4 we wanted to give the player the opportunity to break line-of-sight and regain stealth forcing the enemies into search behaviors fortunately we had some experience with this loop from the Last of Us NPCs are fairly expensive in our engine we've always had a tight limit on how many you can spawn at once next-gen hardware meant that week ago we could double our maximum count from the ps3 going from 8 to 16 so we wanted to take advantage of that of course having this many NPCs meant that it was incumbent on us to give them all somewhere to go and something to do we also had the miner goal of reducing the overhead for combat design and authoring every combat setup

was a work intensive process and it was also fairly error-prone if the player did something that the designer didn't expect then the scripting might fail to respond accordingly so we wondered could we reduce the amount of scripting required for combat so with these goals from charted for in mind I want to give you a brief summary of our existing AI tech does she give you an idea of what we were working with to try to achieve those goals so I'm gonna give you that quick version but if you want a really in-depth look I recommend listening to Travis Macintosh's GDC table from 2014 it's called The Last of Us human enemy AI it's free to watch online you can just google for it a lot of what I'll be saying in the next couple slides is really just a more in it's covered in much more depth in his talk see a few people photographing that so let me do that so skills and behaviors are the building blocks of rai skills are a prioritized list of

high-level concepts like panic combat melee each skill determines internally whether it wants to be on or off and then the highest priority skill takes control each skill also has a state machine inside of it for instance combat skill has an open combat state and a cover combat state the same skill handles both when a skill makes a decision it pushes behaviors on to a stack behaviors are reusable low-level components such as move to a point or shoot from cover behaviors are then the interface with the locomotion controller that actually moves the NPC around and interfaces with the animations now I'll go over the layout let the level mark out we had for the NPC's so here's a combat layout as the player would see it and here's our nav mesh nav mesh is hand placed by the level designers and it defines where the NPC's can stand and run and just do normal traversal across you can also see some special green navmesh Poly's in the bottom left corner

that's stealth grass a mechanic we added an uncharted forward to help the player hide from enemies we wanted the NPC's to be able to take the grass into account when navigating either intentionally avoiding it or like intentionally moving through it and searching it traversal act a traversal action packs or taps allow the NPCs to traverse using special animations these include jumping up and down jumping across a gap vaulting over cover stuff like that so tap animations are authored with an intro a loop and an exit and this allows them to be placed with flexible Heights and lengths nav ledges are a new markup for uncharted 4 to indicate connectivity between climates this is really cool new tech that allowed the n BCS to leverage some of the players climbing animations unfortunately it was mostly for the buddy npcs to allow them to climb with the player the enemy NPCs generally didn't climb unless we had like a really compelling layout reason to let them do it

so our engine takes all of this hand placed markup and automatically generates these posts posts are discrete points in the world that they are going to evaluate they're going to use to evaluate sorry they are going to evaluate to determine where they want to fight from so in this screenshot you can see several different types of posts Green is an open post orange is a cover post there's a couple of them kind of back on the right teal is a perch post you can see those on the edges of the cliffs and purple is a climb post as I said per those climb posts are mostly for the buddies so skills use post selectors to choose the best position to fight from post selectors basically give a score to each post based on various criteria then they select the post with the best score these criteria can include can that post see the player what's the NPC's path distance the post is the post in the NPCs zone etc some

selectors reject posts entirely others are more fine-grained and they give the post to score between 0 and 1 and then all those scores get multiplied together to create the final value so here's an example of post selection in action so here's an npc he's in cover he's shooting at the player and then he decides he wants to advance on the player so why did he pick that position to advance to so if we zoom in here the text is probably still too small to read but these posts are all green because they're scored very highly for the event skill like they're close to the NBC they've got the target lined up just as they want it actually the reason that leftmost post one out that was actually selected is that this NPC is a rifleman and riflemen like to be a little further away from the target so that leftmost post was the chosen post and it was the most ideal of this kind of set of really good options for advancing then as we move over to the right here we'll see a bunch of posts that were technically

acceptable for the advanced skill but they were rated really low they're really close to the target they're really far away from the NPC they're much closer than a rifleman want to be so these are technically acceptable posts for the event skill but they won't be chosen because obviously there's much better candidates out there then as we zoom out we can see some posts that were just rejected in entirely they effectively have a score of zero so these include things like covers that are facing the wrong way or covers that can't see the target this NPC is advancing so we don't want to pick covers that are behind the NBC he's advancing so all these posts just got rejected entirely at some point in the post selection process and it's actually really useful to have post selectors that are just this course that can just accept or reject things it's conceptually much easier to write them and they really kind of let you narrow down on exactly the kind of posts you want so this NPC reaches his post and then

he's gonna keep fighting from cover and he's good to go okay so here's the fun part here are some things we tried early in development that didn't work see here we've got an NBC that's been assigned to a zone that covers this ruined building it's the sort of thing we did all the time in the previous uncharted games it's not obvious but this NBC is standing in the open because he's actually completely broken at the moment why is that well this NPC is in combat skill and combat skill does post selection around the targets position target being the player however in this massive layout the MP said player is really far away and for performance reasons NPCs can't do post selection across the entire layout the post selector is limited to a 25 meter radius roughly centered on the target the post selector checked every single post declared them all invalid because they're outside at the NPC's zone without a post the NBC doesn't know how

to move or what to do so he just defaults to just standing there now obviously this isn't a terribly difficult problem to fix from like a a technical perspective that's that's really obvious but it does reveal a pretty fundamental design flaw we have a mismatch between our scripted zoning and our systemic post selection so if we can't use zones in this large layouts we'll need a new system to help the nbc's spread out an intelligently occupy space we want them to spread out horizontally like across the combat front we also want them to have layers of depth on MBC's a short-range mid range long range etc our first naive attempt at doing this without zones was too strongly enforce the desired weapon range in the post selector really enforcing that we will the pistols have to be a short range the rifles have to be at mid range we hope that this simplistic logic would be sufficient to avoid the NPC's clumping up unfortunately trying to keep NPCs that

their ideal weapon range can make them extremely unstable the player can move very quickly and uncharted so the distance the target is constantly changing these NPCs can never settle into combat positions because their desired weapon range is always telling them to reposition so this naive approach was a blunt instrument if we wanted to evaluate good combat positions our approach would have to be much more nuanced and complex so we started trying some more of these sophisticated approaches again trying to move away from the idea of having to rely on the scripted zones all the time we wanted to see if we could identify strong areas in the layout programmatically so we tried two approaches for this evaluation connectivity and vantage we already have the navmesh mark up that the AI used to navigate and we can use that to generate a map of the layouts connectivity could

we use that information to identify relevant features stuff like pinch points and doorways if the player is in a building could we send an NPC to cover each one of the exits unfortunately when we started running this logic on more of our layouts we found the connectivity couldn't consistently produce useful information while I could handle man-made structures pretty well like houses connectivity struggled to detect anything useful we ran it on big organic spaces like this one that slides really dark it also struggled in our wider vehicle scale spaces there just aren't enough important nodes on this spiderweb of connectivity so connectivity was an interesting idea and it might have worked pretty well in our previous game The Last of Us because it's layouts were generally tighter and narrower but it wasn't a good fit for uncharted Vantage was another kind of programmatic evaluation and we developed it to try to fix a problem with watchtowers let's say

we designed a layout with a watch tower and we put a sniper up there to shoot at the player if the player took cover then the sniper NPC would determine I can't see my target I need to reposition but in repositioning he would jump down off the tower that's terrible so we needed some way to make the NPC's prefer elevated positions even if those positions can't currently see the target to do this we try to evaluate combat positions using this concept of vantage so in this screenshot you can see the player and cover on the right and a strong elevated position on the left how could we make NPCs like that position even though it can't currently see the target so to evaluate vantage we gathered all the navmesh Poly's in a small radius around the player the borders are a little uneven because of the navmesh triangulation but this orange area roughly represents where the player could move in about 3 seconds of movement vantage does a raycast to each

of these positions and counts how many of them a given position can see so in this screenshot green ray casts are clear and red ray casts are blocked you can see that the elevated position on the left has clear line-of-sight two more positions around the player versus the other lower position being tested in the top left this means that the elevated position would have a higher vantage score and the nbc's would thus prefer it so we tried these two programmatic approaches to identifying strong layout positions but they kind of both have the same fundamental problems firstly they couldn't consistently produce good results in every layout we have a pretty big variety of layouts in uncharted 4 we could tune these algorithms to work in one but then those same tuning values would just totally fail in another kind of layout and then we maybe retune there and go back and forth it just never worked consistently everywhere more fundamentally we were making the

mistake of trying to reverse engineer high-level decision-making the level designer had already created a level with combat in mind and had purposefully designed certain areas to be strong we were then throwing out that knowledge to try to achieve the same results programmatically wouldn't it be smarter to bake the level designers intention right into the layout programmatic evaluation also limited that players ability as Arie limited the designers ability to make each encounter feel unique if the NPC's fight the same way in every layout then we run the risk of every encounter feeling the same the designers need more control more knobs if they're going to craft setups that feel truly unique so I'm going to come back to all of those issues in the next section but for now I'm going to move on to search so I mentioned earlier that one of our goals for uncharted 4 was to allow the player to re-enter stealth from combat to do this we needed to develop a new skill to make the nbc's search for the player after they'd snuck

away so our first version of the skill used heat to represent how recently an npc had cleared an area the initial heat was seated from the players last known location areas would heat up when unseen and the NPCs would cool them off again by clearing them out with their vision so you can see in this screenshot that pink means hot and blue means cool just like real life so let this play out for a little bit so you can kind of get a sense the loop here the NBC picks a hot position that's nearby it goes and clears it out and once they've cleared that out they picked the next nearby hot position ABC just faked us all out and went off camera then he decides he wants to check on top of that plateau so the heat based

search looks pretty cool with the debug tools on because from this perspective it's easy to understand why the NBC is choosing these points to move to but it's quite different when you experience this behavior from the player's perspective so heat search has three main problems in heat based search npcs change directions in unpredictable ways without seeing that heat information it's really impossible to find a pattern in the npc's movement there's no consistent line of motion it doesn't look very human and it feels really bad as the player as the player when the NPC you're about to stealth kill just turns around and busts you if the player can't predict where an NPC might move next then the player can't plan their approach in stealth it robs the player of the ability to do clever things secondly he based searched as a poor job of spreading out the NPC's they

tend to bunch up going after the same clusters of hot nodes if all the NPC's are searching in one area then the player can just hang out in another area without feeling any pressure thirdly it tends to pull the NPC's towards the nooks and crannies on the outside of the combat space if there's a little pocket out there that can't be seen Keith's going to accumulate in there and eventually draw on an NPC to go check it out we'd much rather keep the NPC's near the center of the map near the strategically important positions like the exit so all of these symptoms come from the same really fundamental problem which is that we design search skill based on the perspective of the NPC's searching for the player counter-intuitively that was completely wrong and again I'll explain why that is in the next section so if we think about authored and systemic as two ends of a spectrum we came into uncharted four on the far left

side of that spectrum with the design philosophy of the previous games highly authored tightly controlled by the combat designers and then in the early days of Uncharted forest development we swung pretty heavily in the other direction trying to develop approaches that were just highly systemic and relied much more heavily on algorithmic evaluation ultimately we felt that we had over steered away from offered content our aversion to scripted combat was robbing us of an element that made the previous games play so well we started thinking was there a middle ground that would let us avoid the drawbacks of the highly scripted approach could we get the best of both worlds so onward to part two of the talk or I'll be talking about the things that actually worked so I'm going to start by going over the new tech that we developed and actually shipped with it's just one bullet point on this list that this is the biggest section of the talk so so far I've

talked a lot about zones and zones are really a blunt instrument a designer assigns a zone to an NBC and the NBC uses that zone to modify their post selection could we make a more nuanced version so we developed a new concept called a heart point it looks like a zone but it's an object that exists in the world independently of any NPC NPCs can be dynamically assigned to a heart point and they will fight as if it's their zone and I'll explain why that abstraction is so crucial a hard point has five properties that is a name that can be referenced from script it has a region that limits where the NPC will fight from it has a minimum and a maximum number of assigned ABCs and it has an enable/disable toggle that can be controlled from script so here's an example of how hard points were used in one of our encounters the green heart point on the Left covers the strong elevated position in the tower the

yellow heart point covers the open field that has good lines of sight to the rest of the layout and the teal heart point in the back covers the exit door it has a minimum assignment of one because we always want to keep one NPC there to prevent the player from sneaking out too easily so this debug shows the actual region for each heart point I set it off before because it's kind of hard to read but this is the actual extent of each hardpoint defining where the assigned ABCs are allowed to fight from so in this example these two NPCs are assigned to a hard point on the bridge with a minimum assignment of one and a maximum assignment of two so they're happily fighting from the bridge and holding down this position but when reinforcements spawn behind them the new MVC's are going to take over that hard point assignment and this frees up the initial two NPCs to push up on the player so this reassignment happened

just completely systemically just by respecting the min and Max counts as defined by the designer it would have taken much longer to achieve this kind of dynamic hand off with just a purely script approach picking which NPCs to assign to each hardpoint is handled by a code module called the hardpoint coordinator it makes selections based on various criteria such as distance the target weapon type it also generally tries to avoid assigning the ABCs backwards it always looks better when NBC's can move forwards to their objective so I mentioned earlier how we had some goals for combat things like spreading out the enemies and creating layered combat with NPCs at different ranges I also showed what happens when an MP sees skill doesn't match their zone so our solution to all three of these problems was to implement combat roles but before I talk about roles in uncharted 4 first want to talk about the ghosts in pac-man

they're with me so each ghost has a different role and fundamentally these rules were designed to achieve the exact same kind of design goals my information on the pac-man AI by the way is based on the excellent article the pac-man dossier by Jamie Pittman it's on Gamasutra so I shout out to Jamie for letting me use some of his diagrams in my talk so the red ghost blinky is very simple he just Pathfinder pac-man's current position and constantly moves towards him blinky chases the player from behind and pressures them to keep moving pinkey path finds to a position two spaces ahead of the players current heading so this simple behavior creates a cool dynamic where blinky closes him from behind and pinkie gets ahead of the player and cuts them off it's actually really impressive to achieve that dynamic with such simple behaviors Clyde is the least threatening ghost when he's far away from the player he acts like

blinky and chases to the players current position but when he gets too close he paths finds away to one of the corners of the maze this makes Clyde bounce back and forth keeping a somewhat consistent range from the player and a fair defending his corner of the maze finally inky I shall get back to inky later for now let's go back to uncharted 4 so the three core combat roles in uncharted 4 are engagers ambushers and defenders these are the three layers of combat in ascending order of distance from the player engagers are simple they approach and attack the player's current position they can move wherever they want and perform typical NPC combat behaviors ambushers do this they pick a position to take cover and then they wait for the

player to move their way so they can pop out and Surprise them we dynamically generate ambush points 20 metres away from the player along the navmesh an ambush er claims one of these points and moves there to take cover with the intent of anticipating where the player could move next the reason for this behavior is that during our play tests we often notice that the player would be fighting in one area maybe having a good fight you know things would be pretty well paced and things would be good but then they'd you know if things got too hot and they needed to disengage they'd run to a different part of the layout and would just be completely empty ambushers help the space to feel fuller because as the player moves around these ambushers will suddenly pop up pop out out of cover and engage them and it keeps the pressure up on the player and then defenders are the NPC's that are assigned to hard points they will not leave their hard points but they will do

their best to get line-of-sight and take cover within their assigned area so how do these three roles compared to the pac-man AI well blinky is an engaged because engagers chase the players current position and pinkie pinkies like an ambush sure because ambushers head off the players future position Clyde is a defender because defenders keep their distance from the player and they hold down their corner of the maze Clyde's a little bit of a stretch truthfully but it's a bit similar all right what about inky so inky is considered the most dangerous ghost due to his erratic unpredictable behavior to determine inky's movements create a line between the player's position and their current heading then take the red ghost blinky and mirror blinkie's position across that line that is the position that inky will chase to sadly this weird behavior has no equivalent in uncharted 4 so you remember earlier how enlarged

layouts we had a mismatch between our scripted zones and our post selectors while having roles fixes this issue engagers want to fight at the player's current position so they do their post selection centered on the player ambushers do post selection around their ambush point that they've been assigned to and defenders do heart a do post selection around their heart point therefore nobody has the problem of trying to do post selection outside of where they're allowed to fight from earlier I mentioned that we had a code module that assigns the NPC's to the hardpoints well we have a similar code module called the encounter coordinator that's responsible for assigning nbc's to roles or backwards assigning roles to NPCs for instance we have a limit on the number of engagers an NPC can request to be an engager but the coordinator gets to decide who actually gets that role it

selects the NBC with the best rating based on various criteria again stuff like distance the target line-of-sight weapon etc so here's an itty group here's the nitty-gritty of how this role assignment actually works let's say we have a group of you know 15 NB C's we need to assign out well first we assign NPCs to each hardpoint up to chart points minimum that's absolutely crucial we absolutely want to get the minimum for each hard point as the first priority after that we assign a certain number of NB C's to be engages so that number X is actually defined in a script file called the global combat params I'll talk about that file more in just a minute for now it's it's usually about two or three so pick two or three of those npcs that are close to the player and they'll be the engagers then we pick another a number of npcs to be ambushers again that number Y defined in the global combat params usually about two or three pick two or three npcs to be ambushers then from the remaining

npcs we assign them to hardpoints up to each heart points maximum and then if we have any NPCs remaining after all that those NPCs default to being engagers but we don't actually want that step five to happen if that happens that's kind of a fallback that means that the the encounter will have more engagers than the designer was intending so actually if we ever hit that step five during development a big warning appeared on screen letting the designer know that they had to increase the limits on their hard points or you know start adding new ones so here are those global combat params I'd mentioned these params are tuning values that we can customize per encounter this allows us to have systemic combat behaviors while still giving the designers lots of control to customize each encounter and make it feel unique so this slide with the source code is probably a little hard to read unless you're really familiar with Lisp so I'm gonna go through some of these individual combat params on a separate slide so global combat params we control

things like as I mentioned the number of engagers and the number of ambushers we also had to find that a time delay to replace an engager if the player shot one of the engagers and killed them do we want a new engager to be assigned immediately or do we want to have a little bit of delay on that that would impact you know how hard that the nbc's push up on the player in a given encounter we also defined the snuck away distance for stealth how far away from their last position that from the player's last-known position do they have to move before the NBC's consider them to be snuck away and then lost and then they go into search behaviors this tend to be something we tuned kind of per the size of the encounter space we had a timer for the NBC's advancing from cover to cover we had a timer for how long until an NBC would start throwing grenades we also had a timer for how long until we sent out an embassy to lank the player we also controlled the minimum flank path rating so that meant

like how good does a flank path have to be before an ABC will choose to use it if you reduce this threshold the NB c--'s might choose bad flank paths but it'll make them more aggressive and they'll push up on the player much harder now finally we had the minimum and maximum number of shooters which kind of presents an interesting design question about limiting the NPC's desire to shoot so if you have 10 NPCs that are all allowed to fire at once then 10 NPCs are 10 times more dangerous than one NPC and we don't want to scale combat that way because it creates combat that starts out really hard and then gradually gets easier with every NPC you kill instead we want to limit the number of NPCs that can fire at the same time usually capping out at about three or four this gives us leeway to use more NPC's in combat without greatly scaling up the difficulty however we need to disguise the fact that some NPCs are intentionally not shooting so I'm gonna

let this video play out for a little bit I thought this video had sounded it doesn't so we tried to make the handoff between NPCs being allowed to shoot as smooth and as seamless as possible and I think it turned out pretty well so what's the logic behind this so shooter is also a role but it's orthogonal to the other roles engagers aren't necessarily shooters and vice-versa only NPCs that have the shooter role are allowed to shoot and then they give up that role when it's time for them to reload or they lose line of sight etc this allows us to artificially limit the amount of gunfire towards the player without making the abcs look bad here's a still view of that hand off in the previous video the two NB C's circled in white have the shooter role and are allowed to aim and fire and then to MBC's without the shooter role just hunker down and cover waiting for an NB C to give up the shooter role but they

look just fine while they're waiting okay getting back to search let's talk about our what we shipped with for our search solution so our solution for search was to have the designers mark up the levels with spline loops NPCs use these spline loops as a guide and search and will flow from segment to segment being died animate that by being dynamically reassigned as they move they're allowed to deviate a little from their splines if they want to check out an interesting feature or go search behind cover or look down the edge of a cliff stuff like that otherwise they stick to those lines pretty closely so these authored loops help fix some of the issues that heat surge had firstly NPCs maintain their line of motion better which makes them look more natural and it also makes their motion more predictable for the player enabling the player to confidently plan for stealth kills these NPCs aren't going to

suddenly turn around and just bust them secondly the NPC's stay near the spline loops and because the loops are authored by level designer we can choose to only place them near the interesting parts of the layout this avoids the problem the heat search had with NPCs wanting to go comb the outer edges of the layout where there wasn't going on and thirdly we limit the number of mbc's assigned per spline so they never bunch up too much actually having to NBC's on the same spline can facilitate some cool performances like having the NBC's check in with each other or they can pair up and search together for a little while it's really useful how about the NPC's are how how about the NB C's that are signed to hard points how can they join in the search what we don't want them to just abandon their hard points so they're not allowed to go walk along the splines instead we modified the concept of Vantage to instead test what points the NPC's can look outwards from while staying on

there hard points this gives them a good behavior for search that doesn't pull them away from the area that they're supposed to be guarding finally here is a quick grab bag of evergreen AI techniques the stuff that we've carried with us for several games and they've always been really useful so we just kept using them in uncharted 4 so I mentioned that we had a script file called the global combat params it was actually named this way in contrast to the just combat perhaps that are set individually for each NPC in uncharted 4 all of our enemy types are actually running the same fundamental AI we use the combat params to create the distinct enemy types snipers shotgunners heavy weapons etc this setup is extremely useful because it's flexible and extensible let's say you happen to have one combat setup where you need a rifleman who doesn't take cover you can define a new combat purim as a child of the default rifleman and just toggle the boolean that allows taking cover we

actually ran into a situation really late in development where we wanted to create an armored grenade launcher we were able to do so easily by combining some elements of the unarmored grenade launcher and some elements of the regular armored NBC we match those together and created a new combat parameter just worked right away it was great we actually have a nice synergy between some of our classic enemy types from the previous games and our new concept of combat roles in uncharted 4 we made it such the shotgun NPCs or just always engagers so they're always able to push up on the as they please on the other hand snipers are just always defenders we set them up with a hard point with a strong position at good lines of sight and then they just snipe from that area in a way it's a more systemic version of the sniper with the tiny zone that you saw in that video from uncharted 2 so combat in the Uncharted series is at its best when the players take advantage of all the traversal abilities available to them we

really want to disincentivize the player from just hunkering down in one place and playing it like a cover shooter so to gently suggest that the player should keep moving we have the enemies through our grenades grenades or the gentle suggestion this is managed by a simple module called the grenade director and it just tracks how long the player remains in a certain radius if the player leaves the radius reset the timer but if the timer runs out request an NPC to throw a grenade grenade thrower is a role and maybe see what the best clear art is chosen to take that role so here's how it works in practice player remains in the same radius for too long the timer meter you see counts down and NPC is chosen to throw a grenade grenade gets thrown and the player is gently suggested to move so flankers serve the same purpose push the player if they've stayed still in one place for too long flanker is

just another role that an NBC can be chosen for and their job is to get behind the player while avoiding line-of-sight we had a very similar skill in the last of us and for that game we tried all sorts of complicated logic that involved like calculating the exposure along the entire path in the end we actually had much better results with a much simpler algorithm and we carried that approach over to uncharted 4 so we use the combat vector which is that weird yellow oblong polygon it's oriented from the players position players at the bottom of the screen so it's oriented from that position towards the average direction of threat of the enemies so flankers use a path find that makes moving through the combat vector more expensive so that's a really simple behavior but we've really great results with it and all sorts of different kinds of layouts it's just a great all-purpose kind of behavior accuracy ramping is a tuning value that I highly recommend so from

the time they spot the player NPCs have to ramp up to full accuracy over time based on their distance this gives an advantage to players who want to stay agile and keep repositioning they'll have a chance to move from cover to cover without taking too much damage players who just turtle up in the same position will not receive this benefit so you might recall that we had a special kind of post called a perch they were the teal posts that we generated on the edges of cliffs and stuff so I'm going to play this video to show why they were necessary [Music] so perches are a special type of post that we generate on the edges of cliffs and over thin railings they allow the NB C's to play custom animations that lean further out and help them our aim around and downwards we found that without them the default aiming animations really

struggle to deal with these complex 3d climbing spaces now I'd like to briefly talk about some of the limitations of our current systems and things we'd like to improve on in the future so the scoring of post selectors has a lot of magic numbers has a lot of handwritten distance curves and it takes a long time to tune these values in a way that works in every layout and once these criteria have been tuned it's really impossible to go backwards and discern the thought process behind them I really don't have a good solution for this but I wish we could develop this waiting the waiting in the post selectors in a more transparent methodical way the logic for generating ambush points is not very smart it simply looks twenty metres away along the navmesh so in this screenshot the players at the top of a cliff and the navmesh distance doesn't take into account the steep drop these ambush points at the base of the cliff are technically 20 meters away but their positioning below the player isn't very

useful fortunately we had a simple workaround in this encounter we set the number of ambushers to zero using the global combat params but in the future I'd like to develop more a smarter heuristic for generating ambush points and finally there's a combat balance problem that we struggled with and you'll see it in this video so the player can snipe in NBC with a pistol from about 50 meters away and this is because we found over and over in our play tests that players expect the the guns to hit the sender the reticle no matter what unfortunately this limits our ability to make short-range weapons and it makes chipping away from nbc's at long-range very effective any time we had a really good effective play tester it was playing on the higher difficulties this is all they do we did we did do a couple of things to mitigate this strategy we

added damage wobble on the camera and secondly we have an accuracy floor for the enemies it's even an extreme long-range there's a limit to how low their accuracy will go ultimately though I still consider this to be kind of an Opie tactic and it's one that I'd love to find a more holistic solution for all right in conclusion through uncharted forest development we explored both extremes of this spectrum highly authored approached a combat design and a highly systemic one ultimately we landed somewhere in the middle and I think that gave us the best of both worlds my suggestion to you is early in development try to figure out where your game should land on this continuum and be cautious of the extremes and now here are four things that I hope you'll take away from this talk takeaway number one don't reverse-engineer high-level decision-making take advantage of your level designers knowledge unless you're making a completely procedurally

generated game baking designer intent into the layout is often better than trying to reverse engineer it algorithmically takeaway number two design NPC behaviors from the player's perspective when we made Heat Surge we were thinking about searching for the player from the npcs perspective in a sense it's like we were designing fun for the npc it's better to start from the player's perspective what experience do you want the player to have what decisions do you want the player to make then design your NBC Bay accordingly takeaway number three author roles then assign npcs to them programmatically deciding where to draw the line between the authored content and the systemic behaviors can be really tricky the approach that worked for us was to have the combat designers outline the desired roles for an encounter but then allow code to select the NPCs that were actually going to fulfill those

roles this is a great way to handshake between these two parts of the system and takeaway number four if you have a large number of NPCs then only a few enemies should get to directly engage the player in that case the outer ring NBC's need a goal other than also shooting the player or they're all going to look like young kids playing soccer all gonna be chasing the ball it can be useful to think about it spatially if an NPC shouldn't chase the player then where should they be standing that's it thank you very much [Applause] [Applause] I have a little bit of time for questions if you want to come up to the mics I can take your questions hi my name's matt from insomniac i'm

just curious the the the one with the chipping away with a pistol the obvious solution i I would think all would be reduced the accuracy of the weapon like did you try yeah we had a kind of minimum deviation on the first shot of the pistol but any time you know you see the reticle and I guess from one perspective you can think well the shots gonna land anywhere in the reticle that's fine often from the player's perspective that's I had 80% of the reticle over that NPCs head and I fired and I missed these guns feel sloppy and terrible I and and often actually of NPCs if playtesters felt like they couldn't rely on the guns they just went crazy with melee they just loved mailing because they felt at least that's reliable when I hit square I know what's gonna happen so ultimately that was a concession we had to make we had to make it so that the first shot send her the reticle no matter what and then that kind of unfortunately had the knock-on effect of this kind of unfortunate long-range

behavior that yeah I don't know this is something we're still trying to fix okay thank you thank you I'm curious about what kind of iteration times all of these different approaches took before you came today when is it really worked well we had kind of the luxury of being able to do a fair bit of this in pre-production so as before we really got ramped up before we actually were making you know all the layouts for the game it's funny actually in a way when we started heading towards production and we're making you know more than our couple of a handful of tests louts we had when we started seeing the real layouts thinking was gonna have that's when actually a lot of the issues fell out and we had to really think on our feet and come up with new stuff and so I would say hopefully a lot of the things I said up there that we tried and we didn't work we didn't spin our wheels for the on them for more than a week or two some of them may be longer but that was kind of our loss we had the good fortune of kind

of landing upon our solution of like hard points and rolls within like the early period of actual production and that let us go forward like confidently with all our layouts we were making during production Thanks I guess look over here Hey for when buddies were involved in combat did they take on some of the roles the enemy's head for ambushing and engaging or how did you balance that I didn't work as much on the Buddy NPCs I actually the biggest thing like combat with buddies didn't turn out to be that hard we kind of gave them a targets we gave them tons of melee behaviors to help the player that's kind of something that's I didn't cover at all in this talk but as a whole other complex system as melee so combat tended to be really decent that's not a very interesting answer actually it was stealth that was a lot harder with the buddies and that was something that obviously we were sensitive to and we tried to push a lot harder on and we worked really hard to make sure that not only did that buddies

never like get you busted but also like they never as much as we could walked in front of an enemy or like we did all sorts of interesting looking we did like a future exposure map to detect like where we'll be exposed and keeping the buddies out of those but actually their combat behaviors were very simple generally it was the biggest section of the game where you had two buddies with you was Madagascar and generally kind of fortunately that had really big spaces so it actually gave us a lot of leeway to account for them trying to find good positions as well I think we would have struggled to have two buddies through the whole game and through some of the tighter encounters so I'm kind of rambling at this point it wasn't generally that hard and they didn't use the same behaviors that's high for the minimax shooters manager did you ever run into cases where a player just got really close to a certain AI that wasn't able to shoot because he couldn't get a token because it was being blocked by

other AI that Pershing from him farther that weren't as critical to like the the current like engagement encounter that you're experiencing in the guy I just kind of like either stood there ran away or maybe melee but it felt like strange I decided not to shoot so we didn't typically run into that we do do some prioritization so that closer NPCs get the shooter role more readily than further NPCs I can't remember the exact details but we may have done some things as well it's like if the player directly engages with an NPC we really prioritize that giving up the role we also kind of fudge it with giving up the token like I mentioned we you give up the token when you reload but we actually if you count the bullets they're not always completely consistent about like how many bullets they should do before they reload we sometimes say no this this NPC really needs to shoot just let him shoot don't he technically needs to reload right now but he would look really silly if he did so so so we did some fudging like that and then yeah as I said prioritization based on distance which really helped so something I that struck

me when you sort of move from more authored to more systemic and landing in the gray area was the NBC is more or less communicating with one another they seem like they're all making sort of their own autonomous decisions and like this sense of tactics sort of emerges from it but there's those elements like because he's suppressing should I be more likely to flank or because this could this grenade thrower just got selected does it is this a great time to become more aggressive and the NPC's talking to one another even through a manager did you experiment with any of it and did you get any good results or we did things kind of along those lines one of the skills that I didn't go to super high detail on was that like advancing from cover to cover skill and that will do things like okay this NBC is going to be advancing from cover to cover so this other NBC should lay down suppressing fire or forget the other things you mentioned but like we can talk about like flankers like we only send out a flanker if there are enough other NPCs doing interesting things

putting pressure on the player like we try to pick a flanker who is not already doing something interesting we try to pick someone who's doing something kind of boring or sitting and cover somewhere and can't do anything better okay pick them to be the flanker so there's some of that kind of fell out from that but I'm not sure if answers your question but hi I was just wondering how many about how many engineers were sort of supporting the prototyping phase of these different systems five five or six yeah very very small programming team very lean I mean it was kind of the people you saw on screen part part design team part programming team really small thank you Jerris I think that looks like it if anyone has any other questions feel free to grab me and we can go to the wrap-up room thank you very much for coming [Applause]

you