Hacking into the Combat AI of Watch Dogs 2

[Music] thanks Dave get everyone to hear me okay you can have to excuse me I'm suffering I come from Montreal we have lots of snow there I'm suffering from a coughing a bit of a cold my name is Chaddock Eclair I'm an AI programmer at Ubisoft Montreal I've worked on a number of games Rainbow Six Vegas or sup Prince Persia a number of Assassin's Creed games and of course watch dogs to wash dogs to you play Marcus you're a hacker you're fighting social injustice the game has four pillars hacking combat stealth and navigation I'm going to show you know yes so today I'm going to be talking about cover this is an agent in cover behind a air ventilation unit most of the things that the agent does in combat

involves cover so they could shoot over the cover they could just protect themselves they could be throwing a grenade calling reinforcements so it becomes very important for them to position themselves ever looked at a good cover so that they have a good distance and a range and a good field of fire I'm going to show you a video where Marcus is entering a layout it's called home docks and the agents are going to react take cover around him and try and kill him oh I have to do that

so today I'm talking about two different things the first is how the agents position sums in cover and then second the challenges that we faced as we put this system into production and they started making levels and the problems that came up there are three types of characters in the game the first is the main protagonist Marcus there can actually be two or even four because the game is played in co-op the second is the agents these are the main enemies of

the game they can do the full range of actions and AI can do for example full combat investigate search those kinds of things for that reason because they do so much there's a limited number of them there's only about a dozen in the game I'm not not a dozen archetypes but a dozen at one time the last category is civilians these perform a kind of more lightweight version of the actions they might just shoot once they might just hide behind cover or they run away Roxanne spoke yesterday about the civilians their reactions I just wanted to make a point about the agents and their reactions combats a very tight loop and you don't want to interrupt it so we actually don't handle many reactions I think grenade reactions very important getting hit like a hurt reaction but most other things they just continue their normal combat loop the agents can be a member of a number of factions law enforcement street gangs

organized crime what's important watchdogs - or different projects - is that these factions can now fight against each other as in addition to one of the four players this creates complexity terms of squad which is our group coordination detection and combat and cover selection this is the overhead view of the home docs than what we saw in the video it's a much larger layout than watch times one and so the distances are larger makes it harder to choose good position to give you a perspective this is the rooftop that we were fighting on the video they say a normal combat range this this blue square I want to draw your attention to that little red truck but I'm gonna talk about it later so at the start of watchdogs 1 the game designers the programmers and the artists sat down and negotiated what

does it cover what should look like it's one meter why to give room and it's one meter high to 1.2 meters just to give it a little bit of variance and it has to be artistically very or visually very identifiable as a cover so that the player can say hey that's I can protect myself there a normal cover you can shoot over or you can shoot to either side from when you're in cover behind it there's also high cover which is 1.8 meters high you can't shoot over this there's a dead zone in between the two again to make sure they're recognizable and if you look at the image it looks like the high cover looks like just to cover stacked on top that gives a further clue of what kind of cover it is this is the covers in the map just drawn with the debug display you can see there's two different colors the pink covers are just normal covers the purple covers are what we call player only covers these are not really good covers but they still exist for the benefit of the player they might be unreachable

they might not provide any protection to give you an example on the right solid metal sheet that's a perfect cover so the end agent would go there on the Left it's an open railing it would look really dumb if the agent went there but the player has an ability to stay in cover and move along a wall or move along a railing and so we need to support that and have all these covers everywhere for the benefit of the player so as production went on and we started making more interiors and we need to make furniture inside deaths a desk that's in 1 meter high is a bit high and so they had to negotiate a low cover and so we squished down the animation a little bit and you can see examples in the park bench front of the car so there's three reasons why an agent might want to find a cover the first is that combat has just started bullets are flying

everywhere and they need to dive into cover really quickly we call that first cover now they're in cover and they need to shoot at the target and they shoot a few times and then they want to move a lot so they pop out somewhere else and the player doesn't get a bead on them that's called neighboring cover the third second cover is you're leaving your cover to find a better cover either it's flanking the target provides better protection or it pro the range to the target is better this is the target this is the agent there's three criteria which the agents use to evaluate what a good cover is the first is distance to the target so if you have a pistol you want to be very close it's usually based on the weapon you want to be very close if you have a rifle or even a sniper rifle you want a distance this much further away the second is the how far

the agent wants to move to get to this cover the you don't want to run across the battlefield to find your second cover you want to go from cover to cover but also you don't want to find a cover too close to you you want to if you're going to leave cover and move somewhere you want to least move like seven meters or something the third criteria is how much it protects you so the ideal cover is perfectly flat 90-degree angle to your target as you start twisting the cover you start exposing more of your back to being shot there's more of a more percentage of your body that's available to be shot the fourth it's not very criteria it's like more like a condition is a line of sight on the target and the line of sight is not tested from the agent itself but from the cover there's six positions of which you can fire from he's firing from the upper-right position so the covers on the to his left and he's actually stepped out of the cover and is now

firing to the right each of these positions are tested towards the known position of the target you can see three of them have good fields of fire and the other three are blocked the top one the middle one is a blind fire so then we put this all together you take our three types of searches and our criteria I'm gonna cat table and we apply weights to these criteria so some are more important in certain searches than others when you're looking for your first cover the most important thing is that's nearby to you you just want to get into cover doesn't matter if you have line of sight or not just protect yourself then you want to move becomes more important as you're moving to get your ideal weapon range so that becomes a high priority and you need a line of sight now right if you're gonna move somewhere you need to be able to fire on the target if this let's say the neighborhood cover

failed because it could be possible that you don't find the line of sight where you are you'd switch to a second cover you're now going to leave where you are and go somewhere else the distance to target is important the line of sight is important it also becomes more important not like super important just medium that the cover provides a better protection than where you are now again the top view of the roof the target and this is the agent this is what we call the donut it's the it's the distance to target so he wants to find a cover somewhere inside this donut ideally to the middle ring of the donut this is the distance to agent note that there's no hole this is a first cover search there's no hole so for first cover there's no cover to close you just want to find the closest cover and then you try and overlap them based on the weights this is an exclusion zone you

don't want to get close to the target so any covers that are too close you just cut right away there's actually a special behavior which we call emergency shooting when the agent gets into this circle that's too close they just stop what they're doing pull out their gun and just empty the clip in a kind of like Last Stand either targets gonna die or they're gonna die because they're just too close so if we zoom in a bit he started at the yellow circle and he found a cover very close to him that has a good line of sight but each of these covers could have been equally good just the waiting was slightly different for the red one that's the one they chose this is an example of neighboring cover search the agents on the far left far right and they're just moving to the left they're moving a certain amount of

distance if they if it had been longer they would have only moved 7 meters and they again I don't yes that's the cover they chose this one's a bit complicated all the debug display is now shown so there's lots of little circles there but you can kind of see that there's a blue circles the yellow circles this is an example of a second cover search there's an additional element is these purple circles these are other agents and you want to stay away from them so any covers that are within these purple circles we just ignore you don't want them to clump up and I'll get killed by an explosion or something so the agent what's special about this is they're underneath inside the building they're not the same level so they're finding a cover on the roof and they're basing on our distance but this distance actually goes through the building so when they come to move to the cover it's actually quite a long path they have to go outside the door of the building go up the stairs come around the pathfinding

is done last it's the most expensive thing we do the raycast for the lion sire expensive but the pathfinding is the real bottleneck there's two out two outcomes because of this the first is that they're exposing themselves for much longer time and they're going in front of the target making themselves easier to shoot in hindsight maybe what I should have done is hard to say but chosen five covers and then a flood fill to those five covers and then I could have just really chosen the closest one based on path the second thing we tried to do during production is to arrange the path so the points went from cover to cover what we call shadow paths so that it's always behind cover because he's actually walking around in front of the cover here but that would have required more path finding and so we put that aside so you

spend months working on this you can't see your desk you're working at the game designer working with the test or animators and then it goes on to production and all the levels I need to start putting into their Maps and you're like whoa no no but uh all these things come up right coming back to this little red truck in the corner this is actually a scissor lift so the player gets into it can go up in the air there he's up there takes out a sniper rifle it just starts clearing the layout it's not much the agents can do they can't path to him they can't shoot him because he was lying cover we call this unreachable I just like the scissor lift but it happens on buildings other areas that they they don't have an access to that the player does so we had to have a special unreachable mode that's kind of hidden you don't really see it and the ages start acting differently in this case the first issue

was that the layout is so large then they might not even be hearing the gunshot from the sniper rifle however if the player misses all those bullet misses are nearby them and so if there's two or more bullet misses they go into combat and if someone dies they go into combat too and the first thing they do in combat is going to cover this now protects them from this sniper rifle even if they don't necessarily know exactly where it's coming from the second thing we did is give everyone grenades it's a bit brute force but the grenades can be throw in arc reach areas that they can't on because there's no navmesh and they disabled the scissor lift when the agents are far from the player and now this is very most the things I've been talking about AB in generic target because the agents can fight each other but it's this very players specific if the player is actively aiming and an agent is unreachable and as far away they stay in their cover they won't try

it if they're out of cover they'll go to cover they won't try and shoot they won't try and change cover they'll just stay there while they're being aimed that to kind of maximize their survivability this mitigates that kind of overpowered clearing the level with the sniper rifle do is the large-sized the layout when you're far from a combat the first thing you do is move towards the combat lynnster you're in 30 meters you start doing the normal cover search that kind of thing but this creates a behavior where the agents were far away just move directly towards the player right and makes them very easy to shoot so again in this unreachable behavior they don't move directly towards the combat they especially find a cover around the player and move towards that cover instead that creates more of a lateral movement where they're moving across the players vision steps directly towards them this is another example of

layout in the in the city I think it's called hack the hacker there's a number of agents of a particular faction let's say FBI they're protecting a special target which is a star in the house the layout is surrounded by walls these are the entrances either gates or the one the left is just an opening so the player goes in and they starts a combat he gets seen everyone starts shooting they all start doing lots of pathfinding requests to find a cover to find a second cover to move towards the player that kind of thing but the layout also has a large number of civilians in it remember I mentioned that there's a limited number of agents and large number of civilians all of these civilians all start pathfinding not sure you can see it well they all pathlines same time this creates a huge pressure on the pathfinder and now there's a backlog of everyone in the game trying to pacify at the same time ultimately it's good to get the

civilians out of the way they leave all the resources are freed up for the combat it's the way we handle this and the first step was that agents had priority and pathfinding over civilians because they were more actively engaged in the combat and those who were closer those characters that were closer to the player would have their pathfinding go first what this caused is that civilians who were far away never moved you know I used to think that like walking into a wall was the worst thing that could happen that was like the typical AI bug but I got all these bugs about people not even path fighting not even move they would just freeze and then agents who were far away also got stuck so we had to modify the rule and anybody who had the first path mining request their first request would go before everybody else so we still did on distance but before the people were close they got into cover looked like they were doing combat and then they waited for the

civilians and the agents were far away to get their first request civilians would clear out the agents would far away who move in and then combat could continue as normal so let's come back to our layout we have these agents protecting their VIP just so happen that up here there were some gangsters Teske and they hear the gunshots too right and they become interested and maybe they start shooting because they can do factional and faction fighting right and then the police show up so now you have like four parties involved in this combat and they're all doing path fighting and maybe the player decides okay I'm going to hide and kind of drop out of the combat so the test guys the orange guys are shooting out with the blue police to the north right and the FBI are like hey we should go check out what's going on so they have to leave their compound through a door to go up there so they find a gate they go through the gate they go all around the compound but this

compound is quite large and if you're familiar with a star you have to flood the interior of the compound before you can actually before you can actually get out this what happened is we're hitting our max path mining cost and then path lines were failing and then it was just delaying everyone else because the failed one took longer and again everyone was just freezing up so what I did is I asked the level designer to add a door it seems so obvious but the level designer was very unhappy with me because they spent a lot of time playing the defense's placing people and then I just asked for a back door and I said it's really important the path line he's breaking so I had to go back and think about why the path line he was hitting his costs the first is that I was stopping them from going through the walls it kind of looked dumb when in combat when the agents taking so much time to climb a wall they're

exposed it takes longer but I had to reduce the cost of the walls because if you had two walls it would just stop the path lying right there the second thing was the gates the gates especially these large gates take a long time to open and but I still had to reduce the cost of gates but I'll show you this [Music] so they have to go to the control panel first [Music] they have to go to the control panel first wait for this gate to open and then they can go through and leave them standing there in the open easy target to shoot so we did is we kind of cheat in combat and the agents they don't go

or the control panel they just kind of tell the gate to open and this makes it much easier from the use these gates allows us to have a lower cost of pathfinding and that kind of resolve that unfortunately I was not able to go back to the level designer and tell him that he could remove his gate that I hate added this is another example it's the cargo-ship there is one way onto the cargo ship and it's this long pathway so we had this issue that the testers the game designers would complain to me about the agents rushing the player the agents would just run up the player and try and shoot him I was like what I didn't code it like that doesn't exist

[Music] [Music] so it's not the most challenging behavior right like they're just running up to you and dying right and but I really I couldn't figure out what's going on this is a side view of the same ramp they're doing a second cover search you can see all the circles and the cover they're finding is right here at the bottom is the it's a really good cover because it flanks to the target

what was happening is the path was going right next to the player and as I mentioned before when you get close to the target you stop what you're doing and you just do this do-or-die last and shoot at the target but it created this effect that they're running up to you and then just shooting at you so to address this we took three steps the first was that any covers that are behind the target like a 45-degree angle we discard them you're gonna have to cross against the the target to get there anyways right so don't take the risk the second is to just if we do find out cover with a path that goes by the target we just cut it it's kind of wasteful we've wasted that path finding but we need to address the problem the third thing we did was if two or more agents died next to each other it was

probably a choke point and the product the player is probably shooting them and killing them so we cut a hole in the navmesh for a minute patch it and then no more agents can go down that ramp way this forces them to they're seeing their friends die they don't go and it forces them to go somewhere else that might be more expensive or less optimal but it changes the flow of the combat so as we started like realizing that we had these issues with approaches or entrances into layouts we started going looking at each of these layouts and seeing what we could do about it so this is the same cargo-ship we started the looking up the ramp we looked at side of the ramp now we're looking down the ramp from the top and we and we looked where

could we add cover that on their side of the ramp that they could use and so we asked the level designers to add cover here it's more accessible and doesn't have to go near the next to the player and then you can fire down the ramp so I have a good idea and then you're licking all these circles but I have a good idea of what the decisions that the eye are making right but the level designers everybody else don't necessarily know it's kind of a big complex you know with all these weights and circles it'd be nice if there's some like in hindsight if there's some sort of tool that drew green or red on the ground and if it's green on the ground that spot can be fired upon by several covers but if it was red maybe there's no covers only one cover that could fire upon that position and that way the level designers could just turn it on look at their map and say hey

I have this entrance but it's red nobody can fire upon it and then they could say okay maybe I'll add some more covers and it would change the green this is another example of a an entrance again it's a home docks it's it's just another way to get into the same layout we asked them to add covers at the top of this balcony here which they did we're all happy and we come back later and someone has placed these blocks here these crates right in front of the cover and if you can see from the red arrow it actually blocks the line of sight from the agent to the target so it's an ongoing process right so we're like hey add these covers as I mentioned the the beginning there's several pillars to the game right so navigation is also an important pillar right so the player has to be able to

navigate up to get to this balcony pretty easily right so there's passes done I'm doing my combat pass but they're also doing their navigation pass it just happened at the end navigation one out so to wrap it up my advice is communicate to the team why the agents make the choices they do as I said before you're the expert on this you understand all the logic and decision making they're making broadcast as much as possible even if you don't see in the game make sure everyone your team like act like a publicist for the AI and then play the game go through the levels actually see how they're they're acting in the real world you know in there in the real layouts and then maybe there's like small quick improvements you can make that like change the way

the the game is played right like that unreachable or the the rushing we were able to change those things and so it's not really technical advice but I think it is a vice that will help you make better games I don't know how much time we have questions what's your name Neil did you guys consider using a hierarchical Pathfinder at all in order to try and mitigate those large-scale problems you can easily that is such an awesome question we have a hierarchal Pathfinder okay it doesn't take into account the things that change States the gates things that can be breakable so we're like oh oh oh we're so close we so we do do a hierarchal path it does make sure that the before wasting time on path finding but we couldn't use it in a more

advanced way because those links weren't there did you have the vehicles interacting with the combats and did they take that into account planning or have vehicle to vehicle like chase yeah yes I mean there's a whole other vehicle combat when the player gets further away and is it a vehicle they jump into their vehicle and they drive after them the vehicles have cover on them that actually makes it complex because the covers are moving it's dynamic but there is a whole vehicle section to the combat which I just didn't talk about today we are we are out of time [Applause]