NPC Voice Design in The Division 2

[Music] hello my name is Adam Ritchie coming to you from sunny Sweden to talk to you about our latest project the division - in the past I worked at Midway Jagex Lionhead Studios but now I work at massive and I'm here to talk to you about MPC voice design in Tom Clancy's the division - we're going to look at the work on NPC voice systems for the division - covering the role of the voice designer a look at the project itself and our goals NPC behavior bringing context to their dialogue sharing the player experience with friends voice production use of statistics for balancing and wrapping this up neatly with a nice conclusion question we're going to try and answer

today is how can we make non-player character box systemic voice lines relevant and engaging to players how do we make these voice lines more than just background how do we keep them as mood setters but also make them matter how do we use them to bring our characters to life what is a buck I hear you say I'm glad you asked greeting the player investigating a noise throwing a grenade reloading their weapon bark is a word for a voice line designed to rationalize and Telegraph the actions or reactions of an NPC people call these different things battle chatter Zeos we use bark because it's like a dog barking each one can sound similar but it can carry meaning in different situations you'll hear me

refer to them as just vo or voice lines as well barks are Worldbuilders ways to tell a story and to entertain and whose job is it to make these bucks well other people it's the voice designer and what is the role of the voice designer what is this job have you ever heard of it unless you're a Ubisoft employee you might not but maybe you know a dialogue coordinator or voice producer our responsibilities are pretty broad but I tried to condense them into four areas systemic box design what an npc can potentially say that gives them purpose and meaning what should I say as an NPC does context matter what do I know what should I not say will a lack of context for what I'm saying break the player experience what should I not know and not be able to say we also need to consider how they say it are they

talking via a megaphone or a radio or in person or they're wearing a gas mask and how often can I say it is it a one-time thing should I be able to say this thing a lot without getting annoying and how important is the information that I'm conveying to the player we're heavily involved in casting of the actors to play these roles some things we have to work with our location where our characters from where can we record where are the most suitable actors can they do the accent easily the level of casting what the budget is how important that characters going to be and how much we're going to hear them the schedule when do we need to record how intense is the script going to be for the actor do we need to spread it out when the talent gonna be available and the quantity how many actors can we afford how many different voices do we need for the game will there be recurring voices and then

when it comes to recording there's a lot of stuff to decide and work around the technical specifications of microphones prions the studio and other environments that we're going to use the session setup a lot goes into this voice directors engineering engineers being on-boarded training staff with new tools we like to get developers and actors to come to the studio and meet the team see the project we also want to make sure that we've got the right people involved from in terms of voice coaches or researchers and then in the flow and we start maintain and finish a session how much material we can cover and still hit our quality targets and how do we create the best assets in that environment then communication and support when and how do we send the scripts what format what time do we give the actors time to rehearse and can we send extra material for the actors and

directors like trailers images gameplay media references and then editorial process who's going to edit that material do we have time and we don't do we keep breath in the voice lines or do we clip it close to the words and then we've got the file naming and loudness settings to putting the results into the game we have an in-house database which we update the text and audio from the recording sessions reviewing the selects making sure that we've got the right ones updating any metadata and flagging any session changes the may affect other departments and then there's the sound the quality assurance pass on that making sure the material sounds right and that we've used the right microphones the right takes putting this into wise getting the material to play in the right way to do 3d right for lofts by saying how much world reverb is applied we also have processing editing

- animation timing or other tweaks and changes adding special effects and the mastering process and then we've got testing how does this work in the game has it been hooked up does it sound right in context and then we iterate and repeat this process add infinium because when you work on a game as a surface the work never stops so how did we get here in the first place well in the division one Tom Clancy's the division was released March 2016 the story is set in post-disaster New York City during a harsh cold winter gangs are roaming the streets boisterous full of character noisy and aggressive the game suffered from some technical issues at launch but quickly established itself as a leader within the genre some the box that the NPC's said were tied to behavior whilst other barks were driven by timers to

give NPCs something to say and to make combat and safe spaces feel busy and some barks were were often overused by design and then ended up playing a lot more than was actually intended the team had limited ability to fix this or control playback after release and it took some time to fix some of these bugs but appeared at launch what do we learn from the first game after it released the box that were driven by behavior made the NPCs seem clever and enhanced their connection to gameplay but some characters were sometimes talking for the sake of talking and we wanted to improve on that for the sequel players play the game very differently to developers and repetition can be funny but it also breaks immersion when the first game released whenever you killed an enemy from a particular gang their teammates could do a friendly down bark and could call out a name to show that they had some connections that team mate

but due to a bug it always picked the first character on the list which was Alex so players shot Alex a lot you can buy t-shirts with that one now as an achievement named after this in the game as well we embrace this as a company and we even called some of the bosses Alex making fun of ourselves but we also committed to avoiding this in the sequel the Division 2 released in March 2019 it set some months after the first game with themes of regrowth and recovery now we're in summertime Washington DC and New York City again in the DLC we've got extreme weather heat rain storms magnitude of technical improvements have been made since the first game and have got improved behavior they can do more they adapt to your play style better AI means that the NPCs need to feel more intelligent too we also set an

ambitious development schedule giving us a smaller time frame and to build upon the foundations of the first game so let's look at how we aimed to use NPC behavior to make their box relevant and engaging and any American viewers are going to be twitching at the UK spelling throughout this slideshow so working with the NPC AI team and based on our experiences from the first game we developed some ground rules for our system that we hoped would fulfill the ambitions whilst keeping the amount of writing and recording manageable because economy is king when it comes to box every bark will be triggered by an AI decision and will have a purpose and a reason for being said they're not random box should be simple to implement the database will do the heavy lifting of any of the rules and the o-line selection and I should add a note here that we opted not to do a complex rule-based dialogue system barks will be

connected to the NPC AI state for example guarding or fighting the contexts or circumstances will alter which line is picked to play make them more relevant and useful which we'll cover in a later section and NPCs should seem aware of their team and will respond to things set around them they won't ignore their friends this is an AI behavior tree in snowdrop our engine it looks like a simple flow diagram at the top level but it can go quite deep as the a makes decisions about what actions to take we looked for opportunities where a person might say something broadly speaking there are two approaches to designing barks driven by behavior literal barks describing actions or reactions is useful for the player to

get immediate basic gameplay information grenade out advancing on enemy shields being broken and then non neutral box things a person might say when doing a behavior but then are not tied strictly to that behavior we can do this Bravo squad flank enemy contextual spawn the Fallen shall be answered being revived by a friendly you get my rations this week these are things that build the character or the world their mood setters and flavor and a bark that on the surface seems very literal can become non literal with creative writing reloading can be literal or it can give us some insight into their character and the world the writers are very involved

in the right design process and we discover which barks are best written literally or not and even then a lot can change when it comes to the actual writing something that sounded very sensible on paper may just not work for writing and it needs to be redesigned so iteration is central to this process when we talk about barks at massive we group them by a stimulus an individual bark is a line said by an NPC but a stimulus is a parent to those bugs this simple word can be used by an NPC designer to call a bark at the appropriate place in their behavior when we voice the voice team will control the playback of Pat so here we have a design for the stimulus enemy down the NPC has killed the enemy and decides that they're going to announce this we keep the design simple and useful for several audiences the implementer knows

which characters need this buck and when it should play the voice designer knows which characters are involved and which conditional metadata to tag the line with the writers have contacts that they can write for and the several writers involved so these examples are very helpful and the voice director knows what the mt NPCs motivation is which they can then communicate to the actor in session they'll refer to this quite a lot and become very familiar with our barks database the stimuli are stored in our internal database and script management tool oasis on the left is our AI scenes Explorer where all the barks for the different factions and families of NPC are built central is a script writer below that is the excel-like interface where we can amend all kinds of data per line and on the right we see the character Explorer and media player

well we can audition the Robo voice temp all the recorded lines oasis also stores and organizes our story scripts cinematics menu text and all of the other localized languages there were a total of six hundred and forty five different AI stimuli that NPCs could use at the launch of division 2 this accounts for around sixty five and a half thousand AI voice lines or box split across two hundred and eighty eight different characters here's an example of some box stimuli from the first division game each one of these had to be hooked up manually in snowdrop for each NPC archetype and if the stimuli moved inside a wases it could break this was due to the stimulus being called by its location partner Nueces like a website URL the box were also evenly weighted between the build up to combat actual fighting and detection but

players spent a lot more time in combat so the NPC's quickly boil down to a sin core set of combat barks that are played the most frequently for the division to knowing that our NPCs would be spending a lot of time fighting we expanded the combat bark section a lot and covered more new behaviors and special abilities growing the combat barks line count to some stimuli now have as many as sixteen bucks per NPC to help avoid repetition fill Dunstan's AI team also simplifies the classification of a stimulus to just the last word of the location path making it much easier to implement now if anything moved in oasis that link wasn't then broken the designer now just had a simple list of words to pick from instead of a page of URLs and needing to find the exact one we had 288 AI characters in the game at launch however

all pulling from 645 stimuli so this was a huge time-saver once the stimulus exists in oasis and has some box the data runs through a build system into perforce and then can be implemented in the snowdrop engine [Music] the NPC then gets a stimulus that will trigger every time it runs a behavior some of these behaviors are rare but others can trigger repeatedly this is where the dialogue controls come into play throttling the box from spamming let's see a short 30-second clip of some different contextual barks in action and we return to this later to break down what's happening

and why not use a logic or rule-based system for box well a series of contextual ordered rules that control the flow we looked into this early on and it would have been would have given us greater control but it would have also grown complicated and ended up being time consuming to implement we'd have still been developing and testing the theory when we actually needed to be recording and implementing we opted for a data-driven solution with simple controls that attuned centrally and could be overridden at a more granular level this can tie you in knots when you're tuning values but if you stick to the principles that you set at the

beginning this can be a good solution we also tested out conversation systems like an example here from the first game the NPC's had a section a selection of starters which they could play when they were not in combat and they had a larger variety of responses this could yield conversations that varied and was economic with a line count to improve on this force equal the obvious goal was reducing repetition because there were far fewer starters than there were responses so they quickly became familiar we decided to do rather than invent invest a lot of time in an around elaborate rule-based conversation system was to create a matrix system of starters and responses that were interchangeable this proved quite the writing challenge but the result is a seemingly fluid and hugely variable simple conversation system here's an early working prototype of the system

with some owaisi's generated Robo voice nice day isn't it yes lovely day nice day in the example of a cleric to reload in their weapon they'll trigger the reload stimulus and this will then ping nearby friendly NPCs in their fireteam within a radius and pick one that is allowed to speak they will then respond to the reload back with their relevant conversation response stimulus which in this case is cover a light respond furthermore the clever an AI team were able to grab the timing of the start of voice line and feed the length of that to the client which then queues the selected response back to play after the elapsed time so the timing feels natural here's another video of an early test

exposed take him down this is of huge benefit to our localization team because the system doesn't care what language the player is using this is fantastic because it means that we can give the localization teams no strict timing conditions for the box and so the actors have the freedom to deliver the lines at the right cadence and rhythm for their own language without being constrained by the English line length reloading go I've got you covered they're out in the open we've got a target from this angle you can see the debug timing arrow here better and the countdown timer that shows when the response line is about to play there open fire take them down I'm outta ammo go I've got you covered

they're out in the open get him and this is what it looks like in a way sis this column shows the relevant stimulus name reload and then here are the Bark's themselves and this is where we tagged each line with a conversation response stimulus in this case cover ally respond the response itself is a stimulus and so here's an example of what that looks like and the text lines for recording so the starter caused one of the above for example ammo is gone and the response is any of the below I'll give you cover as you can imagine this can sometimes be challenging for writers and to make things more complicated we use the responses in as many situations as we can cover ally

respond can also be triggered by the advance on enemy stimulus for example when an npc wants covering fire when their and move up the idea was to build responses that can apply to as many situations as possible and fill the world with these mini conversations that make you feel like the NPCs are engaging with each other let's see it's working in game as a shootout between some civilians and the true shunt sons militia reaches a conclusion nice shooting is that are you okay there'll be more tomorrow we love to talk about the weather we felt that to make our NPCs more human we should let them talk about the

weather too especially in an environment where the weather is more and more extreme this could have made our NPC behavior graphs even more complicated but with our excellent AI programmers mons and daniel we conceived of the tag system where the server can check the conditions such as the weather time of day or even what object an NPC is hiding behind and change which barks will play as a result this system turned out to be so useful it was this kind of set and forget kind of logic that this these tags are now used for various systems in the game such as contextual animations and the living world system we started by making a list of the conditions we thought would be the most interesting for characters to talk about what were the most relevant to the player things like talking about the recent story changing the Bark's based on the time of day or the weather even target health

miscellaneous events have they killed an enemy recently had they lost a teammate the NPC team made a debug display to reveal which tags are active on NPCs and on the player or on a cover props such as a car crates or dumpsters here's a clip from another test environment what was that take his damn look already and we found that enemies didn't really indicate enough threat to the player when they targeted you so if the enemies could call their target out by some kind of identifier it would be pretty cool we thought about using player customization hats or the size of the gun or a new gear but as we add more customization to

the game regularly we risked having to record new lines with all of the combat actors and there were a lot of actors our game being heavily dependent on taking cover it made sense to involve the cover crops in our barks tanks with the help of the environment our team we polled a list of the most commonly placed cover assets which we categorized in the first round big thanks to Mickey and Sonya and the props team for their help with that mammoth task this gave us a short list of the most common cover props when the NPCs could then call out as they flanked attacked or searched for somebody hiding behind them there's an example of the car tag in action the result the player feels singled out and special and gives you a bit of a thrill to hear them spot

you when you think you're being sneaky the other big point of context for us was how people talk differently alone to when they're in a group or with company if we make 10 bucks for a stimulus five of which can only play when the player is alone and five when the NPC sorry if the NPC is alone and a 5 when the NPC is in a group in either case we only have 5 lines it's useful if down the line a design change means that the NPC is used alone a lot in missions for example then those five alone box will get heard way more often than the other five even stead we don't tag seven lines with any group information and then they can play whenever and we tag three with in group only when they're alone we have seven lines to pick from and when they're in a group we have ten the advantage is gives

us maximum variety for the same line count and it provides us with plenty of options in either case and it is more efficient the disadvantage is it makes three of them rarer and introduces new writing challenges take a specific example reloading why should the NPC say they're reloading to inform team mates that they're going to be vulnerable if they're on their own what should they say is this still a moment that we want to signpost the players so that we'd tell them that they're going to stop shooting for a second the answer is yes so we still want to give the player box to react to but the NPC is alone they're now talking to themselves their weapon the enemy or out of the world at large and the writers got quite creative with these here's an example from the closed beta when tagging was still a work in progress of why it's important to get this right I'm hearing reloads yeah very

tactical of you to scream out when you're reloading [Laughter] here are some writing examples anyway down is a stimulus that triggers when somebody has killed an enemy by using the group tag on the lines we can set the script up for the writers so that they then know what context are they're writing for let's run through some of these and look at why they can or can't play so here the line is got you they're talking to the dead enemy and it works well in all scenarios there is no tag so it can play in any scenario did you see that one they're talking to their teammates it needs a tag to say that I'm only in a group when I can say this line and I'm with my allies so I can play this line yes I need to be faster next time I'm

talking to myself works in any scenario there's no tag on this it can play regardless of whether I'm with friends or not and it will play if I'm alone and hey I got him I'm definitely talking to a teammate here so he needs a tag and if I'm not with friends this line cannot play so wrapping up this section let's return to this clip from earlier and break down what's happening the player kills an enemy and another enemy responds friendly down showing awareness the player uses a health pack and the enemy starts to call that agents healed but it's shot mid-sentence any other player would still have heard this line stop here at the same time a barely audible enemy is also shouting a taunt in the background a friendly civilian

NPC arrives to support and they see the player and say told you they're here to another civilian teammate because they're in a group the civilian starts to attack the enemy shouting insults and using their faction name true sons this shows that she knows who the enemy is and it gives us some idea that she hates these guys this is because the dialogue system rolled a line with a tagged target faction true sons which has a higher chance to play than untagged lines the true son start to plant panic and shout we've been flanked and they've brought reinforcements another tree son shouts get back which is a response to one of

the first two lines another civilian shouts oh no as she is flanked in turn by an enemy the flanking enemy calls out stupid civilians making targets for themselves this line has the tags target out of cover function civilian militia and group in group he's pointing out an easy targets attention to his team at the end another tree son run out of bullets and shouted got to reload and the final true son wants to destroy the players hive which is there to revive down friendlies and he orders destroy that thing but has killed mid-sentence by the player some brief notes on our online experience which i think is one of our coolest yet most under-recognized achievements as mentioned in the i-x

example in the first game most parks were played on the client and we're not synchronized across players in the sequel we want players to experience the same voice lines regardless of location or language the NPC AI team move the entire dialogue system from the client to the server this now does the heavy lifting of VO bug triggering and instructs the local client which voice lines to play this also works hand-in-hand with localization allowing people to share the same adventures in different languages it opens up server-side updates as a way to tune the system once it's online making fixes quick and simple to do and it gives us access to central anonymize statistics about barks playback because it's not coming from the player client here's another closed beta video of a team enjoying the box as a group god I love

the swearing God gave it on people and the voice production our development cycle looks like most games starting with design writing going through implementation and testing and then casting and the studio with an ambitious scope and aggressive schedule we needed to optimize our recording process where we could she've the recording of our 65,000 odd bucks for more than one hundred and eighty nine combat characters across a summer we relied on a well-trained team of voice directors engineers who supervised and ran the recording sessions on a micro level and on an editorial army put in place to deliver

the results to us here in sweden and most of our recording sessions were run remotely it's not as fun it's not always a smooth but it's green it's safe and it's efficient although the director was usually present at the studio as well due to necessity we've worked in lots of different studios over this project so consistency is always a challenge we send reference clips for the actors to make and make extensive recording specification documents though we always have to make some adjustments to suit the studio ideally we'd have recorded everything in one place with one director one engineer and so on but with our aggressive schedule and actors all over the world we came we became very good at adapting Skype for business and other call services became necessary Ally I even wrote an etiquette guide for

our team so they had a structure to our calls and sessions which is a whole nother GDC talking itself we had a more or less open talkback communication in the sessions so I could talk to the director the actor when needed the engineer and in this instance they're all in the same room but we used a lot of different Studios due to the amount of recording we were doing in so many different places and microphones to avoid losing takes and to suit the various ways dialog can be heard in the division too we used four microphones set up at all sessions if we'd had more shoot outs at the beginning maybe we'd have dropped the mkh 40 which was often roomy in the smaller studios here we've got the mkh 60 at the back and forty which was our back up front Mike then Jed posing in the background there

then we've got the cause eleven laughs for head mounted and a u-87 AI or you eighty-seven which is not pictured I don't have a picture that having all the mics set up at all sessions became a bit of a burden but it did save us a lot of time where we might have otherwise had to go back and rerecord something or set up another microphone for another part of the script if there's air on a mic or an actor's off center then there's usually a perfectly good take that's a close match and the mkh 40 often served as a great backup for the 60 or for quieter lines where the 60 wasn't picking them up as well microphone purpose the mkh 60 and the forties are back up the shotguns they were backed off for in world barks and mocap with booms

[Music] and we've got the cause 11 for head-mounted for all of our radio calls you've got a division elephant headed your way this is the real test and the u-87 for our special characters and in any cinematics or voice-over I'd also like to say something about how many

files were cut and sent to us by the editorial team but it became impossible to count it was a lot but the voice directors and sometimes us would call select and alts in the sessions so that we didn't have to get every clean take and go through it all for delivery otherwise we'd have never finished the game sometimes we did have to go back to a session and ask for more alts on a line but that happened so infrequently that that was a huge time saver balancing all the box that NPCs can say lead to a very noisy situation they've uncontrolled spam is definitely not engaging our simple dialogue control system is used to prioritize and restrict the playback of the voice lines from being a spammy mess to a more natural cadence using cooldown timers we balance the ebb and flow of the combat box experience aiming to keep the

listener engaged by hearing fresh content we also have a chance to play the server makes a dice roll every time it calls a stimulus giving some box our higher or a lower chance to play and it gives us another tool to balance playback and reduce the occasion of the more memorable and specific writing lastly a priority system that we can set on a stimuli Shane shines a light on the most important decisions that an NPC is making we can apply these rules at several levels at a broad level just to start with ensuring the NPCs don't all talk at once or that we don't hear from one individual to often then more granularly control of our contextual tags stimuli and voice lines themselves for example here our NPCs love to chat about the weather when they're not fighting so we put a cool-down on hot weather to prevent them from complaining about the

incessant heat which is the most common weather type even though this is exactly what would actually happen in reality it gets a bit annoying in the game and we have situational priorities this is a dialogue restriction rule set that can prevent certain barks from being played after certain situations for example we stopped small talk after a firefight has killed one of their teammates the game server is populated by a living world system containing NPCs that exist and continue to operate regardless of whether the player is nearby their behavior calls barks all the time if the player is here on the map not to scale then the NPCs are nearby will appear visibly and move around the NPCs that are nearby will appear visibly and move around and their barks are clearly audible here is a bark that the player might hear another bark and all over the

server this is happening but the player can only hear within this radius much smaller than that actually so none of these barks actually get past the initial state because snowdrop is very good at managing millions of events and calls per frame this doesn't add any overhead so if an NPC entity is placed on the server and it barks and no one is around to hear it does it make a sound no we early out with the data analytics team we collected statistics from the server and use that data to inform balancing decisions in the dialogue system this provided us with lots of data to sort and filter these are some of the data sets that we collect box per NPC faction location whether it's an environmentally contextual line or not through filtering and analysis we could identify imbalance

box that were triggered too often and using cooldowns on specific stimuli or lines and via our other controls could reduce that likewise we could identify box that we wanted to increase the frequency of and attempt to improve their playback rate some statistics from the open beta most commonly played stimulus is reload playing roughly seven hundred and forty five billion 121 million times to players across all servers during the beta and that wasn't for a very long time even the least common stimulus faction training mentoring played around a million times during the open beta there are a lot of there were a total of six hundred and forty five different AI stimuli that NPCs could use at the launch of the

division-2 this accounts for around sixty five and a half thousand voice lines in one language of a total seventy three and a half thousand including the story cinematics and radio this is split across 288 characters of which 189 of them were combat characters if we shared this evenly it would amount to an average of 228 lines per character but in some vendors and gate keeps gatekeepers we only have a handful of lines whilst our largest characters in the terms of the line count are the combat civilians and they have around six hundred and thirty bucks each and the smallest was a guard with only 11 lines this was indeed a massive project especially when it came to localization so I have to drop a little not here to the team we've done a fantastic job of

maintaining the standards that we set for English for the division to release we localized into 17 languages for text and include English we had 10 languages that were fully voiced our team translated five hundred and forty seven thousand words per language which is about the same as the Lord of the Rings trilogy and The Hobbit reaching a total of around 6.5 million words translated across all languages we recorded over 73 thousand lines of vo per language for the total of six hundred and sixty two thousand recorded lines 1217 actors recorded for us for the launch across all languages and we had 57 localization testers during peak time and since launch after a year of updates and with our new DLC warlords of New York the

game grew a lot we have now got close to 900 AI stimuli which accounts for around seventy eight point five thousand lines of barks alone not counting the story in the missions that's just bugs this is split across three hundred and sixty four characters of which two hundred and seventeen can fight and the total voice lines in the game at present are 93 thousand vo lines per language at times ten for all the other languages at home the total words four hundred and seventy seven thousand that puts are somewhere between the original Lord of the Rings novel or to order of the phoenixes for younger viewers and Atlas Shrugged although we're still a full hundred and ten thousand below in war and peace

and when subtitles come come along half of our players use subtitles around 30% of our players use English and localized languages use about 20% of our players use localize languages subtitles and then the other half our players don't play with subtitles at all so conclusions in closing behaviour we ended up recording much more than we answered up using do too tight iteration time so there was a little bit of wastage which you should plan for statistical analysis helps us to identify and remove redundant data and we're still finding uses for the box that no longer had a purpose at release we recorded box for tanks being aware of grenades when the NPC team made the tanks not reactor grenades we had enemy

to close lines for snipers that virtually never played because they're always at range we went big and then we went smaller with context context robotics played more in some areas and less in others meaning that some are still very rarely heard this gives our players our thrill when they hear a line that they've not heard before after around 300 hours of playing but we needed more iteration to finesse and balance them better we have since tagged more objects to increase the chance of hearing the cover prop box and we simplify the tag usage in our latest DLC to focus on the most common and obvious contexts such as the faction weather and time of day not many people have noticed our innovations in synchronizing box across the server this to me says that is so obvious and fundamental to our shared enjoyment that it should always be like this and it's

necessary we can't all sit in the same cinema watching different films and then expect to be able to relate to each other's experiences afterwards this brings us together across orders and across languages and using statistics helped us to find issues with the dialog system but also with the game as a whole when barks for the wrong character being heard in a mission that doesn't have that faction you know that an NPC boss has been given the wrong barks character and now sounds like a huge armored tank instead of a stealthy sniper if we devoted more energy towards fast analysis of the statistical data that we were fed we'd certainly have been able to find and make better decisions about what not to record or have spotted stimuli that we're not working sooner and those bugs that when they were fixed later could potentially unbalance the local landscape when turned on next time I'll be asking for a

system like this from the start and making time to analyze the data after every internal play test an iteration with the ambitious schedules iteration time is always going to be tight but with an online game your product is also never really finished this gives you opportunities to polish and improve your product after release and with clever tools such as server based AI bias control the job of patching can also be made simpler and easier so what can I do with this what can you do on Monday look at your box as more than just background filler their world builders and they deserve at least as much attention as the main story we've your box into the setting better why is the angry person angry investigate how contextual triggers could enrich your box what are

the three biggest conditions that affect your NPCs consider how you're planning to balance your box experience how are you going to get data and find ways to save time and resources by recording actors remotely learn to embrace it and find methods that work for you it's good for the planet and it will save lives and I'll finish with a broad thank you to everyone who worked so hard in making this game great for our players and for our community thank you for listening