Realistic Performances in Games

[Music] good afternoon and welcome to realistic performances in games fair warning I have been told this talk is a bit chewy so strap in there is a perception out there that the average game is Mindless entertainment with at best a thin story cardboard cutout characters and stilted acting but many companies have been working tirelessly against that stigma at Naughty Dog where I've been for over 7 years we work hard to ensure our performances are believable emotionally relevant to the player and blend with our gameplay as seamlessly as possible I'm Ryan James and I work at naughty dog in a somewhat unique position as lead editor aside from writing and directing some of our content it's my job to keep the rest of the team or to help the rest of the team create and maintain performances that feel like what you'd expect from an edited movie even when they're fully

interactive but what makes a realistic performance a naughty dog it means a character needs to feel vulnerable nuanced and human which means we're often focused on how natural every movement feels meaning how the character moves and how they speak needs to feel emotionally and physically relatable and of course they also have to look realistic which takes labor from everyone to get right which is our entire goal Naughty Dogs games strive to intertwine story and gameplay so the player experiences a seamless performance which we call the active cinematic experience we like to provide the player with emotional investment and motivation behind the momentto moment game play this is largely done through non-interactive cutscenes but because we strive to keep the player in control as often as possible a lot of those drama and character moments end of paying off on the stick as we call it

but figuring out which beats belong where and pulling off the execution of them requires a lot of iteration and in nauy dog one of our biggest strengths is our willingness to retool or even throw away content including captured performances if it will make the players experience better even when deadlines are extremely tight our company culture encourages people to speak up if they think something isn't working and take advantage of our tools which enable quick alterations to try out new ideas this means all of the narrative content that we capture is prone to changing even after it goes into the game and sometimes these alterations happen so quickly that their trickle down effects get lost in the daily Shuffle thus it is up to the editorial team to constantly Chase change tracking and managing each edited performance in its various Evolutions over the course of development in order to ensure that when the player gets their hands on the game our characters interactions still feel like the authentic human moments

we'd intended them to be when we first captured them so today I wanted to share some of the techniques we've used and challenges we've encountered when creating realistic performances in games while allowing them to change over the course of development oh and uh just a heads up spoiler warning for most of our games here at naughty dog we don't develop the story second since we shoot and Implement narrative material in small mates capturing progressively with the actors for easily a year or more our story in fact comes together right alongside design of course it takes many months of pre-production before we can shoot anything our directors and writers first juggle developing story macros with design overseeing the creation of characters that offer varied gameplay mechanics and casting actors to embody those characters but as soon as we have enough material written we jump in head first with our actors to capture story beats via motion C capure and dialog recording

often we focus on a demo of some kind a small slice of the game that establishes our tone and explores the games developing mechanics right away the game starts evolving because of these initial shoots not only do they help our principal actors start infusing their personality into the characters but their choices early on allow the team to adapt to discoveries we've made with the actors on the stage for instance on the very first day of shooting the last of us we began with a scene where Joel and Ellie are ambushed during their cross country Journey Ellie was originally pulled from their pickup truck and pinned helplessly by an enemy making her a quasi Damsel in Distress for Joel to save but Ashley Johnson who plays Ellie had the desire to fight back against her attacker so we shot it that way leading our directors to go back to design and rethink the character's overall vulnerability for the rest of the game though we wanted to reinforced the need for the player as Joel to protect LA from harm it still worked better for her to become far more capable in combat

situations than we'd originally conceived as we continue to shoot cutcenes are initially our top priority since these are non-interactive Beats reserved for the more delicate and emotional character moments they require as much animation time as possible to hit the level of Polish required we've spoken about our capturing process before but there's a couple of things I feel that are essential to capturing the best performance possible because as an editor if all I had to start working with with stale or unnatural material there would only be so much we could do in post to make it better A lot goes into how we create scripts that give actors honest material to play but I sadly do not have time to touch on that today however I would like to point out one strength in that process every phase the material goes through is a collaborative effort from the start our creative directors encourage input from the team and our actors either before the shoot or during our full day rehearsals

especially with our actors we even do a table read to break down the script before we rehearse if something doesn't feel right the director and actors work out what sounds the most natural and try that sometimes when we're shooting we try a variation uh overtakes and see what we like best in the edit Bay later when shooting a scene we primarily capture it in one continuous take unless we need to break it up for additional set builds or to accommodate a stunt lighting camera angles we don't worry about any of this until post- production this lets us focus on getting that perfect Master take with the actors encouraging the spontaneity improvisation that brings life and genuine emotion to the scenes during the shoots the creative director acts as the Director on set partially because that's who's worked most CL closely with design throughout the production process and therefore who can give the most context to the actors meanwhile the editor's job at the shoot is twofold first staying away of the director's notes since then they can

later be more helpful at personally executing or communicating to others the director's intention behind each moment and secondly the editor ensures that we have the material we need to craft the dialogue performances later this includes noting which takes were the best performances on a particular line as well as asking for a wild take to get a clean read of a line when the performance is still fresh rather than trying to recreate it months later in ADR this is especially useful for efforts those grunts you hear when the players exert themselves which often get spoiled by set prop or suit noise while the actors are moving around Drake exertion Mar great all right just give me a hand with this come on give me a hand with this [Applause]

it's some of the most ridiculous things when we have them do Wells once the footage of our actors is in the can the real work begins breaking it into pieces for the design animation and audio teams back at the office the editor lead animator and creative director start editing as soon as possible we found that cutting together reference videos is the fastest and most efficient way to communicate the eventual shape of a performance after choosing which takes of mocab we want to use we create four paneled videos called for UPS that show the reference footage of the actors alongside intended camera angles as we experiment with cameras and massaging the blocking for the best composition you end up having to watch them with a split Focus looking at the face here the body and the camera there sometimes even listening to audio that doesn't match up with the actor's lips this is because we can take advantage of the fact that this isn't

live action we can swap different audio takes if we need to fix something or use those wild takes to cover spoiled dialogue the facial motion capture is even ordered separately after the edit is locked and conformed to match so we truly have the freedom to change anything like swapping only one actor's map for an alternate take to get a cool gesture they did in there or cutting between two different performances right in the middle of a shot normally we hide these edits on camera Cuts just like in live action but if we can't our animators then have to clean up the resulting pop in the map or at the most extreme grabbing the face audio and body all from different sources to Cobble together a moment we don't have usually that's because we discover something a way something could play better in the edit and since we didn't get it originally we've got to fake it but usually the most we do is swap reads of some lines to increase Clarity or subtlety there's definitely magic in a per performance that can easily be

shattered if you stray too far from The Source once a scene's edit is approved that four up reference video is funneled to every Department that will be involved in the further translation of these performances this includes animation props environment lighting sound and design but they all know the scene isn't picture locked more like picture latched because as I warned these edits may very well have to change sometimes A change is required because of a mistake in the edit nailing the timing of a moment can sometimes be difficult since you're essentially looking at mannequins in one pain and trying to sell an emotional transition that you can see on an actor in another when the facial animation is applied and we look at the scene full frame some things might move just a little too quickly so we have to adjust and then communicate these timing changes to all the affected departments another change that comes up is that sets are often altered to accommodate gameplay revisions an entrance or an exit to a space had to

move there's a wall in front of that camera or a cover object sitting right in the middle of a character usually these require minor animation fixes like moving characters or cameras but at worst the environment might need to be reverted by Art and Design the biggest changes that come in maintaining continuity is in maintaining continuity with gameplay since the transition of characters entering or leaving a scene needs to feel seamless their animation have to match what's actually happening in gameplay design helps ensure this by creating pinch points that force a certain action from the player like opening a door or vaulting over something that will blend into the start of a scene and when it's over we often blend characters movements into regular gameplay animations to match on the way out but when gameplay changes so do these entrances and exits often this means recapturing or at least reanimating that portion of the material

like this scene in The Last of Us which first had to have the camera corrected to not be behind the player because that would be too similar to gameplay then the level changed so that Joel would be in the lead rather than Ellie so we needed a pickup uh to a new opening where Joel went in first as these scenes change we have to keep careful watch so nothing slips through the cracks sometimes certain scenes linger in a partially done state for a long long time waiting on a character model or a set that didn't exist when it was shot for example however cutscenes are just the tip of the iceberg since we like to keep the player in control as often as possible we've increasingly relied upon methods other than Cuts scenes for implementing story beats one major way is through animations we call igc's igc stands for in-game cinematic because until Uncharted 4 all our cutcenes were pre-rendered movies so igc's defined the distinction between

animations that happen live in game versus more traditional non-interactive narrative moments like cutcenes now that everything plays back in real time the biggest distinction is that some aspect of the animation is usually interactive in the very least the player might have camera control but that all depends on the particular igc some are as insignificant as opening a door or walking into the frame at a start of a level others are full-blown minic cinematics sandwiched between gameplay moments with no camera Cuts like during this puzzle in Uncharted 4 igc's are also a clever method we use nowadays to blend into and out of our cut scenes the first or last shot of some cut scenes on Uncharted 4 were igc's so the camera could blend smoothly transitioning on a camera cut to the rest of the non-interactive cut scene there are hundreds of these littered throughout the game and they

will often change multiple times being shortened lengthened or recaptured until design thinks they feel right within the pace of the gameplay around them editorial like with cutcenes needs to keep up with any timing adjustments moving parts of the audio around to match the animation as it changes but even though the interactivity of an igc offers great advantages for gameplay the way their audio works sometimes offers additional challenges igc dialogue is similar to a cut scene in that it plays from the first frame of the animation Until the End usually containing several lines of dialogue and or efforts precisely time to the animation however that is where the similarities end unlike cutscenes which play back like a movie where every volume level or perspective shift is crafted and controlled igc's often have to account for a real-time interactive camera so igc dialogue plays directly from the voice box of each character in 3D space space this creates different

mix issues because our camera is the listener I.E microphone by default the benefit is that the position of the character's Voice automatically matches what you're seeing on screen without any post-production work instant realism but the downside is that if the camera is really far away and you want to hear the speaker clearly we have to cheat how that line plays or if the camera Cuts while someone is speaking suddenly the person you hear in your left is now talking on your right midw which can be extremely distracting like in this moment from Uncharted 2 we dealt with both problems it took a lot of trickery to make Nate's voice audible from across the ice cave but when that camera cuts across the space you'll notice we quite couldn't fix the pop in Nate's voice as the camera Cuts ton it's a bridge pull the switch on your side

yeaha but wait there's more cutcenes and igc's are only half the battle our performances would not feel seamless without a lot of other systems that play in between them but before I break those down further I would like to look at an example clip of what it all looks like when everything comes together during this beat of Uncharted 4 Nate and Sam are just starting the next leg of their Journey exploring the ruins of a Scottish Monastery that their rival Rafe and the team of mercenaries he's employed are trying to excavate with dynamite the Drake Brothers meet the shoreline Merks for the first time learn the new Dynamite gameplay mechanic and solve the first puzzle of the level finding a crate to climb past a section of ruins that is until another team of mercenaries arrives it's a long

drop yeah going need to find another way down hey this should hold good call what the hell is all this it's excavation equipment shoreline what look I thought they were just by the cathedral [ __ ] get out

we're good yeah nice to meet you too Shoreline seems like they were expecting us and like they're searching away from the cathedral which means we should get to that graveyard Pronto exactly what is this place monks had several living quarters main one being by the graveyard of course location location location more shoreline equipment at least no shoreliners now what do you bet

yep Dynamite yeah be careful with that stuff huh hey got the door open I was very loud hopefully they didn't hear us hopefully we just blend in with the other explosions look he's a way up through the roof it's too high for a boost though though oh let's find something to climb on yeah hey we can use this to climb out of that building

there you go take that door pretty good with that stuff well I've blown a lot of [ __ ] up over the years [Music] hide I think we got away it's a mellow sequence but there's actually a lot going on here and every single element that's happening is helping convey our story Beats at a glance these elements include igc's when Nate uses his rope and later when he picks up the

dynamite animated gesture layers on Nate and Sam as they explore and Converse oh and nice still going yes animated gesture layers come on a yes A Brief cut scene of Nate and Sam finding the shoreline equipment before they're ambushed systemic enemy dialogue from our Shoreline mercenaries as they attack the player systemic buddy dialogue from Sam when the player successfully stays in cover efforts as Nate knocks out the last MC and then lands from a long drop near those ruins and of course before a and after all of this there is also crap tons of story dialogue playing so about that story dialogue most of the dialogue you hear in our games are individual lines scripted by hand with specific timing we

even have a designer in a scripture 100% dedicated to creating systems for hooking it up they can't complete an entire game on their own since by the time the game is finished there's too much material for them to cover alone but they are usually the ones triggering lines based that play based on numerous conditions some as simple as when a player walks through a defined region others much more complex rather than play with rigid timing like cut scenes and igc CES we cut our dialogue into pieces that can flexibly respond to the player's actions this also means we can adjust the execution more easily as things change during development these in-game dialogue moments actually make up a larger percentage of narrative beats than our cutscenes we feel this type of material is crucial because it carries the story forward by keeping the characters alive between those cutscenes it can justify practical and gameplay driven goals within the context of the narrative likewise it informs the player of any

changes intention like exploration or combat States and it's also a vital tool for selling the realism of the environment in the player's mind when the characters respond contextually to it unlike the cutscenes and many of the igc's most of this dialogue does not come from the map stage that's because it primarily requires the character speaking while in the player is in full control so we record The Lion's Share of this content at an ADR Studio the actors are usually paired up so we profit from their chemistry just like at mocap also they wear the same type of microphones so that the audio audio quality matches that of the stage the scripts for ADR sessions come online rather late as the levels they often reflect are still being locked down by Design This means that most of the time the actors are reading their lines without any time to prepare so we bring a video capture of the level in its most current state to give them as much context as

possible and anytime there is a cut scene that we've already shot and implemented we watch those down with the actors too to remind them of the character's emotional state that they need to match in order to keep the tone consistent because the game is still in flux we record a variety of material and anticipation of ways in which the game might change gameplay conversations make up the bulk of our recording but these aren't always straightforward because sometimes parts of them are optional the actors will perform one version of the conversation all the way through and then pick up the other versions right away to keep the energy consistent such as in that Scotland clip different lines would play out depending on when the player interacted with that yellow crate how we open this door look there's a way up through the roof it's too high for a boost though hey we can use that yellow crate

sometimes we also pick up alternate versions of lines alts for short especially when the level is in a rougher State and we need to have multiple options as design locks things down later this frequently happens in puzzle spaces among others since they require tons of specific vo and animations to cover what the player does and in what order on Uncharted 4 we even kept a wish list of alt lines that design thought they might need and tried to cover them all as pickups near the end of recording this resulted in much better coverage for the final game than we've ever had before we also have a whole category we call cya lines as in cover your ass which are generic phrases like hey look at this and come on this way which we record in m multiple emotional states and contexts just in case we need them at a later date either to help with gameplay changes or maybe to sell a character beat we record these with our heroes and all their buddy I just like at map our actors are also

encouraged to improvise during ADR sometimes this will inspire us to rewrite and re-record a new version of a conversation on the spot and at the end of most sessions we also record wild passes of our levels where the actors watch the play through video again and are encouraged to Riff on top of it though our writers try to cover everything we need in a given section of the game this often provides alternate versions of lines or even completely new conversations that add enormous amounts of spark in character we extract these gems and use them whenever possible for instance Sam's location location location line in the Scotland clip was an improv that Troy Baker threw in and we kept it all our recorded dialogue is then brought back to the office and stacked in time to the video this means we each take of line This Means each take of a line is edited in a Protools playlist and arranged in columns so we can quickly listen to down to all the takes of a line when making our

selects well no one's home looks like no one's home looks like no one's home these picked lines are then edited into individual files with unique names and uploaded into our dialogue database tool editors also distribute scripts from the database to our designers so they have a list of what content they need to implement along with the scripts we also Supply another form of reference movie unlike our cut scene for UPS which are tied to motion capture data these are instead exports of the very movies the actors ADR to with dialogue placed to match the timing of the actor's original deliveries hang in there [Music] Joel [ __ ] okay there's got to be another way

these ref fids help show where and when the individual lines of dialogue should be scripted to play so that it feels and sounds like a realistic performance designers use it as a guideline as they translate all this cinematic pacing and subtlety into pretty much scripted math the goal is to include proper beats for a breath or a natural pause between lines either for the speaker to have a new thought or for the companion character to actually listen before responding when we don't hit this the moment can feel rushed disjointed or Flatout confusing it sometimes takes several passes to get thing get the timing right in game designers can only use these movies as a guideline because often the video's timing was too fast or too slow and is not completely representative about how everyone might play the game they use their best judgment trying to keep three different potential play Styles in mind some players like to explore taking their time stopping to hear every

lovingly crafted line so they don't miss a thing but if we were to time out everything to that play style it would ruin the experience of more average players who like to enjoy the story but keep things moving where we can the simplest solution is just to have certain lines not play if the player is going too fast like in these Scotland ruins the line specifically pointing out that a ledge is too high for a boost doesn't play if the player rushes out there of there before the previous conversation is finished hey got the door open that was very loud hopefully they didn't hear us hopefully we just blend in with the other explosions look there's a way up through the roof oh let's find something to climb on yeah and then there's the Speedy players who Dash through the puzzles to get to the next combat setup we do our best to get our conversations playing naturally but quickly so they don't get lost but sometimes it can't be helped

helped once in a while we have an area too small for the amount of VO that has to play and end up shipping with moments that can feel like a string of lines back to back depending on how quickly the player is going like this moment in Uncharted 3 that just ended up a little too tight compared to the pacing of the cut scene that preceded [Music] it who are we to argue with Sir Francis say we head down there and end this thing once and for all look at this place whole City's resting on this Cavern doesn't look very safe does it that's an understatement we got to find a way down there if we get the timing right it can convey subtlety without needing a specific body gesture or performance to sell a moment like in this bit from The

Last of Us the pause before Ellie's o line conveys a turn her realization about Joel's darker past her follow-up question then gives Joel an awkward pause of his own all of this emotional content managed to fit in one small stairwell even if the player never stops come on let's get out of here how did you know no what about the Ambush cuz I've been on both sides oh so uh you kill a lot of innocent people we'll take that as a guess take it however you want that moment worked on dialogue timing alone but when we need more we now can dynamically add extra body motions to our story dialogue to

increase physical and emotional realism in that Scotland clip when you saw Nate waving his arm and actually looking at Sam while talking to him that was an example of our gesture system in previous games we've had all sorts of contextual move sets and animation that helped sell the reality of the Hero's World by having him or her react to their environment whether it be recoiling from fire stumbling while exhausted bracing themselves against walls ET ET our designers are also able to play additional unique animations on our AI characters to help bring them to life contextually that's what allows Sam to look around and dynamically check out the world around him but now we employ an additional level of detail gestures these are partial animations that play on top of normal gameplay animations like a headshake and arm pointing a shrug Etc the player can be running climbing or crouching and these layers will usually still play in case you're wondering why sometimes an arm isn't moving in that clip that's

because it's not part of the animation's layer meaning the player can be carrying a gun or another object while the rest of the body does the gesture one of the ways we sell these moments is by hooking up a specific gesture to a line of dialogue in script so when each line plays the gestures play on top of whatever the character is doing programming and scripting magic also helps the speaking characters actually look at each other dynamically when appropriate since we don't record specific gestures for every line the final performance isn't as rich as uh cut scene might be but the result is still an added level of life that doesn't restrict the players's movement as they roam around the game World though we can also hook up custom facial animation gestures to lines in the same way we usually don't go to the extra work since the camera is normally behind the player character's head and far enough away from all other speaking characters by default we employ a neat procedural system which generates lip

sync and facial animations using the audio files amplitude and subtitle text what is this place monks had several living quarters main one being by the graveyard of course location location location besides the story dialogue we have another category of lines called systemic vo these groups of lines or these are groups of lines with similar context that can play when triggered by AI driven events this material helps to add life reinforce character relationships and provide feedback in various situations throughout the game mostly during types of gameplay scenarios that use repeated systems like combat when an enemy spots the player for instance they have specific lines that they use to call that out there he is I see him these buckets as we call them offer multiple varieties that can play based on the same situation so the player doesn't just hear I see them over and over and over at the end of our

Scotland demo Sam called out one of his systemic lines to indicate the player is still hidden from the enemy the entire bucket consists of these lines I think we got away all right Nathan we lost them we lost them I think we lost them I think they loost St of us looks like we gave him the slip this sort of dialogue serves a practical function in terms of player feedback but it also keeps the characters alive and reacting to what's going on in the same way the storylines do which is why we don't only use systemic dialogue for our enemies any dialog triggered by AI rather than just script falls into this category including buckets for any major characters the hero their AI controll buddies and of course all our multiplayer characters since that mode is pretty much all combat mechanics the system is controlled by the global

tension state which we break down into four stages unaware investigate search and combat in unaware any threats nearby have no knowledge of the player or their companions enemies can converse with custom story based dialogue like they did at the very end of that Scotland clip the player and his buddies meanwhile will likely be in standard explore mode this is the natural state of the game if they get close to enemies in this state buddies and the hero will use whispered versions of their various buckets so the characters sound like they're trying to stay hidden in investigate tension Rises a bit the enemies heard or saw some evidence of player interference then they try to uncover the source of the distraction now playing their systemic buckets regarding what they saw checking in if they anyone saw anything Etc this is the first level of tension where the player would feel threatened and that they should either fight or get the hell out of them there in search tension is near its

breaking point because enemies saw the player then lost track of them they speak buckets of lines so the player can audibly locate the enemies attempting to converge on their position feedback from the heroes remains whispered including buddies pointing out where the nearest enemy is on your left behind you or warning them if someone approaching is carrying a powerful weapon look out shotgun that sort of thing combat is when all hell breaks Lo and in our games uses almost exclusively systemic dialogue but since we want it to feel as nuanced as any of the quieter story moments we employ a lot of buckets to cover a variety of potential situations enemies are aware of the player and actively trying to H attack him or her with guns fists or whatever other means they have they have lines announcing when they are flanking pointing out the player's exact location or warning each other the player as a power weapon like an RPG or a Molotov cocktail buddies continue to warn the player the

same as they do in stealth but instead of whispering their lines are now shouted over gunfire we also have dialogue that helps emphasize their relationship to the hero I.E that they care about them buddies ask if the hero is okay after they've been shot or stumbled by an explosion they shout out that they're coming to rescue the player if they're grabbed by an enemy and say thank you to the player if they are rescued in return the hero also has dialogue to connect the player to their emotional state like the stress of diving out of the way of a grenade would be oh God or stealing themselves up while reloading and cover DRS is usually like okay here we go there's usually something that plays after taking down an enemy as well in Uncharted it's usually a funny quip that suits the tone like lights out whereas the LA in The Last of Us it matched the world's grittier tone just okay since systemic lines have a more functional purpose they sometimes lack the subtlety or subtext that our longer

conversations do but they always maintain the same personality as the character delivering them so that the player doesn't become weary of hearing the exact same thing from multiple characters over and over for example Sam calls our main character Nathan or little brother in a lot of his systemic warnings and is more ball busting in his tone while soly calls him kid and maintains a more paternal tone Elena calls him Nate and has her own brand of loving exasperation with our hero in her pattern of speech even with personality thrown in though these lines could still sound somewhat repetitive so we have a shuffle system which keeps track of every line said by every AI group advancing the queue for any one bucket across the board when they speak ensuring the same exact line will not be heard frequently if enemies spot the player one might yell I see him the next time that happens rather than here I see him again the player will hear it's him or some other variant not until the bucket has been exhausted

by the shuffle system does it reset to the top again we try to record 10 to 20 different lines per category that might be heard the most to avoid the player hearing the repetition in any one play session sometimes the lines are also written to be modular so they can be stitched together based on a situation not only does this allow for a variety in the simulation but it makes the enemies seem alive and intelligent in The Last of Us an enemy might say Hey you too go search over there each of these is a separate bucket of lines that plays back to back utilizing natural pauses to make them sound like complete varied thoughts and another scenario it might play as hey you go check the alley or some other variant one subtle way we keep things feeling more real is by avoiding repetition not just in lines but in the voices themselves since we have a limited number of actors voicing our NPCs we try to make sure they have different voice types medium low Etc so

that the player won't hear voices that sound the same over and over also as I mentioned enemies are often initially encountered while they're having a story conversation customized to that particular group or area we try to keep track of which actor's voice is being used in each conversation then try our best to not use that actor in the next setup further avoiding repetition a tricky situation that then comes up with these enemy conversations is how to interrupt these lines that are already playing if the player shows up in their Vision cones they have to switch to their systemic dialogue and go into combat so we cut off a line wait a split second and then just play the systemic dialogue as if the enemy had interrupted themselves voluntarily right I would have been They're All Dead they're all [ __ ] dead the hell's he yapping about take a breath who's dead Go Crew the 76 for a long time this rough method was

the only way we had to interrupt conversations however the addition of a drivable vehicle in Uncharted 4 which the player could exit at any time meant entire sequences of lines might be skipped because the player wanted to explore a little this forced us to come up with a more elegant solution for interrupting and resuming dialogue between Nate and his buddies ensuring conversations couldn't so easily be lost forever we recorded more buckets of lines regarding and responding to the player stopping the 4x4 and exiting then attempting to resume the conversation upon return to keep things in line with our story dialogue the tone of these interrupts had to be different depending on the nature of the character's current relationships in the plains of Madagascar Nate is Comfortably exploring with Sully and Sam so his tone is more casual watch Towers Avery was the most wanted man in the world so if he was hiding something out here makes sense he be right back okay so uh you're not interested I am

just checking something out Sam you were saying oh that's right I was uh anyway Avery was the most wanted man in the world so if he was hiding something out here makes sense you need Lookouts on the libertalia island by contrast Natan Elena's relationship is strained so he's more subdued when he hops out of the car well is a good match for Shoreline he needs their muscle and they need his money they do they seem pretty well stock Shoreline got involved in a couple of Hold Up Wait what you see something okay all done yeah so Shoreline got involved in a couple of civil wars that didn't plan out to make it sound natural when the character stops speaking we had the actors record glottal stops the sound that a person's throat makes when it

stops speaking these would play to cut off the lines in as natural a way as possible and but uh um soes were the best sometimes we edited alternate versions of lines that would flow more properly when coming out of an interruption this meant combing through an entire conversation and marking which lines needed alts in case the conversation resumed on that line in retrospect we wish we had purposely recorded alternate versions to restart a conversation rather than just to edit the originals this system is still very new and we are looking forward to building on it more in future releases there's one last category of dialogue to cover briefly efforts besides those during those wild takes we get at mocap we record lots and lots and lots of efforts of various types with each of our actors mostly because they

give feedback to the player while also helping the characters come alive selling all those physical situations like climbing fighting and um dying that happen in our games just having breathing from the characters alone especially in situations that are tense or when they're fatigued also helps give a sense of what a character is feeling without them having to State it when recording we have actors chase videos of a player grappling fighting and falling so they have context for the sounds they're making we often get different sizes small medium large of each sound and play the appropriate Ones based on the intensity of the animation and as most efforts are vocally stressful we always record them at the end of a session in fact we do the same with any shouted material so we don't blow out an actor's voice in the first hour so as all of these systems and animations start coming together that is where the of minor and major changes truly start a cut scene or a

conversation might work in a vacuum but it's not until we get these pieces into the game that we can truly judge what works it's about getting it right on the stick and feeling it out and then making it making sure the performances remain consistent despite the adjustments we continually tweak Swap and rearrange our dialogue and animation sometimes over the course of years all to improve the emotional connection between the player and what's going on around around them first and foremost not all the content we write and Record Works in the final context of the game even if nothing changed in design or animation sometimes an emotional beat just feels false if possible we will alter that vo with an edit by adding pauses or stealing a breath sigh or laugh from somewhere else and cobbling it onto the line it can really make moment sound more natural our systemic dialogue might have similar problems when working in context sometimes an AI bucket will fire in a situation that's tonally inappropriate

like Nathan Drake's bucket of quips he might spout off anytime he knocks a dude out if the tone of that game of the game at that point is more serious our designers flag that area to suppress all the jokey vo until it's appropriate for it to play later in the game the same goes for all the Buddies systemic dialogue another possibility is that a small or not so small bit of exposition might need to change sometimes it's contained in a cut scene others it's a gameplay line whose context was altered by Design usually something this specific cannot be fixed with an edit a new line needs to be recorded there are also times when a line is making things confusing so it just has to go we hold many Focus tests to determine where the pacing drags augmenting or cutting as needed to hit the ideal amount of gameplay between story beats since we overlight overwrite our levels anyway we can usually afford to cut part or all of an exchange and move another bit of dialogue in to F fill the hole as necessary all these adjustments make the

game better but we also have limits on what we can change as the project draws to a close the first hard wall we hit is localization localization for those that might not know is the process of translating and re-recording all text and dialogue into dozens of language for international players games at some point have to lock recording no more new lines and writing no more new text so that all the existing material can go through this process in time for shipping the problem is nowadays the deadline for localization is not after the game is finished it's at least a few months prior which especially since recorded dialogue is often so hypers specific to the actions happening on screen leads to a greater problem when things in the game evolve after that date because once we've reached the Lo deadline nothing new can be recorded this phase is when those generic SE ya lines become really handy we can use them to patch areas of the

game that evolved after the deadline need to direct your players attention to a solution to a puzzle hey look at this does a character shout out this way come on but you want to sell that the buddy is following add in a right behind you in response this type of dialogue has become so essential we intend to record even more lines like it on future projects like on Uncharted 4 we didn't know if the kids Escape in the manner flashback would feel better as a 10 stealth sequence or a balls to the wall chased from the police we recorded dialogue to support support both scenarios as time was running out but most of it was for the stealth version turns out what worked better was the latter we ended up covering the majority of the chase using cya lines but sometimes cya lines aren't enough we need something specific and have to borrow lines from another part of the game so we go digging through our database to see what we have that can cover the request when picking the line that will be duplicated we take into

consideration how far apart the new instance is from the original place the line plays so that again the player won't notice a repeat like this moment in The Last of Us that was covered entirely through cya lines reused systemic systemic dialogue and a storyline of Ellies from earlier in the game [ __ ] oh that was intense you all right Sam yeah I'm okay thanks Joel there are even times we've had to change the precise timing of certain igc's or cutcenes last minute meaning the audio files returned from all the territories have to be re-edited by hand on our end our localization manager keeps painstaking track of all these changes so we know which files need the extra love and by the way tracking all of this madness by any means necessary is the biggest challenge of making things The

Naughty Dog way the efforts of our minimal production staff and various leads are sometimes heroic but as we near the end of production the biggest limit on us is just time at a certain point we can only use animations that we've captured or lines that we've recorded if we don't have the material and can't make it up sometimes that means living with tonal shifts or big cuts in material that was painstakingly shot and Moa back on Uncharted one design made a change to gameplay by inserting combat into a sequence that was originally just quiet traversal Nate and Elena casually walked into an area in the cut scene and discovered a jet ski they weren't out of breath or recovering from a tense combat situation but the game play needed a spike of energy for the player and it worked even though it caused a tonal mismatch with the cut scene that followed still that late in the development process we could not re-shoot the scene so this was logged as as a an example of how we could improve maintaining tone when going into a cut scene in the

future on Uncharted 4 we were working on an epic sword fight near the end of the game and we had planned to introduce those new mechanics during the manor flashback with the kids we even MOA and recorded a whole bunch of material to make it happen but when we looked at the amount of work it would take to polish and script all those animations and the time we had left to finish everything else in the game our directors made a hard call and cut it so the player comes to the Ending Story stify untrained and can possibly be emotionally divested from the sequence as they struggle to learn new mechanics near the end of the game but if we catch an catch an issue early enough it has been possible to make the hard choices required to maintain consistent tone and performances in a sequence that needs adjustment nearing the end of The Last of Us one of our designers pointed out a big flaw we'd overlooked the Firefly Hospital culminated with a cut scene where Joel charged into the operating room to save Ellie in that scene he kills the doctor and eventually Marlene

right there and we smash cut to him and Ellie driving away it was a great cut scene and the performances we got out of it as written were excellent the problem was as the designer pointed out the player missed out on a big emotional payoff that we hadn't even thought about after carrying Joel's daughter at the beginning of the game the player was set up to expect an echo of that event that Joel would be carrying to safety at the end redeeming him some aspect of his former loss so with some hesitation we looked at what it might take to salvage what we could from the existing cut scene by keeping the Marlene animation and just picking up Joel now carrying Ellie we were able to move the main story beats of the cut scene into the hospital parking garage this allowed for a brand new ending sequence where Joel with Joel carrying Laie past dozens of firefly soldiers who we were assured would refrain from shooting Joel because

seriously he was carrying the cure for mankind so that is what we recorded all of our dialogue for and then after localization locked it changed again there wasn't enough tension if Joel could just walk out of the hospital with Ellie so the actions of the Firefly soldiers were revamped so they could attack him but that didn't match up with any of the dialogue we' recorded it took digging through NPC material we had recorded for other parts of the game to make this moment and the performances in it feel as equally polished as others in the game ultimately was totally worth sacrificing some mocap and scrambling through folders of VO to create an experience that would capitalize on player empathy to that degree so we do a lot of crazy [ __ ] and I'd just like to emphasize the way we do it isn't the only way or even the right way to pull off realistic performances especially depending on your team size budget Etc but the level

of flexibility we allow for story and gameplay to influ influence each other during development is why we feel we're able to strike a balance that we're proud of and it's also why executing these performances requires constant collaboration and dedication from every discipline on the naughty dog team writing acting rendering audio concept character programming animation design lighting production even it and Ops it takes a veritable Metropolis of talented men and women to make the performances that the player can feel when the controller is in their hands it is a constant struggle but if we can truly move you and maybe surprise you then for us that's art worth making thank you