Forging Audio Workflow Changes on Destiny: Rise of Iron

[Music] everybody thank you very much for coming my name is Karim Schumann I'm in the audio QA department at Bungie this is the talk I hope you can read it it's pretty big so just a few quick things please silence your phones or put them on vibrate and doesn't look like we're gonna be super PAC so you don't need to move into the middle or anything also at the end of this talk you should be receiving evaluations and your email it would be really great to get some feedback so I'll try to remind you again at the end alright let's get started you got a quick fun intro Bungie is an

exciting place to be everyone's always looking forward everyone's always thinking how do we take destiny and involve it to the future even wondering how the hell you did it Bungie is an audio team has always had a hyper attention to detail that's a key part of the mantra what our team is about we each bring something to it that wouldn't be there if we weren't here we work well together we challenge each other being a part of a much bigger team is exciting because there's a lot more collaboration and iteration on ideas you're constantly trying to achieve something you're trying to get to a higher place it's a big goal and a big accomplishment to create a universe from scratch and then make it sound like that universe I really enjoy working with the people here it's been a fun ride in the ride still movie what a great place to be [Music] awesome so I have the distinct pleasure of working with that amazing team for

those of you who don't know let's quickly go over destiny is an action shooter developed entirely by Bungie rise of iron which we're gonna be talking about today is our latest expansion which was released September 2016 and for the audio professionals in the room who are curious we use wise middleware implementation that's how we handle all of the audio going in and out of system so what are we talking about today this whole project which eventually turned into a presentation started with the conversation I had with our audio producer Victor who was somewhere around here today and so my question to him was what is the most painful part of development for for our audio team and his answer was the mixing process and his reason for that was that our audio

team tends to lose a lot of time dealing with issues that are not audio issues they're running into roadblocks and problems that aren't things that actually have to do with making it sound good and overall because of various time factors and the way that the mix process works it was it could get stressful for our team now for those of you who don't know this is sort of a cross-pollination talk we've got people from all different fields in the room I'm gonna quickly go over what an audio mix process is it basically occurs at the end of development when all of the sounds and music and dialogue is already in the game but that doesn't necessarily mean it sounds great already a lot of work needs to go into making it feel cinematic and natural so for example let's say there's a cutscene of two characters having a conversation and

they're standing next to a river you do not want the sound of that River to be louder than the converse a to the point where you can't you're having trouble understanding what the characters are saying to each other right even if it's a really big river that's rushing by and that might actually be more true to real life you want something that's a fun cinematic experience you want to be able to hear that River but an important part of the audio mix is to make sure that what's important is louder and clearer than what's unimportant even though we want to be able to hear and experience the whole scene the audio makes also involves a lot more than just volume manipulation it involves adjusting timing various cues it involves checking frequencies and making adjustments in that way as well it can even require going back to the drawing board at times which is rare it's sort of a last resort but if certain sounds are regularly

occurring in conjunction and that's not something we were hearing during the sound design process it might be something that's coming up in the mix and that one or two of them might need to be reworked before they can be finalized so because of all this delicate audio work that needs to go into the final mix and it needs to happen at the final stage of development when all of the sound and music and dialogue is already in the game mix requires that our technicians are working in balanced audio rooms full 5.1 surround sound speaker setup so kind of like the image that you're seeing here is one of our specialized audio rooms in the bungy studio and it also requires our team to be playing and experiencing all the new content that we release for a project over multiple playthroughs it's a long and intense process so the mix is handled by the audio leads at Bungie with a lot of help we have help from our head of audio a head of the

entire department our music director our dialogue director there's a lot of input that goes into the process but primarily it's being handled by our audio leads first and foremost we had three weeks scheduled for the mix Sun Rise of iron one of the biggest challenges about this for our audio leads is that when they switch from creating content to mixing content they're suddenly put in a position where they need to be experiencing content like a player so if you take a look at the slide which is just a super quick breakdown of part of what the audio dev process looks like for our team you'll realize that everything that occurs before the final mix doesn't actually require them to be playing the game like a player would they do go into the game engine they go into test maps they make

sure that their content is functioning the way it's intended to but the final mix requires them to actually progress through the game and simulate the player experience and that can be challenging to suddenly be put in a different sort of work mindset on top of all that time is really precious and I can't stress that enough it's the last three weeks of development and that deadline is a hard deadline it's close to ship and everything needs to be as polished as possible so previously what had been occurring in what Victor communicated to me in our original conversation is that a lot of time was being lost navigating in the game and we're going to go into that more deeply in a minute as well as that multiplayer content which our game has quite a lot of proves extra difficult during this process as well and so my project can be summarized into two basic goals in order to improve this

mix process I wanted to organize audio play tests to facilitate and improve our multiplayer experience and mix process and I ended up customizing tools that we have internally to make the work easier overall for our audio leads so let's get started with part 1 multiplayer so rise of iron the expansion included plenty of multiplayer content audio mix decisions in this type of scenario must be based on real gameplay so imagine you're one of the audio leads and there's a new PvP mode that you want to be mixing and you're in your specialized audio room and you load up the game you load into the PvP map and you're running around and you're like yeah all the new sounds sound great is that analogous to a full 6v6 12 player PvP experience on that map and the

answer is no and of course our leads know that and it's just difficult to fully replicate the scenario no one before this had taken ownership of audio play tests and so while they did occur they were inconsistent now to be clear play tests at Bungie are very common we have a lot of balancing play tests and design play tests that occur it's best especially with our PvP team and our raid team a lot of work goes into that we have designated play test areas the issue is that audio mix play tests come with added difficulty on top of that it involves connecting rooms digitally so that we have the play test rooms where we can have large groups of people connected to one of our audio rooms where one of our audio leads would be making adjustments and improving the mix it involved planning a lot of extra time as part of that and reserving those

rooms it also required support from other teams it required having those play testers who handled the PvP content the raid content and getting them on board because we needed their expertise and we needed butts in the seats to really fill out a whole 6v6 experience and get a realistic scenario for our audio leads to make real decisions and so mostly this was an issue of timing and communication and it also involved a lot of explaining the importance of what we were trying to do to those other teams right I mean if you're in the audio department of whatever project you're currently working on you know that you have to be an advocate for the importance of what we do and sometimes it can be unclear to those who aren't as experienced with working on audio so scheduling a big thing and taking time out of everyone's day to say hey we're going to do an audio mix play test for the new PvP experience a large

part of what I was doing was answering the question what why what is that and why why do we need to do that and so previously our audio leads maybe didn't have the luxury of sort of sitting down and making sure everyone understood what we were doing because their time was so precious this close to ship so best example of multiplayer that I'm that I'll be digging into a bit deeper is the raid that we released with rise of iron the wrath of the machine raid so for those of you who don't know what a destiny raid is like it's a six player co-op experience their giant boss encounters rooms after rooms jumping puzzles and a lot of unique content unique mechanics and so of course as a results we have a lot of unique sound design that goes into that and a lot of unique music that's triggered as you experience that content so it was really important this was one of the biggest elements to the release for the audio team with the new expansion so allow me

to show you why this type of content is extra challenging for our team I'm gonna show you video it's one audio lead it's an example of one audio lead trying to mix the final boss fight of this specific raid and they're fighting alone they've got a debug command inputted into the game so they're receiving no damage and they're just trying to experience the final boss fight to make some audio decisions so let's take a listen [Music] [Music] okay so that was a mess one of the loudest sounds in our game is getting shot regardless of whether or not you're taking damage and there are

enough enemies in that encounter balanced and designed for six players working together okay so all of those enemies based on their AI scripting we're just targeting the only player in that scenario right and as a result of getting shot all the time not being able to really move around or hear the ambience the external cues that are occurring it's really really hard for our audio team to experience that content solo and really improve the mix based on what they're hearing so here's a quick clip of the same encounter with a full team this is how the content is supposed to be experienced [Music]

[Music] so that's the audio goal there's a lot of moving pieces as you can see there's specific audio cues and they need to be balanced with the overall experience it's very complex and so the first workflow upgrade was scheduling consistent audio mix play tests it was mostly a task of coordinating and planning it started with us on the audio QA side attending regularly the PvP and raid balance play tests that allowed us to be more familiar with the content it also allowed us to be more familiar with everyone on those teams and we then started to organize joint play tests specifically with the goal of improving the audio mix this meant we got lots of players experiencing the content as a group another interesting effect was

that they were playing like real players this was content that they were familiar with that they knew how players would play it in real life and they weren't running around in the game like our sound designers do where they get all excited about the new thing that they made and they're like oh look we can do this and isn't that great and then you get people who are playing the game like real players and they're running the boss fight real fast in real time this allowed our audio leads to get an improved perception of what the game was going to sound like at full speed play overall this proved very valuable and we continued to expand on audio mix play test today we're starting to do them earlier in development it's really an important improvement that we made with our workflow so part 2 improving the tools to make it the mix process easier for our team overall during mix time was lost just getting around and the reason

for that is because destiny is big and complex the leads want to access all the new content quickly they need to get to this specific scenario they need to hear this specific mission but unlocking and accessing content in our game is challenging so here are some examples of how content naturally unlocks in DES is just some there's more each item that you're seeing on the screen is tied to in the engine a hidden flag when you complete a mission for example you flip that flag you trigger it and it allows you to then access the next piece of content behind that of course working in development we have debug commands that let us specifically trigger these flags without actually having to complete the content and play the game naturally every single day but during development those flags

can change those debug commands can change more implementation occurs as we move closer to ship date the build becomes closer to what is actually going to be released and so it becomes more and more systematically set up for those flags to be in place and functioning where it's more like the player experience and less like the creation experience that we get used to and so leads wouldn't be encountering any of these challenges before they started working on the audio mix what I was saying before that their work before working on mix is not based on experiencing the game like a player they're working in their DAWs they're working in wises they're working on test maps but now three weeks before ship they need to work on the mix they need to be experiencing it like a player and that means moving forward and experiencing one mission or one section

as a player would and that meant dealing with all of these flags quickly and effectively so the goal is getting around and returning to specific States day after day as they iterated and improved on the mix process this is how our devs got around in destiny before rise of iron dev builds all of our dev build builds include command lines the challenge with that is you need to actually memorize every debug command that you want to send into the build that comes with its own challenges most people use our debug menu which is the image that you can see on the screen now the debug menu definitely works it uses controller shortcuts so it's regardless of platform you just have a controller plugged in you press a series of buttons to open up the menu and you continue to use the controller to navigate through those menus it's basically a series of nested menus like a folder sort of structure it can be

pretty slow navigating through it we're gonna have a video briefly just to give you an idea of what it's like to move around in it it can also be hard to find what you need sometimes most of our devs they get used to going to specific audio locations where they know it has debug commands that they need but if it's something new or it's something they're trying to deal with the debug issue that they don't normally deal with it can be very challenging to navigate if you don't know where you're going and also it doesn't include everything when a flag got changed that doesn't necessarily mean the new debug command required to handle it was going to appear in that menu so here's a quick video just to give you an idea of what it's like you can be in-game you can press the shortcut to open up the menu and you can use the keypad to move up and down and hit confirm to enter menu turn on a command that you want hit cancel to go back and select a different one and that's just the general concept of what it's like so to improve the

process I focused on this tool specifically it's a dev tool we have a Bungie called Guardian there are a couple of caveats here caveat number one this is an internal tool that we use I had to sneak out just to show you this much so make of it what you can caveat second one I did not make it i customized it specifically for our audio team and I'm going to go into how in a bit more detail but many thanks to the Bungie tools programmers that we have an amazing team for not only creating this tool but also including with it all the tools necessary to customize it for each team's use what I'm hoping that you'll get from Meeks explaining all of this are some of the concepts and usage and feedback that I received that will hopefully help you with whatever project you're currently working on or will in the future so Guardian handles a lot of different

functions we're not going to get into all of them it runs on a PC that's important to know so it's controlled by the mouse and it connects to your game client regardless of what platform you're on so you give it the name or IP address of whatever dev kit we might be working on and it can connect to it recognize our game client and allow you to make changes specifically there's the element we're going to be talking about with Guardian today is the command assistant and again getting around to destiny is hard so the goal was to use this tool to improve accessing the content and make useful debug commands accessible with a single click this is Guardians command assistant that's what it looks like if set up properly Guardian can replace the need for the debug menu of the video that I just showed you it starts off blank so I created all of the buttons that you're seeing in front of you basically behind each button when

clicked custom scripts send multiple debug commands to the game client that you're connected to and so it's organized and created with the intent of easy functionality grouped based on how the devs actually use those debug commands specifically the audio devs so one of the other excellent features is that as soon as this was finalized and saved I could automatically share the entire page of buttons with the whole office so the next time anyone in the office closed and reopened Guardian it would automatically make this page available from them from the drop-down of buttons and command assistants that are saved into our system before this there had been no audio command assistant page before this known an audio used Guardian pretty much at all but based on my experience I was the one who believed that it could make things easier and faster specifically when it came to the mix process quick tangent

maybe you don't know what kind of debug commands are useful for an audio and so I'm sort of going into all these debug commands and you're wondering why is this so useful and important so if your tools programmer maybe or an audio programmer maybe you can look into creating some of these debug commands and tools for your team's if you're an audio content dev and you've been struggling with your own process maybe you can ask some of the programmers on your project for some of these excellent tools so let's get into it a little bit event emitters this is my personal favorite it's used a lot by the audio team it basically puts if you look at the bottom half of this image you'll see some blue debug text that you will not see in the game normally this text is

includes the name of the event that's being triggered the audio event that's being triggered by wise and it's placed directly over the object that is triggering it in the game this proves extremely useful as we navigate and we encounter issues we're searching for bugs we want to verify that a specific weapon for example is making the sound that that weapon has the sound that that has been designed for that weapon so on and so forth it's extremely useful for quickly understanding why something is not sounding the way it's supposed to sound distance occlusion obstruction basically anything that gives us visual feedback of what is happening in the audio engine that wise is trying to create for us whoa I need to hurry up okay so a distance occlusion obstruction also adds some of that blue text it lets us know how far away things are from the camera as they're being triggered lets us check fall-off distances things like

that environment info brings up a big blue window like the one you're seeing here that tells us what what ambience loop is currently playing what reverb settings we are in based on the environment material info is this exact blue window that you're seeing based on where your cursor is placed or sorry where your crosshairs are placed it will let you know the exact material name which gives us a lot of great information for gunshot impacts and footsteps and we also have a bunch of commands that let you wander around without actually playing the game so things like turning off damage skipping ahead and emission these are all super useful for us especially in the mixed process so incorporating our favorite debug commands into Guardian again function and layout needed to be intuitive I spent a lot of time working on iteration it didn't come out perfect the first time a lot of changes were necessary at

first once it was completed and in the state that I thought it was actually useful and better than what we were currently doing I then had to teach with the help from Victor our audio producer we had a demo for the audio team where I went through with Guardian and showed them went through the paces and showed them what it was capable of then in the next office wide play test that we had while developing rise of iron I went to each of our content creators desks I had them load up Guardian I gave them a couple of examples of how it could allow them to navigate the game and I asked them to try using it if they wanted to use any debug commands instead of the debug menu that they were used to just to give it a shot I also throughout the process was offering to take requests to make new buttons each individual dev might have commands that they used shortcuts that they wanted that would improve their particular process and I was always happy to incorporate that once questions and ideas for improvement started to arrive in my email inbox I

was really happy it meant they were using it and overall just a quick display here it's organized by columns player setup allows you to sort of skip the character creation system when you're loading up a new account gameplay changes allow you to turn on things like turn off damage increase your ammo your ability recharge rate debug client allows us to turn on all of those debug texts that we were talking about in the previous slide music information ammo investment shortcuts these were this is the general assist page that I made that continues to be useful for us throughout development so here's an example of Guardian in use so we've got Guardian page on a PC on the Left we've got the game client running on the right we're in a very specific scenario real quick because I forgot to explain what we're trying to do so let's say you're in the mix session and you

made a change the previous day it's a new build you want to check to see how it's working okay so you change the fall-off distance and volume on a specific sound that only occurs at the end of mission to some monitors turn on and you made some changes you want to see if it's working so this is the process that we're going to go through the first thing we did here was we selected mission two from one of the command assistant pages and so now we've already started loading in to mission 2 on the screen you're going to see the ship takeoff now we're going into player state and we're turning off damage so we don't have to worry about getting killed while experiencing the sound now we're going into the debug client column we're turning on event emitters so we can see the sound names and we're turning on distance emitters so we're going to be able to really check the fall-off rate based on the camera location to where the sounds being emitted from now we're

just waiting for the ship to actually load into the mission and you can already see some of the blue debug text in game this is all fully connected but running on two different streams we've set up the phase jump of the man we're gonna skip ahead to the end of mission two we don't want to play through the whole mission right so we're skipping ahead to the final phase now we're in the game we've got all the debug text we need we can pick up the controller and immediately move to the section that triggers the monitor sound so we open up our goats we're seeing all the debug information playing over the sound so we know what's playing and how far it is from the camera and there's that unique sound that only occurs in

that part of the game we wanted to make sure that the changes we made yesterday were sounding good today and that's the process and an example that video took a little over a minute and a lot of it was spent loading in to the zone skipping ahead in loading the the final phase before Guardian according to my conversations that process could have taken 10 to 15 minutes repeated dozens of times per day of audio mix process the time savings were really high and it reduced a lot of stress for our deaths keep in mind that more mix time overall meant that more content gets polished because that deadlines not moving right so any time saved meant that more content more polish could actually go in our devs could get closer and closer to the final product that they were really proud of so the results were in there were two pages of Guardian custom buttons created for rise of iron a general audio assistant that I went

through and a rise of iron content page that specifically allowed you to access all of the activities that we created for rise of iron quickly the audio team adjusted to using this new system devs no longer bothered with the debug menu at all I'm told today they don't even bring it up they are consistently using that system it also improved audio QA efficiency to the point where it's now crucial we on the audio QA team use Guardian and those command assistants to the point where the expectations of how long something will take have actually changed other teams it's also really useful for when other teams have audio queries it's really easy to point to hey Kareem I think I found an audio bug something weirds happening how can I check to see if it's triggering the right event instead of typing or copy pasting along specific debug command to them I can just point them to the Guardian page and they can bring it up and use it themselves most importantly it resulted

in a better final mix a better overall product everyone wins when the game sounds awesome and so the teams helping us play test we're happy and next time it's even easier in summary if you aren't an audio please no audio devs need support in this specific scenario me and audio QA that's where that support came from but it can come from anywhere the next thing as well as that audio decisions really need to be based on real gameplay and that's part of the reason why multiplayer can be so challenging I also implore you please don't skip on project Tools customization is awesome if you have the time or the resources to create a system that allows you to customize your tools I highly recommend it and it's important to recognize that even if you have a tool that can do something really awesome it needs to be easy to use no one in our team had used Guardian before because it wasn't easy for them to understand and so by

customizing it for them and for their needs and making it intuitive that's when the changes really started to occur development is hard over all games are hard to make especially as you get towards the end right but by working early looking ahead we can really do our best to make those final days as easy as possible overall this project proved to be really successful here's some words from the audio team at Bungie thank you very much this has been my first talk at GDC it's been a real honor to speak in front of all of you and please fill out your evaluations if you can if you have any questions you can follow me to the wrap-up room I know we're out of time thank you [Applause]