Fighting Legos: Audio Choreography for Battle Design

[Music] so hi i'm chris brown and i'm giving a talk today um fighting legos audio choreography for battle design what that really means is discussing the change in how we record what i call vocal sound effects or omnis or or onos kinds of the sound effects that you're going to hear when you're playing battle design so it means if this works correctly uh using voiceover sound effects as a game mechanic we have a lot of we used to on carts right you'd have to cut everything down to its minimum thing um you'd only have a binary choice uh something like this going from this where we're enhancing a moment somebody gives a punch somebody receives a punch it's just a small sound that lets you know okay they've given or received a punch we're going to move now more into a world this is a still from for honor where the voice over sound effects that

you're hearing back from the game are actually helping you to learn how to play the game and to confirm for you that you are playing the game correctly why so why does it matter voiceover sound effects help you play the game well isn't enhancement enough for me i'm always looking for ways to help people understand the world better when i want to bring a change to the table people why why do we have to change why do we have to do it that way doesn't it you know if it ain't broke don't fix it um so we kind of always go back to this what's the purpose of sound design when we're finding why we're going to make a change is it going to serve our greater mission for me the greater mission is always providing the information you need to know to interact optimally with the world and again this talk is audio bootcamp so it's made for people who are coming into sound design which is why we're going to just do a

brief overview when we talk about getting the information that you need to understand and to interact optimally with the world we're looking at things that provide feelings and context for you it helps you comprehend the world it helps you be motivated for me when i hear drums starting i get fired up i'm ready for battle you know you have kind of these classic archetypes right your minor chords of danger and loss strength of drama your breasts of victory kind of the things that are the standard language that we use in audio to convey certain emotional feelings to people to let them understand that this is what's going on in the game in the moment and to help them be motivated to move forward when we look about ambiences when we talk about ambiences this is still from red dead as i'm sure you know you think about the desert this feeling of desolation this feeling of loneliness this aridity this uh kind of you'll hear alone coyote howl

it's always reflective of marston's psychological state this feeling of being sort of the lone person out there maybe the only one who understands the world maybe the only one who understands you know the grim realities the mechanisms of how the world truly functions it helps to convey the sense of being very kind of lost also when we think about ambiences we think also about bioshock feelings of you'll hear the creaking you'll hear the water it's very haunting and you don't players don't really sit back and say why am i feeling creeped out all the time you know you've got these sounds you've got the splicers coming at you you have the you know big daddy groans you have these sounds that are individual sounds that are giving you a feeling but underneath it all you have this bed of the actual pressure not only of the moment but the pressure of the ocean on this thing that at any second could

break through so we talk about sound we talk about music providing you feelings context we talk about ambiences providing you feelings and context when we talk about non-voice-over sound effects we talk about actionable tool info right so you're looking at the lancer you are hearing it you're hearing its power it's kind of exciting it's kind of grindy you're like oh how much can i f people up with this it's very exciting and fun and not that i'd like to do that but um so uh right here's your garage agent 47 it doesn't make any sound which is important um you have uh um oh my gosh why am i blanking on the warthog okay my favorite vehicle uh because it's small it's light it revs at a lit you know a slightly higher pitch it lets you know oh this is more nimble this is not heavy as a tank this is going to let me go up and on crags and on rocks

and it's all things that again without the player really thinking about it it lets them know these are the rules of the world these are things that are at my disposal that help me understand the mechanics of the game design itself that help me move around so in that way your sound effects are helping to fill out level design it's it's pretty neat when we think about voice over sound effects um binary enhancement right give a punch receive a punch this is super punch out which is super fun um but it's it's not telling you really anything other than that you know you gave a punch that was satisfying but it's not helping you really bond with the mechanics of the world um as we moved further along in game design we have now again because i'm doing mostly aaa kind of battle design you have your

assassination so here's the strangulation you have your kills from red dead and then again the satisfaction of killing splicers it is satisfying um in red dead in particular the kill sounds that you hear can be so dramatic and so pathetic that meaning that you kind of feel like an for killing these people which contributes again to his psychological state so you are it's an enhancement it's deeper it's moving into creating emotional states for the player but it is not yet helping the player to understand that the particular feedback that they are inputting into the engine is being reflected back to them the way that we used to record these types of voice over sound effects we would record them as what we called a variety pack so we would make diverse sounds for a single action they were kind of catch-all sort of an imprecise response

they're non-customizable in engine you're not sure that you know let's say you have a bank of 20 sounds coming from a script like this 20 strangled to deaths so you have maybe some wet ones maybe some short ones maybe some long ones um and the engine is going to randomly call them from the sound bank and it's not necessarily always some people will key it to the art but that's pretty complicated from an implementation standpoint it's not necessarily always gonna it's not really gonna happen that much um it's a bank it's gonna pull from a bank you're gonna get whatever kind of strangulation art that you're gonna get you're gonna get a random strangulation kind of sound that you're going to get maybe they're going to match maybe not but it's great that we used to do it this way because we didn't want to tire the player's ear so we'd always say we want to do a huge batch of a lot of different things so that the player doesn't feel tired by hearing the same punch

sound all the time the same strangulation sound all the time and we did that mostly due to space limitations engine limitations processing limitations so now as we're moving forward kind of a year or two ago um with this particular game as an example but with a lot of other games you'll see that people are able to create what i call bespoke or custom tailored interactions so we'll have something where it's not just a punch but it's a combination of things that are that the player is putting in and then the game engine is reflecting back so the player's saying well i want to kill this guy it's not just that i'm stabbing him it's that i'm kneeing him in the crotch and i'm hitting him with the elbow and then i'm beheading him or some variety of does it sound like i like my job um so so uh yeah so

so anyway so what we want is we want people to learn how to play the game in a way that is supported by the sound design that we are giving them so we want to create a sound bank that is going to have a highly highly specific very small sound that is going to tag exactly to the to the animation that they're going to receive so when you have this let's say you're going to elbow someone in the neck it can't be what we used to do with our variety pack it can't be like 10 or 20 different well maybe i'm blooming in the neck and it's wet maybe it's dry maybe it's raining maybe it's you know all these different things we had to figure out a way to create a really really specific script that our actors could do so then every one of those individual pieces in the engine would uh would be consistent and would always reflect back exactly what the art was going to do again why is this good

because believe it or not we did have to have these discussions looking at haptic feedback or looking at gyroscopic feedback as we're moving into vr it's awesome and it's great and it gives you information but it doesn't give you the kind of finely tuned information that i want to provide to the player so that the player understands exactly what they're doing and they're learning through sound league amazing the art is fantastic it helps you but we could do better and that's what i'm always looking at is how can we do things better how can we support the player so that they are understanding what they're doing and having a more satisfying experience accurate immediate reinforcing feedback between the player and the game right you have here's your instruction set you want to make sure that when you are giving an instruction the engine is saying yes i understand you want to make sure that what you're going to get back is this not that um

so how do we do that we talk about voiceover sound effects creating audio pieces um a script is this kind of sample and it's it was interesting um i use for honor as an example there's a couple of other games coming up where we've used the same model because we have so much more processing power and so much more customization available to us the actors are really accustomed to variety pack because we've been doing it for so long so it was really hard for them to kind of be reminded at every take no you have to do it the same the same the same every time and they're like ah it's so okay i'll do it the same and they and they would and they would do it and what was interesting um and this is a conversation for another time but the western kind of european language model because it's not a tone-based language and it's not dependent upon

absolute perfect tonal consistency every time what we learned in doing this is that the actors even if they're doing the same specific length for the same tiny thing there will always be a tiny little micro variation in their voice so it doesn't tire the ear and when you paste everything together it actually worked which was really really neat and as we were watching players playing the game they would say like there was one particular person i was watching and they said oh that's a move i'm like yeah that's a move they're like oh well wait and i can do and now and to watch them learn and watch them learn through sound was really exciting for me and it made me feel as though we could give them a lot more to work with and help them understand their tools so very short talk each element is a building block enough building blocks grant you infinite customization by mixing them up

[Laughter] and it creates a more responsive world it creates a greater bond with the world which is the thing that i'm always after um and the greater bond is greater satisfaction um and happier players that's why we do what we do yay that is my talk so my talk is super super short so if you guys want to talk about anything else i can answer your questions or you can all go get coffee all right coffee time oh wait you have a question oh yeah great talk perfect length oh i think it could be longer and i want to come up with two questions how has this changed your castings a lot of are you looking at actors who maybe can't do a hundred ten very similar variations of 100

moves or do you try to train people into it or is there a point where you're sort of like okay we have to try something different when we're sending out sides or auditions oh that's a great question so the question is how has this changed my approach to casting am i having a difficult time with people who are unable to conform to the kind of new it could be perceived as a limitation or it could be perceived as greater freedom it hasn't really changed my approach to casting but it has changed my approach to working with the actors in the studio it is definitely something that i need to train them into and it's a little bit um difficult because they are depending on you know some actors are trained to do three in a row some actors are trained to for every one in the three in a row they're trained to give you something different some actors are are like i'll do you're three in a row each one will be different hopefully it'll be something that you like then you have other actors who are more method style actors and they'll say i'll give you three in a

row each one is aiming for a particular level of emotional truth and i'm just trying to get there so for this item because it is really simple we train them fantastic thanks yep thanks actually continuing that question uh in the vo session do you have strategies for getting to that point like would you go through a bunch of different variations until you hit the thing that you want and then do that 10 times or how do you handle that oh that's super smart that's a great question so the question is um continuing on to that when we are in the recording studio if we are training someone in an actor and how to do this correctly or how to do it in a way that gives us material that we want um i'm trying to think of how to phrase it uh do we have them try it until we get something that we like and then we have them do that thing that we like

10 times or do we just kind of let them go and we have them do it until we get the thing that we like and then we have them do that five to ten times hi hi um thanks for your talk it brings up a lot of questions we've had the same issue as uh as you we're looking for consistency or petition so what we started doing is actually writing out onomatopoeias and a bunch of them so like and ha and he and same thing with k's and same thing with vs or or even right up silent onomatopoeias and so we have this list that we've been using on multiple games with all of these written out automotives which makes it really easy for the actors to repeat that's pissed that one specific onomatopia and once all of that is done then we'll go through them and see what we recorded applies to what action instead of being action driven onomatopoeias oh that's a great approach

so i was wondering if if you had tried this and if it worked because i i've constantly been wondering should i go more for for when i tell an actor to kick i get that same problem that that you've been getting they do whatever and then it gets really hard to get consistent acting so yeah i think that's a fantastic approach um i haven't specifically tried that writing out the the onomatopias on my stuff i've seen um there is a great i don't know if he's here there's a great sound designer at ubisoft uh who is uh has i've seen do some similar things like that i think it's a fantastic approach um restating he is uh actually writing out the onomatopoeias on the script and then gathering a library of specific sounds and then seeing where they fit the art instead of doing it the way that we've been doing it which is to kind of look at the art and then try to have the actor create something so he's kind of um out ahead of the process

i think that's super super it works really really well and just to add something else that i do is what we do is we we write all of these on a shared screen so that's in our uh on our recording uh console computer and we share the screen in the vocal booth so as they're doing what they're doing if they do something quite different that sounds like an automatopia that it can write i'll add that to my list so i keep track of all the written onomatopoeias and sometimes when they go randomly i'll take notes of exactly how it sounds it's much easier to track so it's just an idea i love this idea it's wonderful yay is it do you guys want me to restate or you guys all heard it yeah okay great yeah that's fabulous thank you so much hi i'm an audio student at cogswell and actually had a question um what percentage of game audio requires the use of javascript because i'm learning a little bit about javascript and i was wondering if i should really pursue like the you know learning how to script

you know more in advance that is a fabulous question and i wish that i knew the answer to okay um i'm sorry that i don't the question was how much of what percentage of game audio requires javascript and is it worthwhile for a student to spend a lot of time learning that so they can use it as a tool and i'm really really sorry but i just do not know the answer thank you hey so i'm pretty new to game audio and everything and especially unfamiliar with voiceover and everything so you're talking about getting real consistency and everything in the voiceovers and i would imagine like saying a fight scene for example or a fight to wins uh you'd have a lot of like you get hit in the same place but you would have a lot of variation because you're not going to grunt the same way so why do you want that oh um i like the consistency so that well there's always going to be because again we're not working in a tone-based language here in the production that we're doing in

the u.s um we want that consistency so then the player this is a fantastic so the question is why do we want consistency if somebody is getting kicked why are we moving away from the concept of a variety pack um and the answer is because we want it to always be the same sound so that that little portion it's like saying um that we're going to let's say we have a particular set of moves we want the player to understand that that is the sound of a kick so that is what a small kick sounds like so then when the player is doing the particular input on the controller for a small kick they hear oh the engine i did the controller right the engine understood that i wanted a small kick and the engine is giving me a small kick so it's more about like the player input opposed to uh like kind of player feedback and experiencing that something like that

um it's so that the player understands that that so it's basically so that the engine is telling them i received your command correctly it's basically just an echo back um and in order to if it was if we were still working in binary like if we were doing super punch out and it was the same thing all the time like oh my god it would be super boring and sucky um this only works when you are working in sort of a for honor environment or an upcoming environment where you're creating you're stitching together bespoke things it would not be awesome if you were working in super punch out because it would be super boring okay thank you thank you hi hi thanks for your talk um so it sounds like you're talking about shorter asset length or correct yeah absolutely um that flow together and sort of fight combos or not exactly not a button masher kind of a situation but um how do you feel that that uh changes

like if something needs to get changed in terms of an animation like the timing needs to get changed or something do you feel like that makes it easier to kind of roll with those kinds of changes or is it that's a great question um the question is what happens after you record all of these teeny tiny sounds that are going to be pieced together and make your little lego dragon or your lego millennium falcon or whatever um and the animation changes uh i don't know that it necessarily makes it easier or harder i don't think that those things uh i think you probably are faced with a which re-record but that's sure if you want to do it right yeah okay that's a good question thank you thanks hi hi um so i one of the things i really appreciate about what you're talking about is the implicit accessibility advantages um anytime that what the player does corresponds

uniquely to what they hear has inherent advantages for any blind or low vision gamers who otherwise aren't getting that reinforcement from what they're playing so a little bit of icing on the cake that is amazing thank you very much the comment is that this is really helps accessibility for uh low vision uh people or people who are blind who are playing the game that i hadn't even thought of that and that is an incredible thing to say it's a really good thing hi so um i was kind of thinking i did see that the direction that you had for some actors was you know like we're gonna do the same sound um with uh with variance uh 10 times but also at the same time with consistency with the advent of middleware now um you know like really running inside games um in particular you know like um especially at the aaa level um how often is it because i i'm sure that

you know like not all 10 assets are going to be used so how often is it that you know like you would put like a randomizer in terms of like pitch and volume to substitute out and kind of get rid of that monotony um well i think that for this particular pro it's a good question so the question is um if i'm understanding it correctly uh would we then use middleware in order to try to create variants is that in order to create variants to avoid sort of audio monotony in the particular sound for this specific practice i don't think that we would because we rely on the actors natural kind of organic tiny variation in order to do it so yeah i'm not sure that we would um some other people might and it might be amazing for us in practice we haven't done that but it's a good question thank you great thank you so much

hey thanks a lot oh so uh the question i have is does this approach change at all for working with creatures or more monster type and less less humanoid creatures wow that's a fantastic question um gosh i don't know i guess it would depend on if you were playing the creature i think if you if you were playing the creature as likely some sort of giant thing that would have multiple moves that would be stitched together where it would be depend where the player would need to understand that what the direction they've given to the creature i think if it's a playable creature character god that would be fun it was a devious laugh yeah that's that is a great question i think if it's playable sure cool um and i i did get the five minute um thing so thank you very very much everyone i really appreciate your time