Three-Way VR Audio Postmortem

[Music] good afternoon everyone thank you very very much for coming to essentially the last session of the conference sort of time schedule wise i'm sure a lot of us are excited but exhausted and a little bit lower in energy i actually did a talk in this very room on tuesday as part of the boot camp and i was full of energy and it was a high energy exciting little session this one's going to be a little bit more low-key i think for all of our sakes so the post-mortems are actually a fairly common and popular thing as part of gdc because one of the best ways that we can learn is to go back to the things we've done before uh analyze them to a point but sort of consider certain aspects of them in this particular case what i've done is it's going to be a bit of a you know a very fast session because this is only a half

hour session um i'm going to go through three vr projects that are very different and and i think if there's any takeaway i want to have today it's that one of the problems we're sort of getting at the moment is everybody going oh yeah vr and they we just vr dump it in a bucket that's what it is um it's not there are so many different things that vr is already and i'm still very much of the belief that none of us yet really have any scope of where vr is going to end up because i don't think anybody yet has worked out the exact best language to use to present vr we're having to relearn cinematography we're we're really having to relearn how we approach the audio of it i've i've looked at a lot of vr projects recently and and i don't want to be nasty about it but the audio in a lot of these is frankly really very ordinary they're not doing a very good job of

conveying the sense of presence of being in a place we can't use just common stereo or mono and in even hrtf which is the sort of new spatial format is not a magic bullet hrtf is just another thing and in one of the ones i'll talk about today i'll go into that in a little bit more detail so this is called immerse impact and in form and the reason why is because if we think of about a triangle and we've got those three elements in a triangle you will tend with your projects to sort of lean a little bit more towards one than the other two or you might if you're dead said you might equally have all three elements to a certain degree these are the three projects i'm going to be talking about and each one focuses on one of those specific elements at the reduction of the other ones so we

have vr regatta which was on the vive oculus gear vr outbreak origins is a location based one you've all seen these sort of ideas of people running around in warehouse they've got a backpack on with a computer their vr headsets their headphones and they're carrying a plastic gun then there was actually a military training simulator that i worked on for the australian army basically the same sort of technology as outbreak origins in that they can move freely around a training area with the equipment on it but obviously a military training simulator has a very very different purpose than anything that's sort of a game-based idea so as i said vr is not a single thing so vr audio must respond to the specific specific needs of the project uh in the same way that we treat the audio that we would do for a film we would treat differently to the audio that we would do for a television series or a netflix series and then we would treat game audio differently again or a podcast so you know just saying it's

audio for media there's a very broad term and we have to approach each of these things differently in vr um i think it's possibly the same way that you might say that well the audio that i would do for an fps is going to be different to the audio that i would do for a strategy game we have to accept that that's the same for vr depending on what the purpose is and yet again the reason why i've done this whole talk is because i've observed that some people are not quite understanding the possibility of this yet so vr regatta is one where the immersion is the critical aspect of the project outbreak origins it was the impact and the military training simulator is purely about informing the personnel who are training with this and so each of them will have i mean obviously you've got to inform somebody so in the vr regatta game and in the outbreak origins there's still some level of informing the audience or the player what's going on but it's probably much lower down the list than it was in the

military trading simulator so vr regatta was a lot of fun to work on because it was a fixed position you were sitting in a boat you weren't walking around you couldn't get out of the boat you could move around the boat a little bit but this actually allowed for a really really curatable experience i knew that you were always in a boat so you're always surrounded by water which means i could place the water and i could really work on tuning the water and making it really lush and lovely same with the wind and the wind was very important because sailing boat so that limited scope of in a boat on a water meant that i could polish the the elements that i did have far more than if i'd had to make 50 different levels with 5 000 monsters and 20 different weapons and all this sort of stuff where you've got you're just spending all your time putting content in i had the luxury of really being able to polish this quite highly now it was a small scope i mean this was a

very small team an indie developer in australia and a small team they didn't have a lot of time they didn't have a lot of budget but again um that meant that because it wasn't one of the things about a lot of simulators like if you've got spaceship simulators and airplane simulators often there's not a lot of you're not rendering lots of trees and stuff like that because you're up in the air same sort of thing with this you're in the water so there's no ai so there was not a huge demand on resources for the rest of things going on and not a huge demand on the audio system from the point of view of not lots of environmental sounds and monsters and ai so i could put everything into the few things that mattered now you might be thinking well that doesn't work like i want really good wind in my game but i've got lots of things to go to to deal with that's still possible you then just need to set up how priorities work or you can extrapolate some of the things i mean as you can see there it's like 24 channels which i've

used for the wind well you could probably get similar effects by starting with that and then cutting down and working a balance between your resource needs and what what sort of sound you want but again you can also achieve this by using a priority system where it's like yeah so i'm using this many channels when there's not much going on but when monsters start coming in or whatever i can start to reduce the use of some of these to allow these things to come in so those of you are not used to working with audio systems there are lots of ways in which you can balance out the need to juggle you know needing lots of channels and only having a finite number of channels to work with all right so i'm going to go through this bit in a fair bit of detail so you can see there uh the representation from above of the the boat and i've the camera would normally be sitting in a boat but i moved it across just for clarity so the the camera there is the standard camera you'd have in unity and what i did was i created an array so where that says ambient array that is

actually a series of sound emitters placed around the camera kind of north south east west and each one of those would basically play a looping wind sound why did i do that one of the key things with vr is the ability to turn your head within the environment every one of us right now are literally sitting within a sphere of the world like we have air all around here and and that air is basically what translates the sound into our ear and it doesn't just come in from here and here it comes in from all directions bounces off areas etc and so what i was trying to do was sort of create in some ways a little bit of that bubble and so by having wind coming out of each of these and each of those wind sounds is just slightly different it achieved what is in my mind one of the most important things to achieve with audio for vr a point of difference when i turn my head i feel like there's

something there and something there and something i feel like i'm turning my head within a bubble of sound whereas with stereo you've got this here and this here but if i turn the head it's like well what's there and what's there and then these other ones are the fixed ambiences which are placed out actually end up working essentially like an audio parallax layer so i've got my array here and then kind of in the gaps but at further distance back and all of these have got attenuation on them so they're all sort of dropping off um those ones at the back just sort of sound like wind over there and so when you turn your head you've got this really nice sort of you know dome of of of wind around you um i actually also the array also had one above as well um so if you did that there would be wind above you and you could do as many of these as you wanted to i this was the balance that i i worked with and i really liked how the result um i then have a mono emitter over here which is a 3d emitter and that is actually the source wind so that will actually all these other ones are just a really gentle atmos like almost like a room tone super super subtle and soft

right they're just there to make you feel like the air is there um like the sort of sound you would have in a completely empty room it's subtle but it's noticeable whereas the mono 3d emitter that was over it would always be sort of 20 meters away from the boat wherever the wind was blowing so if the wind changed to over there it would be over there and it literally just was it was the sound of the wind but you'd be like oh the wind's coming from over there then i did something else i played around with it a lot and i was like okay so but i know that in real life when the wind's blowing when i turn my head 90 degrees into the wind the wind blows directly into my ear canal and it actually reduces so i actually put in a parameter so that when you turn 90 degrees into the direction of the wind the volume of the wind would drop a little bit and it would filter a little bit and i thought yep this is getting closer but then i was like yeah but there's that there's that thing of when you look directly into the wind it's blowing past your ears and you get this really blustery effect over your ears now that's a problem

for hrtf because hrtf is probably quite good at giving us spatial sort of elements all around us up to about a meter once you get inside a meter in the near field hrtf just goes blue i don't know what to do and i was struggling i'm thinking how can i get this effect and i started listening to some sounds and then i found a really really nice stereo recording of wind blowing past my ear like it literally has that you know you can feel it rustling around your ear the shell of your ear i'm like you know that's a stereo sound you can't use stereo sounds in vr because it'll track with the head etc and then i thought except i added it in and it does play it's a stereo sound it's right past your ears but it is playing at zero volume anywhere around the 360 degree arc except for when you get within three degrees of that's where the wind direction is it starts to fade up to full volume it's full volume relatively um so when you're looking directly down

to where the wind is you get this nice near field blowing effect past your ears and as soon as you go three degrees off in either direction it's faded down to nothing so a stereo sound that's locked my head is working because it's only triggered at the exact right time binaural sounds is another one that would be really good binaural are really good for that whole creepy thing really close to the back of you but of course binaural if i turn my head doesn't work but if binaural is a really short thing like think we're playing a horror game and it's a little girl's voice that says hello by the time you heard it and reacted it stopped so i can turn my head and it doesn't matter and so the whole time there's this little voice on my shoulder behind my head giving that creepy feeling but as long as they're really short you know really really short things that by the time i've reacted they've stopped playing so i'm not going to notice them sort of swinging around with me binaural can be super super effective because it's binaural recordings are really good for that near field thing so you can see here that there's a whole bunch of layers there and and then the thing that is really important here

is hrtf is not a silver bullet hrtf is one of many layers that you use to create the effect of of what you want with spatial audio so the the fixed ambiences and the ambient array were sort of basically hrtf the stereo one i was telling about with the wind is just a stock standard stereo sound the mono emitter was a mono 3d sound uh in other areas i think i used a just a mono sort of drone sound so layering mono stereo binaural 3d standard 3d sounds with hrtf is how you get this any of you working on vr or any of you who have audio people working for you on vr really understand that just switching on hrtf is not enough and i've seen some projects where they have just done the normal 3d stuff switched on hhf and gone great and i'm fairly convinced now that with this i reckon even if i turned off the hrtf aspect of it all these other layers

would mean it would probably still sound better than just doing the 3d sounds with hitf so just be aware of that um all right so i've got a lot to get through so with the water same sort of thing we had emitters for the water all around the boats then we had um a bow splash when the boat started to move the actual sounds would change they would be manipulated as we increased in speed to make it sound like more of a wish and they would actually sometimes crossfade to another sound of more water movement right side emitters for port and starboard but i had uh multiple sounds and what i did with the side emitters was i had something to get an increase in resolution because this game was on the vive you you're sitting there you can actually lean your body down to the water and when you did that and got close to the water it wasn't just that i i ramped up the volume or or the volume ramping up automatically through attenuation i actually cross fade it in so you've got your normal just white noise whooshing worship when you put your ear right down i would raise up like literally a

bubbling sort of leveling water sound because you could hear the water you know going along the side of the boat and the turbulence there and so again with vr in vr i can put my head really really close to something that i might not be able to do in a normal game and so having all of those different levels of intensity etc really really brought this this project to life so uh the narrative it's basically just the audience's experience there wasn't a lot of narrative for me to work with uh audio supports that narrative by surrounding the audience and the sonic elements there were there were people who were um giving feedback saying this is glorious i feel like i'm out in a boat and people talk about the idea of setting up a fan so they've got wind blowing into their face to enhance it but a lot of people really really loved that they felt like they were outside they felt like they're in the open air they really really loved responding to the wind and actually being able to tell which direction the wind was going in just by turning their head and listening dynamic nature to the wind and water

sounds are less about the mechanics of a racing boat and more about the interplay between the audience and their environment and that's the point it's not feedback of the boats going fast it's feedback of you are in this virtual world i'm trying to convince you that you're in a believable world by basically giving you elements that really really enhance that immersion by immersing the audience in a virtual world they can believe in all the other elements of the experience and become more convincing and this this is the point if you get the audio right it will can it will provide so much support for the other elements of being in a virtual world it doesn't matter if you've got the best graphics in the world if your audio is not up to scratch for vr it will undermine the entire audience experience all right outbreak origins true freedom of movement literally you can run around a warehouse very linear experience room by room they

literally move on from room to room it's a because in a warehouse they've got like 15 minutes to do it it is linear it is timed etc hugely immersive experience um because of this format of you're running around with a gun you've got everything on you've got freedom of movement it is actually very very a very immersive experience broad audience this is this is like going to the to the cinema or to any other sort of thing this is designed to be you know family friendly sure you're shooting zombies but it's supposed to be family friendly people from all walks of life poor people rich people people who are into games people who are not into games in fact one of their biggest audiences is you know um a lot of males from sort of young adult through to middle aged who have never played video games and they go out for a party or whatever and come along and try these things out so it's you need to be very very careful with how we approach this because it wasn't typical gamers shared experience as well people like they're playing as a team it's like i think four people the two groups of four they they were playing together but they

were moving through in sort of two different groups um unique environments as you move through each had its own reverb i'll get to that in a second but the transition points are really easy you're in you start off in the bunker and it's a concrete military bunker and to end that level you walk into the lift there's just a line in the sand across the thing you walk across that line i switch off the bunker reverb i switch on the lift reverb again point of difference doesn't matter if the reverb is 100 exactly right for that bunker or that elevator the average audience is just going to go hey the sound changed and in fact no sorry they're not the average person is just not even going to notice the sound change they're just going to go i walked into an elevator and that's the point if we do our job properly they shouldn't notice that sort of stuff and the zombies was a familiar narrative and so that was kind of good to work with because people's expectations were never really challenged too much so here's one of the examples of one of the points of impact so we've got

our trigger line so a is our elevator and we've got this sort of trigger line and when the door opens so so they can fire their guns in the elevator if they want to and it goes bang bang bang and you've got the hum of the engine in the elevator and then that's all got that rivered when the door opens there is an emitter just outside the door of the elevator that's basically whoosh wind really really loud and it just feels like the door opens there's wind but in the elevator it's still just they're that close space as soon as they step across the line once again one of those ambient array things kicks in but this time it's really really quite strong wind and they are up on top of like a 40-story building on top of the city and that transition point of being in this nice cozy little elevator to i'm now up on a on a thing and it's open etc was really quite impactful for people now one of the other things i did was um as i said there was reverbs applied for everything so you fire a gun in the bunker it goes bang you fire a bunker in the elevator and you get slap echo when you got up onto the roof uh i did something a little bit different i got all the gun shots that

had nice long tails and i literally went chop so i had about that much of the gunshot that was attached to the gun bang bang bang bang really really dry the tail was attached to an emitter that was about 20 meters in front of them and as soon as you pulled the trigger it would basically you go bang and the tail would start there and then randomly either travel in that direction or that direction and so what it was was bang and it was that echo off the mountains moving the echo was over there and it was moving off in a direction randomly like that the difference that made to standard gunshot sounds was massive especially with vr and stuff because you'd be you'd shoot and move like that and you'd still be hearing the echo trailing off as you were moving around and it was just really it really made a massive impact made all the guns etc sound really quite exciting when you're on the roof then what happens is after you've killed some zombies this helicopter comes along and you can you know 3d model helicopter comes in that's fine and there's another pressure plate you walk over the pressure plate into the

helicopter and it basically switches off that ambience array and gives you one of basically the interior of the helicopter now there's a couple of things that's going on here you could sort of wonder well you know you're just sort of jumping really quickly between these reverbs and doesn't it sound a little bit weird not in most cases no because people are concentrating about stepping into the elevator and with the helicopter even if you are at the top of a 40-story building there's about that much gap and the helicopter's there and you've got to step across that gap and i will tell you even in vr i had tested this time and time again and i know i'm in a warehouse but there'd be times where i'd just be going yeah i am in a warehouse aren't i yes good like literally the verdict the vertigo and sense of threat you get in vr your brain is filling in the gaps going what on earth are you doing you are on top of a building and you're about to step over a really dangerous gap and it really freaks you out if you haven't tried these warehouse experiences they're really worth it for that um and so again you're in the helicopter now you've got the helicopter interior but i'm playing the helicopter exterior in the doorway so you're getting a blend of both of them

so there's a whole bunch of things going on here with arrays and mono sounds there and staring stereo sounds there and 3d sounds there and all of these things layer together to give a really significant impact and we for this particular company um they approached me at one point and said so we whenever we build these we actually get regular focus groups you know when we're getting close to the end and just to sort of get some feedback and things we want to change and we've never had anybody mention the audio ever and we had a couple of people go wow this experience is like three times more impactful than the last ones or spooky or whatever the work the adjective they used you know they said there was so much more uh immersive than their previous ones and in a couple of cases they did actually mention the audio i'm unsure whether it's a good thing or not because the audio when it does its job properly shouldn't be noticeable but i think in this particular case it was a set group of focused people who had played the experiences before and they were like this one is better and they were asked why do you think so and they

sort of said well because of the audio so in that particular case weapons were all dry sounds environmental reverb added and tails uh as i said um subtle environmental details had a huge impact this is one of the other things i do when you started in the bunker there's computers and stuff everywhere etc etc but over in a corner there's like a pipe there and things over there and i thought well people are gonna be standing around this is when they're all coming loading into the level one by one and sort of testing their guns etc and i thought let's reward the players for investigating so the pipe that was over in the corner i basically put a gurgling sound in it but literally you only hear it when you go right up to it and you're within you know a couple of meters of it and so all of a sudden it's like you know and people are like hey this is cool and they'd call their friends over just like check out this as a gurgling pipe because it was it was like a just a mini little prize for taking the taking the effort of going over into the corner of the room um and that's another thing that we can do with the sound of like you know you've got this big area it's like i mean if we were modeling this room there's no real sounds going in this room but if we were going to to make it

interesting we could add all sorts of things to basically reward our audience so the general concept understand that your audience is likely curious give them things to investigate create things that add life to the environment i think audio should always reward for investigation you don't have to be really overt about it you don't have to put it absolutely everywhere but just in general i think there's things that we can do with audio where there doesn't have to be an animated object there doesn't have to be a physical thing there you know we could put a sound in that corner there that basically sounded like maybe there's somebody having a conversation on the other side of the wall you there's no assets needed other than audio we can add to narrative and stuff like that as well so as i said the player takes time to walk to the unremarkable corner discovers the gurgling pipe no not much time um all right i'll go through this very quickly the military simulator was quite different very specific audience and

purpose this was this was to train people to save lives um serious purpose um they they believed they knew what they needed and that was probably the biggest takeaway from this one so very specific audience and purpose limited scope was helpful but a very high level of expectation they they were like oh this is serious this is very very important and initially it was daunting um but they they as i said the client missed some opportunities for improvements and i'll give you an example of this uh again the array for the wind etc but i said to them at one point what about the enemies guns this is training for you guys to walk around and i've given you the correct i've recorded the the military weapons and i've given you the correct sounds and all this sort of stuff like i did with the tails etc that was all good but i said to them what about the the enemies weapons what are the enemy you need to have the sounds for the enemy's weapons and i'm like oh we don't care about those i'm like sorry what you don't i this is not a game they kept saying this is not a game this is not a game actually what they should have said is

the the the honest truth was and i don't mean to be nasty but honest truth was we really don't understand what we need and the game industry could teach us a whole bunch of stuff because from my point of view i've talked i've spent five years in the army i know people who have been in combat and from a survival point of view if you can't tell the difference between a sniper rifle that can kill you at two miles and a submachine gun that can only kill you from you know 20 feet you're not going to last very long and especially in this case we were adding uh reverb to all sorts of things so you hear the sound of an ak-47 ah that's an enemy's gun was it close where it's a threat or did it sound like it was a mile away where it's irrelevant did it sound like it was coming from a building or the street where do i need to look so i was astounded that they just like oh we don't care about sound and it was there was a level of arrogance there which you know i was the client of their client so their choice but i i think i actually wrote a um a blog post about this afterwards and put it on not

specifically naming them but putting this on of like i i underestimate the importance of audio at your peril if you are creating simulators the way we audio our sense of hearing is our critical danger sense so i think in this particular case in this argument is you will always have all three elements uh the waiting will change per the needs of the project so vr regatta was probably immerse then in form then impact because we need to inform where the wind was going for sailing purposes whereas outbreak origins was impact first immerse them in the thing inform last of all there's a zombie over there that's all you really need to know whereas the military simulator was informed immersed to make it a realistic environment but impacts not really it was it was a serious purpose project so i think in that regard um it was quite interesting that when i started doing this i realized oh wow this is the perfect trio of projects to to explore this sort of concept

um so it is important to meet the needs of a project but the project will often define what is possible and feasible you know if the client's saying that's our scope doesn't matter whether it would be awesome if i could do that just you know we've got to work within that we've got to understand that our job is to serve the clients um it can be tempting to add cool things because it's cool but does it really add value to that specific project some cases yes or no you you need to consider that vr requires a new language i sort of started with this to communicate with our audience evaluate the importance of the elements of immerse impact and inform to tailor you and tailor your audio to suit so work out if we we consider this our new language potentially then maybe which dialect are we using for the specific audience for that specific purpose um and so that's the last slide um we have a couple of minutes left and uh the um organizers sort of generally said we could probably go a couple of minutes

late with questions because we've got that bit of time so we'll have some questions if anybody has them but otherwise thank you very much everyone hello thanks very much for the talk um i was just curious on that point about uh adding value to the specific project a lot of the stuff you were describing like on the boat putting in the uh level of audio detail something i as a sound designer would absolutely adore doing was there any point in these projects or others that you've worked on where you had to kind of scale that back because you're doing too much cool audio stuff and the project was not um supporting that granular level of detail no i understand what you're saying no for the two creative ones i pretty much met what was needed i mean i probably did a little bit more than the client was expecting for the zombie one but they were delighted for it they're like this is great um

the military one was the one where they kind of went man we don't care so i just stopped like right at the beginning they pretty much said we don't care so i went all right well there's your gun sounds there's your ambience there's a couple of booms in the background and i just went they've literally said we don't care and so my client who was the one making the thing said yeah they don't want to don't waste time on it so it was lucky that they said it then rather than at the end after i've done a whole bunch of stuff and they're like i'll take it out so that was good at least but the the the the vr regatta one was really about the experience the people who made it were people who sailed boats and they said they wanted their their players their audience to really really really like they were sailing a boat and the overall feedback from that one was people just going we love how this sounds so in these i know exactly what you're saying and i think yes there is a risk of doing that perhaps i was lucky that in this case i went just far enough thank you no worries uh i have a question about using uh when you uh split out the ambiences uh

as separate emitters yes instead of just having like one bed yep uh are those stereo uh or are they like left mono splits and you're using like the left on the so you're talking the array thing yeah yeah really good question so um if i had the option i would potentially use a quad microphone array or an ambisonic microphone like if i was going out freshly to record it i would i would do that and then i've got my nice sort of things if i had stereo what i would do is go oh look i've got a 60 second stereo file of wind and i go chop 30 seconds bam bam bam bam um but it actually can even work with mono like i went through some more ones and one of the wind sounds that i had that i wanted to crossfade to him this is a really lovely wind sound but i only recorded it mono years ago but i went mono mono mono mono because of course you know again if it's two minutes long 30 seconds 30 seconds 30 seconds and it it's not we're not doing true stereo and

you know the nice panty way what we're doing is we're setting up emitters that are playing a sound within a 3d world that has all the attenuation etc so even mono and in fact over the years i've used mono for creating most of my environments because you chop it up and position it etc so you can pretty much use any format you want what i would say is grab some files throw them in and then you know add salt to taste sort of thing just play around with it until it gets to where you want it to be cool and just adding on to that since there are more questions do you ever use the waveform of the audio to move the sound around for example if a sound has a doppler effect or if for example like a wind sound might have on the left channel it swells and then on the right channel it swells and kind of you know alternates do you ever like split that in

in left right channel and then use the the sound to uh move the positioning of the sound so what you're saying is if i don't have a so if the wind is actually moving rather than just being a static rain or if the rain instead of just being rain it's like sort of rain so there are actually so there are kind of accents within if if there are accents within the the files that i'm using do i use those to determine what's going on like if there's two if you have two static emitters yeah and then you know pretending like the sound is coming from left to right instead of manually you know having that scripted you know as a moving sound what i would probably do is i would probably use those separately i would probably still set up my array with my really really subtle fairly flat ones and then what i might do is set up those ones you've just talked about with a surge i might have them as a

a nice clean fade in surge fade out you know so they're clean at each end and then might have them so that they trigger on specific points so like if i turn my head at a time you know trigger one of those somewhere because it just sort of again it's highlighting that point of difference whether it's actually realistic that me turning my head affects the wind and we know that it's not going to in that way it's just that perception of oh i did something and something happened um so there are kind of examples that i mean one of the best examples is um the film wasn't very good and i watched it many years ago there's a film um i think ben affleck was in it that was like the pearl harbor film where they you know the the japanese attack and then the americans you know two american pilots beat them beat them off and what was really interesting about that was um with an airplane you've got if an airplane's just sitting there and you're recording it the airplane just goes you know loudly or whatever the meow sound only happens when you're here and an airplane passes you and it's that doppler effect and yet when you watch that film it's showing scenes of the

camera like here's the airplane here's the cockpit so you can just sort of see a little bit of the aeroplane fuselage the cockpit and the guys sitting in it right and and they're showing you as though it's a fixed camera so the plane's flying like this the camera and the whole time you're just seeing the dude in the in the cockpit right and yet the sound half the time every time they showed one of those things you're getting a sound going and i'm like it doesn't match what's going on visually at all it really doesn't it's completely wrong because it should be a bypass left to right what we should be hearing here is just the constant but obviously hollywood looked at that went wow yeah but a constant brewer is not going to really work so we'll put something in to represent that the plane's there and the plane's moving so it is technically absolutely the wrong sound for that scene and yet who noticed that right so what i'm getting with this is that there's a whole bunch of smoke and mirrors we can do with sound and 99.9 of people i would argue that a whole bunch of sound designers who wouldn't saw that probably didn't even notice it you know we just go oh the plane's moving and that's fine you know our brain is great

at filling in gaps and so i i like the idea of what you're saying but i would use it just you know i turned my head and there's a swoosh sort of thing because it would just enhance that whole sense of i'm doing stuff and the world's interacting with me i mean it's a great idea and i i'd almost be tempted to go home and open that project i'm just throwing one of those in to see what the effect is to be do i need to make it more subtle yeah uh and sir oh final question on your car example uh if you have you're inside your car from the previous talk um how would you set up your emitters so that if i'm hearing the engine from inside that it's coming from the entire panel you know the wall of the car instead of like i might even put two emitters there so they're sort of you know one or make the emitters big wide so they fulfill that whole area um obviously filter it a lot so that you've got the inside the car sound sort of stuff but i would definitely position it there rather than on me here because it's there in the car yeah yeah awesome

all right this probably should be the last question but um great talk um i see from your buyer you've been doing work in ar as well i was wondering if you had any insights on blending sort of real world sounds with with with ar fantastic question um that's almost a talk on itself but i'm trying to give you some information no no you're absolutely right and this is one of the big problems with ar is that we need to be able to hear it but if i'm a star wars fan so let's just say we've got an ar r2d2 there there's a couple of things that goes on r2d2 needs to sound like he's there not here not here from mono and quite often a lot of the hrtfs this is a struggle i mean down on the expo floors look at people going oh we've got this audio solution and i'm like sure it sounds like it's swimming around but it never sounds like it's there three feet in front of me and so you've got that that aspect of really trying to nail where it feels like it is um really judicious use of reverb might

help that a little bit but technically if r2d2 goes beep i should get a direct signal to me here and then i should get a bounce signal off there and that's part of what tells my brain r2d2 is there and all the other reflections and stuff unless you're going to do lots of retracing and stuff you're not going to get that so again smoke and mirrors um you you just have to kind of pretend um if i had the ability to sort of do this one of the things i would probably try to do is um if you had the tech to do it is you know you've got your hololens your magic leap and it's got a built-in microphone i'd want that built-in microphone to basically go i'm going to listen to the environment that i'm in analyze that and then use that to convolve the sounds that's coming out of that at least to try and guess the sort of space that i'm in um but generally i would have it be mostly flat like no reaver etcetera etcetera and if i couldn't do that maybe

i'd just do a best guess again i'd get the cameras that are built into these devices to go hey what does it look like we're in we're in a big space cool apply big space reverb six pre-preset um it is super hard the other problem is that if r2d2 is in this room and you're all talking you know what i mean it should sound like well r2d2 sound is blending with yours and maybe if i'm looking at you and you're speaking i can focus on what you're saying if all of you are talking really loudly and i look down on r2d2 i'm now focusing on that and there are most of the sorts of audio apps and stuff have focal cone sorts of things where the thing that you are looking at will be slightly louder than everything else but again that takes a lot of tuning because it's like how wide what's the angle of that focal cone and then how what's the difference in volume between that and what's outside it and there's a whole bunch of other things so again it still comes down to smoke and mirrors and i think as much as the technology is critical i still i'm still placing smoke and mirrors as

being a better solution than the technology right now or at least sorry i'll let me revise that smoke and mirrors supporting the technology whilst the technology is developed and evolves further and maybe as the technology devolves further we can relinquish its reliance on the smoke and mirrors and dial those down a bit but then again i'll leave it on this point i mean any hollywood movie that we we look at is so devolved from a reality i mean the handguns sound like howitzers and you know something to say you know a dragon the size of my palm makes a noise the size of a 40-foot dinosaur so if if you're wanting to make if you're trying to convince me that something's there that's fine if you're trying to convince me that there's something that's heroic hollywood that's there that gives you a lot more cart blanche to have r2d2 all of a sudden and sound like a giant t-rex thank you no worries all right thank you very much everyone