Creating Dr. Grordbort's Invaders for Magic Leap One

[Music] welcome to creating dr grobart's invaders for magic leap one uh i'm james i've been doing a lot of weird stuff in my career but this one was definitely the weirdest i started a long time ago making pc games from four months on the lithtech engine uh which were interesting uh worked on do you remember what this is yeah there we go it's a few people remember the n-gage we did they did a big tactical rpg for the n-gage uh shortly before the platform went away um worked on playstation 2 era and gamecube and xbox stuff in montreal with some really lovely folks went down to new zealand the first time and did a couple more um xbox and playstation games down there and even did a game for the iphone 3gs which was new and shiny at the time when mobile games were a thing um that was just like what is this how does this work um shift a bunch of stuff in the xbox 360 ps3 era and then um was offered an interesting

opportunity to go back to new zealand i'd gone back to canada was living in toronto working with ubisoft up there we did splinter cell blacklist which was a pretty exciting project and was invited back to new zealand to work on something that was new and shiny and it didn't look like this at the time to be fair this was a few years before the form factor was really finalized so i went to new zealand to work with the crew for on a really exciting game for the magic leap one which shipped last august um so the magic leap just i'm sure everybody's kind of aware but they're a spatial computing company from florida we've got offices now everywhere from ziorek to tel aviv to austin to wellington new zealand um and we've just uh launched the first round of the hardware in august last year and uh it's been been really exciting to see what happens and we work out of what a workshop our crew is in wellington new zealand and workshop is

a physical award-winning physical effects uh and props company so if you've seen anything like all the lord of the rings films and the hobbit films all of the armor and the swords and everything came out of that district nine the guns from that and the mech suits um kong all kinds of films workshop makes really incredible stuff and so it was sort of a strange thing to have a physical props company making a video game uh so we work out of a new studio that we call what a game shop uh which is just down the road from the main workshop because we sort of outgrew our space there and we need to go somewhere and it's fairly typical as game studios go you know you've got desks and those sorts of things what isn't typical about the space is that we have one area that is a testing with two testing areas one of which has got movable walls and furniture a bunch of cubes um really nice to be attached to a props company they can make us all kinds of fun stuff and so we can reconfigure our spaces to try and experiment with the different

ways that people's homes and offices might be set up we have a more lockdowns kind of full-on testing space and then of course our you know areas for hardware storage and things like that but otherwise it's kind of what you'd expect from a game studio sorry so just to clarify a couple of things so spatial computing what do i mean by that um just so we're all on the same page um vr is something that it's a completely digital environment right it shuts out the real world and it takes you to a whole different place it's amazing for immersion into whole other worlds augmented reality sort of knows about it's geospatially aware usually it knows about the world and it can tell you some information about what's around you but it doesn't actually integrate with the world spatial computing and mixed reality really does it's digital content that interacts with your real world and that is where lots of the excitement came for me was how are we going to put games into your house into your office

into the spaces around you and interact with them in a meaningful way so the first game that we've made is dr grobort's invaders it's a fairly large scale experience it's a long form action game which is kind of unusual for the space i'll show you a quick clip from the trailer rather than the full trailer but you'll get a sense of it [Music] the trailer itself is actually a lot of fun but it's a little bit long for the presentation so recommend you check it out on youtube um so how did we get to this um how do we get to that experience um long and winding road it turns out uh we had to build a team as i say what a workshop didn't have a games team prior to this happening right this is this is a new thing for them they tend to make physical products not digital products and so

it actually worked out really well for us because we're making digital things that go in the real world so there's still that sense of this is a thing that i need to be able to interact with it needs to feel real so you take a bunch of people who are really amazing at making physical stuff that feels real that isn't should actually exist in the world like if you pick up a weta workshop raygun and you turn it over you'll notice that there's a serial number on one side there's details built into every aspect of everything that they do and it's that attention to detail that companies keep coming to them for again and again and again we had to build a game which as you all know is you know real easy right no i started to build a team we had to build a game um and we had to help magic leap build a platform so you know no pressure um and we stacked all these things together uh and it was a very interesting very exciting challenge which is why i signed on what was really important for us was that magic leap did recognize games as foundational to the platform right so

our colleagues at magic leap studios in florida made some really wonderful experiences project create is a really beautiful sandbox that lets you do all kinds of fun stuff i highly recommend letting some knights fight it out before you drop a t-rex on them and tanandi which is a collaboration with sigour ross that is a beautiful uh music um it's like living in a music video from an alien planet let's check it out and of course asana invaders and lots of other partners have come along for the ride and built lots of other cool things as well and really what's important to note about this is that there are going to be lots and lots of people coming to this platform right mixed reality and spatial computing as i'm firmly convinced is the future of this stuff and it's going to be used by everyone everywhere but at the moment the only people in the world who actually know how to build stuff for this this new medium are game developers we're the only people who do

real-time 3d interactive content um and it's it's going to happen very very quickly when um as all of these folks start to go we need to be in this space and using this stuff you're going to start to get approached by all kinds of interesting people so i encourage you to please welcome them with open arms because we've got so much to share with them so as i said we were helping them build this platform the purpose of the team really was to be the end point from hardware software operating system as everything came together we were there at the end to go cool we're going to use this and break it and then we will provide you with feedback so to do that we use the absolute latest of everything um him on the big bench there which was like literally you were locked into position so you could make sure that everything was visible eventually some stuff made it to a hard hat at one point then to what we call the wearable device three which jake is wearing down there with the big handles on the side which you'd often be kind of

balancing because you had to strap the thing on yourself and then crank down it sounded a little bit like a medieval torture device when you put it on somebody because the way that you cranked it it just went like look like it was um trying not to squeeze it too tight and joe's hanging out there with one of the early peqs so whether it was the sdk or anything else it was really quite interesting to constantly have this stuff also the conversations you get to have with customs as you come back hand carrying one of these prototypes through the airport um so we were in this constant feedback loop and we were we would fly over to florida several times a year we would spend time with all of those different teams and explain to them kind of what we were trying to do where we were breaking stuff what we were excited about and what opportunities we saw so that we could help them steer their efforts as well and one of the big things that we had to do there was to learn the language and the goals of these other teams right um i'm not an electrical engineer i am

not a uh uh interface designer um who's worried about how the very first experience of like opening the box works all of those other things so we had to learn their language as much as we could so we're able to communicate more effectively with them and one of the things that i think helped us a lot in in doing that and being involved in that process was showing up with something fun this is one of the early magic leap offices oh sorry and this was one of our earlier demos uh on on the wd3 and we set up the the whiteboard you can see over there and people just wrote their names down to come and get a demo with us that board was full for five days everyone came to play one of my favorite experiences i've ever had as a game developer was an electrical engineer who spent all of his days working on low-level circuit board stuff that i do

not understand at all who doesn't play games who just knew that he had one of the world's most interesting problems to go and solve and he was just curious what the heck we'd actually been up to so he came over to play he had a giant grin on his face the entire time he played he kept laughing when he took the device off he looked at me and said how did you do that i just said well with the thing you made dude and he was just ecstatic he was over the moon and every i saw him for the rest of the week and he was walking on air because he didn't have that opportunity to see the end result of all of his hard work to see it all come together and that's one of the beauties of you know what we do in games right is that we take all of this hard work by all these other people and then we distill it down and make something exciting so that was always um really wonderful to be able to go over and share those things so dr roberts invaders is an action game um and to make this game we wanted to make sure that everybody could play because we didn't really know for sure who the audience

was going to be right we were fairly confident that would be fellow developers some of whom would be game developers but many of whom would be application developers enterprise developers those sorts of things so we weren't you know and it's a brand new medium so we don't have a lot of conventions to go by um historically action games have been used on you know input devices like this lots of buttons i think the the dual analog stick control thing is amazing i mean i worked on splinter cell for a while if you want to see my 2014 talk all about how those systems are tuned down to a nutshell go for it but for a lot of people that's a very intimidating input system to approach so what we use instead for magic leap one uh is the control and gesture so control is a sixth off control it's tracked by the device no external lighthouses or anything required to see what's there it's got a touch pad on the top an analog trigger bumper button and the home button there it's basically like your starter pause

button and gesture and the device can recognize left and right hand and about eight or ten gestures at this point with more being added all the time and so for us we our mandate right from the beginning from greg broadmoor our creative director was uh make sure that it's a two button game so trigger for something the thumb button for something else trigger obviously well we'll get to that in a second that's pretty obvious in the hand so simplify this whole thing down and this was really once we wrapped up this is really exciting for me as well because my mom has never played my games my dad's never played my games they appreciate what i do and they like that i'm happy doing it but they don't play them

that was one of the coolest things that ever happened to me my mom got to play my game and had a good time and that was pretty special so i feel like we kind of checked a box on the everybody can play button or list whatever that is so the game itself starts with these ray guns ray guns are the heart of the dr gordwortz ip greg who created it a little over 10 years ago started off with just some paintings of ray guns and then they started making physical ray guns and then comic books and a lot of other things that have come along since um and so we wanted to that's that's really where we started with and so we we want to make sure you you could get it right all right also quite dangerous probably should sign a waiver and the nice thing about ray guns is that once you that's gimbal uh once he hands you that ray gun now it's in your hand attached to the control being tracked accurately and the thing with the ray gun is that we've we made the game at its heart it's

quite simple you are under attack and you have a solution everybody kind of understands what to do very very quickly the game has a lot more depth to it as you get further into it but at the very heart of it at that initial experience it's really like what is this thing in my hand what is that oh no okay and the the you know time to solve is pretty quick um one thing i'll mention is when people get their first virtual object in their hand or the first time they can actually experience a virtual object in the real world in mixed reality give them time to appreciate that thing as i mentioned you know we workshop is really about details excuse me um and so we had like a variety of skins for example for the rayguns and stuff each one of these things has got so much detail and texture to it and you'll see the very first time

someone gets a hold of a raygun they sit there and they just go like what is okay this is a thing there are bits on it moving there are dials there are bobs when you fire it it almost feels dangerous because the whole thing kind of vibrates and shakes the effects are great um so yeah give people a chance to look at stuff um also players will just like they'll put their head in stuff um so you can't control that right so make sure that you sort of consider your virtual content in a way that you don't get that because that's really disturbing what we came up with james brown one of our graphics programmers did this really wonderful technique of what we call an outline shader the device has a clip plane sitting at 37 centimeters so it's just out from your face and that's to avoid uh virgin's accommodation conflict right so we we can render content 37 centimeters all the way up to infinity multiple depth planes so stuff feels like it's in exactly the right place your eyes naturally adjust to it it's great

but we want to make sure that if you bring something in too close it doesn't hurt basically so one way around this for us for all of our content was we used this sort of outline shader to make sure that at that clip point you weren't getting things like the backs of people's eyeballs another thing we found was that putting ui on a raygun didn't actually work out great when people are playing they're really looking out here because and this is going to be different of course for every experience that you make but in our case when we were playing a shooting game where there's objects out at a distance that you need to target people are looking at that and they're doing this and they're going blind blind blam with the ray gun and so we thought well hey this object's going to be in field of view they'll see it kind of regularly we can we can tell them about stuff um so at one point we actually had like a health system that was like a three hits sort of system and we put these like heart bulbs on the back reality is everybody looks past that object and occasionally they use it to

aim a little bit but often of course if you try and do this for any length of time you get this and then this and it gets quite tiring very quickly so what we found was that people don't actually after they're done examining the object it's become a real thing to them and they don't you know object permanence kicks in they don't need to look at it all the time they're like cool i got this so yeah having stuff directly on the control if you had an up close experience i think where you're doing sort of something fine on a table or whatever that might work but for us we found it wasn't super helpful um so with all of these ray guns where we ended up we experimented a lot we had these dispensers for example that like sat in your room and you had to stick your hand in them to get ammo out of them and a bunch of other things we had shake to reload and what we found generally was the more complexity we put outside of the players experience the harder it was people to understand also a dispenser seems like a really cool idea hanging out

but remember you're in the real world so now i'm taking up real estate that could be occupied by a robot or some other gameplay object so it was like what if i don't have a good place to put that and all of the complications come out of that so where we ended up with it was actually putting the variety inside of different classes of reagan so we had heat ray guns that you know melted things and burned the armor off we had force ray guns which shattered things and pushed them around and then we had the grappling hook which allowed you to grab stuff and yank it into your hand and all of these rayguns sort of over time the player learns how to use them through through the different missions and you get this different kinds of challenges inside of the caverns i'll talk about in a minute but generally speaking what we found was keeping things in the player's hand keeping stuff you know centered on the player was generally better than trying to externalize it in an action experience but there are times which i'll get to in a second where we externalize stuff because we're in a much quieter environment

or more general environment but if we give you three of these things how do you use them how do you get a hold of them um we came up with what we call a tool bangle um which basically as soon as you touch the the top of the track pad there it pops out these three um one of each of the classes of a tool that you've got and then you just swipe your hand through it or just tilt even in that direction and it very quickly just swaps over to that people get into that rhythm quite quickly and kind of because it's always heat grapple force so people very quickly get into this rhythm where they're just like oh i need my force and they just flick the wrist and the next thing they know they've got it works really well so we're in this strange new world we're in mixed reality we're throwing robots at the player and everything else so how are we going to teach the player how to you know learn about this space and enjoy it you know quickly one of our primary ways was gimbal and

gimbal is a is a companion character who features very very heavily in throughout the game um he's a companion oh cool you must be my human you look so lifelike um and just having a friend there was really quite someone who just addresses you directly and is able to you know be nice about what's going on around you very helpful um he was uh he's also the kind of the core protagonist of the story and the story teller i'm nice to meet you an introduction is in order i am dr grodbort and i'm broadcasting to you from my headquarters at the dr grodbords factory so gimbal was able to bring all the human characters in the world through a holographic projection we did some fun stuff with these guys they will look at the player so they'll head track to you

there's a few moments where dr g will point at you for example and points directly at you and those are really kind of spooky moments where you're staying there and this character just looks at you and you're like okay hi all right we're here now a couple of little tricks you can get away with when you're using holographic projection uh if you keep walking around him to a point where he can't look at you anymore he'll just sort of flicker and reorient because that way you don't have to do a bunch of footsteps which was quite helpful um and uh it meant that we could kind of pick good spots for him no matter what and and uh the other valuable thing that we found was you can see from gimbal there his the projections coming off of the ring on his neck that means that when gimbal starts projecting you can sort of immediately tell where that character is going to be and follow to it and he's your guide so obviously you know you've got a limited field of view it's fairly large but it's still limited and because we're introducing objects to the world that were not there previously like how are you supposed to know that this thing's behind you

well audio cues help a lot but we need to get you there so one of the things that gimbal does is super valuable is he'll gesture to thing look at it move to it and you naturally just follow him there and he was also a teacher so this is um explaining like gimbal is throwing a wrench at you so that you understand what pain is and the way that we communicated pain to the player was the uh or that they were taking damage was all virtual content would go red for a second the world would actually slow down that's a really really valuable tool you've got access to when the whole world is moving at normal speed but suddenly game time changes people notice that very very quickly it's super super effective and then uh through he'll do an animation to explain to you how to put your hand up so you put your hand up see your shield and now you blocked it so and finally um also a really helpful uh character when you're in combat and

he hides out in his heidi hole and he calls out where the threats are coming from to help you orient yourself yesterday my colleague jordan did a great talk at the gdc ai summit and so if you want to know more about gimbal definitely dive into that one it'll be on the vault afterwards so we've got people moving around we've started to show them where stuff is we're guiding them through this space with a companion um but you're still standing and you're doing all this stuff so our core game loop actually incorporated quite a lot of breaks in it to give players a chance to rest and we loop between two kind of main spaces what we call home base which is where players have this map of the robot planet that allows them to select missions and they can buy ray guns and do other things like that it's also where we deliver story content so that projection of dr grobort for example those are always moments where the player is operating at their own pace and they can be kind of calm about

things and take a moment to investigate stuff when you go into missions now you're fighting robots you're collecting gubbins which are the currency in the game and you're destroying these big caverns and things like that and what we found was that generally speaking in the mission space we put players in there for about two to five minutes any more than that you're you're moving around and it can get quite tiring to do that um and so we and it's also just like stuff is in your face and trying to kill you you need a break so we come back into home base and you spend as much time as you're there as your uh as you want you there and what we tried to get to was a point where players felt that rhythm and they knew that they could kind of whenever they came back to home base they can take a break and they can step back and maybe they'll come back tomorrow or later in the day or something like that after they've played for half an hour or whatever it is because movement just can be tiring not all of us are like our animator jake um

sorry when i saw that i just had to throw it in it's just they were going to do a mocap test and i was like really okay um one of the things that we also again we're you know trying to in trying to make a game for everyone we did a lot of work uh around accessibility for our second update um this is a very difficult space this is the because as with everything in mixed reality this is new right um and so we were trying to get into just think about the way different ways that players could interact with the content um so for example shield is on your left hand um or on your offhand actually so you can play left or right handed doesn't matter so we added in our second update we had the ability to put the shield on the bumper so you play with one hand if for some reason the other one wasn't being detected well or uh you had an injury or for any other reason um we put an aim beam in there the generally speaking the visual effects guide the player shots really well but if you're having difficulty with depth that'll help

aim assist which allows you to interact with objects at a distance so the robot planet has moments where you're plugging things in and grabbing stuff but this lets you kind of step through those things even if you can't quite reach that invincibility was and the ability to skip mission parts it's like people are there to see what's in this experience it's a it's i keep saying this but it's new right so we want you to be able to see all of it um there's no reason not to so why not the other nice thing about invincibility is a great way to stop and look at the art also pause the game because the game content doesn't disappear so if you want to take a walk around a robot just pause the game and subtitles and we're pretty excited about this one i had my doubts but they assured me you were the one for the job unfortunately that wouldn't be the last of them sorry about that the video capture unfortunately cuts off the side but the way that our subtitles system works is that it pins itself to um the speaker if you look away from

them it drags to the corner of your field of view but it's still quite legible and it biases towards the side that the speaker is closest to and as you look back it comes back to them it scales depending on distance i you know i think the steps after this would be you know being able to do dynamically scale the font a few other things that are kind of new standards but i feel like these were good first steps and i'm excited to see what other people get up to and of course it's all quite experience dependent so how do we get all this stuff in the real world um one of the first big challenges is figuring out where to put stuff so at the beginning of the game um the first time you play uh if we don't recognize the space immediately because once once we've got a scan of the space and we know where it is we can just recall that and bring it back up but if we don't we have this scanning experience so a player walks around you can see this shader and it fills in in a really interesting way it feels like you want to try and

cover the whole space and this funky stuff we've got these buttons in the space that just float and hang around and as you walk around and press them they're there to encourage players to walk around and get a better scan and so you can kind of it also helps you understand ah this is the space that the game currently understands uh and so it hopefully encourages people to do a fairly robust scan once we've got that then we use a world search to figure out where to put stuff so this is us just moving the colliders there or the player and so we're just moving the player around and trying to figure out okay that's where the stand can go gimbal's trying to be okay i need to be about here to the player can't quite see it but there's stuff on the walls that's moving around of course the player would never see this but in debug we kind of go all right where is stuff going to fit so once we've figured out where we're going to put stuff we need people to be able to find it one of our primary tools was what we call glint system so you can see that those waves kind of pulling off to the side

at any given time there's really one thing that we want the player to be looking at and we just have a priority system which sorts that and so these glints sit in the corner of the field of view and just dry your eye to it it's quite subtle it's not as it's more subtle than you're getting out of this in-editor capture but we found that that was really valuable and it extended into a few other parts of the gameplay gimbal of course as i previously mentioned we'll always go hey things are over here wait until scene so the lookouts were really great basically oh sorry so that's an example of objects waiting until the player looks at them before they activate so we could have all these great animations of like portals opening and all this other cool stuff going on but if player never sees it that's a bummer um and it's and it doesn't you know it doesn't sell the the um doesn't sell the fantasy as much uh so

what we would do is leave for example in the case of the porthole being closed there we would leave a glint on it and the porthole had a closed state and you'd be kind of looking around going like oh is that oh what's that and then it would animate and open up and we did that with lots of different objects so that players were always sort of aware that this was a living mechanical thing happening to them or around them i should say sometimes up to them spatial audio is super super important um figuring out where things are i mean being able to hear stuff the non-including speakers on the magic leap one are quite good for basically positioning stuff around the player you would hear things coming out of walls and all around you all the time again not a lot to get in there from me but dave shumway is speaking about this on thursday and his examples from create are amazing and he get lots of details there so if you're into the audio side strongly recommend that again stuff happening outside of your field of view the porthole sparks were a really valuable thing so

as portholes were opening and being basically we were like cutting them into the wall and doing that we would cast these sparks off we'd always try to make sure the sparks would cross the player's field of view and that was really effective because of course they had directionality and so it would pull your attention towards them and you'd see them happen and lastly this one's really subtle but it was a really neat thing that simon our vfx guy came up with was ambient moats of stuff in the air so we just kind of filled the space with this little gentle bit of dust and that dust was always being sucked into whatever porthole was open so there's always this sort of like subtle background thing drawing your attention towards where stuff was going on um it's a little thing but it was one of those you know i've gone through like six things right it's like everything on top of everything you just try and layer this stuff in correctly so how does all this stuff actually work now these are all like these weird contraptions right you've never dealt with this stuff before

we've got the robot planet we've got mish cars we've got store all of this stuff these are fantastical objects you've never interacted with them before so one thing we found pretty quickly was making stuff physical was absolutely key every time we tried to abstract something away i we found that people got lost so making it physical making it something where you're just you're really doing this with your hands and your body and you're moving to it was was really important so in this example the player is plugged in to get a mission their mission card prints out and then they put it in the thing so they put it in the briefing stand and then now they can go and set up their loadout of what ray gun they want to use each step was discrete and you could go forwards and backwards in the chain depending on what you were doing and that was really helpful for us because we knew where the player was at at any given point in this set of interactions so we could make sure that you were guided through nice and cleanly

and not confused by um you know basically anytime that you were doing two things at once people kind of lose track of it eventually we'll probably get to the point where you know operating in virtual spaces like this or sorry with virtual content in the real world like this is you know kind of normal and will have some set of you know regular standards and norms that like wasd kind of stuff that people are used to but in the meantime you know play test the crap out of it and you'll be surprised at just how quickly you can overload somebody's awareness of things and you know make it fun oh no there we go um the briefing cards and a lot of other bits and pieces that we had you can just grab and throw around and just like people would spend like minutes just chucking this card in the air and then catching it and chucking it and catching it and people will make a game out of anything right and we just reprint you another card why not and making it smooshable was really important

so smoosh is something that tom hall one of our programmers came up with as a way to avoid object interpenetration because nothing ruins immersion quite like going oh this virtual object just goes right through this visual virtual object and it has no sense of weight or presence that's me using my hand uh the render on the capture is not great but you get the sense of that so each one of these physical objects um or virtual physical objects uh including things like the robot planet what not as you touch them and push them they smoosh they bounce around they've all got physicality and so nothing ever clips through anything else and that was uh he just he came up with that one day showed it to us and everybody kind of went everything has to do this like everything which made our lives slightly more complicated as you can imagine but it worked out for the best and lastly making this thing really

physical um the grappling gun was came out of one of our feature jams uh so we would take some time off from kind of core development uh every few months and even sometimes it was we were doing it once a month to try different things out and the grappling gun came out of one of these jams and again it was one of those things that was like we need this this is really cool it lets you reach out grab something at a distance pull it to you and examine it and if you you know rip off a robot's leg and then you've got this object in your hand all of a sudden you're looking at it and then throw it back at them it's a good time there's just lots of bits and pieces you can interact with what gets even weirder is that you reach into the robot planet through the portals that are in the wall and grab a virtual object from beyond the wall and then pull that back to you and that is quite an experience the first time you do it that also led us to some puzzle mechanics that you'll encounter later on in the game um which are

basically these robot planet security locks so again this is another way to sort of control the pacing of the environment or sorry control the pacing of gameplay you're in there you know later missions and things are kind of intense and there's lots of robots we want to like slow stuff down a little bit so we get into combat we do some stuff and then the robots like lock you out for a second now you pull this thing out of the wall and it's got all these boxes and bits and bobs it's almost like a busy board and you're bopping and changing and moving stuff around and they're just really fun to interact with but again it's at the player's own pace so it slows things down a little bit lets you take that moment to kind of enjoy and appreciate so how do we get these robot invaders into your living room um well this is what we call human scale [Music] combat so you see we've got these these robots that are of different sizes they

navigate the environment a little bit differently um and we're just putting them you know in your space with you and the different robots um we started with our pro bot um this is like a small flyer like a really nasty wasp is the first thing you encounter flying stuff is great because it can clear your couch your table your whatever it can just get around to the player very quickly super helpful it was also actually like the very first thing we put into mixed reality uh from workshop that's sam miller one of the original magic leap engineers demonstrating they were just they were trying to take a picture and go like is this thing really here which is a lot of fun um but yeah i can't recommend strongly enough like if you need something that is quick and can get around look at flying stuff and these use the same flight graph as gimbal so we got a bit of reuse on that space uh the simpletron are these tall gangly kind of like creepy dudes um they're about average kind of eye height for most players

um and we wanted them to be like a melee enemy so they just originally their attacks like they'd walk up and they'd take a swipe at you it turns out that that was like getting really close and really kind of in your face so we gave them a ranged uh like a arranged punch where they wind up and throw this punch out from quite a they can do it from quite a distance um and then eventually when uh as we were getting closer to kind of the final form factor of the device and we were able to move around more we went back to giving them some melee attacks as well that were um like this spinning attack for example where he just like sticks his arms out and helicopters at you and you can hear it coming and it sounds like a million razor blades it's really it's a bit freaky fun fact though everything is physical including that attack so if you sit down it will miss you it goes right over your head [Music] and then we had our thug bot and these guys were waist high they're just like permanently angry little dudes they've got a jet pack so they can clear

obstacles in your room and in the caverns then they had a couple different ranged attack types and you see all these robots are wearing different types of armor the armor for us was really valuable because it let us challenge the player's ray gun choice in different ways so like the rubberized one over there melts when you hit it with heat but will reflect force and just the grapple bounces off of it the um that shiny chrome armor there can be obliterated with a force weapon but again the grappling hook will just skid off of it and it reflects the heat ray guns and these are the fully armored versions but when we introduce them to the player we would give them like just the torso armor or just the leg armor and that let us challenge the player's precision and also start teaching them where you know where they were going to be vulnerable to certain types and lastly i'd say it's really important to keep this stuff kind of silly or at least it was for us because again

we're attacking you with these characters this is really it's in your space it's quite personal so we made sure that everything they they've all got really goofy voice lines they've got these fun little animations that they're they're idols um the uh rockets that get fired from the thugma bots um are these like super angry little things that make goofy noises as they fly by your head also really good for specialized audio and we just added kind of humor wherever he could if you use the grappling hook on the walls of your room like just against the wall there you'll find that you can rip things out of the wall that you wouldn't expect there's all kinds of stuff hidden behind the you know the boards of your house [Music] um so the robots all shared a nav mesh that was made up of grid positions that were constantly being weighed to find the best spot taking into account things like player position particularly line of sight based on whatever attacks they had available the position of the other robots the height of surrounding geometry and

the available movement states from one position to the next so do i need to be able to jetpack over there can i walk there in a straight line so this was a really important and i think it's one of those challenges that you'll find feels like a traditional game challenge until you go i have no control over the world all my level design stuff goes out the window because i don't know where your couch is so being quite dynamic about this is really important and the attack tells so that glint system that we use to kind of indicate where stuff is at for the player um was uh was really valuable and we ended up with that see how um you see how there was those glints there where they were yellow to indicate okay that's the position of a thing they would go red uh when the robot was

about to attack and that's how you knew where the where the threat was coming from if you didn't have to be looking at it and so we had to manage the intensity of all this we built a thing we call the action timeline and that's set up in such a way that we can stack and do base uh sorry we can decide who's going to attack the player and when based on a variety of criteria like their distance to player did they last attack how close to my field of view are they like we want to bias attackers to the ones that you're looking at versus the ones outside create kind of that kung fu fight circle feeling um so the the action timeline basically everything has a buffer of exclusivity and we can adjust adjust that so it says no other attack action can take place during this time um we activate the tell which is that red glow at a certain point there's a wind-up time on everything so how long does it take before the robot's actually able to attack so in the thugbot's case you could see him like standing there embracing himself to take a shot and then the attack itself and so

by dynamically scaling this we could really quite dramatically change the difficulty level of the game and so as you get later in the game the exclusive buffers come down and down and down so that robots are attacking more frequently and that was actually that one variable became the one that we kind of focused on the most um so where are these robots coming from it's the robot planet so we had these caverns [Music] so to sell the idea that this robot planet is this big space where these robots are being manufactured and invading the earth from we had a few of these really big caverns that would open up and you can see if you walk up towards them or you stand to left or right and just see all of this detail and depth inside of them and we also use them as sort of a form of environmental storytelling so they start off in the um in the sm in the furnace where they're like melting down

metal for building something and then in the smelter you can see them manufacturing the parts for whatever that thing is and then in assembly you can see them putting it together before we get to well the bit at the end which i won't spoil but um and these caverns uh all of them ended up with these light what we call um gates which were kind of light puzzle elements like the the heat-based one over there this little robot who just pulls a lever the idea being that you're sabotaging the robot planet each one of these responded to different uh attack types like the the heat the force the grapple and some of them were compound and had two um one thing i'll note about that though is that we were you know sure we're dynamically placing stuff in the environment we're trying to get it all right all the time and we've gotten our stuff out beyond the player's wall but you've got to make sure they can see it because if that cavern happened to get placed up against the side of the wall and the thing that they needed to shoot the gate they needed to shoot was like over there somewhere you can't walk through your wall right so you

can't even physically get yourself to a position where you could see it um so we had to we developed a like a test harness basically in unity that allowed us to kind of look through and figure out okay we assumed that players can stand we had to make a few assumptions and one of them was players can stand about this far away and see roughly this much and the caverns themselves again to adapt to the player's space could also lift up a sill so they started off and they're about yay high and if there was a couch or something and that was just where we had a piece of clear wall we would lift a sill up and then the robots could actually jump over out of it and again you had to make sure players could still see what was in there so how do you get this consistency in diverse environments you know we've got the scan we've got the the world search to make sure stuff is there we did end up with a small minimum spec during the scanning phase ideally players just look around and we get what we need very quickly but if necessary if they hit the pause button we'll show them kind of what the minimum spec is

and so for us that was like uh two 50 by 50 centimeter spaces 150 by 50 centimeter floor space and an 80 by 90 on the wall if we needed that once the scan satisfied those requirements then we knew we could play the game in that space so basically you can't control the level right the level's not there for you anymore that's the player's room you can encourage people to move their furniture around probably i'm sure that there will be experiences that do that but we wanted to make it work so you really need to focus on what a systemic experience is how are you going to govern the play the game's really got to fit the player's space so we focused on mission design right so that was which robots get introduced when what type of armor shows up when uh which combinations of armor the rate of spawn the rate of attack all of those kinds of things and so we were able to pace out the experience so that everyone had roughly the same order of

experience to happen to them the roughly and roughly the same sort of intensity of experience and so i think it's really important to kind of you can create and we did this a few times over the over the course of development where we create a bit of a like a target ideal of like oh the best experience would kind of be this what would we do what variables would we need access to to try and get us to that point and then make that true so we can kind of paint that experience everywhere so we had to plan to be wrong um this was the thing like we've got all these robots and stuff but as we were doing this we were you know we had quite a bit of hardware uh change over time this was the wd3 and you can see here um as richard's playing he's really you know he's standing in one place because he's got this tether to to a pc and you can't can't move around so much and so we knew for a fact that we were going to be untethered our

sixth off would would work with you know attached to it people are going to walk everywhere this was certain this would be true we were helping you know the magic leap team get there and it still blew our minds how wrong we were about all the assumptions that we had around player movement and everything else where we got to in the end you can see how different this is with what jimmy baird one of our lead designers calls a game about movement where really we're trying to get people to move around in this space and they turned into a much more dynamic experience so yeah when you're developing on early stage stuff you can prepare to be wrong you know you're going to be wrong and it still shocks you a little bit about how wrong you are and i will say that you know ambiguity is difficult this was a long road for us um to you know to help develop this help develop the platform and develop the game um we had to really you know conserve our energy um throughout development we were in

research and development kind of phase for quite a long time and we could have been sprinting at every milestone and throwing ourselves up against this thing but we knew that at some point you know the time for that would eventually come but in the meantime we needed to to just be experimenting and be cognizant of it um it was amazing how fast the magic leap engineering groups moved right they were moving mountains to make a brand new platform from scratch like i can't i still can't quite comprehend how they pulled all that stuff off in the time they did um but for us it meant that we we sort of we try to build something to share with them to help them guide their process to get them excited and engaged with with what we were making um but when we got it and but then we still had gas in the tank once it was like okay we've got the final form factor away we go the good news is um you know it's probably not going to be this kind of hard again we now have a platform we

have the operating system we have the sdk and it's getting better all the time just in december we did a game jam and in three days we built nine prototypes and it used to take us three days to get something to work so it feels really good it feels like we're gonna we're in a very exciting space with this so we shipped it we launched dr robert's invaders at the leap convention in october in los angeles we built four of these demo bays you can see they look quite big and quite open compared to probably what your living room would be and that's partly you know it gives the robots lots of space to navigate and everything else but it's also because you can play in a much tighter space that you're familiar with when you're in an unfamiliar space like it's a health and safety hazard because you'll be running around and i i saw some great stuff i saw somebody actually like dive behind that thing and shoot from underneath a coffee table which i thought was great um so yeah bear in mind that if you're demoing things

uh any space that you're familiar with you've got like a built-in kind of proprioception and sense of that space that other people won't have so we try to make it safe for everybody of course because it's what a workshop we couldn't you know just do it normally we had to have lots of folks there you know our scientists were great they were welcoming people into the experience getting them to sign their lives away on a blood spattered piece of paper and then handing them off to us for demos we had lord cox wayne himself there and the lady victoria it was a good time and people just really enjoyed it it was it was really gratifying for us to spend having spent all that time you know quietly uh hiding out in new zealand making this funky thing for people to just go and have a real good time with it um the press reception was really positive which was gratifying and i think lots of folks have said that it's been helpful for them to see what a full experience can be in this medium so just to wrap up

if you have the opportunity to help develop on a platform um communication is key i mean that's true always and forever right of anything we do but it really is be prepared to live with friction this stuff was really hard for a long time and you just kind of have to get zen about it our technical director mike i swear is some sort of buddha um you've got to figure out what's under your direct control and maximize for it so we knew that we could control for example like we're working in unity we're on we're in editor on pc fine no problem we can build lots of stuff there figure out how we take that to this early stage hardware as quickly as possible um but then how do we you know make as much as we can and then push it over make as much we can push it over to spin up and lastly of course bring the fun to your partners if you're a game developer working with uh non-game teams you can you can make their month by showing up with something great so spatial computing is super ridiculously exciting i'm very excited about it it's why i moved back to new

zealand um and i hope you all have a crack at it everything is new again including our audience right the the one of the things as i said at the beginning we're really like people can just get into this you put it on and you're like oh i've got a thing oh i've got a ray gun oh things are happening um and we've seen this with all of our experiences tanandi you just reach out and touch stuff and it's uh it's quite different and so you're gonna you know meet all kinds of people who maybe never played games before certainly never played an immersive game before like this um and so you know be prepared for that um get physical you know we found that we could prototype a lot of stuff by uh you know just basically acting it out you know this is like where does gimbal need to be in relation to the robot planet as the player goes through different steps um you know so there's greg jimmy and jordan acting out like what would happen as you're working as you as the player go through this stuff um you've got almost no control over the player's space

and that's a feature not a bug build to it you know it's it's going to be hard it's going to be tricky but take advantage of that take advantage of really thinking through systemic design and focus on player comfort you know both the physical and the mental you know take like even light physical activity can be tiring over time um you know you think about like the minority report interface where he's moving stuff around there's an interview and tom cruise says one of the hardest things he's ever done in film because he had to keep his arms out there for shot after shot after shot you know so think about things like that um make sure that you're not building something in in a vacuum where you're like cool i've built this on my computer always always always push to device stand up and test move around play it in the environment that you expect it to be played in and defend the planet from the robot invaders thanks very much

so um from wednesday through friday we'll be at the unity booth we're showing the dr gorbort's invaders single player demo and we also have a brand new experience called gore battle which is an early look at our first multiplayer experience it's a lot of fun and it's yeah i don't want to spoil anything but there's also a tea party um so do recommend checking that out sign ups will be open i think from first thing in the morning at the booth also as i say we have offices from zurich to tel aviv to austin to seattle to new zealand and we are always hiring and david martin one of our senior recruiters is here if you want to talk to him about all that fun stuff and otherwise i think we've got about three or four minutes for questions uh great job um i played the game myself actually my mom had the same experience uh as your mom didn't she actually almost knocked down a wall painting

trying to like the number of people we've had almost put their heads through walls trying to look at portals it's just like it was it was really amazing for me to see um and um that kind of brings me to to you know you mentioned the physicality of this and and how you know players get a little bit tired and so forth and i'm wondering um given that you've you know spent years on this um do you have a sense of what you think the breakdown might be going forward in mixed reality between uh these kind of more physical experiences do you think that's going to be what really drives the popularity of these platforms or will it be kind of a little bit more kind of casual uh almost more like kind of how traditional games are maybe something more like the you know the angry birds experience what what's your take on what do you think people are actually going to be doing and do you have any statistics on what people have been doing in the game out in the wild like in terms of play time um you know and so forth completion rates and things like that uh so i'm trying to research question but um you know are we expecting to see casual experiences more physical experiences more seated experiences um i think the answer is we just don't

know uh i think we'll see probably a lot of stuff right like it's just you know just like on your phone or your console you've got everything from a match 3 to god of war and i think people are you know people are going to find and want different things um depending on on their mood so we'll see how that goes um uh second part was about the oh stats yeah so we have some uh yeah we do have some analytics and basically we were looking for yeah how long are people playing for you know average session length things like that um off the top of my head i think our average session length is somewhere between 10 and 40 minutes which is quite a range but we see you know you obviously you see outliers of people who play for a couple hours the battery will go for almost three hours playing so um yeah it's quite variable i think um but yeah it's it's also like it's worth noting i think that because we're so early in this in this stage of the platform like people are people are engaging with it both as players i'm here i want the experience

and as developers going how does this thing work and so like they're just jumping in going like okay buttons are kind of like this that's cool that's a reference for me in the get out so i think our actual like the reality of our stats is probably not it's not indicative of of players per se it's indicative of this wide range of people who have got access to the device cool thank you you mentioned how unusual it was for way to make a video game i'm just wondering how that decision came about like who was like we need a game studio let's go talk to the guys that did lord of the rings swords oh um so the the magic leap when a workshop relationship predates magically so roni abavitz who's the ceo of magic leap um first got in touch with uh wedda to um work on a sci-fi world that he wanted to create because that's what we do and so they were already had that relationship going and as magic leap became a thing he was like we need games we need content hey guys i love this the gold boards ray guns and stuff seems like a good fit in a way they went

awesome thanks all right cool okay hop this way and then this way sorry ladies first okay thank you um this was great i'm wondering what you see different considerations are like net new um things developers should keep in mind for multiplayer when it comes to spatial computing uh so things keep in mind from multiplayer and space computing um space obviously is like the i mean the spatial part is the huge one we've had four or six people running around you know playing with each other and um the inte there's a reason there's a tea party to kind of start the experience to keep things chill before we start moving people around a bunch um i think yeah i think space is is honestly the single biggest consideration and then like what's um interactions with other people are amazing in mixed reality like it's just there's something really special about being able to do something unreal and a little bit weird just together in the real world so yeah i would strongly encourage like anything that's cooperative

or um like direct one-to-one interaction is is yeah thank you cheers um so i played the game and i was really impressed with the transparency it was curious how you guys achieve that because most of the games have all the assets they still look really transparent in the real world uh you have any neat tricks how to fix that um yeah so ways to make the the virtual content appear more grounded um i think it primarily is just our lighting model um that we we tweet that a long time um and i'm unfortunately not the best person to speak to the specifics of that um you probably if you swing by unity booth uh tom hall's there he can probably tell you a little bit more about it cool we got here um all right go for it two minutes grip one uh have a uh two two layer display uh that that contains uh uh two

different focal distance and uh uh what uh is uh is the usage of uh in your game title uh uh near layer is dedicated for dragons or uh if when the robot comes close to the user uh to use a so when are we uh you know how are we choosing excuse me how are we choosing which depth plane to be using um it's driven primarily by um by uh oh excuse me eye tracking so based on what the player is looking at so if you're looking at a robot at a distance we're rendering far plane if that robot comes up really close and you're looking at it right here we're it's it's like a fundamental part of the platform is really to make sure that the rendering is as accommodating to the eye as possible so it's eye track that way so we just we drive it based on the standards that the sdk sets uh is it smooth on switching between our

layouts it's quite i mean it's it's a fraction of a second and people are blinking enough that usually you don't notice it so much but um sometimes you'll see it switch a little bit but that's yeah um it's not yeah don't notice it really okay thank you cheers all right uh real quick um did you ever think instead of like having people like skip certain sections if they're having difficulties uh did you ever think of maybe setting up a sort of scaling difficulty where like the exclusion buffer that you're talking about was kind of like maybe increased a bit or you know decrease the amount of robots coming through or yeah so dynamic difficulty scaling we experimented with it um unfortunately it was it was a really difficult one to get the right kind of information out of it i definitely think that's a that's a place to be looking um especially to accommodate as many players as possible um but yeah unfortunately for us it wasn't something that we never kind of got it to a place where it felt right so we kind of were like

we'll step back and this skip was added bit later to accommodate for anybody who had real challenges with it thank you okay last one cool i really enjoyed the experience um but uh we i'm using it actually as a reference point for something we're working on right now um the thing i was most interested in were the portals right that the robots come out of i was kind of curious can you speak to how you guys accomplished that because you know trying to include the spatial mesh and right i had like the occlusion mesh this is it was it was i i got a solution but it took some work yeah um so how do we get the portals on the wall um it's basically we're uh we're rendering the entire cavern space on the other side of your wall and just sticking it on there it's not like a traditional not traditional it's not like a a proper portal system it's literally like we put the geometry on there and then we put occluders around it so that you can only see through from certain angles um basically one of the big challenges we had was um

making sure that like these the robots are walking with ik and stuff right so having them having the physics transfer nice and cleanly between the real world and the virtual world and having shots travel through that and everything else we tried a few times through different portal systems and all of them at the end of the day were a little bit too expensive and just created so many edge cases that we stuck with that that format um i think in the future it would be really nice to have a like a really sweet portal system so we'll probably build something like that at some point i'm sure but um yeah at the moment it's just like literally there's the level stick it on the outside of the wall try not to walk through the wall thank you yeah cheers thanks very much everyone