Job Simulator to Vacation Simulator: Advanced Interactions for Room-Scale VR

[Music] hello everybody our talk is lessons learned from job to vacation advanced interactions for room-scale VR I'm Devin rhymer CEO at alchemy labs and I'm and reiki CTL and cable Wrangler at alchemy labs and we are alchemy labs we make absurd and highly polished VR games we're an Austin texas-based developer and a few of our games you may know our job simulator Rick and Morty virtual reality and the upcoming vacation simulator and here's a one-minute clip from our trailer to help kind of give you some context for the rest of this talk sports season wondrous

vacation Beach has it all snap some shots of your evolutionary ancestors and cool that overclocked neural processor dry up and spend those seashells by the seashore marvel at the rustic grandeur an unforgettable vacation forests become ones and zeros of simulated nature as you get back to your digital mute missing ox spark in your surge protector there's plenty of fish in our stream scaling away a vacation mounting for a transformative winter experience rotating office temperature chilling your thermal sensors let it go and enjoy all this winter wonderland has to offer don't hold yourself back anymore you'll enjoy all of these beautiful so when it comes to interactions in VR job simulator was kind of the thing that kicked things off for us we had done some VR before that but this was kind of the first point that we ended up building VR that you interacted with the

world with your hands so we started work on this with a very early prototype of what would end up being the vibe and one of the first things that we did was we built this demo where there was four boxes and we put the on the floor in your virtual floor floor and I found myself just sitting there just stacking blocks and at that point realized that this was a very interesting and fundamentally fun way to interact with this world so that was kind of the kickoff point for what we wanted to build so we kind of used this direct interaction into the virtual world as our guide so we knew be one of these deep world interactions we wanted to make games that were approachable by everybody and then the other thing was to make no one's sick so in job simulator we kind of established a few early ideas and kind of basic design principles that we use to this day so for example taking advantage of your physical space to physically move around our physics based approach to interacting with objects tomatoe presence which you see in this video

right now solid object believability floors which are very important but we're controversial at the time and many many more we've also done a number of talks about a lot of these basic concepts and you can see that go to our speaking page and see many of those talks but today we're going to get a little bit more in depth so what we're going to do is we want to dig into four specific case studies on things we did in vacation simulator and more specifically each one of these case studies is going to cover a different set of topics that we've been heavily iterating on since the development of job simulator the first one we're going to talk about today is locomotion so in job simulator how we locomote it around the world was our own body moving in the physical real world space every single job was designed to be exactly one play area size and we moved between different jobs or different spaces by eating this exit brito fading to black and loading in an entire new scene we would use room

resizing to properly size the world to your play space and so we would remake the layout of each play space for various room sizes and that included redoing the models writing custom code for it and and it would also include per platform so PlayStation VR had its own play space and it worked great what we really want to accomplish there was at our was the same experience no matter what she played it on so that brings us to the world of vacation simulator vacations kind of had this expectation that they're about exploring and doing a variety of different activities so for example having an entire beach and you could only exist in one small zone would feel very very problematic so we knew we wanted to build a much bigger world for our next game and so unlike jobs simulator this one destination per zone concept just wouldn't work we needed more space so we knew that we're gonna have to move around so when it comes to VR locomotion we've been pretty

outspoken on this topic and the main reason is VR motion sickness so regardless of what you read on the Internet a significant portion of the population will never overcome simulator sickness when it comes to vection from artificial locomotion Nadja not only reduces the audience of your game but can turn people completely off from VR there's still so many people that have limited or no new experience into VR yet and so we want to make sure the first kind of taste into that world they understand that VR is for them so our number one rule at El Camino make people sick so what kind of options does that leave us and so the first one is like is teleportation and in our opinion teleportation has got a bit of a bad rap in VR we think the main the main reason for this is that up till now for the most part simple teleportation methods have been used and also they've been slapped very commonly into existing content that wasn't designed for it in the first place and then to compound

this there's so many lessons we haven't yet learned and how to do good teleportation it right now it feels very much like the early days of third-person cameras in games and also early development at FPS is on consoles things that we take for granted today but at the time when they were starting to roll out some of them were kind of controversial and weren't quite sure how they'd be solved so this was one of these challenges that we wanted to take on and alchemy so when you start talking about teleportation when the first things you're going to look at is granular teleportation and it's teleport anywhere you point a specific spot that you're going to teleport to in a very precise location up here and you release and you go there it's very easy to drop in and many VR frameworks include this by default and it works with existing content but there's a lot of challenges with it specifically it's very difficult for new users to select the correct spot it's difficult to understand where your play space bounds actually are in comparison to where you are moving in the world so

in this image here you can see that the player is that mug is a beyond the edge of their play space bounds but because of the way they've teleported they can't actually reach that without taking a step back and teleporting again and that's very unintuitive to do that super gonna do to try to explain to a person the object you want to grab you actually have to walk in the opposite direction of it - then get towards it and and granular teleportation discourages physical movement once you get the ability to teleport wherever you want you stop walking around your play space and then standing still is also more physically exhausting than moving around in your space and then lastly we have this concept called teleportation fatigue and that is when you can see a spot you want to go to but you have to do a number of kind of small jumps it gets very fatiguing to do that and granular or teleportation in particular suffers from this so we're looking for a new way to move across larger distances and our games are about physical physical based interaction which makes

Grail or teleportation not a great fit so we started looking to see if there is a way where we can merge the the benefits of teleportation with kind of our play area style design and we landed on zone based teleportation so you look at a zone and hit a button and it's really intuitive for our players because they're now not thinking about exactly where they need to go but more kind of what zone do they need to go to it also encourages active movement within the zones once your since you're only moving between spaces you're still walking around and engaging with those spaces and it allows us as developers to design discreet zones instead of trying to design an entire world which is great so this means vacation sim is much larger than I'm job simulator much larger so this is just a small chunk of the beach or and you can see a bunch of different zones so each one those red boxes is kind of a

zone that we have that the players can teleport to as compared to the jobs which were one single zone so when Jobs came and we had five total zones and in this only the certain part of the beach here that we're showing there's seven so when it comes to zone based teleportations there tell teleportation there's a lot of wins versus granular teleportation so as Andrew mentioned we find it a lot easier to get people to understand and to use there's less teleportation fatigue of spending a lot of time trying to get to the exact point you're trying to go as developers we also know that that area is clear of obstruction in the physical world because we know the bounds of where you are and so we know that if you're inside that area that every object and every device and every interaction is the player can actually perform that and then also this concept of if you can see an object you can reach and grab that object because we have defined bounds of your world we can make sure those objects are always within reach that

said tone based teleportation has a lot of challenges that comes along with it a really big one is that you need to design your game with zone based teleportation in mind this is not something you go back and retrofit something with you have to think about it right from the start so to kind of go through a few of those challenges a big one is 180 tracking and so there is a lot of VR setups that have a limitation of 180 degree tracking when it comes to your controllers most have 360 head tracking but when it comes to hands you're kind of limited into the space so you have to think about how that's going to impact things traditionally with something like granular based teleportation the method that's used to solve this is manual rotations or snap turns the problem with this is that there's constantly needs to be at like a thread running in the back of your mind as a player of where is my real world forward because that's the point right I need to make sure that I'm continuously orienting myself versus where I'm in the

virtual world versus where I am in the real world and when you have to do that you're automatically breaking presents so one of the first things we started thinking about was okay let's say there's a zone to your left that you want to go to but some of the primary interactions are like to the back left for example so one thing we thought about was how about what we do is when you teleport we go and rotate the world for you so it's like a snap turn but we're in control of that and we thought this gonna be a great idea was gonna solve so many of our problems and we tried so so hard to make this work it turned out and every one of our tests and every which way we tried this at the end of the day it was disorienting to the player because they weren't a hundred percent sure what direction they were going to be facing at the end of the teleport we also discovered that it's very important to be looking in the direction you're going to teleport to try to complete that feeling of understanding where you're going to end up if you're looking directly we are going and then you end up just forward that is very very understandable in the human brain so when it comes to

interactions this means that they need to be in that forward direction so we end up laying out our environment such that the front of those play spaces are where you're going to have your primary methods of interactions so like forward and to the left a bit and to the right a bit and so all those interactions need to be within that tracking volume that said let's say something like a 360 a set up is up supports 360 controller tracking there's still benefits with this design because most of those are still tethered and one issue when you're doing a lot of teleportation is getting twisted up in the cable and having to do cable management but because of this approach you end up not rotating all the way around very often at all and so it actually has some benefits there and so as I kind of mentioned and that's designing the world it is very very difficult to design layouts for this but at the end of the day the player doesn't have to think anything about it and one thing we talked about it alchemy all the time is like the best work we can do is the work that a player never notices

another challenge is where to teleport in terms of directions we found that moving in the cardinal directions is best so that's forward back left and right we've also found that certain other angles feel better so the 45 degrees your diagonals do but it's important you're content it's going to feel different depending on the way that your your content is laid out and it's really important to have a clear path that you're virtually walking that straight towards the destination otherwise the players get disoriented you're trying to make them feel as if they moved between two spots and taking a curved path is you know makes that disorienting another challenge we face is where is the edge of my current zone right how do we logically break up these zones and in job simulator we had these desks and you know kind of big physical objects that players naturally wouldn't cross but in vacation simulator we need to teleport and teleporting through a physical object feels bad so we need to kind of

have a clearer more open area and the goal is the players should never question why they can't just take a few steps to get to the next zone so it's like I I can see it I should be able to walk there you need to provide some context for why the teleport is needed we do this by communicating the play area we use lots of techniques to show this so a good example is like visual design and the beach views tells on the ground and we use different terrains so you might have like a rocky area that is the play zone with grass on the edges we use doors and gates to help show that like you couldn't just naturally walk there but when you hit the button to teleport the door opens and gets out of your way so that that's like the action that's being taken we use elevation changes too because you can't really walk elevation changes in VR because we most of us play in flat rooms and so so we we use those elevation changes to help kind of exemplify like why this teleports necessary and then when all else fails we put it in a full-screen

fade and it dims the screen when you get to the edge which is really discouraging to players to be if they suddenly decide to walk between zones if suddenly it fades out it's like okay I know that one thing that we noticed is that systems that do have some sort of chaperone or guardian over time people start ignoring it but a full-screen fade at least even a little bit is a very clear indicator regardless of what you're looking at that oh I've kind of left my the play area so another challenge is communicating the movement itself and so in during the teleport we fade in and we fade out but we want to communicate even more that you're moving during that process and one thing that we do to do that is sound effects during the teleport and so very specific sounds that end up grounding the player in that world and so let's say for example you want to lay your teleporting to like a little bit of a rocky area we might play a rock a rocky footstep sound when that player gets to that destination or play a wooden sound when I like teleport out

onto the dock a very important thing that was a very difficult at the beginning is we started going down the path of footsteps because like footsteps seemed to be a very clear indicator that I'm moving and it worked but the problem is that footsteps take a long time to play out and your teleport lengths you want relatively short and so we ended up having a scenario at one time where we had like three footsteps but what happened is the teleport fade would come back in you'd be standing still and then you'd hear of two footsteps you're like I didn't do that I'm standing still so we ended up having to play a lot of things that sometimes sound a little bit like footsteps but sometimes aren't like for example on the dock it might be a creak of the wood on the dock when you come to the destination you've got to think really hard about the sound length of those effects another thing that we learned is that we can play around with the length of that teleport so how long you sit on black and how many sound effects you play to try to convey the distance of a teleport that something is a little bit longer for you to get to so here's a quick video kind of explaining how all this stuff kind of plays together so you

can see here you start teleporting the doors move out of the way teleport in the water there's a splash sound and all these things kind of play together to make you understand how you're moving around in this world so the next case study you want to dive into is character interaction so when it comes to characters and alchemy we always want to create memorable characters so right back when we were started working in mobile and the PC days we started building all these interesting characters to populate our game we realized we really wanted this in VR but this was going to be pretty challenging and some of the big issues with that is in traditional flat-screen gaming that's a character in a world in VR that's a character in your world so you need to make sure that you end up doing all the right things there and the presence and believability of those characters is very dependent on that how

those characters react their dialogue their demeanor so on and so forth so when it came to deciding what we wanted to do her job simulator we knew that there was some things that were just outside of our reach right we could not build a full humanoid character right because then we have problems like okay we have limbs and how they gonna reach and grab things how are they gonna move around how we're gonna do facial animations and all this stuff because we didn't want to fall into the uncanny valley so we started talking about robots and so one thing that we ended up coming up with was this box style bot that we called them and they have animated faces where we can use really interesting expressions but the end of the day would ended up working with them as people start identifying them as characters but we kind of solve all the problems with that so vacation simulators more complicated as we've been telling you many times and there's tons of teleport zone I mean there's lots of different locations that the player could be in throughout the course of play in the game and so a bunch of new issues cropped up with characters because of this the world needs to feel alive and so how do we place these

characters around the world to make it feel complete while not getting in the way of the players and we also want to expand further on player interaction with these characters so let's talk about personal space to feel a connection with a character they're going to be near the player right they're gonna be in the personal space but they can't be too close in fact the ideal is kind of a yellow zone on this graph which is uh between let's say a foot and a half from four feet or 45 centimeters and 1.2 meters and players need the ability to move around their space as characters interact with them because in VR player agency is really important so having characters kind of up close that are following the players around is not so great and violating an intimate space that red zone feels very uncomfortable and aggressive if it's not something the players invited so for character positioning we found it's best to keep the npcs outside the immediate play space the area that the players can play in that's even better

if you could put them at the very edge of the play space because that's an opt-in zone where players can choose the distance at which their they feel comfortable engaging with these characters we also added a an avoidance system to on the chance that a player needs to lean over or go towards a bot or for some reason they get in a close proximity where the bot tries to get out of the way of the player and tries not to make contact because that just feels so bad it's one thing is like these BOTS they look on the screen like okay they're lovable but if you have this giant mini-fridge style thing flying at you that feels very scary and very bad so we have to make sure that we never end up breaking that so now that we have these characters in place players are going to want to interact with them and a job simulator this interaction was really simplistic you could pass characters items at specified times and specified items characters were very heavily scripted so there was no real agency to initiate a conversation the

characters just kind of came in talked at the player said what they were going to say and then left so when it came to character interaction in vacation simulator we knew we wanted the ability to make it user driven to determine when I wanted the conversation so instead of the bot talking at you that you were saying hey I want to have a conversation in traditional games the answer would be let's just map it to a button and something that we talk about all the time is like in VR is resist the urge you know really resist the urge to map things to buttons there's almost always a better way in VR we've never had an input scheme that allows for so much creativity don't just fall back on old paradigms like mapping to buttons so our solution was waving waving feels very natural particularly in VR and it feels like the thing you should do to get the attention of an NPC so implication simulator you look at a bot you wave at them and that initially initializes the conversation and so it gives a player

any time to say hey I want to have a conversation and on top of that waving feels very fun and very friendly as so much so that I'd played enough vacation simulated sometimes when I pop into other VR games and I see an NPC I'm like oh hello and then like oh that doesn't work that's not like a universal thing I hope it becomes the universal thing because it feels so good so on the note of having characters kind of placed in the environment or in these zones we ended up in countering an interesting problem so when vacation simulator when a bot was there generally they were kind of looked at you and it was great there's like one bot looking at you that feels great but we kind of set things up for the beach it has all those zones and every zone has a bot and then you teleport for the first time into the beach and then all the bots just like looked at you and you're like oh no what's going on this isn't very very very creepy and so we we knew that we needed to do something and the answer was not do anything so first

thing we did was like let's turn that off and then you had the bots just all staring out into nothing because it's very clearly even though these are just like BOTS right you know exactly where they're looking and you kind of like guess that they should be doing something and so one thing that we ended up doing was creating these idle animations for the bots themselves so here in this video this bot is like bouncing around this coconut right at a distance this also helps us give character to those carrots before you even get close to them so you're like oh this is probably the bots that like sports and likes to kind of play around which you totally get and then when you teleport into the zone then they kind of stop that animation and pay attention to you just like you'd expect and so at the end of the day what it does is it made our game feel way more alive and our world way less static than it did before so the next thing we're gonna talk about is systems and some systems we built in in our game and specifically we want to look at temperature because we think it's a good example of kind of our

systemic thinking so temperature as a concept is a common human experience right many of us are probably either too hot or too cold in this room or somewhere in between and players have an expectation that things react Ennis Civic way when they encounter hot or cold in job simulator we had items that required and and spaces that required interaction with hot or cold like cooking in the kitchen hot dogs in the convenience store and players expected to be able to heat or cool those certain items and we had certain devices that could provide heat like cups the grill toaster microwave pot hot dog roller etc but it again it only worked on very specific items so those fluids and some food so when a came duplication simulator we wanted to kind of rethink the way that we handled this system so we came up with this much more broad concept of temperature zones and so to kind of break that down one example is each one of these destinations has its

own kind of climate its own temperature so for example the beach is hot and it actually has a temperature associated with it the forest is more moderate the mountain is cold but the mountain is only cold when you're outside when you're inside the lodge then it's room temperature and that's kind of at a higher level and then we kind of drill down after that and so for example when we ended up building the the mountain we end up putting snow balls in that's great but then players would take the snow balls to the beach and expect them to react and so they needed to react and so here in the video someone took a snow ball to the beach and it melted in their hands and so that kind of like all has to play together but there's more so now when you start thinking about melting and what kind of that yields here's an example of someone taking a snowball into the the lodge and melting it above a cup you expect that snowball to turn to water and then you expect water that's falling to be able to land inside something like a cup but then if you set that cup on top of the fireplace well then you kind of expect that water to

heat and but then when you've heated that water then you kind of expect it to boil and when it boils then you kind of expect it to evaporate and so we kind of went through all these things and as you kind of like grab a new one then you're like well that's that's an expectation for the next thing and you kind of like walk through all these things and so in our game temperature is used for all the things like cooking and boiling and freezing and all that stuff the interesting thing is once you kind of set these systems up then you can use them for more and more interesting things so let's talk about cooking and we have a number of areas in vacation simulator that can be used for cooking like the grill on the beach like you see here and it works by using the temperature zones we talked about to systemically interact with the objects so in our cooking system unlike Java simulator was rudimentary where there was just kind of temperature as a concept this is actually we actually started programming a system that uses heat transfer so an object is closer to the surface of the grill or on the surface of the grill it actually heats up faster than if it were

above the grill and this is great for believability and rear of our game and to go kind of deeper into that concept once we have cooking we extend it further so any food item in the game should obviously be Coco ball because we cook food and it should have a corresponding state chains and then we even let the object be burnable and all of that happens not because the objects interacted with the grill but it happened because the internal temperature of the object provided by heat transfer changed in a way that changed its state to one of the other states and we made all of that work and it's great yeah and so like to kind of add on to that in job simulator we cared a lot about surface temperature but that's not really what you expect in cooking what you expect is like the internals of this thing heats up and so it actually does absorption of heat and now on the opposite side and you're like freezing it does the opposite and so a lot of those things play together and then to like add on to this we're talking about food here but every object

in the game also has a temperature in itself so you can do things as silly as like oh I'm gonna go put say a container of milk on the gun on a barbecue and it'll heat up and then I'll pour in a cup and then the cup will be boiling milk and there's even we can go even further so we have the freezing items at multiple places where they can heat things up and as an example we we love pushing our systems to their far limits so one of our developers was like I'm gonna make matches and so you can in fact cook a hot dog in vacation simulator with a match it will cook very slowly because there's not a lot of heat transfer but you can hold the match match up to the hotdog and it cooks and so important thing is like this is one of the systems that we kind of douve deep into there's there's many more this is kind of an interesting one to kind of see as you like step through the process and you're like okay I added this then what does the user expect you do next and kind of kind of finding the bottom

of that the logical bottom of that is very important and so a great example kind of going further is sandwiches we're going to talk about sandwiches now and you know what is a sandwich that's a huge debate if you've been on the Internet so ignore that we're not gonna think about that right now we're what we're gonna think about instead is to job is to sandwich right this was great so we a job simulator we split it into four jobs each job lived in its own separate world and sandwiches were a big part of the gourmet chef job at the time which made sense but that didn't cross into any other worlds it just lived in gourmet chef and make me a sandwich seems simple on the surface because in your real life is simple to make a sandwich it doesn't take a lot of skill sorry sandwich artists and VR there's a significant technical but in VR there's a significant technical challenge to overcome to make this sandwich happen so when you started job simulator and start of making the kitchen you realize we wanted to make sandwiches but sandwiches

are very very difficult to try to convey and to do in VR and so our solution at the time was something we called the sandwich maker so the way that it worked is there is this platform with a skewer and you put food items on top of it the skewer gets taller and you keep stacking and then at the end to determine that your sandwich is complete there was little all of you put on top and then it became a unit of sandwich and then you could pick it up and he could place it on the plate and kind of send it away for job simulator this was a kind of a close enough approximation to sandwich to make it work but it left a lot to be desired but the trade-off at the time was that it was technically possible for us to build because we were a very small team and did not have a lot of time to focus just on sandwiches and so kind of explain a little bit further this is a video of how that works toss all your things on that little skewer gets a little bit higher time sandwich Hamlet Sandridge happened unit

of sandwich a big problem is that with this method you couldn't eat the sandwich which is like pretty fundamental but it was one of the things that we decided we just needed to do a very early prototype is like well just make it a single bite but then someone would pick up giant sandwich and be like I'm placing it on the thing and then all of a sudden it would vanish and they're like oh I ate it I have to start again and so that NVR coming very awkward so it's like okay this is just not something we can solve we'll put it aside and then to kind of this kind of stack on top of this we started learning all the affordances that come with sandwiches luckily for us in job simulator we only had to contend with the affordances in the kitchen and not the entire game on this note affordances it's one of the most talked about topics in our office we talked about affordances all the time and a big reason for that as this image describes

is we like to put a lot of interactions into our games but every interaction you put into the game impacts its affordance any affordance of everything else so in VR we like to think about affordance and kind of like three major parts so the first part is the player's real-world experience so if this object is X in the real world it should do X and VR and this is something that is quite a bit different than traditional games because of the limitations of what you're doing but in VR because you're interacting with the world in a very similar way you bring a lot of those experiences along for the ride the next thing is input and so the track input of your hands makes people think about how they would interact with the world with their hands and so everything ends up needing to be grabbable untouchable and I am and kind of solving all those problems in the affordance camp and to lastly here it's the expectations of the game itself and so if I did X within the

game already with one input that similar input should do X in other places in the game so the entire game actually plays plays together and it's very challenging to think about how you could change one thing in one part of your game and it will technically break your game but it might break people's thoughts of what should happen in other parts of your game so let's do a one sentence explanation of this and that is the world should work in a way that makes sense to the player given their prior experience the input and the inferred rules of the virtual world and so some examples of that are if you can see if you see a ball you can pick it up and throw it if there's a bottle of water you can open the lid pour the water out or drink it and if there's a drawer you can open it so that's kind of giving you an example of what affordances are so how do we afford a sandwich we want sandwich to be in sandwich making to be

intuitive and an understandable task and players expect that the world reacts in similar ways you could see you know this coffee is like the player expects that they hit the button it pours into the cup they can bring it up to their face and drink the coffee and they want these objects in the virtual world to match that expectation so look constructing a sandwich and vacation simulator the world of Ford's making sandwiches anywhere unlike the machine that existed in a specific spot in us in the specific job and job simulator the player should be able to create something that also that looks and feels like a sandwich and those two points alone create a ton of affordances a sandwich or sandwich like object requires items that are stacked into a single unit and we need an in-universe understandable and intuitive way to assemble that sandwich because as we were saying before we don't want to

end up putting these sandwich makers all over the world and that's just not the way it works in real life you should just be able to construct a sandwich so we said we weren't going to talk about this but here we are let's talk about what a sandwich is first we place some starting item bottom bun bread etc then we place contents contents of what you want your sandwich on top of that and then we place a last item on top it doesn't have to be bread but you're gonna have some item that you consider the last item so was that so hard look sandwich is defined delivered on that promise and one other important note hot dogs are not sandwiches and we have the code to prove it we will be taking no questions on those two slices so now we're setting off to do this very simple task of making a sandwich so the very first thing we got to think about

is attaching is essentially what we're doing is we are trying to attach more than one thing to something else that could be more than one thing attaching by itself is actually a very complicated task and the tech necessary has absurdly complicated and now we needed to do the most absurdly complicated thing of the most absurdly complicated thing so let's talk about how kind of our attach system worked in job simulator so here you can see we bring the cartridge near the slot yellow highlight appears you release it snaps in and it snaps in with a defined location and orientation based on what has been setup this allows for very simplistic interactions we did a few of these inside job simulators like a cork in a bottle for example and this one I was a few places that we used it it was not so great but it kind of proved out that this was an interesting method of interacting with the world so we realized we needed to kind of extend detachable Zout there was a lot of use

cases for this so even before we started like leaning into the sandwich problem by itself we started working on more complicated ways of attaching things so here for example we're building a sand castle this is complicated but it's a little bit better because essentially this is a fixed plane that we're dealing with I mean kind of do the math on top of this but the one thing that we realized is that every system that we build for attach needed to support the concept of also holding the thing in your hand when you're also trying to attach things and in beer this is quite complicated because you have very limited interaction you don't have fingers to go and manipulate things you're essentially doing a grab and release so kind of what do you mean so let's talk about stacks of attached so attachable items can be marked as this like stackable state and so one of the concessions we made was let's think about some things that we called like sandwich starters some things that you

would kind of expect to build on top of and so for example here it's like there's a bottom bun it's like okay that's a thing you want to stack so when you end up bringing something that is can be attached to a stack of sandwiches near it you'll see a highlight in the final destination and then when you release it'll attach and then that item then parents itself underneath did now new sandwich object that you can then manipulate and so here we're taking like stacking through it it's like okay now here's some cheese and then here's like a splat that we'll talk about in a second and so this also in this example is happening on the counter but we need to deal with the fact you could have this in your hand you're like tossing things in it and then trading hands between that and kind of what your intentions actually are so now that we have stacking we kind of this concept of sandwiches okay now we can actually assemble a sandwiches a sandwich itself so designing the stacking like this ended up being pretty intuitive for the players they just wanted to put ingredients on the stack that said that

there were some problems with that so in job simulator every every edible item in the kitchen could be in a sandwich and the world afforded that kind of like incredibly goofy and awkward level of interaction as you can see by this like raw steak egg and whole slice of cheese sandwich and so the interaction paradigm was very very basic there but in real life sandwiches aren't created by just taking a giant hunk of cheese and slapping it on some bread you create it from slices of food so we created this slice Ematic machine to address that affordance and it should take in edible items and output a slice of it not just a slice of it but a nice-looking useable slice so in this case you see the avocado going in and instead of just taking up a chunk of avocado it's actually taking out this nice three three slices of avocado and once again back to affordances once the

player understands how the slice ematic works they assume every item in the game should be sliceable and this is the hell prison we built ourselves in that the artists everyday come to us and they go oh no another item can it not be food we're like no you have to make more slices and so you can see here we created slices for every edible item and you can see like the cucumber getting cut and kind of all these different items tomato and a bunch of other ones and the nice bespoke art we created for each and every one of them thank you John some items don't afford slicing like ketchup that would be super weird who slices ketchup so players expect they react in different ways so for those edge cases we had to address them individually because once you have the system again its affords a certain thing and players expect that so we create the concept of this splat and when you pour a liquid on top of a stack on top of a

stack a splat appears if you try to grab that splat off it turns back into fluid an earlier version allowed you to pick that up that felt very bad it felt very weird to just have like a handful of ketchup like so you pick it up it vanishes so we also addressed the Education marshmallows where you could see they're like in a s'more a marshmallow would flatten out so we kind of deform the mesh to make it afford what the player expected out and add on to that when we ended up building the splat thing it worked great and we're like we solved it and one of the first things a player did was instead of just building their burger and then pouring the ketchup on top or the barbecue sauce they would put the burger on the grill and then pour it directly onto the hamburger I'm like well that's obvious but let's not a sandwich that's a burger and so we ended up having to do things like going through the games like which objects afford splats when they're not sandwiches and so going try to like deal with that and then also deal with the case of like put splats on that and now I want to put that on a sandwich and have that

transfer correctly and take the splat off yeah so we have stacking we have slices we can build a sandwich but we still haven't solved the problem of eating it so let's talk about that all right yes let's talk about eating so obviously sandwiches afford eating it's the most common use case for sandwiches before and job soon as I mentioned you couldn't eat them so we needed to address this kind of in general so eating should be as simple as bringing something to your mouth food should react to this scenario and that's what we cut did in job simulator there also should be some feedback for that before that so for example you bring this ghost particle effects shoot off and you can do like multiple bites as you're going through food let's talk about how to eat and so in job simulator we ended up playing around with bite States not everything

had bite States some things to dot what we discovered though in practice was that an object should have at least that's edible should have at least one bite State so there shouldn't take two bites to complete and part of this was because you need to understand what's going on if I put something close to my mouth and it disappears and they're just a particle effect it isn't always clear it's also much cooler to do buy things in two bites so that meant that we had to go through the world and make sure every food item also had a bite state and so you can kind of see here this is some from food and this is what it looks like when you bite it and those particles that end up coming off your mouth location we also need to make sure we're coloring them related to the color of the item of the food that you ate so we play the sound effect this kind of crotch and then it kind of plays all together and this is for every food item and oh do we have a lot of them so here's a video of kind of our basic eating mechanism in the game home you see the color particles coming out the red and the brownish and coloring them

sounds trivial but then you start thinking about different ways you can recolor things in the world and it becomes pretty tricky so edible items can come in many states cooked slice stacked fluid splat burned all sorts of things and sandwiches can have all of these various states on them so all the systems are going to have to work together to be believable and so if you thought stacks and byte states were bad check this out we have a whole tomato a bit and tomato a sliced tomato and then a sliced bite of that tomato all of which have custom art so sorry again artists we we did it oh and by the way they all have special textures for their burned and cook states and now but doing this was a huge win because every item can now be bitten every item can be cooked and it all of the world affords it and it all works because we created

this generic system to deal with these things so putting our whole sandwich together we can leverage all these systems to make sandwiches work because we've built them all as these kind of generic systems and now our eating a sandwich becomes an order of operations problem so the player brings the sandwich to their face as you're supposed to do when you're eating it it's through the stack of all items and bites them it we then ensure the stack has is still intact has proper dimensions is properly rotated everything is done correct and we spawn bite particles and play a sound effect and if you look there you can see all these items on this picture are bitten as some of them are sliced items some of them are not sliced items and you know it just works they kind of like expand on some of this here I've also to do things like oh there's like maybe a mushroom in there that in the game is poisonous is going to make you vomit so when you're iterating through there you're like oh yeah that's one that's gonna make you vomit when you eat this

and other things like you can see in this stack there's a watermelon slice we actually had have a conversation is a watermelon slice a slice yeah can you slice it can you slice it and then would that be a watermelon slice slice the end of the day was it's better for a watermelon slice to be a slice which means that it should have to bite States to work which me that the final bites day is just eating the entire thing which actually is kind of awesome so let's take a look at kind of the whole the whole sandwich system coming together and here you can see us cooking a bunch of food items including slices pouring a splat stacking it all together our lovely avocado toast and then crunching so everything in the world needs to play together we keep talking about like the affordance of the world but when you end up building all these systems there's like an interplay to them and so here in this image and kind of see like all those things that we talked

about are kind of like being spun together so we'll leave you on this to make engaging VR you must first build the universe or at least part of it thank you so we left some time for questions that people want to come up to the microphones hello hi great talk wonderful information to share I have one question about your zone system so in the Rick Rick and Morty game you guys had this really clever mechanic of having two mr. meeseeks come into play and allowing you to grab hard to read up hard-to-reach objects our right out of your play field clearly that the matically doesn't work with vacation simulator do you have another way to resolve that sort of issue another set of systems that yeah yeah so yeah that's

a very good observation so yeah Rick and Morty we did kind of the very early stuff of doing some zone based teleportation but in the grads there was only three zones so we didn't have to deal with a fraction amount of problems with vacation simulator and the the UCT and toss you like to kind of grab things outside that was kind of one of our our workarounds to kind of deal with that but yeah I was very like themed to that when we kind of set out to make a vacation stand we're like that's clearly not tenable and that's clearly not a system that's going to work well and be like super intuitive particularly over lots of spaces so we ended up doing is we keep track of all the volumes in the game so let's say I have a beach ball and I have a beach ball in this zone it knows it's safely within that zone and his grabbable by you so now let's say I go and toss that beach ball and it rolls outside your zone we were like okay it's now in a state that is outside of your available grabbing space so we don't want somebody like trying to lunge for it and like run into a real-world wall so then we do is we determine once it comes to rest or close to rest or times out what's the

closest zone for it to go to so sometimes it'll like pop back and we have all these little markers of we're good safe spots of them for them to be but then that still means I can toss stuff into other zones and even if it just like trips out of the edge of that zone it's like no you actually meant to be in that zone we'll put that in there but to compoun this problem is sometimes people are like I don't want this object go away and the last thing you want is like that object to be like come right back to you so we also have this concept of lost where it's like okay clearly you intentionally got rid of this and we're like just let it be except for the fact that what if you were like oh I want to build a ketchup sandwich I took my ketchup and I threw it out in the ocean it's like okay there's certain things that we want to recall in those states and that's just kind of the surface of that problem it has been a very challenging probably work through the entire project on I'm pretty happy with every ended up but and there's object Holmes as another good example like if you're you know the ketchup bottle you expect it not to spawn in the safe spot but to spawn back where it came from yeah yes thank you very complicated

subject of ketchup can you stack splats yes the stack splits at the same time question is can you stack splats yeah can you stack splats you cannot stack splats of the same topping but you can grab two toppings and alternate stacking splats we tried stacking splats the same topping but it's like it's weird to understand in like a stream of particles which is kind of how our fluid system works like which one like it that's not a very precise thing so yeah you can kind of like take ketchup and mustard be like ketchup mustard ketchup mustard ketchup mustard make like a really huge disgusting sandwich filled with ketchup and mustard splats we found is that from a foreign standpoint generally people like I want to put lots of different things and so that's super intuitive it just works that's awesome thanks did you guys do any experimenting with more complex interactions like enforcing two-handed interactions with heavier objects or things like that yeah so the question is

do be experiment with any two-handed style more complicated interactions the answer is yes the answer is yes and on that's April 9th you should participation simulator but no yeah we did we did it we do actually have a few two-handed interactions we also are trying to make sure we're accessible so we're kind of like trying to think around those things but yeah we have we have done some experimentation with with that stuff I don't think we're ready to there's one really good example that very recently rolled out of the project was the ability to open books so back to the topic of affordance in job simulator all of our books were just a box right you couldn't open them and so we followed the same thing through invocation simulators like it is there that but we discovered that as our world got more rich and more interactive that more people were like why what am I doing wrong that I can't open this and I was like okay now that the world affords this we kind of got to redress this and

we've been on and off working on books around three plus years but we kind of think we finally cracked the riddle of how to open a book and it is a 2n interaction I grab a book and open it up and stuff like that and we discovered it we're like yeah and then we talked to the artist is like wow I got twice the content to build now and it's like great but that's just one of those that's one of kind of many of the experimentations we're doing in two-handed interactions books don't sound complicated but they're absurdly complicated yes in that same vein I'm curious how you guys decide where to draw the line like I usually don't grab stuff off a barbeque with my bare hand so I notice you guys didn't have spatulas I'm kind of curious yeah so some of that is there yeah where you we're how do you yes I'd internally do yeah so where it is so the question is where do we decide to draw the line internally and and there's there's like a multi-part answer to that so one one big thing is like we can rely on humor a lot but like we can rely on the weird awkwardness of the world which is why

you can just grab things off the grill because our world affords doesn't you know require spatulas for its affordance because it's like oh this is this rough approximation but the other thing too is we we draw the line a lot of times at the holy we aren't gonna be able to code this and lest I make this work so let's just do a joke and like in Rick and Morty there's a great example we're like there's a door right and it's got a it's got door knob and so you know we couldn't get rid of that in that game that's an iconic part of the world as weird as it sounds like this door is iconic but it is it's in this garage and so we're like okay how do we deal with the doorknob and there's another part of the world that there's another part of that universe that has this concept of real fake doors so you go and you reach you grab the doorknob and the doorknob comes off and says real fake doors and players laugh because you acknowledge that they did something so sometimes just the mere act of saying we see you as developers is enough for the players to go like oh that's great it's doing nothing that's bad and then other times we don't know where to draw the line and then we tumble down

a rather and we make water boil from from from a snowball so yeah any other questions oh hi you spent a lot of time talking about sandwiches and I was just curious if you think a hot dog is a sandwich we we did say we have the code we have the code it is definitely definitely not a sandwich it's not sandwich there's no there's no starting stacked item yeah it's different yeah fundamentally and at a core level don't don't add us all right with that thank you very much [Applause]