Animation for VR

[Music] before i begin i'm liz i'm an animator i've been animating for close to 15 years now um and i recently was able to work on a vr project um and i learned a lot so i want to share tips tricks for uh how to think about vr when you're on your animation project before i get into the vr part i want to point out how animation principles don't change across mediums there are of course tips and tricks to work within each type and style of animation but i'm not going to get into the views on those but now for vr these again are tips and tricks how immersive vr is and what draws people to it with any medium what is the story you want to tell and

what you are trying to say are two questions we should always keep in mind with vr because it is a limited audience we should always be asking why vr for a number of experiences vr is used to put the audience in someone else's shoes traveling while black carne arena uh are great examples um they use this in their storytelling this documentary style pushes the audience to feel and experience something they may not have had an opportunity to feel and experience we are being put in the shoes of someone we should empathize with it's a great storytelling doodle traditional theater has been around for a good amount of time and there are some impressive things we've done to set the stage

we've learned over the years how to direct the audience to look and take notice of one part of the stage while the other part might be in the process of being set up sometimes it's a lighting change or the curtain is drawn or a backdrop lifts and to reveal a much larger stage sometimes the actors walk in with their set pieces there are lots of tricks used to help bring attention to where attention should be in immersive theater stage techniques are used but the audience is now on the stage as well how the actors move around and take into account where people might be is an impressive show park theater part improv and these actors know just how to move to convey that they need where they need to go right where you are you get a chance to see an immersive

theater piece i highly recommend going it's kind of hard to do right now because of covid but if you ever get a chance go sleep no more in new york is probably the most well known it is not the only one i it's definitely an experience that is hard to describe uh in wolves we worked with an immersive theater company uh third real third rail projects um based out of brooklyn um so if you ever get it please google all of these um how do you storyboard or plan for vr since there is no camera storyboarding might seem like a huge pain point how are you supposed to possibly show

what is going on when there is no camera how do you show composition what do storyboards look like when dealing with vr it's a question that has no single answer and i've seen many ways that people plan storyboard previes it's all of the things um it's not a simple answer for vr there is no correct way to board for some vr experiences the action and characters are in one general direction at any specific time forward if you would you could board these types of vr experiences more traditionally they're obviously in front of you the camera may change slightly depending on how someone is standing if they're sitting or if they're standing up if they decide to lie down on the ground

it will change how the camera looks but the general composition can stay not the same but similar um for others the characters and action is happening all around these types are more difficult to board but not impossible when thinking about the pre-production phase for characters development is key with or without boards understand your character or characters is going to be extremely important since you won't have traditional boards to reference on wolves though we did not have storyboards we did have character sheets and we spent a good amount of time developing our characters that meant creating many facial poses as we could to reference throughout wolves our facial library was a great starting

point for our animators to utilize and because we didn't have locked cameras even though we had that library we used it as a foundation to build up from just because a pose looks and works well at eye level does not mean it's going to work well once our character is sitting on the floor looking up at us um as for broad body movement poses matter but a pose up close will look different and feel different than opposed 15 feet away for wolves we relied on third rail projects to help us choreograph those movements and it is choreographed there were several rehearsals and back and forth with the director of what was working and how they wanted the audience to feel the actors took our base our foundation and expanded on what we could do and their rehearsals were filmed

and then edited as reference we have uh some people would call it a board-o-matic but because it was live filmed reference we have the actors coming and entering in different directions and our camera was the audience the player at center and would move based off of where we wanted our attention to be drawn so it was extremely useful extremely important that we had this and it also helped us understand where our pain points were going to be it whenever you get a chance acted out right um preference is super important um any animator can tell you that how do we map out where our character is and

where they are going um we used top-down view to map out where our character was and where they were going you can imagine a blueprint of that one two three bedroom apartment that's what we did we laid out where everything would be placed and where our characters would be at the start and at the end of each scene so we could have the scenery transition around her instead of her leaving and walking into a brand new scene on wolves we focused on the sea where she was and we designed the set around her movements that doesn't mean we didn't start building sets beforehand we definitely made sure that we had an idea of what our sets were going to look like so it was a general look and we would have maybe a table or a chair um a couch

and based off of the actor and what they were doing we maybe shifted things so that lucy's actions the actor could really just do what they're best at and act and not have to worry too much about where something was placed in 3d [Music] that said on a mocap stage when you do have actors interacting with a couch a table you want there to be a couch and table but we were also very flexible with where things were placed so plan but also understand that you have to be flexible in order to get the best performance you can um and really to to make your your project as seamless as possible our blueprint

was a foundation but it was also flexible so my advice is to plan as much as you can but be prepared to change everything we also had to film ourselves to figure out some design issues we were having there is one scene where there are four humans and four wolves all running around we knew how everything was supposed to start and where everyone was supposed to end up the middle was a big question so we went out into the parking lot drew the player circle which is um just a circle on the ground with someone standing in the center so that we knew how close we were to the player to the audience and we set up a camera on someone we set up someone's phone up high to look at us uh running around and see a bunch of people running around

and outside him i think this looks like utter chaos it wasn't we all knew which role we had um even though the characters were running we walked it out first and we based our movements off of each other if we saw someone was going to be in the way of someone's path we adjusted um and we rehearsed that a few times and we then took those paths through our actual blueprint of where those paths would be and set that up on the mocap stage for the actors and even then we adjusted so we planned just plan adjust um and uh

of course even though oh again we planned we still had to make last minute adjustments for the mo-cap stage not all scenes were this complicated uh most were pretty straightforward one character where do they enter the scene or room and how does the room appear around them where are they in this space um speaking of spaces when you're designing your animation with vr and someone in the player in the audience position you have to think of their personal space right so there are three different types of spaces intimate space is really close up it's it's think of hugging um very close to a person

we try to avoid that with vr mostly because if you get that close the character is right there gets that close to you the basic human instinct is to like retract and like pull yourself away so try not to get too close um the other space is personal space and this if you think of it's within arms range and you want to make sure it's not too close so [Applause] it's yeah it's arm's length um and and you can enter this space your character can enter this space but be careful with big movements if your character all of a sudden runs up to

the person in vr to the audience and they're not ready for it they're gonna pull that and it's possible that that's the reaction you want um but in general we kind of don't want to scare people too much with our characters right we want them to feel comfortable we want them to enjoy but understand that you don't want it to be too close so personal space is a good area but you don't want to it's the same rules but you and you can bend them a bit but you want to make sure that the audience is comfortable so maybe in that space your your

movement for your character is not these big grand gestures maybe it's more subtle maybe it's how normal people interact with each other next to each other if you can imagine your friends standing next to you what's an okay and acceptable movement for them if you can imagine someone who's 20 feet away who's trying to get your attention yeah baby it is that grand big movement so be aware of distance um and if you have a character running up to you maybe they slow down those last couple of steps um so that they're not gonna run into the audience speaking of subtlety um in vr because we are maybe very close to a character we have to take into account how people

react so let's say your character decides that they want to whisper to you they're going to get really close how do they do that do they do they jump towards you or do they like slowly move in and i need to tell you a secret right so how how you plan that how do you um feel like it's an invitation versus um just do you want your audience to lean in do you do you want them to just stand and watch so how how interactive do you want them to feel in this if you want them to come closer baby don't go all the way in right maybe have your character

stop at a certain point so that they have to lean towards your character um it's a lot to think about and there's a lot of subtlety that we we just we didn't know we needed for wolves um there's a a good amount of interactivity for wolves in the walls um you're handed something some objects from our character and some of them she hands them straight to you forward right in front of you but there's a great section where she's looking through this chest and she doesn't necessarily have she's trying to look for something and she's not looking at you at all but the actor on stage does something so brilliant that no matter who she hands

an object to or every single person we play tested every single person would grab for it and it was this connection that we couldn't have predicted but somehow the actor knew they just understood how people react and so she's not looking at you at all and she just hands you something and you're like oh i had the reaction to grab that like it just felt like such a natural motion so we don't always have to stand up front and center to hand an object to someone we can play around with how we do that and so when you're planning your animation when you're really thinking about it like try different things but also play tests you want to see how people react to your

different animations and play testing can be as simple as it doesn't have to be in vr it could just be acting in real life with a friend and seeing their reaction because vr is so personal it is so it's a one-person show right like the person you are creating this animation for it is an individual it is not a huge audience so how does one individual react and don't just try it with one person try it with like everyone in your family and see how they react um and and that's kind of the the best way to like start play testing start understanding your how your animations can play out um

and so when we were building our animations and we thought we knew what we were going to do we play tested in real life but then we also play tested in vr we had people come in and see what their reactions were and there was one section we were having troubles with because nobody was taking the objects that were handed to them and we were just like so what do we do how do we get people to interact how do we progress the story with people not taking these objects so we had to change up how that section went and so we and ended up um creating more animations for our project than we thought we needed and it just it shows you that um play test early and often and and

see where your problem points are um and this was a design issue but it was also an animation issue like how do you get people to take something that is like placed in front of you um and so we want people to grab for things but maybe maybe wiggling something in front of you that's not the reaction you're gonna get you're gonna swipe it away so that created a a different chain of animations we have to make so um plan plan but also play tests um it's i can't i can't stress that enough um the next thing i want to talk about is staging a camera vr has no specific camera it's wherever

an individual chooses to stand and whatever direction they choose to stare at so whatever they look at a big thing with vr is once someone's in there they the first thing they do is look around everywhere right and the other thing they do is look straight up so for some reason it's amazing we have the sky above us but in vr we have to see what's above us and we're it's instinctual um so for cameras in vr um i feel like i've mentioned it a bunch but there is no camera right there are a few ways though to get people to look in the direction you want them to one way is with color and saturation you can have the whole set black and white and then there's color where you want them to look

another is sound spatialized sound does wonders people hear something and it's subconscious that they look in the direction that they hear it um we could also use different lighting cues effects maybe an animated character or object directing the audience to look in a specific direction so as to not miss what you want them to see can be extremely difficult most people will look all around them like i said in wolves we used a few different techniques we used spatialized sound lighting and vignetting the scenes certain areas would pop up and the rest of the set would be in total darkness um essentially creating a spotlight for people to look at very much a traditional theater technique

with that there's also ways to do that with your character animation those are all ex like other ways to get people to look um if you have a character that is like looking in one direction and you see them and notice them and then they walk in front of you and across you you're gonna look the way they're walking that direction is bringing them bringing your eye to where they are going so be aware of personal space and where they're walking um i've seen some projects where the character arcs around a very specific player they don't come into the personal space but you can have that character walk into the personal space be careful you don't want it to be too close but you also don't want it to feel

like there's too much movement you don't want them to overtake you right so be wary test it out with your friends and family in person and then test it out in vr have a character come that close and see how people do with that movement block it out as it were um you don't have to have it like finely animated in order to get that feel um you would be surprised how many people are very comfortable with the character walking across them um just no big movements right and speaking again of camera i like to say that um the bigger the gesture the bigger the character is moving the farther away you want them think like a wide shot or

a medium shot how much is their body actually moving right um for like a medium shot their feet are probably planted and they're just like wiggling around right for a wide shot they could be doing cartwheels they could be doing a lot more bigger movements and for a close-up shot they're maybe moving their shoulders they're staying pretty like on like on their mark they're not like because if you start doing that um unless you want to show off a very like anxious character um people do tend to step back um so be aware of your distances there um and the the farther back you have a character the more you have to read so if if the character looks this big to you in the distance

they need bigger facial expressions right i always i always like to like point out um like the like stage theater um a lot of the actors movements on a stage they're not meant for the people in the front row they're meant for the people in the back row right what they do on stage has to read all the way back there like all the way up in the nosebleed section so you you really want to like play to that there is no camera there right like on a theatrical stage there is no like well there's several cameras and it's in each individual seat right so you just want to be aware of that um and yeah i think i'm running up on time i did want to talk about the pros and cons of keyframe

versus motion capture i think people already know those already though like keyframe you have a lot of ability to you hit those poses and it's really freeing um uh it's you know there really are no bounds uh boundaries you have full range of what your character's poses look like but the cons are you probably maybe need a very large animation team uh for mo-cap uh the iteration cycle is much quicker right like if something happens on the stage and then someone sees something and like well maybe if we do this you can take multiple takes and then make a decision later on which one you want to follow through with and even if you use motion capture as just reference it's really easy to fastly iterate that

way um the cons emergency capture can be that um it's cumbersome like the amount of editing you have to do with mocap can be mind-boggling at times um and then characters have different proportions then they're actors for wolves our characters had these very large heads so if the character raised their arm you know we had to do a lot of editing and retargeting so that that arm would not go through their head so it's there's lots of pros and cons um whatever works best for your projects i highly uh i like both i think vocab's a style so yeah um and that's my talk on animation for vr i really hope you enjoyed it uh if i didn't say my name before my name is liz cover

and uh it's been fun [Music] you