The Animation Process Of Ori & The Blind Forest

okay hello um so I'm J's Benson I'll give you a little brief rundown of my life in the games industry so I worked on fable three for a little bit that was fun and then I worked on PayPal news ill-fated boy simulator project Milo so that was a laugh and then that blew up so I was out of a job but it was all very sad and then I went indie and it was a great time to go indeed Meat Boy and limbo and all this stuff came out was fun fantastic and so I sort of de facto became the lead animator a very tiny studio with like you know four people at the beginning which was a bit of a way situation because I don't made a 2d game and I started making a sort of lush painted 2d game and on fable and Milo I basically hadn't read anything I just sort of sat around and you know kind of tweaked some little things and fixed some bugs so I worked on Ori for four years and on

the last year I was transitioning off I brought on a really awesome woman called to help an old who came and helped me finish the project and whatever and now I'm giving a talk about it so what's already if anyone doesn't know which is probably almost all of you or it is a 2d painted open-world Metroidvania game with oodles of character animation lots and lots of scripted stuff and it's just very very content heavy game as I mentioned the team was very tiny so it wasn't really even a case of oh we've got one person of each discipline we had sort of like about half the disciplines that you'd ideally want for example no technical animator and I have very limited skill set I'm not very good at many things so for example I can't rig fangs and I also can't draw which is a pretty bad if you're making a TD game okay so this talk is basically about how to overcome personal failings okay so I've got a bunch of images here which

you may recognize this is Princess Mononoke also this is spirited away and then mana locally again so the important thing to note about these images is these backgrounds like ridiculously rich and detailed you can see all the brushstrokes and it's gorgeous and the characters all sort of awesome but they're really simple they're essentially just silhouettes and there's like a little bit of rim light but they're pretty much just flat shaded blocks and then if we go to skip to this bit this is a shot from one and okay where there's like a curse on this guy's hand and it's these kind of riding snakes and it's actually all 3d rendering but they do a really good job of tricking you into sort of not really noticing that at the time so the reason I'm showing this is we wanted to hit the like kind of same aesthetic tone of these films and then also the kind of emotional tone and the moral ambiguity and that kind of thing and I also wanted to kind of pull off a technical trick that they're doing and kind of take this sort of snake arm type thing and do that through basically

every single asset in the game and make you think that the 3d stuff was 2d yes so with that in mind this is what I ended up making with the photons people used to do

okay so thank you so you know what we were trying to do and that's basically what we ended up doing and I'm just going to run through in excruciating technical detail the pipeline and you know how we did it basically oh and also I always forget to mention it when I'm practicing we made this in unity so you can make this game in unity engine if you on team okay okay so this is oh I skipped ahead yeah this is three D or E he's 3d model he's got one texture which is just a diffused texture it's 100 self-illuminated there's no lighting on him that's kind of all he is it's really simple his entire design is about being

silhouette I'm basically doing the same thing hopefully that like akadama looks like in Mononoke there's the little white spirit creatures so there's a few animation concessions in his design we gave him a tail so that I could always show you the direction of travel that he's moving in we gave him double jointed legs so that it's a platforming game you could coil them up like a spring and then having bounced up so yeah and also the zoom of the game is usually a kind of like a meatboy level sort of zoom so with all these super detailed backgrounds having him just be pure white silhouette helps him read really really clearly at all times which is important so like a much earlier I can't rig luckily 3ds max which is to our use for thing comes with a really good approach system called cat and you don't have to be clever you just click click click and you make a rig is awesome and I can skin so basically that kind of took care of all of that side of things and also coincidentally happens to be by far my favorite rig to animate because it handles scaling really well

in a sec so it's awesome and I used carriage and so for rendering we render Ori with an orthographic camera from the side it's a like a small detail but it's actually very important that we don't render any of these things with perspective because in the same way that not having any 3d lighting information is useful not having any of that like mathematical perspective information is useful because having less of the detail makes it easy to trick you into thinking you're not looking at a 3d asset so that's crucial yeah we also do things like uh when I'm running the frame I bake a nice object motion blur into it because it's essentially free I'm already paying the memory for the sprite so yeah and there looks a little orange jumps at fun isn't it okay so if you throw a sprite into the engine using that sort of rendering style it looks vaguely like this in most cases okay so we can do a bunch stuff to the frames before we start making them into atlases and stuff if we want tend

not to because it's better to do effect in engine because it you know it's more useful and you can iterate better and stuff but we do some things like a gift should play here yeah so when the Owls coming out the camera here and baked into the sprites is a depth of field that's blending out which would obviously be a relatively expensive thing to do in real time and we just get to do it for free so once we've baked whatever effect once of atlases we stuck it in the engine it gets made oh yeah it gets made it's a big atlases the point being to save memory so there's a few sort of things to note though that are quite cool but the Atlas system we have there's a dynamic chroma keying system that we have where it samples all the pixels in the various images that you're throwing into the machine and it'll say okay well blue is the least used thing here and makes it a blue screen but it also does really smart sampling where like if a character has for example red eyes it can tell that's important and even if red is the actual like least used color in all

those images it'll make sure not to make the eyes transparent because that would completely up everything so yeah there's also a lot of other oh these are some atlases that's not very thick is it yeah this is not a listing tool but some other cool things about the totally have like you can see maybe if you can read it we can specify for each individual animation the scale of the sprite and game is going to be this the camera is roughly going to be this kind of zoom and this is basically a massive bug prevention tool like the system will analyze okay this is how many pixels it's going to occupy on any given screen and you've given us source frames with you know whatever resolution and if they're just going to be too small and look shitty in gay then it just says you can't build this atlas like go back and make your frames again please which is actually really awesome and just stops you from you know wasting time when you're making the game which is good okay so the next layer on top of that is metadata and this is kind of where the this sort of hybrid of a 2d and 3d pipeline sort of comes into its

own and becomes pretty awesome we're obviously we're making sprites but the assets are all 3d assets we have all the bones etc so we have a layer where we just once we put an animations game I can grab the whole rig and throw that as an FBX into unity and we have another proprietary tool where it just spits out a little cleanup set where I can track the eyes I can track the feet so we can do sounds of the footsteps oh we can do particle effects and blah blah blah we can also apply route motion to all the sprites because it's just 3d and as if you had the actual rig in the engine you can do route motion which makes making cutscenes and scripted events like very very easy so for example as the leaf is like tumbling around the wind here it's a relatively kind of complicated shot to do but it was really like kind of easy to do because the leaf is just using the route motion from 3d we have all like the camera that's following it is just like following it in 3d because we know exactly where the things moving we've got these particle effects and the trail

and the glow and all this kind of stuff and it's just relatively trivial to implement something that looks fairly complicated yeah and then once we've done all that stuff the kind of last part the pipeline is they developed the programmers that means used about a ridiculously awesome unified shader system where you can just do all kinds of crazy stuff you can apply awesome animated distortion you can do really really nice like real-time global effects and things and also like super clever nice things like you can put like a additive multiply mask on one side of the character that sort of lights the character you know if there's a big moon behind them all ever and then if the sprite flips or whatever it'll still keep in that direction so you can kind of fake real-time lighting in your scenes and things so if you look at these gifts here on the top right you can see Ori affectionally within the the in verse he's kind of made of light so in a dark area like this we want him to kind of light up everything around him

so he has a kind of dynamic like a contact shadow with the ground which is made of light on the left here you can see this kind of ghostly figure and that's using loads of nice little shady things as well that all just kind of comes for free and in terms of implementing that stuff it's literally kind of like pressing some tick boxes on this awesome proprietary tool so that's good okay so that's the whole pipeline I was going to say the what's good about the pipeline now okay so the first bullet point is essentially the most important thing in the entire talk and the entire animation system you get to do perfect non-uniform scaling of Bones which is ridiculously ridiculously important and the two big things you get from this essentially you get squash and stretch which is obviously great every animator wants to do squash a stretch but way more importantly being able to manipulate the bones to your heart's content essentially means you can draw whatever shape you want what if I could draw and

actually could get a pen paper and do it properly you know I'd be like very careful about the separation of all the elements and the silhouette and all the kind of stuff and I can just do that same thing in 3d by just absolutely like you know tearing the bones to pieces and you can cheat perspective I can for 2d sometimes you don't want to show like both arms when the character's running or something like this and you just like shrink the arm down to a little nub and do all kind stuff also things like breathing you know rather than just having like the spine kind of rotating like this you can just be like I'm going to grab his chest and be like and it's that's awesome like he's actually breathing now so yeah but as an example on the left here you have like an actual frame that is like a nice frame I'm happy about and on the right you can see what it looks like if you just turn the camera 90 degrees and obviously it's a Horror Show so as you can see all of the elements of those models are completely aligned along one oxygen only makes sense there and that's basically the whole trick of all of the animation in

the game I've got another example here with courage owl yeah yes again she looks absolutely terrifying so we have like the nice silhouette of the Arrowhead kind of shaven blah blah and then she's a complete monster on the other thing so oh yeah and so for the sort of animated benefits of that I don't know how well you know she sees on the left you got Horry added like the top of his jump he kind of crunches down to about half his size at the bottom of his jump he's like twice as long sighs the frame when he hits the ground he becomes like ten pixels and then also with his tail like as he does little spins it becomes like three times its normal length for a few frames and all this kind of stuff with Naru she's actually jumped like her belly he's kind of doing secondary motion like when she hits the ground her legs crunch down to nothing as she's falling her neck kind of elongate like a draft for a few frames and it all just kind of helps to make it look a bit nice okay this is a slightly more controversial point I guess but using this system you have complete control over that you know the game can't render

a frame that you didn't draw right obviously if you're building your game like to make use of real time turning of bones to like for some kind of inspection thing or lever or you're doing lots of clever 3d stuff it's great to have the rig running real time and engine but a lot of time you find like I'd be doing dailies on fable or something and you'd have your play blast right off okay that might call combat animation it looks awesome and then you put in the engine and all of the edges have been rounded off through like compression and nothing that I'm forgetting whatever oh yeah through interpolation like you made a 30 FPS animation and it's now 60 and like half the frames they're not frames that you made anymore you get a kind of machinist to a lot of animation and you know you don't get that if you're rendering sprite another thing is there's no such thing as a system like this as an a blend between two poses that's like the computer is doing so on the one hand it's like okay well I've not got to do a

ton of work myself because it's all just going to snap horribly if I don't put in individual transition animations but on the other hand if you now do something like this and you do all the animation transitions by hand if you record like 15 seconds of you playing your game and then just like watch it back it's going to essentially look like the linear cartoon perfect version of you would have animated right if this wasn't an actual dynamic system and I was very blessed at moon that the program is really really understood what I wanted and they gave me the absolute perfect system for this and I get the all kinda cool stuff like you know in the arc of a jump I could be really really extreme with all the different key poses that I would animate because I could specify well there's a transition it's from frames you know 0 to 6 and I let go the stick or from 7 to 14 or from 15 to 19 and I could just basically go nuts with content which is what I'd like to do it was also really easy to debug and see like what's happening in the system which I find a lot of time you

spend your days just want to work out what your assets are doing in the engine and in this system that's not really pace I just run it all where I click on the character I click record and then you like play and trend repro whatever issue you're seeing and then just spits out all your states and like for example it's really easy to tell if a pair of states don't have a transition because the main gameplay relevant states are in white and if you just see two bits of white text next to each other it's like that's missing a transition you know so it's a little simple things ok and then a smattering of other random benefits like I mentioned earlier we've got the motion blur which actually is pretty enormous it makes a really big difference there's things like the darf and we do glows and certain things you can see wherever in After Effects you know and then this is kind of a huge one because you're rendering you know from the rig in Max or maro ever there's no compression on those bones so it's sort of the equivalent of if you came in to

work on your 3d game on Monday and the program has said hey we're disabling all compression in the game on all your animation it would be just like Christmas and birthday or all into one and you can treat your assets like film assets because it doesn't matter like whatever you're doing it's just it's offline rendering so you can have whatever Polly's you want you can do 10k textures so ever I'm not clever enough to do lots of like fur simulation and fluid and things I don't know how they all said but if you know how to do it then you can bake into sprites and it all works and so that's fun as an example his core oh one of the main character in the game and she's whatever she's got laid - turbo-boost musing all sorts of nonsense on her she's got a hundred fifty bones and for some reason I like to like individually animate every feather on her wings maybe don't do that up Steve she's perfect I like to do it it's fun so you can do whatever okay this is more like a workflow thing even things simple sometimes in a game like opening a door and I actually want my hands to attach

to the doorknob and twist in boa it gets weirdly complicated like okay now we need an IKE a solution above wah all that stuff is really really simple in this pipeline to give an example on fable three they wanted to kind of upgrade the way they did like emotes with the NPC's so instead just waving at someone you could like shake their hand or kiss them or ever and the way this ends up being implemented was that there'd be a fade to black when you interacted with someone it would fade from black they'd have had like a new rig with all of the characters in the same rig they'd play the thing and then they do another fade to and from black which is obviously not like the ideal solution right and it kind of shows that even a massive studio with like millions millions dollars that's just you know these are hard problems to solve so here's a random gift of a situation like that in Orry he runs up to his mum here she picks him up when you jump into her little Barney box there throws him on her back they do a little cutscene thing and the now you're actually playing is both the characters at once your control like the rigs are now like the same thing and the reason it's so simple is

because they're just two sprites and we have all of the metadata for like where they should be and it's just really easy like if I was going to make this now in you know a traditional kind of 3d way I'd be like pulling my hair out like I don't know I'm going to do this yeah so it saves you a lot of time there's no such thing as check how it looks in game if you like what you did in my or max that's it you did it well burn you finish the animation there's so much time obviously there are enormous benefits from doing cool real-time 3d stuff but when your game relies very heavily on blending and additive stuff and layers and other and you spend a lot of your time like not working on making your assets good but just working like what the are my assets doing in the engine like I don't understand I'm like trying to repo things just don't have that thing if you're working with mics which is lovely okay so now this is the bad part what are the cons of the pipeline hmm okay so essentially it's one big con which is

texting memory you're going to pay tons and tons of tons of texture memory for to render all these sprites especially like an Ori we have this owl character that just fills the screen she's huge we're running like 2 K textures and it's like 1500 frames for this shot which can be absolutely insane and you you start competing with the environment artists and with sound and with everyone for very precious memory and there's like not a game in existence that isn't constantly fighting for memory so that's essentially the big cost there are the things they like you if you decide hopper to your production like oh the character should have horns like you can't really do that right because you've got to render every single frame in your game again it's not quite as terrible as that if you actually hand drawing everything it's an enormous problem because the assets are still 3d you can spend the time to like make a batch script that re-renders everything and you can swap in a new instance of the model in a blah blah blah but obviously that the reality is that takes

programming time that takes time for you to wait for things to render and you're going to create bugs and it just isn't ideal you also can forget about various clever real-time 3d stuff so you can't adjust the feet of the ground you can't create any gameplay that relies oh this isn't really applied to already but like la no art type thing where you pick up a pencil and look at it from all angles you can't do that kind of thing and you can't dynamically change the model like if you want to say that there's an upgrade where he gets like cool new boots like that's not going to happen because you'd have to double the memory of the entire game by like having a boots version of the character you also don't get blend trees so the classic thing in any game would be there's an idol there's a walk and a run and I'm going to dynamically interplay between those based on where the analog stick is you can't really do that in a minute I'll say how you actually can do that you can't do things like the head tracking of like always following an NPC with his head and all that kind of stuff renderings issue as well just a time issue if you're doing this like a jump 30 frames that's probably not going to

take any longer to render than just like exporting an FBX file if you're rendering a cutscene it would like to play textures and it's like 2,000 frames whatever that might be like ten hours and so you're going to have to come with some kind of solution a render farm or something and all you're going to have to be very patient and wait a long time for rendering so did any of that actually stoppers and impede the game anyway kind of probably mostly not is the answer so for the foot placement example there's some gifts here of different slopes we basically tackled this problem with like two things the first thing is the character it's just rotated to the normal of the surface that he's on which is kind of like halfway there but he obviously just looked like he's going to fall over because the animation is not built for that so then it becomes a Content problem so we do a version of the idle where he's looking up slope and looking down a slope and that's relatively simple except that then you have to do all the transitions but if you will like

making content and if you have tools that make it very easy to make content like we did you can do some of this fairly easily and now it's pretty robust and basically there's no surface in the game that or he can't stand on perfectly the other nice thing actually about this is that I mean it's cool if you can dynamically put his feet in the right place but the fact that this is an entirely bespoke animation I can completely change his posture and make it you know much nicer so blend trees this is kind of fun one we feel like the idle walk run problem what we ended up doing was all of the animations that effects we rendered at four times the normal frame rate and then we would dynamically speed up and slow down the animation based on where your stick was so the you know 120 FPS version of the walk is playing at that threshold where you're doing the fastest version to walk and then we slowed it down to 30 if you're doing like the little tiniest bit of stick and then when you hit the threshold from the 120 walk to the 30

jog we just have a transition that covers it and it's not bulletproof I've got financing it's not bulletproof but it does the job really really well and isn't a big deal for a different version of house like circumvent the blender issue for for the swimming and we to do a kind of like 360 degree like Rayman Origins type thing so what we did was we had like a swimming idle things going horizontally and then we additively made like eleven versions with his back twisting like concave eleven versions twisting convex and then the engine all times is sort of simultaneously running all of those versions and depending on the angle you're going to and the angle you're coming from the programming magic just picks dynamically through all these different animations and you never realize that it's playing like 15 different versions of things it just looks like a fluid blend tree thing but look at targets but you could do the same blend tree like have a million different versions the idle that didn't end up being a big deal we just made like it's a Content issue again here's 145 degrees here's one looking upwards

here's one in the background and that kind of served our purposes so his Clues ones like okay so I had four years to make the game if you have like a year or 18 months this pipeline may just be completely unfeasible the guys at moon built like completely custom tools for absolutely everything and you might not have the time or the inclination to do that also just personally as an animator I like building content I like building tons and tons of content and doing little clever silly weird things that maybe someone else would think was kind of strangely overdramatic and pointless I think it's fun to do that kind of stuff and also I want total creative control over everything I'm seeing in the game I don't like giving leeway to okay well they're gonna tame animation do this with it I just want to know that it's exactly how I designed the thing to work and to look and maybe not everyone's as insane as OCD as I am so not everyone to

explain it's the perfect pipelines me I loved it I would not want to make a 2-d game any other way and I was very generously supported by the animator by their programmers they kind of just did whatever I wanted and they entered the kind of quality I wanted that's the talk thank very much okay do we have time sorry is there are the QA thing do we have any time for QA or is it over that a perfectly time I talked for two minutes all right today one want ask question won't be offended if no one want to ask a question sorry oh yes and also I was told to turn off your cellphone's but it's too late for that now isn't it um yeah so if it almost died specially if you screech the mic I'd be great hey um I was just wondering if you guys tested with having your characters made up of separate pieces and animating even animating

those separate pieces but using a bone system in unity yeah um I've messed wrong so flat in the past the problem is you just can't turn your character around I mean like in the video that I showed the little sizzle reel thing when he's hitting on those balance pads and kind of gosh I'm spinning the entire couch around and I know I keep going on about you don't make a look 3d but there are certain elements of being able to see the character from any angle that you're very limited by if you're doing a paper doll thing and I didn't want to be and that mean it would have been completely impossible to do that I do so much stuff where I'm telling characters in everything it would just not possible um hey what's the question you're talking about here is a benefit of unity versus the guesswork flow of using 3d assets represent several sprite and so how much of like this pipeline is just because of unity um so be honest unity just really you can kind of do whatever you want in it it doesn't come with like

a super structured pipeline for things there's like a relatively basic set of things that comes with and I certainly at the time like makanan didn't exist and like this when we started it's like unity 3 so there wasn't even like the 2d toolkits and stuff so it really was just like create it all from scratch because like we kind of have to but we also kind of want to so yeah thank you you