Maximizing Animation and Cinematic Content Workflows for 'Just Cause 4'

[Music] as this slide states i am brian vanisky a senior technical animator at avalanche studios i'm located in the new york city office right in the middle of manhattan our other two locations are actually both in sweden there's our og studio in stockholm and then our newest studio which officially opened last year is in malmo our most recent games that you may know about are the latest installments in the just cause series with just cause 4 haven't been released this past december mad max rage 2 which comes out on may 14 and notable self-published games such as the hunter call the wild as well as generation zero which actually comes out this coming tuesday march 26th this is actually my third time speaking now so hopefully i've settled in and i can give you a good experience with this presentation today i'll actually be talking about the content workflows that we've employed for both the gameplay and cinematic teams that began from the ground up

during just cause 3 and evolved over the production of just cause 4. i actually wanted to call this talk maximizing animation and cinematic content workflows for a triple a project that utilized two different dcc software packages with a team size smaller than a typical aaa team but the full title wouldn't quite fit on the schedule board so i had to make do in all seriousness know the reason why i wanted to talk about the work that we've been doing over the last few years is because i truly believe that what we've been able to do with the scope of our projects as well as the team size we've had is quite remarkable it's not only a testament to the tools and processes that have been employed but by the fantastic teams that i've had the pleasure of working with i'm going to set the expectations going into this to better help you understand what i'm going to talk about consumers will look at your product they're going to see shiny things they're going to rate your graphics how you compare other games so on and so forth at the core overall at the core of it all though everyone here is producing content in very similar ways using workflows and methods underneath that shiny finish

a large focus of mine over the years has involved a lot of process and with that said i'm not going to give you a groundbreaking new piece of tech during this talk as it is in fact about using and manipulating things that do already exist while i love sharing my thoughts and process i'm not saying this is the definitive way to do anything but it did work well for us now while a good portion of this talk is focused on cutscene work it's not a narrative talk so i won't be going into detail about that side of cinematics although certain parts of our pipeline were created to allow for the extra work that came into narrative changes it's also not a talk about the overall look of anything or the in-game tech behind it and it's not a talk that discusses creating high quality uh high quality content or how to hit a certain level of quality it is about managing large quantities of content and making the lives of the animators a little easier through the through decisions and work done to improve the process of content creation during the production of just cause 4. and i wanted to make sure that no matter the position that you are in whether it be a small studio large studio or maybe just yourself that you can take back a

few things with you and without the need of a programmer or some special tech do anything that i presented to you as you don't always have the luxury of programming support or an army of tech artists or tech animators by your side i also wanted to make sure that this talk was accessible for any level of knowledge because of this i'll probably go over things that some of you may already have may already know or have done yourselves in order to better help those who may not know about those particular topics this is even more prominent when i get into motionbuilder specifics because of the greater lack of support that you find out there compared to say maya slight heads up though uh this talk is about to be a bit of a hodgepodge because i have some odds and ends that i do want to share amongst some of the bigger tasks so hopefully you don't mind a bit of random tips or tricks mixed in with pipeline and workflow in an effort to keep this talk as accessible as possible i want to make sure i define some terms because i found that in practicing this talk a few different disciplines across the studio weren't quite sure what they were i had animators asking me what json was and programmers telling me that i don't need to explain what json is

i'm just going to lay this all out here for all of you so dcc refers to digital content creation application and for this talk the dcc's that i will be mentioning are maya in motionbuilder mobu is shorthand for motionbuilder json stands for javascript object notation but simply put it's a language independent data format that lets you store data to be accessed in a very straightforward and logical manner if you're aware of what a dictionary is in coding json looks and behaves very similarly on the surface headless refers to running maya in standalone standalone mode which means that you're using full-on maya in all of its features but it runs in the background without actually opening up its visual component baking and plotting as animators and tech animators should already know is the process of converting animation data to keyframes on every frame since this term is different between applications i just want to make sure that everyone knows that baking is the term for this in maya and plotting is the term for that in motionbuilder ta in this case is short for technical animator mocap is short for motion capture and

when i say jc3 or jc4 i mean just cause 3 or just cause 4 respectively now that you know those key terms here's a brief overview of what's to come over the course of this talk i'm going to examine the two main dcc packages that we used maya in motionbuilder this includes what we're working with from the jc3 pipeline and what we did differently for jc4 additions made to that pipeline and the tasks that we tried to automate as much as possible and speaking of automation i think it's worth looking at the batching process that was set up at the studio a standalone ui was created that allowed us to quickly set up customized batch scripts for any tool or process we had in maya finally i'll end up by going over our cutscene tools and how we set up the content for running the pipeline as well as get a little in depth on how the actual transferring of content between maya and motionbuilder was achieved now in order to better understand a bit about my job at avalanche studios it's important to define the role of a technical animator at the studio as every studio within the entire game

development world seems to have a bit of a different definition for what a tech animator or even a tech artist should be to illustrate this better i've thrown some buzzwords for you here up on the screen and the tech animators at our studio all at the very least should know a little bit of everything and have proven the ability to pick up new unknowns and very quickly adapt and learn for myself alone i've dug into everything involved with the animation process to some degree i've built a ton of python based tools and scripts supported both the gameplay and cut scene teams dug into state machines in editor content and have worked with each and every department in some way from the vehicle team ai designers graphics programs almost everyone because of this a ta at our studio needs to be ready to roll with the punches at any moment they also need to be able to solve the overall major problem which is how will we maintain so much content that are when we are limited in our time and personnel another aspect of being a ta at avalanche is that each project utilizes dccs and workflows in a bit different way now this leads me to the question for

jc4 specifically maya or motionbuilder while we were mainly using maya for just cause 3 we did have some tools created during development that allowed for animators to push an animation from maya into mobo and also take an animation from mobu and bring that back into maya however this was not extensive enough to have motionbuilder be anything other than a supplemental tool as opposed to a standalone as a maya guy it did take me a while to really appreciate what motionbuilder could do but now i do realize that the things that does well compared to maya it does really really well i've also seen first hand some passionate debates between folks on which one is the superior product you'd actually be amazed at how heated they actually get um just kind of an anecdote i made a twitter poll recently while prepping for this talk to kind of get a gauge on what people use and the results were pretty staggering at how much maya is loved but if you actually go back and check this thread there's some really really heavy motion builder lovers there especially if you see brad clark he's always up on motionbuilder um you should

check out this thread if you want to see kind of the debate that goes on so when asked my our motion builder why not both it did almost seem like a death sentence to say yes to building up a pipeline that could truly support two dcc's considering at the time there was only one tech animator myself and six animators to support and two pretty different pieces of software but our team was pretty split up on preference and expertise so i decided it was more beneficial to allow them to all work the way that they wanted to and were comfortable with considering the lack of time that we actually had to complete the project especially since we were dealing with so many different factors that work better and one in the other since many other studios out there do in fact use both motionbuilder and maya we knew that building up this pipeline wasn't a crazy idea but the challenge would be to develop something that was robust enough with our lack of people power now in order to try and explain how overloaded the team actually wasn't working with a triple a open world title such as just cause 4 i do want to start

off with a few quick facts on the gameplay side of animation we had six animators to support and no they wanted to have tech animators and like a king solomon cut the baby in half thing uh that's just me quantifying that i was the only tech animator for the first few years of the project and we brought on tech uh ta help a little bit later on uh about a year or two a year year and a half left in the project in the end we had around 7 200 unique animations and among six animators that's quite daunting to manage it equates to about 1200 animations per animator for the entire project very roughly 2.7 animations per working day over the course of roughly two years which is kind of the amount of actual production time we had now factor that in with the amount of time necessary for things like polish accents capturing throwaway r d work and meetings that everyone loves to be part of now as far as preference for those six animators we had one person who was 100 motion builder never touched maya one person who was maya never touched motion builder and

the other four were sort of a mix in between as far as the pipeline goes we did have to follow this structure which i know that many studios who utilize both motionbuilder and maya actually do follow you start with motionbuilder to handle any sort of mocap data using the tools it has such as animation stitching pinning and the take system for organizing content you then move that content over to maya from animation polish and finally and then from maya you export to your game in our case since we did have an animator that only used motionbuilder he would stay in motionbuilder until he was ready to export to the game and from there i had a process set up that will allow that would allow him to pass his content through maya when exporting and we needed to do this to make sure that all the correctives and the custom rig features got picked up before they got exported to the game now keep in mind that we weren't using any sort of runtime animation or rigging so it was extra important to make sure that we stream everything through maya on export to ensure that quality control all right so let's dig into maya

i need to think about what we already had from jc3 what we needed to do for jc4 that would get us to where we needed to go we already had many basic tools that you'd probably expect to have in a aaa game pipeline but after working with them for an entire project cycle we were able to pinpoint the pros and cons figuring out what to improve on matching parent spaces and ikfk limbs as well as setting up constraints have been very standard for years but how could we improve them or even use them a little differently than before did the tools we have over complicate things in the case of foot planting animators on jc3 had to manually tag the content which when this was occurring how could we automate that how about all the scripts and random tools that animators find online install or use and then they just end up breaking maya for all the tech animators and it's a joy to fix there could have been a better way to handle that and essentially animator proof things there were also attachment tools that would help set up weapons and props for animators but they ended up being being so cumbersome with too many buttons and options that we actually just got rid of them

animators tended to have their own workflows and setup attachments in their on their own ways and a lot of this was in part to outside tools that they had found or just processes that they were so used to that i didn't really have to worry about that we also had a file browser for the animators where they could find and open maya files from within maya as well as multi-select files and folders for batch exporting animation finally a tool from maya to motionbuilder and back to maya did exist but it was very limited and we didn't really have a good system to transfer multiple takes back and forth now just visualizing some of this right now we had a lot of those processes broken down into one button clicks for jc3 thinking that it would be easy and awesome for hotkey but in many places this really wasn't efficient this did allow for a few things like space and ikfk matching to have the ability to use as hotkeys but the physical ui was sort of a nightmare and a hassle to navigate all of our tools also had their own dedicated ui window which was a pain to keep track of as well as organized visually

in addition to these tools it was sort of the wild west when animators started to download these third-party tools and install them themselves or create random shelf buttons for mel and python scripts going back a slide we did have animator organized third party tools but they were completely unsupported by tech animation essentially we gave the authority to one animator who oversaw this process he used the network folder to add scripts to and then called them in a custom user setup file that any animator could grab it worked well enough but it wasn't really something that i could oversee or easily quality check so because of this oliver supported as well as unsupported tools came together like that shoebox full of cables and wall chargers that you keep in the closet just in case you might need them someday now for just cause 4 instead of trying to add more the focus was to see where we could scale back like those constraint tools what else didn't we actually need what needed a restructure what tools needed some serious code refactoring when it came down to it we almost started from scratch on many things there was also a bit a lot of refactoring and some decisions made to

either completely get rid of things that that were barely if ever used as well as to push to automate as many things as possible this led to the main goal instead of trying to solve every little issue we would give more power to the animators we realized that we didn't have to try and make every little thing a button or try to solve every single problem we also realized that the animators in the end didn't want to work that way so simplicity became the goal now in order to reach out in order to reach our goal of simplicity we started focusing on efficiency speed user preference and customization the first thing we did was tie all of our tools into one main toolbar that can be set and customized as the animators themselves want after watching other gdc talks or checking out online resources i realize that many other studios tie their tools into one specific animation toolbar so i wanted to set out to do that first but also give it a bit of something extra if the animation tools are open on maya close they open when you reopen maya they can be docked or undocked you can pop out tools if you only need one but don't want the entire ui in your way

if you like the ui but you don't really want to use certain tools you can completely hide them from the toolset you can also rearrange the order of the tools if you use one more than others and prefer it at the top of the stack instead of having animators click a ton of buttons or use convoluted interfaces to set attachments for jc4 we created a simple one button attached detach system that could easily set uh set up specific attachments as you see here with the parachute wingsuit underneath the hood of this process it detects that which rig is active and then when the attach function is run just sets up the active rig accordingly further thanks to the power of space switching once attached you can quickly set specific as to how you wanted things like handles or buckles to follow the character now ikf case and space matching are pretty standard these days so what we set out to do instead of reinventing all this was to evaluate and refactor the way that we ran these tools to create quicker times for both running ikfk matching and space switching single frame matching almost happened instantaneously now and if you want it

to match for the entire timeline it really only took a matter of seconds now something that was introduced back on jc3 to our rigs that further enhanced quicker speeds of space uh and further enhanced by the quicker speeds of space matching was what we called the driver root control you can see it here being that giant brownish colored control above the character's head when it's not selected um that is controlling the entire character now say you want to animate the entire character here dangling with both arms and legs and ik you can easily place the driver route to the position you want to rotate them set the space for hands feet and the cog control to this driver root animate it say tilting left and right and then space match all of these controls at once back to world space where you can then polish each one individually i kind of like to call this a locator on steroids because it functions the same way that many animators do like to use locators to bake and transfer animation but having it built into the rig and ready to work with the space matching gives you immediate out of the box functionality the second you reference a character into your scene we also introduced rotation order

matching the rigs themselves were set up with best practice rotation orders on all controls but adding the ability for an animator to change the rotate orders or orders of the controls on the fly gave them even more control we also gave the animators the ability to create custom selection sets on the fly that were also simple to make hotkeyable multi-select the controls you'd like to group together click create selection set and that's it now you've got a custom group of controls that you can grab instantly even more power was granted to the animators to be able to set up their own custom pick walking for anything on jc3 we did have pick walking on the character rigs but that's it in order to set those settings we would have to hard code it into the rig this time around the character rigs did have default pick walk settings but the animators could customize this any way that they would like and also add pick walking to whatever else they may choose

we also added a custom scripts tab to the toolset that tangled mess of third party scripts and snippets were turned into official unofficial scripts the tas can add third-party tools officially and per force for deployment and animators can locally add anything that they want to their maya scripts folder and then magically populate that tool nice and neatly here there's no need to override or create a custom user setup file it's all here for you to pretty much use out of the box all those random mel and python scripts that were now nice and neatly organized also were also labeled in a way that anyone could see and understand all custom scripts have an auto create shelf button the ability to hide and show what you would want just like our official tools

in a customized help popup that includes a unique command which through the use of metadata create custom runtime commands that when sourced when sourced to allow for the animators to easily apply to hotkeys and marking menus now i want to show you one specific example of workflow improvements that actually took away the power from the animators but it was not power that they really wanted so here's one of those on jc3 anytime we wanted to grab footstep data from a file we needed to have an animator go and manually tag anytime a footstep occurred they had a tool that easily allowed them to scrub through an animation and click left plant right plant left plant right plant so on and so forth you can see here that the red and blue foot placement markers that are getting set for all the frames where a foot will hit the ground while this is a pretty simple and easy process to do it's really monotonous and it takes up a lot of time that's time

that takes away from say actually animating on jc4 i wanted to take this out of their hands and automate that process completely all it took was a pretty small script now don't worry about trying to read this i'm going to break it down for you i just wanted to kind of give you an idea of how small that script actually was the first thing that we need to do is find the distance between the ankle joints within consecutive frames for example let's say you're looking at a run cycle and the left leg of a character is swinging back down to hit the ground on frame seven you grab the position of the left ankle joint in world space then you grab the world space position of the left ankle joint on frame eight you can then calculate the diff dis you then calculate the distance apart that the ankle joint has traveled from frame to frame here's a simple function that can return the difference in distance of two points here we're just saying that we want to evaluate the left and right foot joints and then depending on an animation being a walk or a run there is a predetermined threshold that

is slightly different based on the fact that the distance each foot moves depending on the gate of the character locomotion is pretty different if you look at a run cycle the ankle joint will have a further distance between frames than a walk cycle so what did this threshold actually do we know that when the foot is planted its position doesn't change a whole lot from frame to frame this value simply says do not bother even looking at anything greater than this it was sort of a hacky but efficient way to filter out any non-obvious any obvious non-planted values as sometimes those can otherwise mistakenly get tagged by the character's leg movement when not planted now here's the bulk of the script that takes these factors into account we analyze twice here one time for each foot the analyzation goes through the timeline finds the current value of the world space position of the foot on each frame and compares that value with the world space position of the foot from the previous frame if a distance value is less than the

frame before as well as less than the frame after that we tag the frame the foot is planted by tagging the smallest value of movement in a particular range of values we assume that the script has found a planted foot for walks we also added a filter that made sure a foot wasn't too drastically rotated up and down or twisted since the slower nature of walks meant that the distances of the position of the feet weren't as drastic from frame to frame so surveying the pitch and roll of the foot gave us another good indication that a foot was actually planted and flat on the ground another really nice thing that we could do from here is take the footstep data and depending on if an animation is a walk or a run determine when you'd hear a cloth sound the character almost always hits the point where their legs cross each other after a step for runs this was two frames after a step and for walks it was five thanks to this consistent behavior we can automatically gather cloth sound triggers and not only are we gathering footstep data but cloth sound data this

made the audio team extra happy as they do not have to worry about manually tagging all of this in the game they were already benefiting from the automatically tagged footsteps as was vfx as well so hey we just made the animators the sound team and the vfx team pretty damn happy to the surprise of the animation vfx audio teams and myself this method had a really good success rate i'd say about 85 to 90 of footsteps and cloth were tagged correctly just because of this the rest was manually fixed as sometimes extra footstep tags snuck their way in or a tag here there was missed but the amount of work to do that small bit of cleanup was a lot less than actually going through every single animation and tagging those footsteps by hand now let's see this in action here's a run and by quickly scrubbing we can see where the foot is planted the right foot there is planted on about eight or nine the left on about 17 or

18. write on 28 so on and so forth now we run the script and it cycles through the animation as it evaluates each foot when it is done evaluating we get our auto generated footstep tags let's actually check and see how accurate this really is we got a right foot step on eight or nine a left foot step on about 18 or 19 a right on 28 or 29 a left on 37 38 right on 47 48 and a left on 57 so it's pretty accurate and just for fun let's evaluate four feet and run this script on a llama which is the newest of the animal heroes in the just cause series all right here's our auto-generated data now

there's a right rear footstep about six or seven there a left rear about eight or nine right front about 11 and left front is around 12. all right that's pretty good oh quick moving over to mobu now here's what we had custom on jc3 yeah nothing custom while we did have a way to send motion builder uh maya content to motionbuilder and then motionbuilder to maya that entire process on the motionbuilder side of things only involved making sure that you save your motion builder file with the take that you want to transfer over to maya saved as the active one the only maya specific tools that we really had were just the maya tools that grabbed fbx files and transferred animation onto our uh that grabbed the fbx files and transferred animation over to our maya rigs or would bake down an export animation content in maya out of fbx our goal for motionbuilder was to

actually make some tools now we did actually have some animation tools that my counterparts in sweden had developed over the years but there was nothing that existed for project workflow project and workflow specific needs for us on just cause 4. so now we're developing actual motion builder tools and allowing animators to use it as more of a standalone tool we wanted to extend the default hik rigs to allow for more from within motionbuilder in terms of things such as retaining root motion data weapon attached data and any other reference aside from the default hik controls we also needed to drastically improve the process of sending content from motionbuilder to maya it became necessary to send multiple takes from within a motionbuilder file over to maya which then handled animation content on a one animation profile structure in addition to multiple takes we also had to send full animatic sequences full cinematic sequences from motionbuilder to maya just as we have to do the same for maya out of motionbuilder now i'll detail that a bit more later when i actually talk about cutscenes what we had on jc3 was the basic default

hik rig with no additional features what we needed for jc4 was to add extensions to the hik rigs so that we could pull that data retain it and modify if necessary we actually start off with maya we actually start up in maya with the apos character t-pose it and characterize it now this is our base to work on from here we can either build a maya rig or an hik rig from within maya with the motion builder rig we actually took that and brought it into motionbuilder as an fbx and that fbx that was generated that was brought into motionbuilder has sort of an auto build function that we run in motionbuilder that creates the rig extensions to things like the root bone there that you see it's a yellow arrow and a bunch of key reference joints possibly you can see here i put a cursor on the the right hands plus sign cross looking thing there's a left side one and there's one on the chest for aim target where animators could uh animate the motion data for where the character was pointed so now they actually had controls on these things to animate those in motion builder and it's easy to actually port

that data over to maya now the team has a custom motion builder rig that they're animating with now that the team has a custom motion builder rig that they're animating with they need to be able to push that content over to maya on jc3 animators would have to save their motion builder file with the take that they want to transfer to maya as the active take since animators were utilizing mobu much more this time around and especially for the animator on our team that only used motionbuilder this really wasn't going to cut it some files could contain as many takes such as this one so we needed to make a way to quickly get any and all content out of motionbuilder we developed this ui that allows you to pick and choose whatever animations you want which then get plotted down and completely stripped down so that the result is a simple skeleton that is very abe that is very simply able to be quickly brought into maya and transferred to the maya rigs now it may not be a ton but these changes did make motion builder much more accessible as a standalone tool for the team also i don't have it listed i don't have it

shown here rather but i have it listed there's a batch tool for the xs content to bring into motionbuilder but it's literally just a button that kind of asks what directory you want to import so for sake of visually not being too pleasing or interesting i'm just listening here to tell you that we did have a batch way to bring in xn's data to our animation or to motion builder rather now this section isn't quite as long because as the others because it was at the core but it was really at the core of everything uh i'm gonna restart that sentence this section isn't quite as long as the others but really at the core of everything was making everything quicker and more automated for both the tech animators and animators this batch processing tool became the central core to running heavy processes into anatomia everything was streamed through here the way this works is that the user selects a batch process from a list of json files that we've made for them this could be an animation export or a mobile to maya transfer when they select a file any options pertaining to that file show up on the

right the user can tweak those settings as necessary when you load a script any directories with that with files that pertain to the script also pop up on the left when you select a folder the files within that directory show up for you to multi-select and run the batch on when you run a batch a tab pops up with the ui that runs a maya standalone thread and batches your content headless you were able to run different processes at multiple times at once without ever actually opening up maya this is optimal for computer performance and speed of data processing it's also very optimal for being 100 animator friendly as they do not have to worry about anything from command line and could potentially customize these batch options themselves now quick look under the hood we have a very simple json dictionary where we state some rules and list out all of the options for the user in this case we're looking at our my animation exporter we tell this file what file types to look for which directory to look for them in which batch process to actually run

the version of maya standalone to use and all the options that we want to list here now using as an example animation exporting these options let you create things such as toggles for maybe turning on or off the generation of a debug animation file or a line edit for customizing what you want to call the perforce change list that all your animations get added to we set the type of ui widget what to label it the default behavior and the command that we will use to pass the state of each option through the python batch file when running the process which then then magically populates their your options uh here as seen a minute ago whenever you hit that select select and whenever you hit that select batch script option you can choose your specific batch you actually see it that here i have a few batch scripts uh that i've set up for myself some of these are actually official scripts that were checked into perforce some really aren't there's like a cutscene anime export 02103 here that were just kind of local things that i

quickly set up for testing but essentially you can either do anything for yourself locally or do something you could check in and allow animators or any other person to use so in the end thanks to this nifty batch tool not only was i able to set up anything for myself really fast to start churning out lots of content through headless maya it opened up a ton of time-saving possibilities for the animators too and not just for exporting animations in fact even though our pipeline called for content to be passed through maya before export to the game our motion builder only animator used dispatch tool in conjunction with the fbx takes batch exporting tool that we had within motionbuilder and he never once actually opened up maya not once we just had the batch script set up for him to select all his fbx exports here and then the tool would go through each batch each file throw the animation onto the applicable rig and export that to game and now just with a few months ago motionbuilder 19 coming out with a headless mode i plan on actually upgrading this to adding headless motion builder into the mix a world where we're pushing motion

builder content to and from maya exporting it out to the game all without the need of having to open up any visual dcc is really a world that i want to live in all right onto cinematics we had two full-time team members as far as managing the actual animation and camera content goes one of these was myself and the other was our cinematic artist in the end we had about 56 unique sequences this number also doesn't include our single camera single cut in-game scenes that were used to quickly set up missions our smallest sequence had about five shots that made it up with our largest sequence having up to 60 cuts while all characters and cut scenes were bipedal humans we had 12 unique rigs of various heights and sizes to manage and make sure that they were correctly used in shots between all of these characters and sequences we had about 65 minutes of actual cutscene footage

yes sorry that was a phrase in the question we did have 65 minutes of cutscene footage um now actual hands-on production for cut scenes didn't even start until about halfway through the project and we literally didn't have actual content to work with until about six months before we went gold so that's looking at the process this may not seem too daunting when you look at what we're working with because it's fairly straightforward on the surface in creating jc4 like any of our other aaa projects at avalanche studios we work with outside vendors for all motion capture and content solving when it comes to cinematics to start the process we have a narrative team at the studio as well as a cinematic artist who worked on the story and dialogue which then those ideas were transferred into storyboards and animatics to try and lock down things like pacing and camera work once this was nailed down and our actors signed on we had a shoot the shoot involved both our avalanche studio cinematics team and our vendor working together with the actors to best replicate all the factors that our animatics had tried to prove out

when the shoot was done the vendor provided us with all the takes that were shot during the shoot and our team at the studio went through and selected which content that they wanted to use and have the vendor solve for us after this there was some back and forth between us and the vendor to make sure that our character rigs were set up nicely for things like good proportions based on the active proportions and then the content was solved in our rigs polished and delivered to us as baked down skeletal animation in the fbx data format each shot was delivered as a separate fbx file with camera characters vehicles weapons and props for that shot the content was then brought straight into maya transferred to our maya riggs and then exported to the game from there so we're done right in a perfect world yeah but making games is hardly a perfect process right going back to that gameplay pipeline we were already working with uh we had this pipeline once we started getting cinematic content from our vendor

we added that content in a similar fashion to what we had already done with the gameplay content this transfer is quite easy to do we set up a batch script that takes all the delivered baked down skeletal mocap data and since the skeletons are a perfect one-to-one match with the maya control rigs we transfer and bake that animation over quite easily now of course since we're making games random emergencies or unknown needs may come from out of nowhere needs that with the luxury of time in a larger team can sometimes be met handedly but time and numbers were definitely not on our side so yeah it did not really please us to go this way but for the sake of brevity i'm not going to go into a postmortem right now pertaining to this issue i will say that as a technical animator my most important role is that of a problem solver when working out any issues the first thing you do is assess the situation and come up with a viable solution based off of that so going back to our cinematic content integration process

we needed to be able to do this after we got our content into maya on our game ready rigs we needed a way to get our cut scenes back into motionbuilder to allow for dealing with changes tweaks and critical fixes or updates now some of you may be asking why would you actually go backwards and i get that it's not ideal it's kind of why i paused a little earlier but simply put our cinematic artist who had all the camera work sequence edits and any cut scenes set up was 100 a motion builder guy 100 just like that other animator this guy did give his best try to learn maya he really did give his best effort but the amount of time that it takes to make edits using the tools that motionbuilder has not only mentioned the great frame rate that it gives you for playback uh remember animation caching and maya just happened a few months ago we really couldn't do what we needed to do without being able to take our maya files with that solved motion capture data from the vendor put it back into motionbuilder and then send that back out to maya for final approval and export to the game once any new edits and changes were made

so in order to best explain how we tackled our cinematics workflow i want to break down one of our cut scenes that we had put through its paces so let's start by watching the full cut scene load up the truck we're leaving in 60 seconds did you organize this ambush see most of the stuff you recovered is standard black ammunitions but i also found some of these we're not sure what they do we didn't want to mess with them um there's a bunch more left in the truck good fine so gentle this could be useful they won't have a screwdriver are you adding that to your grappler no hold us you have to make do with whatever you find what are you going to

do with that [Music] listo i'll drop the truck i've got a chopper nearby waiting to extract this make sure you get there in one piece we're moving out all right so that cutscene shows our main hero rico meeting with one of his supporting pals sargento who gives him the part that he adds to his grappling device that allows the player to use the airlifter in-game to pretty much turn anything into a giant flying mass this particular cutscene has a good mix of characters vehicles weapons and props so we're managing a lot of different types of content to start let's look at the process that we need to take in order to get our maya content into motionbuilder we have our maya files shot by shot but motionbuilder doesn't really work this

way we needed to take these individual shots and transfer them into motionbuilder in one long sequence so that they can be set up properly for editing in story mode so i love json i use it all the time to store data settings for anything and everything i can it's everywhere in our pipelines it's probably in yours too even if you don't know it it is at the it is the base at which our cinematics content workflow is driven using the power of json files to organize all of our cutscenes was the solution that not only bridged the gap between maya and motionbuilder but it organized our content in a way that's really easy to parse and locate with a simple call every cut scene had a master json file for the cutscene data which included the scene name the campaign that it was part of all characters props shots vehicles and weapons that appear in the entire cutscene at any point

then we had a json file for each separate shot that told us the specifics for that particular camera cut we again have the campaign and scene names the shot number the characters that are actually in that cut the props in that cut vehicles in that cut and weapons we also have the start and end frame numbers stored now just to better illustrate what this means for us let's just take a look at a few select shots within this cut scene in the first shot we have just rico in this dirt bike since there's nothing else in the scene uh since there's nothing else in the scene that isn't already part of the environment only these two items are stored in the json file for this particular shot the second shot is only rico so he's the only thing that we store the third shot sets us back to a wide angle that reintroduces the dirt bike and brings trajento into the scene with those two trucks

i'm going to skip over the fourth shot to the fifth one since this shot includes a little bit of everything um all four characters are seen as are three vehicles you can actually see the motor the dirt bike a little bit behind that tree um it also includes a prop which is the crate in the back of the truck and if you can't tell the rebel there is holding a weapon in the middle now in an effort to not completely force you to look at code here's what this looks like in a more visually pleasing way because we have all this finer detail stored the way that we do i don't actually have to open up maya to find out what is in any given shot which i'll say is even far easier quicker and nicer than trying to find this stuff on a spreadsheet i can also run a cutscene export headless and our batch tool and filter it down to only say sargento because maybe we made a small rig update or added a secondary animation pass on his jacket this batch process that we set up will parse every json file for the scene and instead of wasting time opening shots where sargento doesn't exist to

check that he's there it will immediately go straight for only the files that he's in for a small team like ours this time savings when you think about the number of characters scenes and shots that we're working with was a huge huge win now this also made it easier to shot by shot deal with our maya to motionbuilder transfer now let's start our content passing process on the maya side it's time to take our entire sequence that is caught up in the maya that is cut into multiple files in maya and stitched that all together into one giant sequence of content so that is mobu friendly for a cinematic artist to use with the story mode and camera switcher a short look into this part of script that does it for us i promise i'll go quick um when we run our process we open a fresh maya scene let's call this the master file we immediately start at negative fifteen thousand for a frame this just ensures that each individual shot is having its contrast transferred over and it doesn't trip up the shot before it before this actually though the json file for the whole cutscene itself is red and all cameras characters etc that

exist are referenced into the maya scene and now that everything exists in the maya file that we need we loop through all the individual json file shots referencing in the corresponding maya file shot uh and adding all the animation data for characters weapons etc sort of in chunks so if you were to transfer that data over one by one you would kind of do it with like shot one goes here shot two goes here shot three goes here and it's kind of stacked one after the other once all the shots are fully transferred into this master maya file there we go once all shots are fully transferred into this master maya file all keyframes are grabbed and moved up to frame 101 so we're not dealing with crazy negative space on the timeline you could easily just make zero your start frame but i personally like to leave frames 0 to 100 open in case any necessary padding might be needed now as you can see here though we have all rigs cameras

and everything else kind of nice and neatly spliced into one now there's another script that actually bakes all this animation down cleans up everything and only makes sure that we have skin characters and baked down keyframes on the bones once this file is exported to fbx it's ready for some story editing and motionbuilder so picture this the maya file you just saw was baked down transferred into motionbuilder it's made its way to the cinematic artist this person's made all the necessary edits to the cameras character positions whatever this person goes up to you and says brian it's ready so now we're there i just wanted to skip the whole part of his editing here's that motion builder file since we're ready to send it back to maya we need to split that up every we need to split up every single one of these individual clips that exist in story

and then we need to export the mountain to each individual file into individual maya files as per our pipeline in order for us to finish off the scene and get it in game in order to do this we need to run some commands in motionbuilder that export out data that can be brought into maya quickly and with little to no load times using the send to maya command or opening an fbx straight up into motionbuilder means that you send everything all at once into maya and it's not necessarily the cleanest result so we don't really want that file sizes get really huge as evidenced by the fbx scenes here on the right and these can take forever just to open up in maya or motionbuilder even to avoid these long and arduous transfer times we break down the motion builder scenes for transfer then fit them back together in maya not only do we have to break each of these clips down and export them separately but we need to make sure we do so in a manner that doesn't export out large file sizes since we're getting close to the end here i'll spare you more long code explanations and just sort of briefly into this process

we first create a component list for everything in the scene then gather all the start and end frame data in each lit in each uh in a list for each shot then we plot all animation to the skeletons and once we've plotted we grab the root joints of all skeletons using our component list to search from then we iterate uh through all of that making sure to grab the entire hierarchy of each individual skeleton as we create clips per character and export out a separate file per character per shot our results look like this and include file sizes that range from mere kilobytes for cameras and anywhere from about a half a megabyte to 10 megabytes per character depending on the complexity of the skeleton and the length of the shot rico would happen to have the largest files with background characters having less for example this is much much more manageable now than those giant motion builder files that you found these files truly contain just skeletons with baked animation data sort of like what you saw before when we were exporting our fbx's from motionbuilder into maya

for the gameplay side they do look exactly the same as that and shot by shot we can now open up a fresh maya file referencing only the rigs for the assets that actually exist in each specific shot and it only takes a matter of seconds to reference these small files in which we then transfer the animation onto the maya rigs from which then kind of obviously takes a bit more than just mere seconds to do but it's a lot easier to do it's all stream through the batch tool the transfer script for taking those exported pieces from motionbuilder does all the referencing baking and file cleanup in a snap we can process multiple shots scenes whatever much more quickly than by using the default built-in transfer setups i did mention earlier that maya was necessary for secondary animation policy i believe and here's why one awesome content tool that we had in maya was the motion generation tool it could use a number of different solvers from maya dynamics such as spring or bounce and then take in some user settings and process that based off of the root motion of the characters

we could batch through all uh through entire cutscenes in maya and run an auto secondary animation patch on all characters very quickly now i think our first pass maybe took about a week to add secondary animation to all those cut scenes you can see here rico's straps are not really moving mirrors here nope now it's moving because we just applied the motion generation to it all right the extra special thing about this tool is that we could run it through a playblast script that we had in conjunction with batch and we would actually get a directory of mp4 video files for each maya file shot to see the the secondary animation after the fact this was super useful for going in and making sure that certain shots might need to be fixed up or certain shots were fine after a run to wrap this all up what we were able to accomplish included better workflows more power and customization a lot more automation and less manual work a full 360 degree pipeline between maya and motionbuilder and all this helped us churn out a ton of content that we wouldn't have otherwise been possible unless we had a small army

to help us now while there are other studios that have similar tools and workflows i do hope that this talk at the very least gave a new perspective to those of you that are familiar and for the rest of you i hope you've enjoyed a bit of the behind the scenes and under the hood that i have presented before i really conclude i do want to extend a huge thank you to these three guys here who all help me in some way um brad clark he's the co-founder of rigging dojo he's there on the left he was always available to help uh answer any motion builder specific questions that i had as i was pretty new in setting up more advanced features of motionbuilder um he still does he's really helpful and you can probably ping him on twitter and he would be happy to talk about motionbuilder with you he actually made a quick video demo for me just on how the camera switcher worked out of his own time it's like a 15-minute demo and it was super helpful um john malaska he's in the middle there he actually gave a talk this past monday at the animation bootcamp about freelancing for animators so check it out on the vault if you aren't there um he came on board actually for a few months when i was cold alone in desperate need of help um he really made

it he was really an integral part of a lot of the motion builder setup that you saw especially the exporting take system and kind of working with animators on where workflow worked now finally cole there on the right he was responsible for many maya optimizations including a bunch of that custom script stuff that you saw that's really really cool and that motion generation tool also they aren't listed or shown here but my animation team's awesome despite all their crazy crazy demands they truly make me a better ta so i really want to thank them and also thank you to the conference associates in the back who have helped out if you see the red shirts thank them this really wouldn't be possible without them all this organization and stuff that none of us really would want to do they do for us and it's super awesome um and hey thank you to you so much for coming to my talk today um looks like we have about six or seven minutes for questions if you want to ask any questions or you can

just see me outside of the hall i believe across the way or something there's a overflow room somewhere but yeah follow me on twitter i'm at techanimator my email brian techanimator.com yes i have all those um so yeah thank you so much i was wondering it's very low i was wondering if you had a headless version of motionbuilder as well or uh we do now i mean we're looking into moving 2019 but it's literally headless motionbuilder just started happening like a month ago okay and also um did you have something to generate all your json file for for your cinematic stuff um we got json files from the vendor at first but we actually did have to do uh quite a bit of just kind of error

checking and manual work just to make sure that they they kind of were set up properly but kind of once you set those up manually the first time you're kind of good and it was also a really nice way to sort of catalog and understand what was in those scenes by hand okay and last one uh what's the reason for having different files for all the shots in the cinematic for the json uh no like in your engine you have like different maya files for each shot and couldn't you have like one big animation for oh that was more done just for frame rate reasons in maya because when you have content that's like a five minute cut scene you really don't want to work with you know multiple characters and all that when you have like a giant maya sequence of say 8 000 frames so we just kind of broke those down into quick little files that you could kind of go in and update one the idea between that was that if you're in maya you're in the polish phase and you're not changing your camera framing or anything like that and you're literally just polishing content okay thanks

hi azo talk and i just want to ask and the name of the extra tools you use in maya from rigging from sorry rephrase that question the extra tools you use in maya for rigging for rigging yeah oh we build our own what we build our own we have all auto rigging tools that we build in-house so everything is all python scripts that i sort of started from just cause three just from scratch so it's all automated stuff we have a bunch of scripts in house and we build blueprint files using json as well actually where i just kind of state this is the name of the control this is the size this is a color sort of everything offsets all that and then we just run that blueprint and it just automatically builds our rigs and by any chance can you send me an equivalent equivalent in commer and right and to build rigs yeah uh equivalents to build rigs right now um i know a lot of people use something called m-gear um which is uh pretty robust um

i can't think of anything else at the top my head right now all right all right thank you very much yeah thank you anything else all right that's it thanks