Thwips and Hugs: The Animation of 'Marvel's Spider-Man 2'

[Music] hello everybody um my name is Robert codington I'm a senior animation director at uh Insomniac games um and the title of this name by Lindsay is uh thps and hugs and it's very appropriate because those are two things that happen quite a bit uh when you work with us um but yeah I'm an animation director I'm not the only one or no that's just more more me there's uh another uh animation director and I'm going to talk a little bit about the the structure that we set things up and you'll see why in a second um so Danny can't be here today but he should be should be right there um so the way our animation team runs is uh broken into uh very specific pods um these each one of these uh pods will run for pretty much the duration of the product uh having anywhere from three to five people per strong

leader um so then that would be traversal combat boss encounters missions uh Living World and then story cinematics is that's a really big team so it's kind of two pods even though I only made one icon here just go with me on that cool um so yeah like I said in each one of those pods you'll see that there is a very strong leader in a group that they work with um and then today uh I wish I could bring all of them here today that would be the perfect thing in my mind but uh we have four of them so uh Stephen Cella James ham Lindsey Thompson and Bradley mclin and they're going to uh tell you a little bit about what their life is like as an animator and as a uh animation lead or associate director at Insomniac games uh in front of these pods and the first one up is Lindsay Thompson all right um hi uh my name is Lindsay Thompson and I'm an associate animation

director at Insomniac games for almost 11 years now well not in that role but yes I've been there 11 years uh and my focus on Marvel Spider-Man 2 was our main story cinematics one aspect of that included the logistics behind bringing accurate representation of deaf characters and ASL to the game uh these cinematic shoots that we did and the construction of the scenes worked a bit differently than our typical CES so some of our key pillars for doing ASL work was uh we had ASL consultants and interpreters we had deaf actors and body doubles on the mocap stage um a different and more extensive animation polish work uh Pipeline and then we also had playable Haley who is our deaf character in Spider-Man um so our ASL consultants and interpreters uh Josh and Jules were our Consultants from hypernova they're both

deaf and work in entertainment to Advocate educate and consult on ASL they were there for any of our questions regarding Deaf culture consulted on the scripts and how to interpret them into ASL and they worked with us on stage to make sure performances were clear and then worked with us to go over every scene and shot that had ASL in it and make sure that our signs were correct and clear so for each hearing cast member we hired a deaf body double we made sure we hired diverse role accurate actors to double as our main cast here you can see some of our deaf performers uh to the left there's uh Justin Fulk is in the foreground there uh he's miles is double John Kim uh in front of him in the middle is uh for our gany double and of course we have Natasha oili who plays herself as Haley our main cast Naji Jeter and Griffin Pau are to the right so we had to figure out the best way to shoot our hearing cast spoken

performances along with our deaf body doubles that would work best for Natasha to play against so for this we ran them side by side so Natasha had signing people to play off of but could still see Naji and Griffin doing their speaking performance each de cast member had their own interpreter so we could all communicate on set and we also had the hypernova on stage to help interpret the scripts with the cast and plan out the signing performance we ultimately determined that it really depended on the each scene sometimes it made sense to have all of our de cast standing and playing against each other like you see here but other times it was more important to have Natasha face to face with Naji while his body double Justin would sign next to or in front of him so Naji could match his performance so in some case uh in some cases our casts were even able to sign themselves on stage seeing uh seeing as they were playing a more inexperienced signer it was authentic for them not to

be signing as fluently as a deaf person would we had to consider each character's experience and knowledge with ASL so gkey uh miles's best friend is working very hard to learn miles is probably the most comfortable and experienced in speaking with Haley Rio miles's mom is clearly just beginning to learn as miles has just started dating Haley spoilers um so we made sure her double didn't sign as fluent unexperienced so for example at the end this is a spoiler sorry if you haven't played the game uh at the end when Albert Moon says nice to meet you in ASL uh we approached it as though he had literally watched a YouTube video before coming over and saying something to her so our interpreter was like well that maybe wasn't quite it but that looks like somebody who's just trying to say a nice phrase to somebody so um yeah it was really cool

um okay so here so in some instances uh our main cast like Naji and Griffin were able to perform their own signs but for more longer and more complex conversations we relied on our DEA performers to make sure everything was accurate so you can see we had cameras shooting from all angles on stage which became important for animators during polish work here we have Naji capturing the emotional performance with Natasha um oh sorry sorry sorry yes uh we wanted them to connect so behind Naji you can see her interpreter is actually signing out to Natasha what Naji is speaking and Naji is facing his double Justin uh so that his voice can match the Cadence of the signing it took us a minute to find the most elegant solution for everybody's comprehension and the best performance uh and you can also see our consultant in pink uh behind Naji uh making sure the signs were clear coming from Natasha and

Justin uh you may also notice that normally map gloves don't have as many markers on them it's harder to capture intricate finger movement most of the time and we often key frame our fingers after the fact but for our ASL scenes we added extra markers to our signing actors so that we would start off with a better more accurate data so once we shoot the data for our scenes we then move on to layout our animators do everything they do cameras layout polish key framework we a department full of directors so this scene I personally took on myself so I was able to direct it on stage bring the data into Maya and set up the cameras do the layout and edit and I actually polished it myself so combining the two performances this was our final result in this particular scene we layered Justin's arm and hand movement onto naji's body and face performance so after laying it out and getting data combined we finally work with the hypernova to go over this performance and video footage so that we

can get clarifications on any signs that may be confusing we would often go through it sign by sign and there were many times our animators had to go into the performances and add or remove signs after the fact to make it clearer this one was really wordy so at times I had to hold the signs for an extra frame or two to make sure that they really stuck and that you could read them but without feeling too staccato and as much as the additional markers helped on stage I would say that we definitely had to touch or improve nearly every sign in the game particularly when there was quick finger spelling uh finally just a quick touch on our playable Haley Mission our designers and writers put together a truly unique experience as a deaf character from the audio design to the emotes to the text conversation they found unique ways to interpret emotions and a story without a voice and bonus points uh there's cat on this Mission so it was really awesome and it's one of my favorite missions in the game look at it um

so thank you so much uh I'm going to hand over to Bradley next to talk about it more thank you Lindsay hello I am Bradley mlin I was an associate animation director on Marvel Spider-Man 2 doing cinematics along with Lindsay uh today I am going to be talking about our process of shooting performance capture um on Spider-Man 2 but more specifically the benefits of having our main cap and stunt actors mirror each other on stage why does it matter um keeping the timing and voice acting perfectly in tune with the body performances is a crucial aspect of putting a cinematic together um how does it work running stunt actors alongside our main cast keeps the performances synced and makes them feel more grounded and authentic here's an example of our stunt actor Seth Austin mirring um sorry uh mirroring our Spider-Man SL Peter Parker

actor Yuri lenthal thanks T miles I've been calling you for hours nothing you know miles has been hunting Lee I'll find him I'm calling the hospital I said I find him I've got this under control all miles talks about is how to be a better Spider-Man not school not his his friends how to help you go help him so the challenges of shooting superhero performance capture uh superhero actions frequently exceed the physical limits of our main cast but we still need to synchronize all performances uh running two performance two performers together introduces a greater physicality

provided by our stun actors and allows subtle performances provided by our main cast uh the best part is everything is synced making it easy to swap out the performance and keeping the timing of our Video Edit let's find out then and here here it is in game let's do this my way this also leads to another Advantage animators get to choose for multiple performances uh sometimes an animator might want to choose from the options I mentioned in the last slide uh you might want the physicality of the stunt actor gives you or sometimes times you might really like the subtleties and

performance of the main cast uh another benefit is we can shoot more uh scenes throughout the day not having to break up stain uh the stunt and main cast performances here is an example where we ran sorry just one second yeah there we go here's an example uh where we ran the stunt actors doing um fighting choreography then had our Craven actor Jim Peri um do a head slam followed by the Sun actors doing a more fiscal uh take of the head slam uh so as stated before uh running stunt and voice actor togethers creates a seamless performance uh that is easier to edit and and what makes this possible

is collaboration um when the stunt and main cast actors collaborate it uh helps each of them become even more invested in the performance and creates a more grounded and authentic character uh the collaboration on set is it's truly amazing um it it's really the best part of working this way the collaboration on set is uh again it's amazing and it really feels like we're all moving as one unit when it's time to roll so uh shooting this way can bring up some challenges one of the main ones being audio interference uh there's several ways we can work around this um we can Implement wild takes which are real-time playback of the stunt performances that the main cast can do voice over on the stage we also uh can just shoot ADR in the booth if needed and we can offset the performances between stunt and main cast uh while still getting everything in one take and here's an example of of that uh offsetting performances so we're

going to have the stunt actors go first then we're going to have the main cap uh main cast uh go and we'll capture their audio facial performance uh and the body performance so I'm going to show a version of it on stage and then what it looked like in game M cast Harry Harry not as venomous as they say and here's what it ended up looking like in game

not as venomous as they say all right that's it for me uh thank you for your time and I look forward to your questions up next is Steven [Applause] Cardella thank you Bradley uh hello uh my name is Stephen Cella and I'm an associate animation director at snc games uh I was in charge of leading our boss pod during Spider-Man 2's development uh and I wanted to give you a quick behind the scenes look at one of our more complicated boss encounters our giant Sandman during the opening sequence of uh the game uh so fittingly with a big enemy come bigger challenges we'll start off from the very beginning with a glimpse into some very early exploration tests we did with our giant Sandman uh this video shows some really early uh gam plan and cin itic spectacle moments as well as a quick

gameplay test we did with our giant Sandman moving around in the city and what it would feel like to swing around him shortly after we knew Sandman would be in Spider-Man 2 which was very very ear in production we quickly knew we wanted Sandman to be absolutely massive in our Spider-Man 2 encounter obviously this would bring with it some technical challenges we'd had to find solutions for we knew that Sandman was going to be heavy and fittingly slow once development and animation started we arrived at about a 20 to 25% speed decrease from the captured data to give us a good starting point for his movement we the speed on a case-by Case basis though attacks for example were given an exceptionally long tell window that were slower than most of his typical movement uh throughout the rest of the mission but gave us the breathing room to provide that adequate speed during the attack portion while still maintaining the weight that we needed uh for him during uh the rest of the mission once we knew it would be our opening Mission though it brought with it some additional challenges we knew the experience needed to feel unique but the controls needed to be familiar to Old players and teachable to new players

that also meant that the player would have limited abilities at their disposal so we needed to make this encounter work with basic abilities in combat with that ever uh ever important weight and slow speed that we were trying to hold on to we quickly ran into a pretty big design pickle Our Heroes even at their slowest speed from a brand new game could literally swing circles around Sandman and be virtually Untouchable after some attack tests of grabs and Jabs we knew large sweeping attacks that covered huge areas would provide real threats to the player but we still had to get a little a little bit more technical help with our amazing uh from our amazing programming team even with the massive sweeps that saman was covering the player could still easily swing to safety without much thought or even by accident to make sandman's attacks pinpoint accurate our programming team and animation collaborated on using a a Technique we call Focus tracking to slightly aim sandman's shoulder and ultimately his weapon arm at the player by subtly manipulating his spine and clavicle

during gameplay that was overlaid over our animation um I like to call this reversed sync aming since we normally sync the player to the enemy uh during synced moves uh but in this particular case we have Sandman focus on or aim at the player's sync joint and then capture the position once the attack portion of the animation starts if the player Dodges uh the player's sync joint which is sandman's current Target would remain fixed in space until the Dodge performance is complete uh this our low our Spider-Man to barely consistent barely and consistently skim over the weapon without colliding and giving them the dexterity only a superhero could have so here in this video what you can see is uh one of sandman's main attacks which is the the arm sweep and you can see a debug view of this this arm sweep attack uh the pink arm that you the pink Arc that you see is the damage Collision moving through the sweep range and the focus Target that's labeled on Miles and the end of that white line uh is sandman's intended target as you can see on the Dodge the white line pointing to

the intended target remains stationary on player input and throughout the performance of the Dodge all throughout the attack dodged or not sandman's spine and shoulder are being subtly manipulated to ensure accuracy to the given Target the other end of the white line uh uh that's pointing at the focus Target is at the start of sandman's manipulated joint chain which you may be able to see is about midpine on Sandman uh what this tells us is that everything from the midpine up to the clavicle is being subtly adjusted to make sure that that weapon arm ends exactly on that sync Point uh this gives you a general idea of how we handled most of of the attacks during the Sandman fight now that we knew that the player could get hit accurately and consistently we needed to ensure that the hero's reactions match the intensity and presence of our Sandman with Sandman being so large we had to ensure the resulting performance from a hit felt just as large and visal to really make the player feel player feel like there were consequences to their actions to do this animation programming again closely collaborated to employ what we called

super knockbacks these would launch the player far and away from Sandman with a quick recovery to keep the player in the fight this video shows a couple of variations of our super knockbacks these were intended to take into consideration the surrounding environment when Sandman landed a hit these knockbacks would Target markup placed in the environment to steer the player while flying back after a hit either into a building or free space if the player was within the target cone of the markup the player was then steered toward the target an animation would cover up the majority of the visual Oddities uh from any steering that we were doing through clever camera movement this video here is an example of a building super knockback that has our debug markup Vis visible to help visualize the system and how it works the small blue and green squares dispersed across the area are Target markup green for building and blue for open space the large red cone that we have highlighted is the target cone in the intended target uh each markup each markup target has its own targeting cone the large Blue cone is a represent

representation of the hit Direction and where the player would be expected to go or the acceptable steering angles to where the player could be directed the intersection of the of the hit cone and the target cones in the vicinity would determine the knockback target the player would be directed to there was still the very rare case though that these checks would still fail and the hero animation would ultimately fall back on a simple air flail hit react that would keep the player from unnaturally colliding into the environment this usually happened when the player was in in a position uh usually really close to close to a building where the trajectory to the intended target would have him actually penetrate through a building um it was still rare but we needed to have that in place just in case uh these super knockbacks ultimately fulfilled the fantasy of getting your block knocked off by our giant Sandman so we think we we hit the mark on that one uh we knew the Sandman Mission though was going to be huge uh both figur figuratively and literally but we had to keep in mind that this was still the opening tutorial Mission this meant we had to make sure the new a new player

would easily digest the control scheme and that they would be familiar for the rest of the game we knew we needed some basic inputs that would give the player spectacular results basic melee combat and basic web shot the inputs necessary would me Were Meant to mimic the standard web shot functionality as well as the Cadence and responsiveness of a basic melee combo but we also needed it to fit the encounter appropriately to match the weight and size of our boss that ultimately led the team to what we called the wrecking ball combo which would have the player use chunks of debris yanked out of sandman's face to pum Sandman during the boss fight cuz that felt pretty good there were several challenges we faced while creating the reing wrecking ball combo though we had to ensure the player could understand the sequence of controls as well as the actions being performed by the hero and we also had to make sure that the combo remained responsive and felt very close to our base melee combat due to his size we had to make sure sandman's blending was in perfect shape because any pops or misal alignment during these moves would be tens of meters would move tens of meters in distance rather than a few centimeters that we would get with the normal Siz enemy while in practice this

same principle this is the same Principle as Norm as a normal enemy making sure sandman's reactions didn't pop and the mov flowed Together made this a bit more complex than typical combat finally making sure the characters and props rign in perfect alignment uh while traveling through the air took quite a bit of tweaking to get it to work correctly we had to switch to the syn location mid combo to ensure the player remained in relative space to Sandman rather than being stuck to follow sandman's head reactions as during each move that would cause the player to shift back and forth with his head testing and feedback obviously were a huge part of making this combo work we had issues with combo with the combo being too slow with engagement distance as well as the tricky note of not making it feel like we were ripping sandman's eye eyeballs out of his head uh this that's ultimately though this combo uh these combos ended up working really well and gave the player an exciting payoff while still keeping our base combat control control scheme in play another goal that we set out to achieve early on in Spider-Man 2 was was to showcase the power of the PS5 and

what kind of Marvel hero spectacle moments we can achieve seamlessly during a 10 pole boss fight the giant Sandman fight seemed like an opportune moment to Showcase this it was early in the game and we knew we had to push the streaming speed of the SSD and what better way to do that than for saman to grab Miles throw him half a mile away through a building all the way to Washington Square Square Park stream some civilian and traffic data in and then send them all the way back into gameplay seamlessly our boss pod and design teams worked work on getting the sequence to function technically and our missions animation team stepped in to polish it up to a final product ultimately it was up to our VFX and optimization teams to handle this technical streaming the technical streaming aspects and put the finishing touches to bring this sequence home all the traffic and pedestrians seen in the sequence were streamed in on the spot using open World Systems this moment clearly showcased the speed the PS5 SSD could handle and also gave the player a try truly epic Marvel moment to witness all while keeping them in the action we hope this brief glimpse into sandman's development shed some light

onto the challenges and spectacular collaboration needed to bring our massive photo life while animation was a large part of it it was just as much a technical environment character Arts audio and visual effects effort to bring this massive boss to life thank you and now I'll hand it off James P hello thanks Stephen uh my name is James ham I was also an associate animation director on Marvel Spider-Man 2 and my focus was our mission cinematics and qtees so in Marvel Spider-Man 2 there were multiple types of qtees these are a few that we wanted to highlight there's the fail forward feed of strength and play Anam Feast of strength qtees are a staple of the franchise but we wanted to find new ways of incorporating them into the game so instead of just smashing a button we started using them by pulling joysticks and a spefic specific direction we also took advantage of the

new adaptive triggers to find The Sweet Spot and here are some examples of their uses so on the top left we use The Sweet Spot mechanic to take advantage of the triggers depending on the pressure applied um it would reflect in his progression in the bottom left we used the mashing setup to show progress as the player gets closer to completion now the ones on the right are a bit more complex the top right example has moving Parts on both ends they need to respect each other by staying in sync as well as try to show some progression as the player can feel the change over time

and the bottom right has one of our more complex setups as the coaster has multiple moving Parts people in the coaster uh our hero webs attached to the part to to the coaster on both ends so everything needs to stay in sync while also showing progress so lots of moving Parts just to get this set up that way people aren't floating outside the coaster webs aren't flying everywhere so we had to make sure that everything stayed in syn our fail forward term is one of our ways to have branching moments in the game going into any qte we ask the question what if the player doesn't press the button by failing forward we allow the player to continue the story without having them replay the same thing multiple times it allows the player to stay in the moment without breaking momentum and helps give the player an experience we would like for them to have granted it take a bit more time to figure out and create but it allow also allows us to tell slightly different stories an alternate version to move the sequence along so on the top left here we have miles

getting ready to get grabbed by the lizard and if the player succeeds miles is able to dodge land on the lizard's back and then continue moving the story if the player fails to do that lizard will then grab Miles drag him underwater but we still ended up getting into the same direction to continue to story forward then we have uh an alternate version of that which is just multiple setups where here the player is has the ability to grab arrows coming at him Dodge them then you have the option to use a firework and hit enemies with it if the player fails to do so Spider-Man will then get hit by arrows and then instead of using a firework he'll find another way to keep the movie keep the story going along this in this version just yanking the enemies down play anim qtes are one of our more powerful tools where we can do full anim on characters without branching to different shots this allows us to play

individual animations on characters while still maintaining a cinematic feel these are more complex due to having a lot of moving parts and here's another example so in this map shoot we have our actors the two on the top left of the platform are our Spider-Man miles on the left Peter on the right the person on the ground on the left is also miles and then there's another person on the right that's playing as Peter and the person on this back is our Venom this is spoilers so yeah um but in this sequence here you you're going to see our Spider-Man fighting Venom doing a combo so you can see we have a breakdown of Miles throws a punch Peter throws a punch and we have venom reacting the reason why we have people on the ground and we have two versions is that way we have multiple variations in case one data comes out a little bit better also our actors on the ground aren't fighting gravity and unnatural natur way so it might give us a little bit better data

to plug and play with which one's going to work the best in the overall scheme and then to finish that sequence off you'll see the same actors are in the same positions we have the two actors on the platform being Peter and mouse then we also have the alternate versions on the ground and we have venom in the middle and this is to finish off that sequence and it looks like a mess but the end result is go miles

all right and thank you so much for watching our presentation we'll uh now open the floor for [Applause] questions yeah if that wasn't clear we're uh we're open for questions to the end work hello uh we have some mics to this each side hey hello um I've got a question question about this QT I think that is like an incredibly elegant solution to like a common problem of that replayability did players notice that there was sort of this uh I don't have a more graceful word for it but like unfailable sort of thing and if so what was their reaction like to the sort of twofer is my mic working can can you guys hear me oh there you go check check cool there we go um yeah I think players started to notice we started implementing that feature in miles and

we noticed it when players started finding out that if you failed the certain qte when miles we call it a save the baby moment in the mall if you don't save the baby and you like go to photo mode and you look behind Peter saves the baby he gives miles like a little thumbs up um but I think players started to notice it and it's and it it helped it with with uh replayability where it's something new that you can find you can also see a different side of the story but also you're not stuck with the oh I failed it have to replay it again and then you're stuck doing that for 5 minutes and it just kind of helped move the story along so yeah it's less frustrating I think I think it makes game play and progression more fun yeah agreed thank you so much I think even if they don't notice if they're happy yeah if they're happy happy we're happy hi first of all thank you for being here I appreciate it my question was for the workflow with ASL do you think that this did the um people who helped you was this a tried and true method or was it more more of a work in progress I think uh I think it's

definitely a work in progress I mean it was we've had Haley in previous games and we were certainly developing a pipeline and a system for it then um but I think as we've worked more with our Consultants that we've done it more times thank you um uh we're getting better and better at it so even as the shoots went on during shooting our ASL we were able to kind of get up and running a lot quicker uh knowing everybody was coordinated we knew where which direction people needed to face where their interpreters were but it was certainly uh a learning process and something I I hope that we continue to improve on absolutely do you feel like you're going to increase more lip sync animation and I'm I'm deaf so to be able to lip read was a huge deal to me in the game because I was able to to not read the captions but look at the performance do you think that's going to improve I I hope so I mean we always strive to make

it better and better and that's that's something that did come up with our ASL Consultants it you'll notice in a lot of our ASL scenes we M we tried to make sure that whoever was speaking on screen was on whoever was speaking was on screen whereas with other cinematics sometimes we'll take more creative license to not show their faces um that was something in those scenes we tried to make sure people could read faces a little better thank you so much thank you hello thank you for the presentation I I have two questions um the first one is about um ALS as well and specifically about localization is this something that you ever considered um at least using procedural animations or something to kind of uh work with the alss in different languages uh because you know obviously we have the English ALS but also other languages and changes and how they work um that that something we learned as well that with different languages there's obviously there's

different signing there's black uh American sign language which is also different um the dialects there's dialects there's uh things like that we did have to stick with English when it came to sign language for our game but we in addition to subtitles we also had a pretty robust audio description system in the game um to make sure that we could localize that as well um but yeah we we were limited in terms of signing that would have been really a lot of work uh to do so yeah unfortunately we did have to stick with one language there but we hope that our subtitles and audio descriptions kind of backfilled that absolutely um and the second question I have is about cinematics and seamless transitions between gameplay and cinematic I I think Spider-Man is a prime example of really well done seamless Transitions and I'm wondering if you have any moap tips for how to shoot these and what your ideas

are when it comes to seamlessness hand it over to our cinematics uh gurus over here uh yeah um I'm not sure the most important thing you want to do is you uh in this position you're between your your narrative uh your writers and your designers and you're going to have to really talk to them and you're going to have to basically IM IM your way through that um where there's a lot of different sometimes you need to get rid of the hero and bring them back in for it to be seamless and then sometimes you're going to hand them off obviously uh I think at the beginning of every project we all kind of go they're got they're all going to be seamless all the time um into and out it's going to be amazing um it's so much cheaper to cut is so much easier to cut just show a reverse or show something else and have them walk back on screen and a lot of times uh we go to that because maybe it's the right call for the story or maybe it's the right call for our schedule um but yeah we we

do try to imagine um where what the needs are from gameplay into the Cinematic and make the most interesting and seamless way possible there's you know of course all different ways to do it where you'll you can funnel the player to a certain location there's ways that you can sort of tag the first shot to play Hero relative that one doesn't get used a lot but it's super powerful we should probably use it more um there's and then and how you get out of it that's the one that tends to be more seamless and it and it is a lot of work the animators have to work with the designers figure out where that handoff is if we can fudge it or have them follow our position or if we have to find that exact location from gameplay put them in the pose and then even worse when they have to hand off to it's easy to say and then he hands off to a like swing air pose but there's all kinds of velocity whether or not he's going into it or up or down and um of course they're all on the front line of that so they all have little tricks for knowing all right if you do this and you have

them coming down at a certain speed it'll work out okay I don't know you that you deal with that all the time yeah and I mean the the real thing is making sure you hit your core poses when uh you know you go back to you know when you call cut they're they're hitting their core poses that's like 90% of getting you there I think I think another important we we've gotten pretty good to the point where we prep these shoots as well as we can we prep them extensively by working with designers and going how is this how are we going into this and how are we going out and I think we always kind of cover our asses now and always plan to have a seamless transition whether we use one or not so there should be a picture of what the gaml out is while you're shooting and you can show it to the actur and you'll be standing here like that you know right foot forward but yeah it's hard work but we do maintain like every every pod or multiple pods meet sometimes twice a week and everybody just talks about it just like you would now and does that

you know it would be cools even if you can't afford it and uh it really gets ratcheted up by everyone you know there's always somebody going you know be cool is if you like flipped over backwards and did that and usually it'll be quiet and they're like you're right and someone will try to you know make it better they make each other better constantly so so thanks thanks for that question thank you hi um I want to follow up on that thought and um especially with the ASL shoot so you have your uh main actor you have your stunt actor you have your voice actor um and you have these poses that you have trying to match up right so you have all of these layers as an animator how which one do you prioritize when you're working with physicality or you know face performance or this ton well I think I think the goal is to strike the balance and so that's why we I feel like we've kind of shown it in

multiple of our presentations that we like to have options so it was really important for us to shoot our main cast alongside our uh DEA actors as well so that we could pick and choose from those performances um a lot of times our main cast actors would put in work to learn the signs and it was a much uh easier data to bring in and just use and improve upon um but a lot of times we had some pretty extensive conversations and we wanted to make sure that we had the most accurate data so it was always striking a balance always having options and always choosing the best performance I mean we're a performance forward Studios so we always want the most emotional and um authentic performance but that's where it comes into collaboration like our our death actors and and our main cast you know they were treated equally on the set and we we want both of their both of

their performances so yeah it's it's a lot of just to follow up on that so you in your prep work you are shooting everything you can at the same time yes okay so that we keep our timing the same so if I need to move in if I take the first three seconds of this performance I can pretty seamlessly as long as they're looking at each other and kind of Performing the same thing you know I when he was doing a lot of fing a lot of finger speaking in that scene with Haley at the end he was looking right at Justin his uh our de actor and he was kind of speaking to it and trying to kind of move his body at the same time so that the inflections were the same so that I would be able to later move in and out of those performances when need' be thank you yeah lots of layers and lendon layers and to Lindsay's Point like when you synchronize that and you're running them together it's really easy I think Bobby you you kind of set that up uh on layers and you can just

really dial in and out kind of between performances and since they're synced up it it's pretty de that's right we were we were animating the layer slider in Maya just sort of bringing up basically just on on a layer baked the the the ASL performer uh onto naji's performance which looked as bad as you would think like they like obviously their hips weren't doing the same thing but like you know if you track it a little bit um and then just sort of animate it on and off but yeah we just like Lindsay said we prioritize the emotion of the story the actor's connection the you know the we can dub the dialogue if it was you know unclean or something but it was it a a heartfelt performance like that first and then we could still fix you know uh hands and body or his hands not following him because somebody's moving and the other actor didn't like we we're animators so we approach it as what can I fix and what can I not fix you know well uh my question is the kind of follow up for that discussion uh first

Stephen when you recording all the the actions at the same time I saw that you might be using some real time I wonder if is unreal is it you using real um for like visualize what's happening everything at the same time is that helpful for the process do you do any like after the session during the session do you cut the mve I don't know what program they use to to show the real time set on the stage yeah I'm not sure what they see the video in on the stage I could yeah I could see something I was wondering if it was UN real engine they they have the they can our studio can run unreal we don't use it I don't know what program they use but it's all the same to us like we don't we don't go with lighting or or anything it's usually untextured but 100% whatever whatever condition the set is in at the time of shooting we use it we we measure everything to it we take the time it's really worth it to tape off feet and

make sure people are hitting their marks U to us using that stage well is the most important thing like our our actors are allowed to sort of use it like a like a theater and we actually figure out our cinematography after the fact a lot of the time so we'll go in knowing we'll have drawings and uh an idea of what we want but we don't complicate the shoot day with a cinematographer who's also being MOA or anything like that we just let them live the space as best as possible um so yeah they don't have we're not doing visors or anything they can see themselves on the TV and get used to the space but they'll just have like the gray mockups in the room but I feel like and this is like it's this is an opinion but we get the best work and we don't complicate the day with cinematography we just have actors live in the space and act it to me it works but you know we can fight in the parking lot if if any we do use the realtime playback a lot of times we'll we'll we'll do a take and then you know if it's like a crowd uh and they're

reacting to something we'll use that for timing so we'll take the real-time playback play the previous take and the actors will then act against that take so they're you know using the same timing as they did in the previous yeah and they can even look at their old self cuz you you know you might have four to eight people on stage and uh if they walk through themselves we'll put a rope on the ground like that's where you were don't be there okay I I was just wondering if you do any adits during the session or like right after just to see if things are matching since it's all the actions at the same time that's only when we need to okay yeah yeah we're very cocky I guess take a question from this side I think they were like here before me all right what you got amongst yourselves what um hi um so you talked a bit about being performance first studio and I've another theme I'm noticing in the qts and the knockbacks is you seem like very concerned with the um how the animation impacts the game feel for the player um

so I was just wondering like in addition to those like if you could elaborate a bit on the sort of like Direction pillars that you have as a studio of like how you what you want to get out of the animation sure I mean uh most of our main pillar whenever we're working on the Spider-Man titles is always to make it feel as close as we can to a Marvel movie or what it would be to experience a Marvel movie so that's that's really how how we plan our action and we want to make sure that we're giving that to the player as best as we can while keeping them on the sticks as much as possible um for those those action type moments so like the super knockbacks we knew we needed to have that type of reaction we've seen it before in the you know MCU and so we needed to figure out a way like how can we make this work with the environment and also keep the player with giving the player that that freedom still of being able to move around and not making it feel too canned um we needed something that was quick but also gave the player the feeling of

like I just got rocked so that's yeah I think uh a lot of the things what Stephen just said is accurate like we go for the feeling a lot of times it's like what does does this feel good and then does this serve the story elements of what we're trying to tell um so going into that with pretty much every script every qte we're like cool what are we trying to accomplish with this how's how do we make this feel good does it reflect the gameplay buttons that the players normally press um we try to make sure that's like the first thing we hit and then also then for us animation we're like how do we make this look cool you know it's like it's easy to say this goes from point A to point B but what's all this stuff in the middle you know and how do we make that look good within certain time constraints so as long as it feels good for us feels good for the player that's ultimately what we're trying to get for go for it thank you hi um my question is about blending animations so when blending between animations during gameplay during cinematics and especially between

cinematics and gameplay and vice versa um how much was relied on animation blending that already came in unreal and I know that um you guys also have like an internal engine that you guys use so how much was of that was custom built and were there any specific challenges I mean I can speak go ahead yeah yeah so we have our own internal engine and kind of what they talked about previously like we have certain core poses that the player needs to hit so especially going from cinematics to gameplay assets um we either have the characters like all right they're getting ready to go into combat we'll have our aggro pose we need to make sure that is the correct pose but then especially for this game because we had two Heroes uh we had to make sure it's the correct pose for miles or the correct pose for Peter and then there we do have blend settings to say like if they're we want them to be in that specific uh pose but if they're close enough we have some blend settings like blend over this duration of time and that helps smooth things out going from cinematics to gameplay and then we also

have a couple of events to help out with like making sure that the camera stays where we want it to be that way we can go into combat setups and have these cinematic sequences to start the combat setups up or if we're going to traversal making sure that the camera is at a specific distance going into gameplay and that takes a lot of debugging saying like oh the camera needs to be 5 meters away with this focal length and everything so we try to sink that up with the gameplay assets as much as possible but we do have some blending duration times to help filter that out and help smooth it out as we transition uh yeah I can uh highlight that a little bit more and for the Sandman fight um we ran I talk a little bit about blending during the Sandman fight and it's really contextual um even for for hero type animations like we usually touch blend times for almost everything um because everything kind of changes based on the Move that's happening uh but with Sandman specifically he also took up a lot of screen real estate and he was massive so the issue was that if we used our typical blend times that we would use for any other enemy or for Our Heroes

you'd see him move drastically and it would look really unnatural um so sandman's blend times were like usually three to five times longer than uh what we would do elsewhere and that was just kind of feeling it out and making sure that everything would perform as as we wanted it to um so we just slowed everything down thank you I have just one more question um I didn't see that in the video but how much of the camera like was moaed or if it was not moaed at all no MOA camera not at all what well it I made a camera that has um motion that's been extracted from Human hips and shoulders and that has been put into um a loop um and I made this just just to clean up my life CU When I I worked on a title many many over a decade ago where where it was all MOA camera and it was a nightmare and you know there was like

people would give you like 40 takes of MOA camera and they weren't synced up to it and you're just like oh I don't even I can't even find what this is supposed to be looking at um so I made this thing that basically was just Loops of of motion capture data at different um frequencies and so there's like swell which is like a big ocean it'll just do this to the camera you know and it does it for a while and it just keeps going down until you get to Jitter which is like you're sitting on a washing machine um and so you can animate these up and do like little shark fin ads in the graph Editor to like camera shake and fade it off you know and it's it's quirky it's not a perfect thing you get used to like how to use the the rumble cam um but at the end of the day like you could take you know two keys I'll you know set a key at the beginning set a key at the end and then push in on you and then just add that that uh drift that you get from uh wave or shake or not shake shake would be way too high um and then you sort of Just Dial that in

and there's like it's just beautiful there's like me two two two keys you know not not a bunch of crap in your graph editor if someone needs you to change it you don't look at them in horror like there's I can't change it there's a thousand keys on it um so it's just really simplified and it looks like map data and it uh like I said it simplifies the shoot day not to use it and there's always that learning curve you'll get new people they're like I've never I'm not a cinematographer but like you will be you you will by the end of this project you're going be very good at it um and I I think that just it just keeps it simple and beautiful and easy to manage like yeah right thank you so much hi the final QT is really really awesome yeah thank you and what is the walking pop line from the idea to the final uh product and because um is there any suggest suggestion that we can do at the first before we shooting because um we can know if it works um at the

first yeah thank you uh we did like a previs we had I had like a week and it was like Hey previs this whole thing and I was like I have a week I don't know how far I'll get but here's an idea and that whole final qte sequence changed a lot over the duration of the game especially like the last what 3 four months it was like hey it's this thing and it's like no it's not anymore it's going to go with this yeah he was trying to add even more yeah there was there was a lot more there was a there was a whole bunch of truck sequences and everything then they're like it's too much James and I was like okay so it was cool I was sad but I was like I did um but a lot of it is just kind of like figuring it out and I kind of learned a lot on the go of like how I can manipulate this and kind of what we were saying before like we wanted it to feel good we still want to have as much of it on the sticks as we can so like even as they're like punching Venom um a lot of that's kind of faked we didn't really have a system for that but I was like oh if I make his reaction to something look like an action so Peter's reaction to miles was prepping for a punch and kind of alternating between

the two we found out that that could work and they just kind of tweaking things to see what felt good and then just a lot of play test like hey does this work good I worked with Bobby a lot on the cinematography where I was just like hey put the camera here we want to be able to see this in the background we want to see this we want to see them going in the right screen Direction so we with Bobby and Bradley quite a bit on that just on that aspect of it but um yeah it was fun to make but it was very hard pick it up as you go and figure it out figure it out as say yes and figure it out get some help that's the pre Prez U previe combined with the game play or just animation uh the previous I tried to get some of it in game but for the most of it it was just kind of a a movie set piece of like this is what it could look like is this what you guys want and it was like yes and I was like crap I got to figure out how to make it now um but yeah a lot of it was just in a previs of Maya just like a play blast then I exported it out into the game

just so we can see where certain environments are because trying to bring the city into Maya makes Maya super slow so I would do a rough draft put it in game just to see like oh here's where there's building is if I want them flying this way trying to figure out what the path would be and then ultimately we were just like just shoot them up into the air and I was like that's easier so we went with that okay thank you hi um in a production that's so high value with so many different um uh disciplines coming together um I imagine that you have to uh concept A lot of things out ahead of time so how does that impact in your experience your artistic abil like your freedom um once you like have a performance like do you have to get a lot of your ideas out in the storyboards and then do you feel like that constricts what you can then do um once you're actually animating you haven't talked in a while yeah uh I

mean honestly on on this on this uh project we didn't do a whole lot of uh previs or or anything uh a lot of it is you know getting together with the writer the designer uh everybody getting into room and talking uh things out and then uh yeah then we will might pitch something uh and everyone's like that's awesome go make that um and then yeah uh I I never really feel that stifled uh artistically um at Insomniac it's yeah it's great and usually people uh I think we've built up enough trust in the studio they're like you make awesome stuff just go do that yeah I think that's been been one of the coolest things as we've worked on you know three of these games now is as we've you know our departments our cinematics department we've kind of proven ourselves as we we kind of know what we're doing now so they kind of let us go and try the things that we want to

try and of course once you actually shoot the data you're more constricted but sometimes we're like this isn't working we wanted oh this would be so cool maybe we can find another piece of data that we already shot or maybe we can get back onto the Moab stage and do this um but generally yeah we we kind of get to pitch our ideas and and make them work so no I I I've never felt more creative L fulfilled than as I have at this studio so the trust is off the charts like most of I think all of us have worked together for anywhere from 10 to 15 years in one capacity or another or even a different company so we we are a family um and people trust us now which is nice because you know when we when we kicked out Spider-Man they were like can you guys even do this and they don't ask us that anymore which is this is great um so yeah there's a super high amount of trust we do we do still previs some things if you know it's going to be a mess if you know you you you can't you

don't even see it clearly in your own head we will send you know our creative people after it to sort of figure it out but never at any point there are storyboards as well but we don't really use them and most of the time just because of the way games are developed the way missions are developed they're kind of obsolete by the time we start to make them but whatever it is we we use it and and we put it together you know if it someone has a good idea that the whole good ideas come from everywhere thing um that's that's all true for us so we'll we'll take as as many sources are available to try to put it even if it's concept art they'll have a picture and like that's rad we'll try to get that in in one of the shots um yeah uh yeah I mean that that's kind of the you kind of touched on that and that we don't we don't always have a good idea of what it is yet like we can write it in a script as many times as we want to and we can talk to designers and things like that but sometimes if somebody has a really good visual in their head and they put that visual out um that's kind

of what makes something real is like getting somebody getting the rest of the team excited about it um so like you know James's awesome finale like he put that visual out there and then everybody got to kind of like rally behind it and then you kind of push it to the final product spider cat that's how spider cat came somebody made it and we put it in the game yeah I mean and even to that like we especially on stage like we have awesome stun performers we have a lot a lot of awesome actors and like sometimes we'll have an idea and I might be like hey I have this idea here's I want these specific punches and stuff but I'm also like hey you guys think while you're doing this and if you come up with something better let's shoot it because I'd rather us have the data and that way we can pick and choose cuz sometimes we might have data and it's like it works but it could be better and we'll filter out and a lot of it's just scrappiness like well let me look for data let me search kick what kind of kicks do we have what kind of options can I have can I is this something that we can key frame real quick and get a better visual so I don't feel like we really stifle oursel we try to push creativity and

that's also one of the benefits I really enjoy about like not locking into specifically having a camera going to a map shoot because sometimes discovering what that cinematography is comes from after putting the whole sequence together and you're like oh if I put the camera here this is dope versus if I go into the map shoe I might feel like a this is cool then I realize it later I'm like well I'm locked in now you know so it it just gives more freedom for creativity and it's like that's something that we really push for yeah I'd say you that that goes with every decision we make you need to show up prepared but you need to not be uh that that preparedness should not lock you out of all the great ideas that everyone else is going to try and bring to it yeah know I know we're getting close to time but we could probably get another question um hello yeah so this is again regarding ASL uh so in the slides say saw that you use motion capture data and then refined it to create uh readable

ASL but in the future or like in this project did you consider or would you consider building uh a library of isolated animations for hands and like signage that you can reuse and omit sort of create like a ASL Google Translate for your animation pipeline or would that affect readability since it's more mechanical and not motion captured that's a question I I think that that's something we definitely thought about moving to the future I mean that's something that we do with poses we do it with facial poses we create libraries so that we can quickly do hand shap we do it with just normal hand shapes you know fist this that uh Swip um so having a pose library of uh sign you know letters and things like that could be incredibly help helpful moving forward if we had more in the game um and that's yeah that's absolutely something to consider going forward speeding up that process

cuz it is a very labor intensive process so um that would absolutely if we had the time and the and the ability to do oh yeah thank you oh drivea that something also that doesn't happen in cut scenes our engine does have that but it's not the one that I made cool so yeah like so this would be specifically for writing out of my a camera not uh for creating an in-game camera and applying shake to it right yeah sometimes I have to check cuz there you know yeah we have procedural events that'll that we will drop on specific animations so like sync moves or things like that uh finishers there's procedural Shake events that we drop on um those animations but they're

not driven from Maya yeah the only the only times we would there's a few certain circumstances where we would do that and that's like finishers finishers have custom cameras where we can introduce that and it uses the gameplay camera but lets our cameras override it but majority of the time we have a different system for gameplay stuff uh I think I think we are at time I'm sorry but thank you guys for all the questions [Music]