An Explosive New Spider-Man: Creating VFX for Miles Morales

[Music] hi my name is brianna lindsey and uh we are going to be talking about an explosive new spider-man and creating visual effects for marvel spider-man miles morales i'm one of the lead effects starters at insomniac games and i got to work on this project and i'm really delighted to be here talking today with you about uh some of the effects that we made for this game uh so this talk has a couple of themes uh some of them were um one of one of the ones we intended right from the start was uh finding a voice for your hero through visual effects and really telling their story and making them unique an explosive new spider-man so an explosive new spider-man was one of the pillars of this game and it's something that the effects team really glommed onto we found a lot of fun in making miles our own hero and finding ways to differentiate him from the hero of the 2018 game marvel spider-man another pillar of this talk is uh technical tips and tricks that we

learned working on a cross generation game so there were a lot of things that we picked up along the way in terms of learnings when it comes to pushing next-gen visual effects as well as working on a cross-generation title working on something that had to be on both the playstation 4 and playstation 5. um and the last theme was something that kind of evolved as i was working on this talk and that one is really pushing forward through the unknown taking risks and working together to create something new and great there were a number of uh effects that you know when we were starting to work on this game we weren't entirely sure how we were going to pull them off and i really feel like we learned a lot by taking a risk uh by challenging ourselves and by pushing forward uh through the unknown um and it was uh really wonderful to be able to to work on that so i'm gonna give you a quick overview of this talk uh during our time together we're gonna be talking about uh miles powers um that includes his venom powers that's his bioelectricity that generates

uh within himself that's also his camouflage ability so uh his camouflage is when he turns invisible we're also going to touch on creating spectacle so spectacle is a really important part of this game and it's something that we sort of had to work together as individual effects team to push we're also going to be talking about the enemy powers uh that includes programmable matter uh so miles is his own explosive new spider-man needs explosive villains to be able to uh fight um we're also gonna be talking about his webs so uh spider-man uh isn't really spider-man without his webs and i thought it would be important for us to talk about that um we're also going to be talking about an all-new accessible game we had a philosophy if anyone can wear the mask while working on spider-man and as an effects department we were really able to make a big difference for players with uh visual and cognitive needs with their contrast options and lastly we're just going to talk about uh challenges for a new generation and some of the learnings and takeaways that we had while working on this project

so let's get into it let's start talking about miles venom powers they're central to the character and a pillar a part of the pillar of creating an amazing new spider-man um so this is uh our next generation shot that we made for the ps5 reveal and um this is uh in the scene we see miles draining electricity uh from an overloading reactor and as the camera moves across his arm we see his suit peel away as his arm flashes with electricity and as the camera continues his mask eventually disintegrates as well revealing who he is to the player and uh for these shots we really had to figure out um creating a soft body simulation pipeline uh just so that we could get that suit disintegrating and coming apart we also had to figure out how we're gonna get that x-ray kind of effect on his skin so you can kind of see his bones and his viscera through that we also had to work on the glass particles flying at the camera and of course the the suit burning up so these were all um challenges that we were eager to dive into as an effects

team so when we kind of started on this this was really just a pitch on paper we had this really beautiful script that was written for this that honestly sent chills down my spine all we really had was this piece of concept art and uh this script and you know in this script we hear peter parker talking uh to the player and telling the player uh you know miles this is your chance to go be a hero [Music] and uh that script uh you know it sort of described uh you know we sort of see miles put his hands into the reactor we see him absorb this power into himself and this is really his defining moment as a hero this is where miles decides to save his uh his neighborhood within uh new york city and um it's he it's a moment of self-sacrifice because as he absorbs this into into himself it's a really dangerous moment where he could potentially die uh so when i when i heard this script i mean it honestly sent chills down my spine this was a huge opportunity for the effects team uh to sort of push ourselves and to create something new

and amazing however i was also really nervous when i heard it because uh you know we didn't have the tools to pull it off we had to develop those along the way so uh one of the big problems that we needed really high quality venom electricity so that electricity that crackles uh down his arm that's something that we had to develop for this um a little bit of electricity it's biological and uh it's something that comes from miles himself and this really differentiates him from peter parker uh the spider-man in our previous game marvel spider-man because his abilities are more technology based we also had to figure out how to do soft body simulations we didn't really have a bot a pipeline to be able to do things like simulations of fabric tearing off you know so that was the question we were asking ourselves was you know okay well we really want to do this we think it's really really cool how are we going to pull this off so just so everybody watching at home understands when it comes to

this shot the effects team was responsible for a great deal of it when you remove the character from the equation you can really kind of see the soft body simulation in question as well as the glass particulates um the electricity that kind of crawls down we need to do that for both arms of course and um you know since i took the character out you can't see it but there's also that sort of x-ray effect that's going on on the character so you can kind of see his bones and his viscera inside of his body uh because it's really important for miles as a character that this is coming from him so uh some of the solutions that we came up with to pull this off uh we ended up leaning on some current gen techniques to start we painted vertex colors to drive that sort of slow reveal of the electricity our artist jenna um did a pass on this and she did an amazing job on creating this electricity so one of the things we found was that we could afford more vertices on next gen and we really tried to use that to our advantage where we could uh by using more vertices we were able to sort of push that fidelity on the next gen console and we were also able to um use the you know everybody

sort of use this technique i'm sure where you use vertex colors uh you color a vertice like red green or blue in this case i think we used red just to be able to drive that precise wipe so there's a material nodes in our material editor vertex color nodes that we uh sort of lean into to be able to drive this this is kind of what that looks like in the particle editor so as you can see we were able to get like a lot of precise control uh kind of leveraging a current gen technique in addition to that we also layered on uh some of those little particle quads so you're able to sort of see um those uh additional tertiary like lightnings kind of at the ends um so we did sort of layer in some particle quads in addition to using meshes um but i really liked that we were able to essentially model this out to get more fidelity for this shot next up we're going to talk about simulations and our uh what we call our anavert pipeline so we needed to develop

a pipeline for bringing in baked uh soft body simulations that really didn't exist in marvel spider-man uh so our solution for this was the anavert pipeline the end of her pipeline can support tens of thousands of soft body polygons so this is kind of what this ends up looking like this is um something that our artist matthew simulated in houdini and exported out of maya this is actually taken from maya we were able to support uh 29 000 verts which i thought was pretty uh pretty awesome and impressive for uh this next gen shot um and this uh invert pipeline uh we sort of had a prototype version of it but we really fleshed this out and finished it out for our trailer and we really gained some useful new technology that we were able to leverage in other parts of the game so uh just a huge credit to our entire programming staff especially our animation programmers who were able to help us flush out this pipeline because i really do i'm really just impressed that we were able to pull this together for this trailer

and then we also used paint vertex weights to drive the burning edges of the sleeve so it's really important for us when we were working on this shot to you know have the audience reveal and understand who miles is while they're watching it so uh you know as the camera is moving you know you're not entirely sure who it is until uh you know the camera starts pulling up closer and closer and you see you know oh at the end it's it's it's miles it's who miles is um so what we did was we sort of used vertex color again here we just sort of painted the uh using the red channel to drive that erosion um so this is uh this is actually a gif and it moves pretty slowly because this uh this shot took place over the course of a couple seconds um but using that that red channel we're sort of able to erode out uh the simulated chunks um and then also by using an additional color channel in this case the green channel we're able to erode out uh we're able to put that hot burning erosion edge in addition to uh eroding out the alpha

so uh that's kind of how we were able to layer those together to create that suit burning up effect next i'd like to talk about that x-ray skin effect that miles has so this is a sort of a rotating gif of his arm meshes and it sort of highlights his uh bones and tissue and viscera and the these sort of reveal in that sequence uh they kind of flash on at certain points and we drive drove this through something called parallax occlusion maps but this was a really important part of expressing who miles is the character through visual effects because this is really a power that's coming from inside of him we thought it was important that you know the audience and the viewer be able to see inside of him so um this is uh his skin texture that we use to drive those parallax occlusion maps um this is sort of a current gen technique i'm sure many of you have used grey packs before i've also heard this called texture packing but it's a you

know simply where you put different materials inside of your red green and blue channels because you really only care about that black and white information on those channels [Music] so basically we use parallax occlusion mapping for bones and veins that's essentially kind of like a height map that uh doesn't displace the surface of an object we use it to fake depth um it technically uses ray marching and it is expensive and it's sort of painted and authored like height maps but a parallax occlusion maps i mean you essentially want to use these um anytime where you really want to get that sort of fake depth it's really good for kind of pushing downwards it's really good for craters but we kind of saw an opportunity here this was um actually uh one of our artists jenna's ideas uh just to be able to use it to fake depth on a character so you can kind of see um the inside of them uh so at least when it comes to the material graph the parallax occlusion maps are actually these uv modifiers that go into

the uv texture so you can kind of see here the parallax occlusion information is kind of going into that uv channel and um you know on the blue channel we're using that blue channel texture to drive just some of the cloudy noise inside of miles uh we're using the red channel to drive some of the squiggly veins inside of his arms and then the green channel is the one that's primarily being used for his bones then this is just another example of this we're using more of more parallax occlusion to kind of drive that squiggly sort of look so we actually are using some uv translate in this as well so that as it's moving you get some additional motion so you can kind of see that um uv parallax occlusion uh uv modifier sort of feeding into the uv translate nodes on the bottom and this is a you know just a really really useful technique to kind of get some additional motion into the depth uh just by using some uv translate nodes to sort of get that additional motion

um next i kind of want to talk about mega venom blast so uh this is sort of the pent ultimate uh venom power that miles has and it was really important to us as we were working on this that it looked like the comic books um it's also biological nature so this is something that comes from inside of miles this differentiates him from peter parker who uses more technical gadgets it was important for us that it felt web like so this is uh something that is spider-man's electricity this is not you know electro's electricity this isn't the electricity of some other hero it also needs to work in 360 degrees so we really need to be able to see this mesh in game play and this is really the ultimate in miles powers and the expression of himself and it really needs to support our game pillar of an explosive new spider-man so this was one of my favorite pieces of art of mega venom blast from the marvel comics you can kind of see here it has that really nice sort of graphical look

uh spider-man is um just in the air and as he's letting this power out of himself it sort of explodes into this web shape and this was a really powerful image and you know it's really a challenge when you're looking at something like this it obviously works really well in the 2d sort of graphic state but you know the question we have to ask ourselves as a real-time effects artist is okay well this is wonderful i love looking at this how do i get this to translate into three dimensions so this was the one that we ended up having in the in the end on our game as you can see it goes on 360 degrees we have a little electricity on miles and then he jumps up in the air and he does this mega venom blast it does have that kind of web shape to it that we layer in a couple of different particle quads as well as this large venom mesh that we use to sort of drive that look so this is sort of what that looks like in our editor and um as you can see we

use quads sparks we have an effects light on there um i believe on the playstation 5 version that light is also a shadow caster we have all these elements sort of working together along with this 360 degree web mesh so when you look at the mesh by itself when you isolate that this is really the heart of mega venom blast in that explosion and it's driven by this uh this mesh that we used and we found that we could use more vertices on next-gen especially but it's driven by this mesh and it's driven by a material sort of coursing through it at the end of the day i mean it's a relatively simple technique but what makes it effective is that we have more fidelity on this mesh than what we could have on previous generations so this is what that looks like in maya this is just the measuring question from maya and really we kind of find on next generation platforms that we can utilize measures more thoroughly and i'm really hoping that uh for anyone looking at home you know just wondering how can i push visual effects for the next generation uh they you know you consider just using

using more meshes you can probably get more fidelity in your visuals that way the other benefit to building effects in this way is that it also sort of minimizes overdraw so you know when you have uh more of this negative space that's actually negative space and not um you know just alpha and out on materials it kind of saves performance in a different way i mean it is more expensive on the vertices side but you're not really paying those overdraw costs that you would you would pay if this was all quads uh next i want to talk about uh the venom sleeve so one of the things that we had to figure out was having electricity crawl down miles his arm this was and it was really important that it felt biological that it was kind of you know inside of him that was coming from him so this is a different from peter's technical gadgets so miles as his own spider-man he is sort of generating the generating the power from within whereas peter parker creates gadgets uh so we we really had to ask ourselves how can we get this venom electricity uh that comes from within miles morales working on all

of his suits and how can we get it looking good because one consideration with the spider-man games is that the player can unlock different suits uh they all have different silhouettes and different uh different shapes to them so you know we really needed something that would fit very very closely you know basically be inside of inside of the mesh as well as give us that fidelity across different suits so i think this is one of my favorite examples of the venom sleeve in action this is a kind of right before that large uh reactor shot that we already talked about but here miles is absorbing that reactor energy in terms into himself and uh we also sort of use color storytelling in here uh this was actually one of our artists jenna jenna's idea but as miles is taking this blue uh new form electricity into himself it changes uh into yellow venom electricity as he can train uh gets mastery over it um some other details in here that i really liked that we did um if you look closely at his rib cage you might see like a bit of a silhouette of a rib cage

in there also his skin is kind of flashing again with some parallax occlusion uh you can kind of see hints at his skull just to sort of give that that feeling of you know this is really affecting him and this is a really big moment in the narrative as his moment of sacrifice uh so the way that we sort of did the venom sleeve we created this mesh that goes over all of spider-man's suits so you know it's something that our character team ended up modeling and skinning and implementing on all of the different suits and then we just sort of had a universal texture that and material set with a different wipes that sort of applied to all of those uh those same meshes we also worked with our gameplay team so that this suit isn't technically on all the time mostly because we found that it got expensive to have all that alpha on the character at all times so our gameplay team very carefully uh cleverly put this into look groups so that we were able to turn this on when we needed to and turned it off when we didn't need it

this is um kind of similar to the parallax occlusion maps with like the bones of the viscera we used a grey pack texture here to drive the different veins for the uh venom sleeve reveal so um that was just really beneficial because as this was a cross-platform title you know we found that texture memory is still concerned on ps4 especially but you know this was still a useful technique on ps5 as well um so we basically split those different visceral channels out into you know we use our red green and blue channels separately uh you know in here we have the mirrored texture kind of being split and we can sort of select the left and the right side of it and then here are sort of the controls that sort of drive that wipe uh the progression node in the bottom left that's sort of the main driver that we use most of the time uh for driving this reveal over time it's exposed to your gameplay team who has control over how fast or slow that

needs to be depending on when and where we're using it we also have some additional uh knobs in here for some key moments where we need to uh tweak the speed of venom so another thing that we did within the venom sleeve that we found to be very beneficial was using a displacement just a little bit of displacement to prevent clipping uh so that was kind of helpful to ensure that that the sleeve works on all of miles suits so uh you know anytime he bends his elbow or deforms his shoulder by offsetting a little bit in the material and by pushing it a little bit toward the camera we're really able to prevent any clipping and to make sure that this is uh properly visible at all times another place where this is useful is also in the fingers uh just because there might be a lot of clipping there all right so uh thanks for talking to me with me about venom powers now we're going to move on to camouflage and invisibility which is another one of miles key powers so this is an early piece of concept art this kind of shows miles turning invisible

so there's a lot of things in here that we ended up using in the final version of camouflage things like the fact that he's tinted a little bit blue he's invisible of course uh he has a fresnel on him he has the chromatic aberration inside of his body we didn't end up using everything in here we didn't make you know the enemies red we didn't really do like a full screen effect like what's in this concept but a lot of these early ideas did end up making it into the game and this is uh the in-game version so this is sort of where milestone is fully invisible and this basically took us three different materials to pull off the chromatic aberration that was in that earlier concept is something that you can also see here so you can really see that the green and the red channels just if you look at the windows in the distance through miles you can really kind of see that those are a little bit offset from each other and you can also kind of see that a bit on the snow through his legs but i think it's most prominent on those windows there's also that transition so he kind of starts you know fully opaque and uh

he sort of gets that kind of like patchwork transition where he turns invisible um so the way that we sort of pulled this effect off uh we used some things called pre-composite and post-composite materials to drive that transition uh pre-composite materials are materials that have rode out of mesh so these sort of tell the pixels to not be drawn and those are applied like you think about your render order those are applied basically before the materials are drawn the post composites are materials that apply after the main material is drawn so the post composites are materials that uh basically in the render order we draw everything we put the post composites on after so that's how you can kind of think about pres before post is after that's why they're called pre and post composites in general these materials are really useful because we can apply them over entire meshes without them being material overrides so they're reusable we can put them on all sorts of objects in the game um you know they're they're something that we utilize a lot in our workflows and uh the camouflage effect they

require both a pre and post composite shader where after the transition is complete then we go to a complete material swap and take a sip of water so this is the pre-composite material you can kind of see the alpha erodes out it erodes out the mesh before the pixels are fully drawn so with the pre-composite this is sort of a yes no is there is not their kind of setup um we don't really have like shades of grey with and that's just sort of because of where it sort of comes in the material rendering order however it's still really useful for cases like this where we want to fully erode out somebody this was the post composite material you'll notice that the erosion pattern on it this mirrors the erosion pattern and the pre-composite material so they're basically the same except the post composite is you know inverse so as the pre-compositor rodes out the post composite erodes in and it contains similar shading

information to the main shader that has that chromatic aberration in it you know it's blue it has that fresnel the math is basically the same between those two except the post composite also has that erosion information because it's used for that transition between the solid and the invisible state once the erosion is complete we do a swap at that point for the main invisibility shader and the reason why we do that is because uh having the the pre and post composite shaders is more expensive if you do that wholesale material swap that ends up being cheaper and that was more performant and you know because this was a cross generation title especially we needed to make sure that it ran in frame on the ps4 um and then uh this basically applies over the pre-composite when you do that swap it allows for the illusion where miles turns invisible while also having solid parts so when we set this up our gameplay programming team is wonderful they set up these configs for us so that we can implement our effects into the game using them uh so we we did kind of

need some uh some help from our friendly neighborhood programmers to make this happen um and then oh yeah chromatic aberration so the chromatic aberration is really uh offset driven by offsetting the red and the green channels from each other on the frame buffer so you kind of see we have two frame grabs there they're eventually being combined into one where the red is being combined separately from the green and blue of the other frame buffer and then uh you know on the earlier on in the math we're just offsetting um one of them by like .01 and we're offsetting the other by negative 0.01 so those offsets are just a little bit you only really need a little bit uh and then that ends up being combined in the end to create some chromatic aberration uh so we also found as we were working on camouflage that we needed some special one-off variants uh so there were really some cases in the game where miles had these emotional beats and originally we didn't set up a camo to really have any other

timing except for that zero to one however we really wanted to make sure that we were including these moments using visual effects to sort of help the player understand times where miles was doubting himself or when he was scared or when he wanted to hide and this required a few custom setups to make it happen but it was totally worth it to help miles become his own explosive new spider-man so uh one such example this is actually the first time that we see camouflage in game uh in this moment miles is threatened by rocks on forces and uh we had to slow down the camouflage transition to really make this feat feel good so the transition is usually about a second long um and it's uh basically that zero to one parameter material we just talked about um but our programming team exposed some uh some knobs for us in the cinematic so that we were able to really slow this down and i love i love showing this beat too because especially when the camera is looking right through miles at those rocks on officers you can see the um

you really see that chromatic aberration there uh another time where we had to do this was uh this moment where miles uh this is right after that uh the bridge explodes um and he doubts himself as spider-man and he's not really able to control his camo powers so his hand by itself turns invisible and uh in order to get this beat like like the the camo ability was sort of designed to go over all of the character it wasn't really designed to go on one part of the character uh so our character team sort of split that hand off to be its own match and then we made a custom material so the pre-composite material is really only routing out his fingers and this was really nice because it kind of gave miles that moment where you know he's able to question himself before realizing that yes i am spider-man and i can do this uh this was um another camouflage beat that we did where miles tries to disappear this is right after he's captured uh and um the roxon guards are hitting

him and we had to speed up camo in this speed we had to make it really fast just so he could like try to go invisible and then he'd get hit and then he would come out of it and this is a sort of building up right to the moment before we do our first mega venom blast so uh it was a it was a really fun beat to work on all right next we're going to talk about creating spectacle so marvel spiderman miles morales is a game that really requires intensive spectacle beats so that miles can demonstrate his heroic abilities so one of the challenges that we asked ourselves while we were working on this was how do we up this for a new generation for what we accomplished in marvel spider-man one of the solutions to that was coming up with a new simulation pipeline so adam chunk is our new rigid body pipeline and we developed it to replace the previous pipeline we had in marble spiderman this new pipeline that we developed thanks to the fantastic work of our core team it allows us to get in high quality large to give simulations more quickly and large large part why we developed

this is because spectacle is overall very important to the game so a really good example of this new pipeline uh comes in from our bridge trailer and also our bridge mission so uh this particular sequence um is in the bridge mission it's when the bridge kind of comes down and um miles in this sequence is uh you know it's really sort of a moment where he develops himself as a hero and this particular simulation this had over 250 000 vertices and um i was really proud of our art team for working on this including matthew bennett um jin choi and alex uh heligo they were able to really push the the next gen fidelity and spectacle on this next we're going to talk about uh one of the simulations that we did for our winter break mission this is actually the opening mission to the game this particular sequence was a really good example of using both of our new simulation pipelines so this includes both the rigid body anim chunk pipeline that we just talked about as well as

their soft body and invert pipeline so the soft body can really be seen on the streamers that fall down from this uh happy holidays sign as well as the uh and invert can really be kind of seen on those um chunks of the sign as it comes down uh one thing that was really great about the sequence was that we were able to uh push the um the fidelity in terms of uh sparks and uh debris particulate coming out uh in this sequence um on the playstation 5 uh even more so than we could on the playstation 4. um and this was a kind of a really great place where we were able to really leverage that hardware in terms of creating that spectacle and then um another sequence that i thought was really great that we were able to really push for next gen is the sequence where this power plant explodes uh the next gen hardware afforded us the ability to have more particles as well as shadow casting lights if you look at some of the lights on these explosions uh on the ground especially in this this part

right here they really added a lot of drama and the next generation hardware was a place where we could really allow that to shine next i'm going to talk about uh programmable matter which is um essentially the uh energy that the energy weapons that miles nemesis the underground used in our game so with programmable matter miles morales as a hero we kind of needed to give him a really new enemy in order to complement him as a explosive new spider-man so this is where the tinker and the underground come in um they used a nanoparticle-driven weapon tech uh called programmable matter in the game and uh of course uh we as vfx are decided to come up with a solution for that uh so we had to come up with new and different effects systems to be able to support this to make it unique and different from what we accomplished in the original marvel spider-man so uh one of the first things that we did when we were working on programmable matter was come up with a list of rules for it so that we sort of understood what it was and how it worked

um so we decided that programmable programmable matter required nanotech printing uh it it formed an internal lattice so it had a support structure inside of it as it printed uh it always printed from a source so this never came from you know the air it wasn't magical it was always something that was tangible and came from a specific place it is technically a metal so programmable matter it is metallic it's not plastic it has metallic properties to it it's always created by finn mason who's uh the tinkerer in our game uh so this was a early pre-visualization of programmable matter uh that we did to sort of help us cement what this looked like and um originally programmable matter was green of course we we ended up changing the colors uh while we were working on it but this kind of uh demonstrates some of those early principles that we came up with um for this so you can kind of see it it comes from a source it creates an internal support structure as it prints you can kind of see the particulate sort of

spread over it as it comes into its into its final shape and this was a really kind of a great uh guiding pre-visualization for us as we were to translate this into real time one thing that we wanted to leverage when we were working with this was gpu particles which is something that we didn't have on the original marvel spider-man um so this is uh kind of what we had in game uh in the end so you can kind of see that a lot of that original pre-visualization made it into the game so we do have that internal structure that kind of prints using a material graph that is followed by support struts on the inside which kind of prints the the skin on top we leverage a lot of gpu particles with a little bit of noise on them to kind of help fill everything in at the end and then of course we also use particle lights just to sort of make sure that everything kind of ties into the scene correctly um so the inside of the programmable ladder

uh mattis the programmable matter lattice is controlled um sort of like a material wipe this is driven much like we drove our sleeves uh so this is driven with a wipe that is controlled by vertex colors defined in maya and we used a lot more geometry for detail so one big thing about next gen is that uh you know you can use more vertices and leveraging vertex wipes really allows you that precision to kind of give you that uh really controlled sort of look you're looking for especially in any kind of a cinematic beat where you really want to have control over that the inside of these lattice models also contained an additional mesh so these were sort of the interior lattices that went inside of each of those segments as they sort of snapped into place and we use this in the um the cinematic as well as the gameplay versions so a lot of segments are sort of driven with the same sort of technique where you have the vertex colors controlling a precise

wipe and then when it comes to timing that uh this is a just sort of a custom input value so we have the um the x and that custom input node that is something that we uh is exposed to us that we can manipulate and control and keyframe in the particle editor so we use um custom input values quite a bit while we work because it gives us more precise control over the timing of when those come online another thing that we use quite a bit was gpu particles so a lot of the underground attacks are driven by gpu particles so you can kind of see this is a gauntlet enemies aoe impact when the enemy hits the ground it creates this awesome shape they actually destroy their programmable matter technology as they do this so gpu particles were new to miles morales uh since we had a cpu driven system uh only in the first marvel spider-man so that because this was new technology we really wanted well new to our daughter game engine because of something we hadn't had previously we leveraged it heavily when it comes to

the underground attacks and we use them to help bravo matter feel more threatening and unique and different so this is the main gauntlet enemy attack um you can kind of see that he jumps up in the air and he punches the ground he breaks his gauntlet mesh and he reprints it using one of those detailed material wipes and another thing that we use in here um if you look at this you can kind of see that it has a mesh that distorts and it creates some really awesome looking shapes as it as it spreads out as a shock wave and it's really using gpu particles in this instance helped to make it feel a lot more unique versus what we had in our previous game next i want to talk about webs i can't really have spider-man without web so i thought it would be important to talk about that so with webs webs are one of the things that makes spider-man spider-man peter and miles share webs and we didn't really change these much from marvel spider-man however it's still kind of worth uh

reviewing how he did it webster team effort between vfx and many other departments including animation rigging rendering and gameplay and it's one of the things that we get asked about the most so i wanted to make sure that i included it in this talk so the first thing i want to talk about are the traversal webs so these are the webs that miles uses when he's swinging across the city so you can kind of see here this is in the bridge mission where he's swinging around it's basically just those web lines that he uses to to get all over the place these are basically splines they're programmatic programmatic in nature on the effects team what we do is we control the shader and material that goes over the webs as well as the web knots that are placed over the spline so with the traversal webs the knots are basically meshes that are placed procedurally across that spline and then the web line itself is programmatic so we didn't really adjust this much from uh marble spider-man but i still think

it's a pretty cool system and worth talking about another web system i wanted to talk about are the web jackets so these are webs that the players can put on enemies during gameplay they serve a functioning gameplay they allow the player to immobilize enemies and essentially because they are webs they do need to be skin tightened to fit over the character so with these kinds of webs we have face webs as well as the body webs uh each of these kinds of webs are skinned in maya to each character model so we have to get that character in the proper bind pose and then we create layers of meshes over it we do have our meshes set up in such a way that they play nicely with the uvs and the materials in our engine for webs and the materials for the webs they basically drive the terrain and the appearance so when these go on and off they kind of erode in and out a lot of that is material driven uh next i wanted to talk about the stuff together web so these are the webcones and web tunnels you can kind of see them here where miles is trying to

stop this bus from going over the bridge he places a number of web tunnels on the bus and these are we kind of kind of call them state stuff together webs but they're really sort of a collaborative effort between departments when we create these uh they're modeled in maya using a custom tool set up for creating them and the materials are then applied in engine to these and uh after that they're rigged and animated by our tech animation and animation department so on the effects side a lot of what we do is mostly just generating the initial meshes and then we'll generate the materials that go with it however when it comes to controlling them and animating them and placing them a lot of that is uh done by our animation team and webs are really a team effort so a blueprint that i thought was really important to touch on in this talk was accessibility accessibility was very important for us for next generation and at insomnia games we had a really big push to create

the most accessible game possible so many departments including visual effects decided to pitch in and see what we could do to make this game the one of the most accessible games we've ever made so we have this philosophy if anyone can wear the mask basically we thought anyone should be able to be spider-man and we wanted as many players as possible to enjoy that fantasy of being spider-man so released within the effects team we sort of drove what we called contrast features in marvel spider-man miles morales and we found that we could have a really big impact on the game in this way and i'll get into what contrast features are in a second one of the things that we created for this was the high contrast background so that is designed to help low vision players enjoy our game this is essentially a full screen quad that we put over the screen that desaturates the world and then after we desaturate the world we'll flag relevant effects for combat to draw on top of that full screen quad and we'll also add high contrast shaders to enemies and combatants and relevant objects to gameplay in order to help them stand out and this is kind of what that looks like

in action um this is uh you can kind of see the sky boxes darkened the enemies get bright shaders relevant effects to the combat draw on top of that background so this is a really beneficial mode for anybody who has vision issues anybody that might have cognitive issues you know they might benefit from this because it basically reduces visual clutter and visual noise and it tells the player you know what is important for them in combat and we found that this was very helpful for a variety of players and part of why this is so important i'm going to get a little personal with this uh but this is a a video from the original marvel spider-man and this is the fisk boss fight so this is the first boss fight in that game and uh my fiancee is a low vision player and when he played through this segment of the game when i when i watched him play it at home he was unable to get through this segment of the boss fight and the reason for that was because he was uh

unable to tell the difference between the boss and the ads and looking at this as a visual effects artist so i know this is a problem that we can solve especially as a real-time effects community i think that uh it's something that pretty much anybody you know could uh come up with a solution for so um this uh the solution that uh well this is just sort of a simulation of um what that low vision looks like so this is actually a simulation i did of my fiance's site so he's losing some of his central vision so for a low vision player you can imagine it's very frustrating when you're playing in a fight like this you're not able to really tell who's the boss and who the ads are but you know thanks to the contrast features that we added to marvel spider-man miles morales my fiance i'm glad to report was able to play through that game entirely by himself without assistance and part of that was because of these contrast features that we added and at least for me personally i think this is probably uh one of the things that i thought was the coolest part of creating a next generation title was

really thinking um holistically about game development and thinking about ways that we can make games more accessible than ever to new players and uh honestly this is probably one of the things i'm most proud of so uh one of the tricks that we did for the high contrast background was basically making the skybox really dark it's in order to detect the skybox so that you can color it something other than skybox colored you just have to compare the depth buffer to the far clip value and we can just do a lerp really easy that darkens the skybox will keep in the foreground intact so another thing that we did for this title was develop contrast shaders so the contrast shaders are designed to be bright and easy to spot for low vision players they're also customizable so players can select different colors for ranged enemies versus melee enemies versus bosses and it's really important to us at least for our game that players are able to distinguish those of course everybody's might have a different solution for their own titles worked for us and spider-man might not work for

every game out there so hopefully you can kind of look at this and think about ways that you could develop a similar mode for your own game so the contrast shaders are sort of set up in a config file and this is a driven by our gameplay team who is wonderful and set this up for us and this ties into uh our ui for easy player customization it was really important for us that the players were able to pick different colors and customize their experience so lastly i'm going to talk about some of the challenges we encountered for the new generation and some of the takeaways and learnings that we had so uh some of the challenges that we faced with this title was that um miles uh marvel spider-man miles browns really needed to be a next-gen title will also at the same time being a cross-platform title it had to release on the ps4 too uh so one of the challenges we had was uh how can we make a game perform on ps4 while still finding ways to push next-gen quality uh one of the solutions that we came up with that was uh i think really helpful was that uh within our internal editor we basically set it up

so that we could split uh some of the particle emitters perf platform so in this example this is actually a smoke emitter and the smoke emitter is flagged to be ps5 only and that's because we wanted these this particular smoke to cast shadows and it's disabled on the ps4 and there is a similar ps4 only emitter within this effects package that doesn't have shadows on it so this was a great solution for us to be cost effective in terms of not needing to re-implement some of these emitters multiple times and it really helped us out in terms of delivering that next gen quality so uh with this system with the ps4 ps5 emitters it allowed us to selectively add a bunch of particle upgrades for both the p for the ps5 that might have been unperformed on the ps4 some of those included making more of our particles lit we were able to afford uh more particle lighting on this generation which you know definitely added more drama to our effects more shadow casting particles so you know those large columns of steam we're able to have those cast shadows

we're also able to have more shadow casting particle lights so those are for instances when you want to have a light inside of an effects package that casts a shadow we could afford more of those and switch between the ps4 and the ps5 i were able to afford denser smoke and fires so you know in order to uh get things performed on the ps4 sometimes we couldn't afford quite as many embers or smoke as we wanted to whereas on the ps5 we could afford more so we leveraged that system in those cases and we can also afford more gpu particles in some instances which was wonderful uh some other limitations that we ran into the ps4 had more memory limits compared to ps5 part of that meant that simulations had to be smaller for the playstation 4. unfortunately uh we were able to afford more on the playstation 5. um texture memory was more of a concern on please on the playstation 4 so we were limited to how many you know smoke maps we could use on that console versus how many uh you know variety of maps we can use on the playstation 5. loading times were also something we had to be mindful of so the playstation 5's

solid-state drive meant that we were able to load scenes very fast unfortunately on the playstation 4 um you know we had to uh pre-load in simulations a little bit earlier just so that we could have that play correctly on time uh some other uh findings um the playstation 5 it had more memory in the solid state drive so that was a huge advantage so we were able to load in our simulations very quickly on that platform um the gpu uh they definitely needed more space for ray tracing so one finding was that although we could afford a little more gpu particles that didn't mean that we could go crazy right um you know just because we have a great new hardware doesn't necessarily mean that we can uh you know go to 11 with gpu particles we also found that we still need to watch overdrawn next gen so alpha and overdraw is still going to be expensive that's one of the reasons why leveraging meshes with negative space worked out so well for us uh and one of the big wins was that we could afford more vertices on the playstation 5 versus the playstation 4.

and that was a huge benefit that we definitely leveraged and that allowed us to use more meshes where it made sense uh however we still had to be careful to watch vertex density so whenever we had like really small splashes uh we couldn't you know put like 10 000 verts on something like that we still had to uh we had to be like somewhat mindful of like how how dense that vertices the vertices got um so some general learnings from working on this project uh just you know the non-technical learnings the ambition of next-gen was a little nerve-wracking uh but one of the big takeaways for me was that you can't create great things if you don't try and it's definitely okay to be a little nervous uh when you're feeling a little nervous it means that you're on the right track and you're getting outside of your comfort zone and making something wonderful we had to an example of this is when we had to build our an invert pipeline to make that reveal shot a reality uh because the simulation it was really important that we saw the fabric tear away so we could reveal who miles was and if we hadn't really taken that leap of faith i don't think we would have made a shot as great as that shot was

and uh we wouldn't have developed this wonderful new pipeline that we can use in our engine we also took a chance on accessibility features and we delivered a much more accessible game so one thing that was really wonderful uh working on this title is that you know i didn't really go into this title thinking that you know it was going to be the most accessible game ever but uh the more passionate we got about it the more excited we got about accessibility and the more the studio rallied behind creating the most accessible game we could um i feel like we really kind of pulled together and made something great and it was really personally satisfying to see uh someone i care about be able to play through the game from start to finish uh by himself um even though he is a low vision player and one of the benefits of this too is that you know we found all kinds of players not necessarily just low vision ones benefited from the contrast options uh we were also able to give miles his own unique powers that were different from peter's through visual effects so it was really wonderful to be able to tell that story and to tell that narrative of miles becoming his own hero and miles having his own experiences

through visuals uh some other additional learnings when working on new hardware old techniques can be leveraged in new ways so uh using vertex blending uh with detailed color usage to drive material uh transitions so you know we were able to leverage more vertices and that was kind of new but you know vertex painting i mean that's kind of an old technique uh but blending the old with the new with the new was kind of a big discovery on this project i also learned that it's really important to thank your programming team for all the amazing work that they do to enable you to push toward a new generation of greatness so a lot of what we did and what we covered in this talk wouldn't have been possible without our wonderful programming staff they really enable the visual effects team to do their best and be their best i also learned that over communicating is good you know on this project we really had to kind of rally together to create something wonderful and you know the more that we uh we worked together and uh problem solved together the better we were able to deliver this uh project

and lastly um you know i really love this project uh the people i worked with were some of the best in the industry and i'm really proud of what we did together and i'm really uh glad that i was able to talk with you all about the effects that we did for this game and uh thank you very much let's have some q a [Music] you