Destructible Environments in Control: Lessons in Procedural Destruction

[Music] [Applause] hello and welcome to control lessons in procedural destruction my name is johannes victor i'm principal the effects artist at remedy i used to work in film as a fact supervisor and artist for more than 10 years and i have moved into games um in 2019 now i don't really have to introduce remedy because it's quite well known already but for the ones who don't we are based in espo in finland that would be up there and uh remedy has been known for over 20 years for iconic uh narrative driven action so max payne would be a thing max payne too will and wait quantum break and of course not at least most recently control um introducing control uh control is a supernatural third person action adventure and you play as jesse faden director of the federal bureau of control which you suddenly become right at the beginning of the game

and you take over this government agency that is investigating that supernatural the agenda for my talk today is uh well i'm going to talk about the mission the challenge that we have put out uh and i'm going to introduce sort of our core principle for a lot of the effects that we've been doing uh on controls or the principle of granularity i'm going to talk about workflow and implementation and then highlight some lessons learned and a little bit of an outlook on things that we probably would like to do better in the future and to start here a little destruction background breakdown of control

[Music] to the challenges that we faced uh a lot of the mission statement was about the location so the whole game plays inside this government agency's building which is a bit of a supernatural building so they're both the walls are moving and everything is a little bit different than uh than what you expect and it's almost a character in itself and um it's key it's sort of it's very well it's the brutalist architecture it's very suitable for government building um and another key was the believability because it's a government agency that thousands of agents all about and it had to feel like a place that was sort of inhabited by so many agents that go you know about the day-to-day work they have telephones and coffee mugs and their copy machines and they have their desks in their places where they are in the different departments and that was quite crucial to sell that and to tell that uh story of that place um the lighting had um a very cinematic look to it and that

was the desired style control is very big in the whole rtx ray tracing um and uh well obviously the lighting educator for that and had to be quite cinematic and uh well brutalism is all about exposed materials so you have that boarded concrete you have lots of wooden glass and you know the sort of very suitable agency building kind of style now obviously uh when we talk about destruction we talk about the haptics so we wanted to sell a rich reactive environment conveying the sense of being able to consistently interact with anything inside and um obviously limitations uh are another challenge because you know we have target platforms we have to hit so we were after realistic rewarding physical impact and we were no go wild create havoc wherever you go but please don't do that all too much because obviously there's performance and memory that we have to hit and the ai requirements and um while we're a very very small team to achieve all that there's sort of an overarching um principle that i just coined the principle of granularity and that is really

that uh we decide to represent every level of natural detail and every effect that you create and it's actually not a thing that's specific to what we did in control um this is also a very very common principle for any of the effects work for film and you try to sort of capture nature because nature isn't quantified nature is sort of a continuum from very big objects down to dust and very very thin veils of smoke and those kind of things and all that you always want to encapsulate to create this richness this rich gradient across all those different um scales to it now if you look at that how to do this in a game engine uh we're looking at an overlapping representation in three different layers and the first one is well you can do rigid body simulations you have chunks and parts of props and props and environments and all that is rbds and the environments are you know like a big static kind of mesh you can collide with and objects are like chairs or tables and then you've got to start maybe representing them with something else because the smaller you get the more of them you will have on screen so you know looking at something like mesh particles or rigid body hierarchies or material decals even is to sort of

sell a certain material richness on those layers so we're now moving you know from objects to chunks and almost into the debris already and then the last layer actually is pure particles particle sprite mass particles for ember spark smoke fire splinters gravel sand all that kind of stuff that plays a big role when filling filling all that kind of gradient now for control i chose this sort of particular angle inside our research sector um and to sort of demonstrate the the different aspects of this gradient how we fulfilled it or how we filled it in in our vfx work and now what you're looking at here is the static environment this is like all the stuff that can't actually move and um it looks like okay cool you know it's a bit empty and a bit fast and then there's sort of another layer and it's a bit underrepresented in this particular angle but you see like that the railings have been added for example and this would be sort of concrete parts of your environment that you sort of can interact with and you actually see

actually not all that much it's quite surprising and then the big difference comes in when you start adding all the props um and really for that kind of part it just shows the richness the game had to sort of have in terms of like filling everything with bits that you can actually interact with and move which are the props and all those kind of additional items that can be placed and kudos to our environments team they've done a marvelous job to sell uh sort of the richness of the place by using all those kind of elements and um for our workflow it is actually quite a standard thing how you would expect it you have an environment build you have environment artists that sort of provide level geometry modules kits and props and then that goes into the effects department for destruction setup and we had a lot of systemic setups at proper x and also there was cinematic animation for specific aspects of what we were doing and that then goes into northlight which is our in-house engine where everything is sort of running on and but we had to choose a certain approach to this and we went procedural

and like what does it mean what does procedural mean everyone says talks about it all the time what does it mean it's a rule-based processing and interpretation of real data that's really all it is you get some data you have some rules you apply the rules and change the data that's sort of the uh the key of this and how that looked like for us and for this kind of environment process well we got the art models in the world and that was our data and the data was actually amended or augmented with a bit of extra stuff extra metadata that basically told us what everything is made of so we could say hey those cushions are fabric and that concrete is well concrete and the plant is made out of plant and once you know that for an object then you can start actually applying rules on those based on what those objects are made of so you could say well grass spawns bits of leaves when you shoot it or concrete breaks into sort of smaller concrete bits when you shoot it and spawns dust and you shoot a metal pipe it spawns like a water that drips out of the hole and then you add like a bend decal you sort of sell a little bit of a deformation on there and you have a finite set of rules that sort of encapsulates um that whole process and sort of make sure that everything sort of has a

reaction to whatever can happen in the world and then that is fed into the engine um and then they impact vfx and breaking the effects and rigid body behavior that is sort of driven by the material and that sort of creates that interactable environment now why would you do that why would you go procedural because setting all that up is a little bit uh more difficult a little bit harder than just sort of kind of doing it but the thing is you need a consistent and fast turnaround and we also needed a predictable behavior within very clear defined matrix metrics so we needed to know like how many things uh something can break into etc and uh we had hundreds of assets and variations so at the top in that top image you kind of see like all kinds of building blocks that make out rooms and walls and pillars and concrete railings and all that kind of stuff and all the lots of astroplane things in there and at the bottom you see a selection of props which is like desks and toilet cubicles and walls and dividers and planters and plants and computers and telephones and all that kind of stuff and we had so much of them but we only

had a team for well you could see on average one to three people actually working on all of that so we needed something that we could just unleash on certain things that come through the pipe and when they attacked in a certain way something specific would happen to them that we have predetermined but he couldn't do it per item because it was just not possible and it doesn't scale so if you look at the material driven behavior now what you would want to do is on the left you see the concrete you know you should concrete concrete stuff happens and you um look at the middle and that's sort of like wood uh wood when you shoot wood it breaks into chunks so it breaks into those the splinter pieces and then they're like splinters on the floor and on the right you see what happens when you shoot glass and that's sort of like three different uh uh materials are demonstrated here uh so the material-based interactions on dynamic objects and then you see the particles and decals sort of reacting also to what the material is and i'm going to go a little bit more into detail on this so the first part of this is the geometry layer um we are looking at a piece of railing here and it's made out of concrete at the bottom and then a bit of metal in the

middle and then there's a wood on top and from the left to right it's sort of you see a hierarchy of how that stuff breaks um so you see level a that is sort of broken up the concrete it doesn't have decals on it because you don't want to see all the cracks when you break an object and then you see the wood is sort of going onto longer kind of splintery things and the metal is a bit bent and then level b already doesn't have metal anymore it doesn't break any further but the wood does and the concrete does and then even level c also optional uh depending on the material would break even further and now it might look a little bit funny because they're like oh that's like super regular kind of you know this like screams voronoi right there but you've got to imagine that you're not actually breaking all the parts of the object at all times so you're literally shooting a certain corner of it and that's where it would break into all the pieces now we had different simulation entity entities and this is like nothing new like we had rigid bodies that's just you know the physical objects that can be flying around the world and then we had a system that we call chunks and trunks could have bonds and really what that is that is rigid bodies that sort of share a compound collision and in some cases it's called a composite some call it aggregate

it depends a little bit and then we were using joints as regular and uh so chunks as i just described chunks are sort of just a compound and they're created at initialization time so like those two things belong together in the share collider and they sort of move as one until they break so close proximity leaf nodes are basically bonded together and um if you look at joins uh they were just created on geometry hierarchy based on the metadata description so this rigid body and this body are connected by something like a revolute or a hinge joint to be a door or to be a drawer or you know whatever else we needed structurally to represent a prop or part of the environment in the game and they were dynamically breakable based again on an impulse strength and there's something special about the breaking of the joints because uh internally when you create uh sort of a joint between two rigid bodies what we actually wanted to do where we actually did is that we actually connected the two lowest level chunks uh uh inside of that which essentially means that you can break a door um and there's a big hole in the door but the door is still

attached to it on an inch it's not like that the joints would actually break as soon as the the parent box like the rb1 box as soon as it breaks into parts it wouldn't actually fall off the hinges it would stay on the hinges as long as the chunk that is on the hinges is still around and that actually allowed us to have a little bit richer interaction because you can punch a hole in the door and the door still opens and closes like a door for example and then uh help to not create this kind of you know fall apart on impact scenarios but uh which is quite prominent in in in some games where you sort of feel like okay as soon as you touch a thing once it sort of just disassembles into all its pieces so it retains a bit of structure and that is sort of how that was achieved and on the simulator on the simulation site inside the engine it was sort of all north light driven all in-house uh software driven basically the data and hierarchy handling the breaking logic and what events and particles respond around all that kind of process and then internally it was physics that did the simulation of the rigid bodies and the constrained as constraints now the destruction tool chain how to create all that was again you wouldn't

say like that's kind of pretty standard so we have the geometry that comes in we had to do a bit of model prep in some cases we created some sort of bonding geometry that would say hey these parts here of the thing can break and these parts can't break and that went then into a destruction tool inside houdini that's a pretty big sort of hda kind of setup that would then do the whole material based breaking of everything and write out the data to disk and sometimes you had to manually patch some of the physics metadata to just make sure some of the settings are correct especially whenever you did more complex things that contained joints and this kind of stuff certain values had to be set manually in there and then that went into the engine that metadata and the mesh data sort of was used by the engine to then create the world and uh simulate it and the destruction tool looked a bit like this so you had an input geo that comes in like this concrete kind of block and then we were sort of defining which areas can break which is those two wing shape kind of things on the side and then uh we decided oh yeah it's concrete it's tagged as concrete so it would do the concrete breaking and operate it through and sort of create all the different hierarchies that come out of that in terms of render and collision geometry

and make sure that this is sort of all um within the metrics and the style that we kind of wanted to it would add all the decals and 2000 would add like rebars in terms of for the concrete part for example etc and it would do that for wood glass concrete and all the other materials that were supported by it and they would export into engine and then engine it looks like this you end up with a hierarchy it's literally a hierarchy of notes and they are you know abc level uh under a chunk and the name drove with the material would be and also other things about if they were static or not and some things about the joints and the type of joints and all those kind of things and uh so yeah so the hierarchy was represented by the by the layers by the by the sort of the depth um in the hierarchy and the physics properties were different by naming conventional material assignments and all the physics then would be driven off that kind of stuff by the engine when it was set up named correctly as well um and you know that's something i'm going to get back to later on um yeah and then it looked like this basically uh this is the uh oh sorry this is the

rigid body video um showing that scenario again just uh with the rigid body simulations so uh jesse's doing the shooting and then we have this little tool you click on things and then they just explode so that's what i'm doing here i'm basically just destroying everything and you kind of see this is sort of you know physics sim is happening looks a little bit empty but this is all the rigid body stuff that is actually happening in the scenario so now some things about optimizations uh uh on the side especially when it comes to rigid bodies what any point in time we only had about 200 active rigid bodies on screen because that's our our sort of the the budget we had to operate on uh to make it run on consoles etc etc and that meant that we sort of kind of started culling things that were off screen and make them uh disappear um there was a collision delay on big events so whenever there were big forces sort of impulse waves implemented that would essentially disable the self-collision objects for a bunch of frames um to make sure that you don't do two

crazy collision calculations of objects that are fast moving in very close proximity to one another because you don't really need it because if you put an impulse somewhere it usually means it's quite clear where things are moving and mostly in an explosion they move apart so you're not really interested in those collisions so that's why i've used collision delay um while the term collision delay itself i know from the film world as well we did about the same things it helps a lot with other issues like self-interest sections and stuff as well um and then we use the sleeping thresholds a lot so um and well luckily if things made of concrete and drop onto the floor they can fall asleep really quickly no one would expect them to bounce around a lot and funnily enough the more realistic you go the more you can actually bank on the whole sleepiness so you don't have a lot of bounce when you drop something on the floor it really has to be marbles to really have them sort of do do a longer longer lifetime across the floor or something and so we had a really really high sleeping threshold and it's also the reason that we could reliably stack objects on top of each other which is actually quite fun

when you do that in game well and then of course we filled in the blanks with a lot of particles which brings me to the next section um which is uh the particle simulation that we did is everything is systemic and event driven right so the particle events that we have a bullet impact so you should think something happens that's what that is driven by the material what will happen what particles to sort of play back we had one breaking so when chunks disconnect rigid body turns to disconnect they would have a bone breaking event where particles would be released which is nice because then you have like cracking uh you know cracking concrete sort of pebbles and all that kind of stuff you can do with it and then the d spawning so whenever we actually made objects disappear we didn't just pop them off we actually made them disappear um in sort of by breaking into into particles that were predetermined based on their material which if you would see them despawn on screen you would actually see them sort of feel like they're breaking apart and that was very very effective to sort of hide that kind of optimization and this is here just a quick view on how we edited particles this was all done uh sort of

it's a live edit it's a hot loaded live edit this is the game running and you can place down a specific uh particle system and then you can go in and change things and i'm really just going in here changing like the emission frequency for the sparks and i think i'm like oh maybe that is actually too many sparks maybe i go down on that and then i change that um a lot more rocks to the simulation uh and the cool thing is you can literally play it back uh live uh uh so you really get an instant feedback and then you can go around and through the thing and see how it feels like and then go back and keep iterating and that that really fast-paced iteration loop really allowed us to polish that kind of stuff very very very much um until it really sort of felt about right and that was very very beneficial another especially specific part about the particles is standard simulation but we also hijacked the rendering sdf at times so you could optionally collide with the sdf which is while so much faster than colliding with anything else and that sort of took care of a lot of the particle collisions not to have things either fall through the floor look really weird um so we use that

now uh this part here now would be the particles uh and the rigid bodies at the same time um so now it's the same thing as before i was shooting and then i'm going around and just exploding everything and you you notice instantly oh my god this is like so much more responsive and so much more fun because there's stuff happening there's dust in the air it makes it instantly much richer and that sort of uh well it's all for the additional particle layers so that fills in that missing that gap in that gradient of granularity that is now covered by that and the last aspect is material decals uh so we have a lot of them we dynamically spawn them so if something breaks it would sort of spawn sort of ground decal of like broken sharks or the kind of stuff on the floor or if you slam the floor it would crack the floor by decal uh they were generally generated sort of in houdini or substance or things like that and we could dynamically spawn them based on the material they happened on as well and that sort of helped to really also include that quite big portion of static geometry you

saw it in the beginning there's like a lot of kind of static stuff around but as soon as you provide an impact on that it really makes a difference and this is how that looks like right so you know if you hit the floor it still is just a single single polygon like a single quad but it really looks uh much difference when uh when you start adding decals i mean decals are kind of awesome in general and uh yeah we could sell all kinds of effects on that and it's quite surprising how much you get away with although you don't really have a lot of texture budget even just a few of them really instantly make something sort of connect to a destructive element like throwing something on the wall and then you kind of have a dent in there although you know it's concrete it wouldn't really make a dent but it sort of helps with making it feel just about right and we're talking superpowers right so and that's now when everything comes together so we have the particles the rbds and the decals and i had to trick a little bit because just the explosion tool wouldn't actually spawn that many decals so i think i started to launch uh uh something up in the air like i think i take a chair and throw it and then you kind of see that i make a dent in the floor and the floor still is

just static dormitory but the decals make it so much better and uh yeah this is sort of filling the last bit on that missing gradient and the other part is custom props and hazards there are a lot of like things you can throw around like computers and lamps and stuff like that that wouldn't be fully procedurally generated they needed some more custom events and custom things that sort of artists created to make uh just feel rich so you have the fire extinguisher that you know obviously can't be kind of missing same as a computer having sparks and all those kind of things and the cable attached to it and stuff like that so lessons learned the four main things i want to talk about is the quality of geometry of input data because procedural is driven by data so it's all about that name based convention kind of the same the alley really monolithic destruction tools and performance monitoring are the four items i really want to pick and the first thing quality of geometry well that is a problem i've dealt with uh my entire career in film and now in games inconsistent incoming geometry and it can go with while it's scaled the wrong way the orientation is wrong but it can also be the material assignments are wrong you

know something that is actually looks like wood but it's sort of tagged as planned or is tagged as flash or something um but sometimes also the mesh quality being too low and that could mean that the subdivisions aren't nice or it's sort of really single-sided and that kind of stuff sometimes you break things and realize well there's nothing inside but there should be something inside because we can break it and that's that's sort of a difficult part of communicating standards and making that all work and obviously we're using procedural tools to break that kind of stuff so it has to be of a certain way and what can you do to fix it is improve the incoming data standardize across the ccc geometry pipeline making sure that everything is sanity checked and export so at export the the exporter would say hey this doesn't fulfill the criteria it needs to fulfill please fix it before you export and that would sort of help a lot uh to avoid that that constant feedback loop between different departments even and sort of put the problems exactly where they actually coursed it can be fixed at the same time um also it would be good to have embedded tools to simplify the setup so you literally have someone who's

modeling a prop being able to instantly see what it would look like if it gets destroyed but obviously that puts a lot of strain on creating more tools with more user-friendly interfaces and all that kind of stuff and that is a lot of work and then you sort of have to find the right trader for that but we're definitely looking into that kind of thing name-based conventions uh well we name things that then get interpreted to be something specific um and the issue with that is names can be wrong there are obviously apparently there are 17 different ways how you can spell concrete for example and and it's not really to blame anyone because if it's a manual thing that artists have to do it just it's a given that there will always be a certain percentage of data that is just not correct and you can also not express a lot of the range that you need in especially physics definition stuff uh with just naming things and uh yeah and then the mesh hierarchy might not be standard confirmed because you know what happens if you do things in maya you shuffle things around and you do all kinds of operations on it and then you higher you just goes bust um so while all i can say about this is

don't ever do name-based conventions abandon them use physics authors straight in the dcc so make people tag things directly where they need to be tagged um and for that you probably need some kind of unified metadata api so from whatever tool the artists are using to creating their props they are able to actually author that type of data and information as well so you don't have a translation process in between you literally export the data that you want to export straight where it's being created monolithic destruction tool that's more almost like a houdini specific thing but in general well you don't want to have the one tool that does it all um and quite quite often it happens when you start with one thing and then you start adding something you start getting something and the more you add the more powerful it gets and the slower and harder it gets to maintain and very inflexible uh the answer to that is you've got to be more granular but more granular comes with lots of challenges because we're looking we're looking at a lot of individual interchangeable components and the problem is you have to maintain them somehow you need to make sure that you version them correctly you need to make sure that even two years into a project you can open something from the beginning of the

project and you're still able to sort of work with that uh although the tool might have changed 20 times in between so for us it means an improved standardized hda management that we're now operating on and making sure that everything is namespace and everything is properly sort of distributed within the artist so you will never actually lose any type of version of a tool you will always be able to rerun something that you did in the past especially for long production cycles that's very crucial and also what is important that you're making tools that you can fully automate and you just run them but actually use exactly the same tool if you would do it manually um so there aren't actually any differences in them and that will really that is very flexible then for you to say hey actually this is automated really nice but then i go in and tweak it and do something different to it in the places where you need it those 10 of the cases where you kind of need to do something with it but as long as they have the same back end everything should be consistent and that is quite important now performance and testing is almost the biggest aspect of it we had no automated testing a functionality so it was really like you put something in the level run around you shoot it a bunch of times and you

see if it works and it's kind of nice but then something changes in the engine the backend changes something's being optimized and all of a sudden you have to kind of retest and that is very dangerous because you will always forget something to test and the effort is quite big so you will have a potential aggression over time and that is something that needs to be improved to make sure that we don't run into this kind of problem the second part is performance testing uh so we don't have any tangible metrics like what do you it's like you know you can't really go by fps uh it's not as easy as that or like milliseconds spend on calculation because it's not a thing that you constantly do in every frame right so it's a you gotta you can optimize for a specific scenario but it doesn't really cover what everything is and so quite often performance issues only flared up when they were spotted has to do with well we make the props breakable and then they're being put into the level and you know sometimes the props are so nice they're being put into the level a lot and all of a sudden you end up with areas that have sort of lots of props desks computers placed and then well someone in gameplay it's a bunch of enemies that throw grenades and then you earn a pickle

and but you don't want to actually make everything break a little bit less nice just because of that so somehow you need to find a way how to deal with this especially then that you can only back to the full picture very late in the project and the the two things that we kind of can do about this maybe is better performance metrics uh so be able to sort of even if it's magic numbers something that falls out of the system that gives you some number which means if it's bigger it's worse if it's lower it's better you can then start optimizing against that kind of stuff and also to use those magic numbers to establish production boundaries sort of like a physics bandwidth because saying oh it's 200 rigid bodies it's too coarse it doesn't necessarily mean it's bad if you have more than 200 it's only bad if you have more than 203 hand grenades and that's that kind of stuff is something that needs to be balanced out correctly but if you have that kind of metrics you can push the boundaries a little bit more and the second part is automated testing that will help with that as well and an automated assessment of assets through a testing environment um so you can div outputs and see the impact of engine changes another thing that can be done is just performance issue counter measures

so maybe something that can adjust the physics performance at one time and obviously you can't change those destruction hierarchies necessarily but you could say hey maybe this guy goes from breaking into level two straight into breaking into particles and despawn and not into another level because oh my god mayhem is already happening hand grenades everywhere and um so that's one part of it uh but for that actually it would be good to actually have better physics performance metrics because right now it's very hard to get that kind of information really out of physics for example and then the second part well zoning of area is basically expected to look that's an idea sort of to say hey you know you might not have known that in the beginning but now we have an area with lots of really awesome breakable props and we have guys in hand grenades and what do we like enemies and anthony's what do we do and maybe you could put a box around this and say hey in here you know heavy load physics maybe you know ignore the last level of destruction hierarchy or stuff like that and that would sort of help us to manually zone certain things where we know things could get bad um to avoid having to downgrade all the assets for the entire game just because of a

certain area and that could be another thing that could be done about this well to a conclusion to come to the end well control's reception was very well it has generally been quite liked and especially being quite liked for its destructive environments and for the feel of launching an object into a bunch of desks and enemies and have everything go to bits um so i think we are quite happy uh that we achieved this um the workflow held up better than expected almost so it is it went really really well considering the small team that we had and uh we are quite happy that this actually worked and we are quite inspired and excited to see where we can take this one next and uh what remains for me to say is thank you uh to matty hemline and who did the greater part of the destruction for the destruction tool and all that kind of stuff and then samyakiran on the northlight side doing all the physics uh programming and then your first work for the particles and also the render side but also the entire vfx department um at remedy that sort of obviously partook in all the different moving parts of control and mated

made it what it is today so thank you to them and thank you for tuning in [Applause] you