Procedurally Crafting Manhattan for Marvel's Spider-Man

[Music] my name is David Santiago I'm a principal technical artist at insomniac games working with procedural systems in pipeline and there been many talks by insomniacs at GDC this year and if you missed some of the ones early in the week I'm sure they're recorded and I will be mentioning some of the upcoming talks during my presentation before I get started and run out of time I want to make sure to thank everyone in insomniac games for their support and preparing this talk and I want to thank in cymet for being such a great place to work and insomniac e is hiring so visit our booth on the show floor to talk to people at openings here are some of the current openings I can show this slide later and we do I believe still have openings for a rigging in turn in both studios this summer and I'll be on the at the booth on or off the next couple days so I'm going to talk about a game that some of you might not have platinum yet and I might reveal some secrets of

the game that have already been published in every language on the online so is show of hands so I do really want this information of who hasn't seen any spider-man open-world content at all okay as well so I will be sure to play the video even though it stopped working I will just get out and play it how many people have not played the game alright well that's pretty much all the information I need so so if everybody else playing the game so how did it start and how do we plan on making this game let's jump back a few years there was a meeting where Brian Andujar of creative director revealed the next game that insomniac would be making for the PlayStation 4 and I actually wasn't at that meeting but I heard that people freaked out I remember the moment I was told you're gonna be working on spider-man and I was like what what why we're doing spider-man you're kidding and it was hard to

believe but I was very quickly convinced that's what we were going to do so I was being tasked to help build the cities and the technology I first developed on Sunset overdrive creating ground buildings and markup but for Manhattan and now that's a little more real estate to cover but but I had a plan I reviewed Brian's announcement presentation which set the game's pillars and goals for every department and it was a mind-blowing amount of content to create in a relatively short amount of time and then in 2016 the public rumor started that oh there's a spider-man game coming and fans started sharing their expectations online so III 2016 we opened on a shot of the city transition to this street with traffic and pedestrians and hot dog carts and we cut to this card and people got excited and while I'm on this card I'm remind to

thank Marvel games and Sony interactive entertainment for their support and most importantly I want to thank my family and the family of micro families and my co-workers for all their support and enthusiasm as we threw ourselves into making this game so the 2016 III crowd they really erupted when we cut to the shot of spider-man there and then he started swinging down the city but now for all of us at insomniac this was real and the quality bar the minimum quality bar was said so from the start we work to make sure that all of our demos ran real time on ps4 hardware and from the beginning we had a strong plan which actually got us to this 2016 III reveal but that plan wasn't yet strong enough for the challenges of the next two years and speaking of that Brian in ihara the creative director has a talk pretty much right after this 11:30 in the South Hall so you might want to catch that but my talk today is about the procedural systems and how we integrated them into

the part pipeline as part of that plan to meet the challenges so this is that video that last week didn't play and then it stopped started playing and now it's not playing again so let me play this real quick if you are able to see it do I have to put it over somewhere else do I need to quit the other presentation maybe let's close that alright so since it's not so for those of you who hadn't seen anything so here is spider-man swinging through the open world don't need to play too much of this I think he Ola you'll get it alright back to the presentation it's great when with PowerPoint ah

shift f5 alright so let me give you an overview of my talk today so we're gonna talk about what the open world planned content was just for a procedural content what the challenges would be I'll give an overview of the procedural systems that we mostly ones some of the ones we used on spider-man I'll describe our original pipeline plan and how the project evolved as production happens and how we updated the pipeline to handle it like I'll mention some late additional challenges we met to finish the game and then most importantly what we learned so here is our marvel island of Manhattan but what is spider-man what is what is in this game so it's an open-world game of approximately six kilometers by three kilometers of designed playable space and we have the nine districts of Harlem Upper East Side Upper West Side Hell's Kitchen Midtown Chinatown Greenwich financial district and the raft and

Rikers Island prisons and Central Park there are 544 roads in 1202 alleys those are mostly fictitious alleys either actually very few of them in Manhattan over 8,300 distinct buildings in 3250 edifis prefabs and an edifice it contains the contiguous buildings or structures within a block more than 350 different types of storefronts which are uniquely customizable more than 3000 crimes more than 3000 vignettes and I think there's a story missions and a hero so how will we do it well to make such a large game we new automation and procedural systems would be used on a really large scale and in some net games is not a big company I mean I think during spider-man we're about 250 in the entire company and we're always working on multiple titles so we knew that automation would be key using macros scripts etc and then procedural systems

so what are procedural systems they are methods for creating content algorithmically rather than manually and so data flows through a set of operations that manipulate the data based on parameters attributes and other input now this is not real-time procedural I mean there's always some small real-time procedural systems but these are procedural tools to author content and our goal is as much as possible to ought to procedurally create a procedurally authored open world Wow it's it's a marble talk you can't have to do something like that so and I just thought of that like two weeks ago so pan crafting is very important at insomniac games designers and artists handcraft the game experience as much as possible throughout the entire game and procedural systems are a tool to help author a lot of content quickly and give the artists and designers a big head start so the goal for spire man was to procedurally generate an open world

early in production to give the designers and artists the rest of the remaining of our production time to polish the game and our target was 80% useable procedural content and I'll have more on that later so we started with a foundation of systems and tools from sunset overdrive and we had to augment that foundation to create the basic tools and systems for spider-man sunset overdrive we built roads on terrain and placed a relatively small set of props and traversal markup and there was a building system to help block out areas of the game but the building Sun sense that were so unique that it was just more efficient to constructed by hand at that time so we had a plan for what the main procedural systems we needed so I'm going to be using this east up orientation for the island throughout the presentation so it fits our ratio better and so hopefully you get used to it it took me a while to get used to it because I we weren't working this way either so at the time in

pre-production the planned procedural systems included ground system for terrain streets sidewalks some decor and integrated markup buildings with markup the traffic system for pedestrians and vehicles and a first pass of propping the open worlds really spamming the props out there so step one for the ground system our roads and alleys roads and alleys define this city everything is dependent on their placement size and lane configuration now a big hint for this talk dependency is very important so from these lines and curves and editor we will generate the streets sidewalks grey spaces and the sea wall around the island ground modifiers affect how the environment is constructed so this yellow overlay is the contents of our ground modifier zone now these are additional curves polygons or volumes that cut deform modify add materials etc to a particular areas of

the game there was also an analogous design modifier zone this is the ground modifier zone a design modifier zone to affect a design content such as crimes traffic BOTS navigation etc so we're not going to generate a giant model for the ground we're gonna make models for each streaming region so our open world is cut into square tile streaming regions 128 meters on a side I must say it about 128 meters on a side no it's a 128 meters on a side so if you see here there's some double lines I mean they're not important but it's an interesting detail in our system they indicate a history of scaling and offset of streaming reasons regions for various reasons and we actually preserve that history so we could validate content to make sure that it was fitting into it's appropriate streaming region now there are very few actually scaled regions in the final game and mostly they're on the coastline just to reduce the tile account you know just make it a little bit bigger we can fit the content into there and you'll notice that Rikers

Island and the raft prison do not have streaming regions but that's as far as procedural Sims are concerned because there is no procedural content in their final version tile IDs so each region has a tile ID which is a map location from a o1 to x-48 and each tile hierarchically contains all the zones for its open-world content for that region and you can notice that lonely tile in the top right corner there that's X 48 and it's our test region that was there throughout the entire game so here is a procedurally generated ground it's typically 16 models per tile so that each is a manageable size for physics and other systems and this is actually a debug version of the ground which is why it is not complete in some areas and you also see remnants of Rikers and raft up on the top there which was later completely handcrafted also note that Central Park's ground was hand authored

to fit the character of the park in the area allotted so eventually all of those manual iterations of Central Park were procedurally processed in order to integrate them into the game the last part of the ground system is kind of actually the beginning of the prop system it's where we add some ground details lane lines crosswalks per district materials and such and it's also we also do a little first passive Street props such as manhole covers and sidewalk props such as traffic and streetlights and a few things like trash cans trees just to fill it out for designer testing so when they're swinging through the city up don't hit that tree onto the building system so buildings were constructed from primitives such as boxes cylinders and polygamy volumes to define the outline of the floors of the building and then these simple buildings are then processed by the procedural system two instance assets from several building kits onto the faces of the gray block and then there were roof models that

could be sized and tiled to fit a roof or a custom roof model would be created automatically if the procedural system couldn't couldn't make it out of pieces and then some rooftop props were automatically added to these building edifices at this time so things like the HVAC s and those water towers the Prezi store system also added all the markups Winky and volumes and various types of ledges ledge paths for traversal and then as we were laying out all these buildings we notice that anytime we change the streets we might have to change the buildings some talks related to this doug's talk concrete jungle gym the building traversal and spiderman that is tomorrow morning and tomorrow afternoon programming talk about our tools our in-house tools editing with immutable data by by Ron so our building tool was originally designed with a recalculate markup only

mode so if a building was hand modified or if it was complete built from scratch our system could automatically read that edifice prefab generate the markup and that markup was always optimized by joining all segments to minimize the number of curves minimize the markup actors so that's pretty much the building system and on to traffic system so from the road pedestrian and vehicle traffic were procedurally generated so as far as the pedestrian traffic system the first thing we did is we placed excuse me static pedestrian volumes on sidewalks and so those volumes just populated the sidewalk with a certain density of pedestrians and they were they would just be standing there to start with but then they could get promoted to the pedestrian system to actually walk around so after you placed those volumes then we would create a network of paths that would connect from corner to corner so corner to corner along the sidewalks corner to quarter across the street then we would integrate any paths from other assets like bus stops and subways because they

had their paths that we have to come up the stairs and around the subway or around the bus stop around the bus shelter so those had some built in curves they would be integrated into the pedestrian system and finally we'd integrate any hand authored path so there was a lot of hand authoring of paths which pretty much just curves through parks and plazas and maybe some other spaces so those were all integrated together and then they would generate the pedestrian traffic nodes so that the actual gameplay system would use and that would include the correct traffic phases for each link so that when the light turned green they could walk across and one day it was red they would hopefully stop actually would turn yellow and they would speed up and run across the intersection well except sometimes you know they just stopped but so that we had this debug with the show with the color all the face is to make sure thing was all working correctly and the pedestrians as they II reached each node they could decide on whether they wanted across the street or just turn around the corner and go down the sidewalk again so now on

to vehicles it's very similar to the pedestrian traffic system so but first the parking and driving lanes had to be chosen from a variety of the lane template templates from each street size so each Street would could randomly if it wasn't assigned a specific template of number of lanes and and in direction and then the crosswalks from the pedestrian system became the stop lines for the traffic at the intercession so just like we populated the sidewalks with pedestrians of the volume we used volumes to populate the parked cars in the parking lane the only difference is we had to create multiple volumes to avoid bus stops hydrants alleys and other exclusion areas where you shouldn't be parking your car we also created a network of the directional paths that would take a vehicle from a intersection intersection and then at the intersections give them the legal or desirable turns that they could make then we would integrate the alleys as limited paths so if it was a drivable alley so see it back in the distance there there are those magenta lines

those are alleys and they come a come in too and they are integrated and I will talk a little more about their their limited use later and then we would create the traffic nodes with the corresponding phases and links to other nodes and the lane markings based on the traffic pattern would then get get applied in the with the prop system so here you can see the route choices so these are two two well there are two two lane one-way streets that are crossing each other and that lane on the Left which is actually the right lane going down I don't know if it's south east or whichever way it's going but they have the choice of turning right into either the lanes or going across the intersection and the other lane has no choice they're only allowed in this traffic situation to go straight across the inner section now here's a more complex area with pedestrian vehicle paths and so it's multi Lane streets so these are these are avenues and so it gets a little more complicated as to what people can do if you look at that lower intersection in the middle there you can

see that all the lanes that have the green light so they can turn two lanes right into that one-way street and the other ones can go straight across and then when the traffic phase changes that left turn lane going from right to left they're allowed to turn onto that one-way street and then here the same the pedestrian traffic for that area you can see the hand authored paths through that plaza there and yet pretty much all they had to do was draw those curves so they intersected a procedural path and they would automatically be integrated into the system so now the dekor system which is pretty much props and and decals and then and then some ped prefabs cosplay explained in a minute so we procedurally optimize placement of the street props such as manholes skid marks tar asphalt patches and such and sidewalk props which include street lights traffic lights mailboxes hydrants trees and on and on and on now also pedestrian prefabs were part of the

system so pedestrian proof shows where things were like bus stops subway entrances hot dog carts sign spinners cab hailers so there were assets with integrated actors that could be promoted to bed at pedestrian behavior so say people are waiting at a hotdog cart they would wait there and then they could be promoted to walk along the along the path or flee from a crime if necessary so whenever there was an encounter the pedestrians would go into a flee mode and then we could also integrate any hand place pedestrian prefabs of which there were also many they need to be integrated into the the pedestrian prefab system in order to populate the game appropriately at proper times we also did a little bit of procedural lighting and I'm not going to talk anymore about that because today at 2 p.m. x-ray is going to be talking about our procedural lighting tools so also in our original design our tools already had the sophistication required for the garb debris propping of act three maybe I

should act how many people made it to Act II of the game where there's stuff everywhere alright so the challenge there was to make it look like there is litter and debris everywhere but not block any path needed by vehicles pedestrians crimes vignettes missions etc so that was it's a little tricky and I know there has been much interest in puddles in the last nine months or so those blue circles there procedurally placed puddles if wetness reaches a certain threshold in the material which uses a random seed a puddle is place in the world and active during rainy atmospheres and then it will splash and ripple and such so at this point we had all the tools to place the props now Impostors a fairly early in pre-production it was decided that impostor generation should be part of the procedural systems since we generated a lot of the content whether it would turn into an imposter so here's an overhead view of the imposter version in the city and it contains every

structure now imposters are lower res representations of every structure in the city and we generate a simplified model from the high-res geometry in the entire edifice prefab and then we use that simple model for our imposter generation to project textures and make an atlas and then we populate the imposter zones with the correct updated imposters so that we can put them in the right kind of containers for optimizing a memory and and such and then we can pursue these early regenerate them when ground structures materials or lighting changes so pretty much we're generating imposters over and over again and here again are all the imposters and showing that the entire city is always present whenever you see a shot of the city it's pretty much all imposters so at most only twelve zones or ever ever loaded high-res and even most of those knowns are going to be in their lowest LOD which is very similar to the content of an imposter so that's it we've designed

and implemented all the systems we need to create open world Manhattan in less than a year of pre-production and now we can generate all the procedural assets let the other teams polish it up and generate imposters and we're so it's party time it's party time we can just sit back and make the game but the design team I don't I don't see the design team at the at the party at all what what's going on well justice we feel like we were going to be able to handle all the challenges for the open world the designers wanted some procedural system love so we had some new challenges the first set of challenges were dynamic encounters the dynamic encounters are crimes and vignettes now we place 29 different crimes procedurally throughout the world and 25 different vignettes for a procedurally placed using different rule sets another challenge at that time from design and gameplay were additional desired markup now that we were getting some new hero moves and then bought navigation so an example of the rules

required to place a crime or vignette like a jetpack a jetpack mission or jetpack crime so what we do is have a four jetpack crime and have to find a roof in the city with 64 square meters of combat space available that was within 40 metres of two other rooftops with 64 scuitto square meters of combat space available which were within 20 meters up and down from that original rooftop and there was a clear path from each roof to each roof so I could put in nav and then also so I could put nav from each rooftop down to the sidewalk or down to the alley below so the system would run through the world find all those possible places based on the buildings that we're currently in the game and then based on the density that was required or neat or desired for each district it would place all of those crimes or in this case those jetpack crimes and create all that markup for the for the actors in the bots so if

we're going to procedurally author all that content with all those dependencies and have it be handcrafted we need to figure this out and make a pipeline so we have to think about that crime in vignettes are dependent on the environment and changing a road or alley will affect contiguous blocks defined by those roads and therefore all about everything in those tiles so buildings crimes etc and even changing moving a crime effect nine tiles because of its depending on its complexity because crimes drive prop placement we want to avoid props or static props in the world where there's going to be a crime and then changing building will affect traversal crimes again lighting and an audio did I did I mention audio audio wants procedural help so just to start with audio we generate some procedural systems to generate the reverb volumes for audio and also be able to distribute audiences that appropriate locations throughout the city and speaking of that tomorrow the Otto team designing the

bustling soundscape so alex and blake will be presenting on audio for the game now back to our story and all of this stuff that's a lot a lot of procedural systems a lot of rules a lot of dependencies so we need a plan and and that is to create a pipeline that completes procedural content in phases that we can iterate approve and lock content for all the handcrafting I mean because we're going to go from this to this it's just going to be a little more complicated so we got Houdini's so although Houdini from side effects software is our main package of choice to develop and manage our procedural systems in production the algorithms can be implemented in any software package and in fact we have plenty of code for generating procedural content that is independent of Houdini or any other third-party DCC software now here is a circa 2016 implementation of our open world pipeline involving procedural

systems and it's a very simplified diagram without every possible error or box because it's just unreadable and it's broken the roughly three phases so in phase one it's the main world layout we're predominantly designers place roads rough out buildings to find Park areas and mission spaces and so forth and then we move on to Phase two where the majority of procedural content is authored and then finally phase three the polished stage where the procedural systems will just help finish the game and let me quickly point out that Impostors and lighting data are being calculated during every iteration of each phase and during phase two and three audio and UI data are also being procedurally generated as well so the game is always playable in its most updated state so here's the first phase the first step is designers layout curves to define the roads well actually the very first version of the city was from an illustrator drawing of the map for our Manhattan and the procedural system could ingest that image and then create the curves in

polygon primatives in our editor the procedural system would also randomly place a rectangular box buildings in different zone to block up to block out the city and it would actually read color data that the designers put down in order to vary the range of building sizes for the different areas so once they got that to a certain state we were just going to work just in the editor so the designers would modify curves in the editor to redefine the roads and they could specify road directions or dominate direction if it's a multi-directional T Lane bi-directional Street Road width and the road width was from outside a sidewalk to outside of the sidewalk Street width is from curb to curb they could alter curb height Lane configuration and and so on so initially only the main avenues were specified and the procedural system automatically chose street sizes directions and lane loud layouts for all the rest of the streets so we just told it something like we want 70% of the

streets to be one-way streets and 70% of those should be one traffic lane with one parking Lane and then the system just went through the city to create workable traffic patterns and we got those alternating one-way streets as you might expect but as time went on the designers would specifically choose sizes and Lane configurations for many the streets to accommodate story and and chases and such but I would bet that most of the streets sizes and directions are determined procedurally so also to start we had very few alleys there were some blocks that designers knew they wanted to break up but until we had buildings in place it didn't really make sense to try to place alleys so then using those road curves just like I kind of talked earlier we proceeded you generated the ground that's just like that so then the designers place the parameters to define the buildings and in this first base so the rough elevations and profiles are created and procedural systems can modular eyes the buildings using those

basic architectural sets and then it would any would apply the markup so now designers could swing through the city and test out their space so his swing traversal possible is desired Billa wetson changing urban landscape lead you along the desired paths so we have the correct variety of density and shapes for the different districts and neighborhoods so there was another talk here at GDC that actually went over all those decisions the one thing to keep in mind is that this early in production spider-man did not have his complete move set of web zipping and parkour or pork point launching I mean it was some pretty basic stuff so we were trying to approve this layout with some limitations so as we litter ated in this first phase there obviously should be an arrow back to Road curves so as and then as also as separate buildings were used to fill the block the procedural system would automatically create the alley curves in between them and you might ask why do we actually need the alley curves instead

of just saying well the space between buildings is an alley well again some of them are drivable used for crimes and chases and they can be even used by regular vehicles their Street is is blocked by a crime or a vignette so they could take a detour down an alley we would also use those Kirstin form the procedural systems for change in the ground surface props lighting audio needs etc so after this first phase we should have the streets pretty much done and the rough buildings can drive our next phase phase 2 is where the bulk of the work for our procedural systems happen and there should be arrows going from each block to any previous block because any change based on a dependency requires some reauthorization Dazz possible if nothing is breaking the pipeline we will generate everything in his face based on all the changes to keep the game update updated and playable that last bullet is very important it means that during this

phase artists and designers should not hand anything that a procedural system generates they must use the controllable parameters and inputs to the procedural system to affect change if they don't they're at the risk of losing work so here is the what we call the art polyline pass and that's the opportunity for in the environment artists to replace the primitives in the designer zones with multiple polygonal volumes to further define the shape of a building and the artifice the artists could also specify the architectural sets that they were wanted to use for the building in this phase the artist could also modify or delete the alley curves and saying that they didn't want an a drivable alley there and then we could recalculate the ground for them and in reality this diagram this phase two represents two almost identical phases that we call the two stages of environment modulars ation passes now iterations of the environment arts first

modules a modular ization pass would yield the structures with their final layouts and shapes but the architectural styles and propping could change in the next stage the next round of phase two or phase three now after this face that were we should have accurately at the end of this phase I guess after the end of this first stage of Phase two we should have accurately placed and approved the pedestrians traffic vignettes crimes props and accuracy is within the scope of our procedural system design so the goal was 80 percent completion using procedural systems so that means that 80 percent of procedural content would be ready to ship or completed to 80 percent so that art and design would have at most 20 percent of the work they would traditionally have so that means it's something that used to set be a week of artist time should be a day of artist time five weeks one week so what we're trying to do is generate content to rules and metrics and cull to Ballpark but not the final spec because the manual polishing passes

would take care of the final adjustments so at this point phase three design has approved everything and there's arts sign-off after the second round of phase two and every team is hand crafting their parts of the game and the procedural systems will just date things like markup Impostors lighting audio etc and there should be plenty of production time to develop procedural assistance to add any new traversal or markup or crimes or vignettes and then we can also start working on optimization and validation tools to improve performance and help find bugs so this pipeline was created to work within a specific plan and that plan is to develop the island uniformly through several stages of completion and we need to lock the rough map layout and structures and define the key spaces before we can work through any detail and so we got to start with the island-wide systems so again we're gonna finalize these roads in the map layout

so oh this was yeah this was a like a little clip showing the changes throughout the that the production after pre-production but it's okay it's not that interesting the so the generated cond we then there we do have the generated grounds which did find the key spaces in landmarks so we're gonna work one district at a time adding detail in iterating on dependencies within that district to bring it to the next level of completion and that was fun in our pipeline design so as long as we worked through the island district by district maintaining uniform levels of polish we could treat each district kind of as its own island and because of the unique attributes and establish boundary conditions for each district we already were handling the shared edge tiles in a special way so for example here is Greenwich district in phase 2 stage 1 so at this point we've gone through phase 1

so we finalized the road map and the rough buildings defining the silhouettes and traversal spaces and they've been converted into prefabs of instance architecture for the whole city so now phase 2 stage 1 art does its modules the first module ization pass they finalize the shapes of the buildings artists are assigned neighborhoods each artist assigned a neighborhood they do their stuff they work through the district then a district is a level completion where we can run our procedural systems to place the crimes vignettes and props and art moves on to the other districts to do their first one of Phase two but we found as artists work through the neighborhoods and districts they needed to adjust roads and alleys for the character of the space so the placement direction or size a road or alley in one current district will affect traffic flows from multiple districts so we might have to change even more roads so at least to a domino effect of changes and that brings us all the way back to block 1 of the flow chart also due to different milestone missions story development items not

only were different districts at different stages of completion but one tile next to another tile one could be in rough and one it could be in polish and but in order to place our majority of our content a tile has to be completed in environments Art's first module ization Pass and gained several approvals and that wasn't happening so there was too much work here and not enough work here and that smooth progression was was upset for our development now it's not that the procedural systems couldn't regenerate the content but manual work could be lost so for example ingesting an alley just just adjusting the alley width in one region could affect many surrounding tiles such as the UV continuity around the block because a block could span many tiles proper placement proper proper placement it's a tongue-twister proper proper placement and the placement of crimes and vignettes in that alley or vicinity I never mess that up when I was rehearsing so that created

a domino effect beyond what we imagined now honestly we or III expected it to happen at edge or coastal piles where there was minimal gameplay content and I knew those would get neglected and fall behind so they were just crazy amount of dependencies every every tiles regions content is dependent on at least its neighbors and best case they're in production in parallel so not only are districts being developed to the different levels of Polish but the individual tiles as well and there was new content for dioramas missions demos and other test areas that we're all over the city and so things like adding a median marinas more roof top features they required at least update of additional markup for the hero and bot support and we also relocated several story related locations and I was personally a little bit stressed and frustrated that we kind

of weren't sticking to the pipeline that that we worked on ah so what do we do we get the leads together and we figure it all out now it didn't happen in a day several meetings on different topics with the right team members over several weeks at least and some growing pains were had by all I mean we needed to know the targets and goals for art design and gameplay one of the main systems the ground buildings etc are going to be at a level of completion that we can move on to everything that all the other processes that depend on them and then when it's work on a tile neighborhood or district gonna be at a stage where additional polishing of the art crimes missions etc will not affect any downstream processes because then we can more confidently hand off work to the next procedural system with the less fear of having to go back up the flowchart and then at some point we'll get to our final imposters lighting audio and UI and and we'll be done

speaking about more about how we work as a team Jason our environment lead has a talk tomorrow case study and empowering your artists so knowing what everyone needs to finish their work we make a new plan for our procedural systems and where possible we branch the dependencies so that each tile region can be independently edited as much as possible then each procedure law ization operation should be modularized based on its dependencies so for example traffic lights traffic lights they're really only dependent on the roads and the sidewalk corners once we know a direction of a road and where the corner is we know exactly where to place a traffic light so at that point that we have all of its dependencies set that's when traffic lights or anything with the same dependencies should be placed in that module mailboxes for example they have additional dependencies such as avoiding alleys bus stops crosswalks things like that so anything with those dependencies should

be its own in its own prop placement model better rules so we need to also need some better rules to catch the corner cases always lots of special corner cases we want to get these rules so we can definitely exclude the invalid content now we don't need to be 100% accurate with our placement of content since we know there will be handcrafting passes but we do want to make sure that at least 100 percent of the planned procedural content is in the game we will placing some metrics and then we'll indicate assets that are in the ballpark but with which we are less confident of their proper placement right because it's easier to try to place these a hundred and sixty crimes out there and know that every people have to adjust them and put them in the right spot other than just saying oh I can only place 120 of them and then relying on the artists and designers to count and remember that they have 40 more crimes they need to place and placing them in the appropriate areas so this system for

verifying confidence of placement it morph into a validation system that was applied later to a lot more content which was whether it was procedurally generated or not so bringing order to the city bringing order means getting the teams on the same page and as many contiguous tiles to the appropriate level of completion for our new modular process so originally each district was a set of contiguous tiles one now it's like I don't care just give me some contiguous tiles so we can work through this game and the the new procedural processes that preserve additional handcrafting would be interleaved with the artisan designer work now that's easier said than done but what we got it done then came a few more challenges so we had some wall crawl issues and on some new assets especially some of the new storefronts so we needed some additional ledge path markup and wall crawl collision added then it had to be added by the procedural system because it was

particular to which buildings kits the storefronts were added to also a little bit later in production a little bit late in production we implemented the act system so we knew that that Spiderman was gonna have these acts and different situations and times of day but later what is late in production when we created the system of overlay zones that would control the state of the world at different times of day or weather conditions or different acts so that way stores could be open in the day closed at night closed during act 3 there could be different groups of people out and about depending on these situations and we could modify what was in the city and act to and act 3 such as all the garbage and in different vignettes so that solution was proposed in October 2017 we designed it in 2000 December 2017 I went on holiday break and then with our implementation was completed in March of 2018 and then the

content was complete well before we shipped later in the year so there were also some late mission space updates that required reoffer and open-world gameplay in several areas and I was pretty happy with how the new systems handled it and then did I mentioned we did audio so we had to generate a bunch of custom audio collision models to match the visible textures that were being presented by a material and that was a significant in Central Park so lessons learned I've talked really fast so we already knew this we're the case a strict workflow is best but we need flexibility for production we want to lock procedurally generated elements with their dependencies as soon as possible as soon as something's locked we're done with it we can move on we also want to harka Klee modular eyes the procedurally generated content to reduce

dependencies so that's what I was talking about the traffic light once we know the the dependencies to place that traffic light or any other content that has the same dependencies we want to do that then we can use that as that might be a dependency for other props or other content that would come later so modularizing it will reduce the number of dependencies if you have all those all the props in one module then you've got every possible dependence going in there you don't want to over define the goals of our procedural system modules now you don't you don't need to create the perfect rules to place something first a lot it takes a lot longer to make the perfect system I'm at I think it supposedly takes in an infinite amount of time to make the perfect system and then second if you if you only play something with strict criteria you're not going to find that many locations that meet the meet the criteria so again I was like I was saying before it won't be it won't be out there in the world to be nudged into place it'll be something that will have to be hand added later which takes a lot

more work so it's easy to if something's there to say oh I'm going to just nudge this to a good location also are the perfect rules or are a lot of an overt over defining your goals and is increased constraints and constraints for dependencies and we're always trying to reduce dependencies for each stage of procedural authoring because we know things are gonna change and any time in so we want to have a limited number of dependencies so it's a limited number of changes now we can create procedural systems that preserve manual work so once authoring hand authoring has begun we want to change to a mode where procedural ism is a pride applied appropriately so whether it just maintains the hand authored work or it augments it or processes in another way also compound dependencies should be split into separate processes now compound dependency is when a single asset has distinct sets of rules of replacement or has a significant level

of branching in its rules so for example say today we have a crime that can be placed on it needs to be placed or can be placed on a road and we want place it like in the middle of a block or in an intersection and then we just to orient it with traffic but then also that same crime it can be placed on top of a rooftop well we don't want to have all the rules for placing that we don't want that all the rules for placing that crime in one module we should separate it out so okay the once we have the road dependencies we'll place those crimes with that there and then once we get to the point where we're the building dependencies we'll place that same crime based on based on those rules so those are the compound dependencies and then again compound dependencies they limit your options for a single asset and will will increase dependencies for all the assets if you have compound dependencies in you this big Network then it's going to create unneeded dependencies for for the neighbors we want to tag procedural content well we want to tied everything but knowing if something is procedural

means you can confidently regenerate it again and again without destroying any manual work now this can be a little bit tricky since an incorrect workflow can allow an artist to prop out around a procedurally placed an instance without creating any indication of the dependency so for example an artist can augment a procedurally placed vendor cart and they put additional items better there outside of the proper hierarchy so procedural system has no knowledge of any of those additions and so if it decides oh I'm gonna move this vendor over here it's a better location well you might end up with floating soda cans and pretzels in that spot because they're not within the correct hierarchy so that particular simple case didn't happen but there were some similar cases also knowing if something is completely procedurally authored means that it might benefit from Sam hand crafting so it's like oh that's just in its in its procedurally generated State let's see if I can custom customize it too it's it's part of its specific location we

want to perceive lis validate as much content as possible now we want to find content that is not placed as metrics would dictate so for example a spawn point is inside of a building that doesn't have an interior so we want to delete nudge or modify automatically only when we're absolutely certain of the result so if that spawner is no longer usable in any way we know we can delete it but if it could be usable outside of the building we'll try to adjust its position as defined in our in our systems rules knowledge and then if we can't be confident of that procedural adjustment we need to indicate the possible issue for the artists and designers so that they can make the decision now what we did for many of those cases for dynamic elements is we put those assets in a state where they would not build or spawn unless someone got around to Hannah and modifying them so ladies later in production the procedural system was a lot smarter about placing all the breakables and throwables for crimes as we had to do it over and over and over

again so the procedure assistants were actually doing a lot of that work and we would put those assets that we couldn't find a proper location for in this state so we lawn learned a lot of things that we can do better and this afternoon ilan has a technical post-mortem but overall I think we were satisfied so repeating myself the original goal was 80% of procedural content would be ready to ship or completed to 80% so that Art and Design would have at most 20% of the work they would traditionally have now we exceeded those 80% goals but all that additional time went into additional iterations and additional handcrafting of content and polished yeah we did it would we didn't have there was no leisure time you know trying to get it get this out the door so now that you know about the dependencies and what has to happen when we change the ground I'll tell you that that road curve zone was locked in late January 2018 and the last

procedural change to the ground occurred in late June 2018 a little over two months before you could get the game in stores I like to thank you all for coming I really talked fast so there'll be lots of times for questions thanks Sony Interactive Sony Playstation Marvel games side effects software Jeff Hannah my advisor and everyone at insomniac games four have a have making it such a great experience a great adventure and now I will open it up to questions so there's microphones on either side I can put all those job openings if you like hello okay I worked on we happy few and we have a procedural system a much smaller system it was similar structure

and one of the problem we met is communicating with the artists the consequences of a change into a handmade tie because they are clearly as you said dependencies between handmade child and the procedural generation around it it was really hard for them to imagine all the consequences of simple change and so my question is how did you explain what could be the possible consequences because Houdini is not that simple it's a complex pipeline and so well so at first you know there's always a little hesitancy for the artist to give up this generating of you know offering their buildings you know they want to make their buildings but as soon as they saw that they could just change a safe and then boom and it would instance all the stuff for them they're like oh oh I like this I like this so they I mean sure they're there some bumps in the road but

they started liking that they would just do their polyline work and get the structure they wanted and they could instantly our tree they could test it that it would could go through the whole you know through the whole pipeline and then later on I mean it took a little bit but they had to they did have to learn and and kind of trial and error as far as what they could change and what they shouldn't change what could be preserved and what wouldn't be preserved different things about how to it just I mean it's particular to your pipeline but how you do override so what things could be override what things can't be overridden just there's all those little details of the dependencies and the hierarchy in your in your data structure it's just it's just things that have to learn and I mean we had a small team so a really small team so getting everybody up to speed especially up for the open world it was overall not that not not that challenging I mean it's we just had lots

and lots and lots of iterations and and they learned when to when they should just stick stay with the procedural systems and when they're gonna cut it off so sometimes it at some points right they would just say I'm cutting this off from procedural systems at all and I'm saying I and our system say okay fine and you I mean do you have to do everything hmm and that would be fine okay thank you there is so much going on in that city and I can see from your total that used in posters and you use lot systems but I've noticed that you're building some made of different blocks with and as you go to the city they change Elodie's yeah like small pieces not the whole building or theater sometimes yeah and I'm just wondering how did you manage having so many objects and what are the decisions you had to make to make the procedural system handle so many objects in a performant way well for the procedural system every every building there were only certain building sets building kits that were would be used by the

procedural system when it was originally authoring the city and each of those building kits had required pieces right and all that stuff that the artists environment artists designed the building kits and then they would export an entire building kit at a time and the appropriate pieces would be put into the procedurals system library but then they had all these auxilary pieces they could use later when they wanted to polish it so as far as the the procedural system it knew about a certain sets of pieces with different architectural styles but it didn't it had no other knowledge other than their dimensions and if there were special offsets or things like that and then it would just based on metrics you know place them on so the the procedure system didn't have have that much knowledge and then later on when they did things by hand they could actually kind of they can quickly replace architectural styles on their own or put custom pieces or things in themselves now as far as all the content in the game Howie it was a lot of

content in the game I don't know if there's anything else to your question about how we managed all those models I mean everything had L Odie's and that was there's all these requirements through how you would have to author everything I guess one thing with the imposter system so the imposter system could actually if could actually see an invisible LOD in the jamb in the model if it was available so if there wasn't a possible lower resolution state that we didn't want to be in a standard LOD we created a special one that the imposter system would see so that when we took those prefabs of buildings and simplified them up to make a single model we would use that even more simplified geometry thank you hi a great talk great game by the way thank you very much I wanted to ask about the iteration time and like how long did it take to to regenerate all the city how did how long did it take to regenerate like one one block or one area and were you able to do it run time

for example in the editor generate and see immediately change or did you have to reload the editorial or the game the the artists had tools in the editor that they could if they did a pot a new polyline of a building they could say regenerate this building and regenerate the building or regenerate the imposter there were some things that they could do it wasn't like real time it would you know we have to get processed in the background then it would pop in for them so took a minute or something but most of the stuff they would get to a certain completion and then it would run through the whole procedural system at some point just because there are so many dependencies you know and then you don't know who has what checked out so there's and there's a whole thing with permissions and when you can run changes and when you can check things out that's yeah it's what I'm forgetting I want to forget about that we could actually regenerate the entire open world overnight and in the early in production we were because there are so many rapid changes let's see so the the

ground system what it would first process to create a sparse set of data that would actually describe where what the UV continuity need to be and where the materials were defined I don't know if you noticed but every single district has different materials for its asphalt and for its sidewalks that's fun and and so all that stuff there's a pre-processing a processing time of about an hour to create that sparse set of data and then each tile itself each about sixteen models per ground tile would take just just for doing the ground and and and creating the Associated data that other steps would need took about a couple minutes a tile but if we ran it all together so that would that's about you know five hours of generation or so but overnight it was not then all the other processes pretty much to process things around the whole city would like to do a processing pass of saying a crime placement or things like that it was twenty five minutes

good to deal with so every time you would do something worldwide I mean I would probably everything has all of these two pennies so processing the whole world for something probably about twenty five minutes as far as a placement pass okay thank you one of the questions is saying sounds good and question about your tile states so you have probably I mean I'd say you have different acts and things like that so that could be in different states how did you solve it within posters do you have like multiple in posters as well for your now now it's just one one one set of imposters and we had this thing that we calculated lighting at a distance so there were actually really was some lighting out there from turn they would light those imposters based on based on captured probes and such and memory footprints was also taken into account when you like please thanks so as we as that's the thing the imposters were put into the zones in order to preserve appropriate memory footprints because we were you know guess it gets technical we were making sure that we were stacking these things tightly next

to each other in one location in memory so we could have them all in then also an you go to you know an instance interior and they're gone or but it's a your an instance interior that has a view out the window then certain of those imposters zones that are visible have to be on right so that all that stuff was how we how we packed them and when you had a toilet for a high lot did you for example had a fixed budget of memory you could use for a certain tile to place things or so yeah every every artist knows their tile budget for stuff I mean they can at the end of the game they've got it up and they're doing stuff and they go like oh no what am I gonna take out okay but and then the procedural systems as I would process things I mean we optimize lots of things optimizing LEDs optimizing collision yeah this is a lot of things we try to do so every tile had a fixed budget every tile has a fixed budget until we break the rules for some reason yes because the Nile next to it has a little bit less you can take a little

bit more now now we didn't we didn't treat it that way it's only if this has to be in this tile because a lot of gameplay actually depends on this and the open world has to be in this particular state okay thank you yeah yeah my question was related to something that you just brought up in the previous question about the file locking and working between different artists you know kind of run into that as soon as you start to scale you have these problems between authoring assets and then as soon as you introduce a process with received generation there my head's kind of spinning so just my question around that did you have what was your process for coordinating all that together and are you also on top of that do you have kind of like a meta tool that is over the top of all these processes and tools that you use to coordinate between people or was it just knowledge of the process I message them and say let go of that tile no no no so

well some there's some of that but but no we we generated all sorts of data caches based on the state of the world at different of pretty much every day so people are supposed to check in their stuff when they go home and and whether they do it or not because we actually grab that data per tile we would we've just this is the latest state of the tile as it's checked in and then we would we would create that cache data I mean x-rays here but we would create that you know we could do it every day but as often as we needed to have an updated state to run the processes we needed so but as far as yeah you know perforce and and checking dependencies or having watch you know having a watch daemon that's checking to see is this ready for me to go it's just people were supposed to be done at certain amounts of time and then that brought the processing would happen and if they weren't well because at the end of the pipeline we were able to sort of run

everything in parallel tiles in parallel then that's how we would treat it so then that we that we'd get the that updated tile I would process that tile and its neighboring tiles or I have that tile inherit its neighboring tiles properties right so the procedural systems would use which neighbors instead of what was originally authored in that tile just so we're all blend in yes does it work yeah hello I have a two-part question um first part how did you deal if you had to with complex intersections like 500 intersections or very close together intersections and how did you deal with if you had to like overpasses and like underpass yeah so the system would if if it knew about the the street itself it would try to figure out a solution so you'd have cars going driving right through each

other and and then we would they were to dedicate well it was just when our production support guys who did a bunch of technical tasks he would go through and clean up those intersections so I would I mean so it's an it's another part of this process that you could just go through and kind of hand author in a different way in our editor and then it would get processes by the system then added to this system and then preserved so anything that was like so a crazy intersection where it's like we can't support those right those left turns there because they're gonna cut another traffic we don't have enough track we only have two traffic faces right east and east west north and south so is there a way we can make it work alright so we can't we have to exclude this from being happen so we're gonna delete that path and they would also the ones they needed and then you could we knew it was manually authored and that would be preserved and then that's how that stuff was happened happened and the same thing with overpasses so a Grand Central Station I mean pretty much to author author any path you just draw a line

and then you give it the little metadata which was mostly by naming convention of what you want it to be and then my system automatically integrated it in okay but did you have like roads going over each other or was it a hundred percent 2d now there were over there the grand social station has that raised deck and of course we also had the bridges where you so for the bridge pretty much they authored they clicked on the ground here and they drew a line they went click growing up the bridge and then they clicked and I think made it come back the cars go up the bridge and then they turn around in the middle and they come back okay thank you very much the pedestrians actually go down the subway they go under the street and then they come out the other side anyone over here yeah yes thank you for the talk so did you have any art driven rules for instance in terms of color palette color variation or illumination of certain parts of the town I yes yes art drives at all so we

had a lot of there was a lot of there's a lot of vertex paint driven stuff and vertex color so like something like like the puddle so in the wetness of the streets so that was all built with by building the vertex colors into the models themselves so that was already static but then artists could go through and vertex paint and so create this other cloud of colors that could modify all that and for if it was on something on the street the optimum thing to do was actually read their vertex paint integrate it back into the street so they could concurrent control it that way as far as lighting and lighting directions stuff I mean pretty much I mean you can see x-rays talk but pretty much the only thing was that was any alley alley lighting but pretty much we were just placing this stuff out there but yeah we're I mean we're always the the rules include any artistic artistic choices that we could follow oh thank you hi hi okay

in last year's talk of a sunset overdrive your colleagues suggested hex great for organizing yes and the streaming a large procedurally generated city but I it's very inspiring and very useful to us but I noticed that you go back to regular square grids yeah yeah tell me the story and the reason behind this okay so the hex grids were originally designed because our procedural system was going to use a a premade set of puzzle pieces in order to do all the roads in layout but once we started having all the crazy elevations we it was impossible to have a fixed set of pieces to do all the work so we use the procedure assistants to make all custom custom streets and so the hex grid actually allows you a lot of flexibility when you're using those pre-created pieces I mean New York sort of built on a grid so it I'm gonna close

this New York sort of built on on a grid so using the grid system for streaming regions kind of just made sense that the hexes did not make sense for for doing New York City okay thank you so we can move to a a wrap-up room and if you could tell me where that would be we tell everybody where that would be