Rendering 'Assassin's Creed Shadows'

[Music] So, welcome to my talk. I'm Nico Lopez and today we're going to talk about Assassin's Creed Shadows. So, today is also the launch date of the game and you have no idea what it means for me to be here for this special day. We made it. Thank you. So I won't ask you where space plays the game. I guess nobody yet. Uh but I want to take a minute to thank the whole and buildering team. They've been awesome and we made a ton of tech and a lot of it is missing in this talk because it it would have been a day long. I also want to to thank the assassin shadows production team and everything discussed today is really a team effort. So first if you want to find me online, this is probably the photo you want to take. with Twitter, Blue Sky, and

MO tags. And uh these are some of the games I worked on. And I will show a trailer of the game to uh give a bit of context to this talk. How can we build a land of peace on the backs of a thousand dead? You'll have to kill them all. Do you understand? Heat. [Music] [Music] Heat.

Fore! Foreign! Foreign! Foreign! [Music] Huh? Fight for what is right. for the freedom to seek justice. Together, we will be [Music]

unstoppable. So today I'm going to introduce you to the Envil engine or reintroduce you to this engine. Then I will dive into ascript shadows and uh some of its rendering pillars. And finally I will talk about how we approach scalability and performances in this game and in the engine before concluding this talk. So let's start with anvil. So Envil was born with Assassin's Creed 1 in 2007 actually 2004 if you consider the development time. It's an engine designed for large systemic open worlds and long rendering. and systemic gameplay as well. Sanville is mostly known for being used in Assassin's Creed, but it has also shipped many other big franchises such as Gust Recon, Foreigner, and Rainbow Six Siege. And you probably wonder what the competitive competitive FPS, a meteorite shooter, and SS screen have in common. And me

too, it's complicated. So over time at Ubisoft we developed many variations of forks of the same engine. It all started with Simita the assist engine. Then it forked into blacksmith mostly known for Rainbow Six Siege and Sillex that is well known for Ghost Recon open world. So eventually we made several uh observations that would change the way we work at Ubisoft. We had so many engines at Ubisoft between Envil, Snowdrop, Dunia, Voyager, UB Art and I'm probably forgetting many more. But we also had too many folks of the each engine like you see there were three for Simita, the same amount for Duna and even more. And so it led to a multiplication of effort and developments and we started realizing that big developments have a long life. It takes multiple years to develop big systems and they can survive up to a

decade in a in a codebase. So we don't have many opportunities for full right of big systems. And ultimately we believe the multiplication of efforts made us less competitive. So working on the same feature system in in so many engines didn't make sense really anymore. So today we work uh with Unil as a shared engine across multiple games, genres and brands. We work in a monor repo, one single codebase and our teams are scattered across the world between the tech teams and the production teams working all together. So now let's dive into shadows. So when we started working on shadows, VA had just shipped and was our benchmark for open world ACRPGs. SC Shadows will be the first nextG uh truly nextG Assassin's Creed game. So Bala was a crossgen title with little scalability really. It scal quality and performance modes had two

very similar frames and the main driving factor for scalability was it resolution. As you can see on Xbox Series X in 4K native, we were only using like barely above 20 milliseconds of the GPU at 30 FPS. So looking at this, when we started working on shadows, we we knew we wanted to push the scalability to the next level. So what is the scope of shadows? So it's made with anvil obviously it's a 16x6 kilometers open world and has a time of day. So so far so good. It's val. Then it has also massive destruction, retracing, virtual geometry, and I'm not finished. It also has systemic weather and seasons. And I swear I don't work for this guy here, but when I saw this list at the start of the project, I probably made the same face. So, it turns out it wasn't so crazy because we shipped it. And here

you can see various dimensions of this complexity with time of day, systemic weather and seasons with the winter. So in um another view of the same thing with different time of day snapshots, some weather states and finally the four seasons. So AC shadows is a gen five only title as I said and we have three modes on consoles between performance, balanced and quality modes with various target resolutions as you can see here in this uh spreadsheet. We decided very early in development that rate tracing would uh would be favored in quality mods. So now let's dive into rendering. Env is built for longrange rendering from the from the ground up. The world is partitioned in cells and the engine is built for multi-user editing in mind. The data itself is organized as streaming grid layers. We have several of these grids and their definition is fully

datadriven. So this is as a large summary how the grids are organized in in AC shadows could be different in any other game with our engine. In a nutshell, we have what we call LOD grids with the short range, main grid, and long range grid. And the difference between these is basically most of the assets will go to the main grid. Short range will uh will have mainly props that can disappear closer to the player because they don't really matter for the general rendering and then game play. And the long runing is mainly for big assets or point of interest that we want to retain for a longer distance. Uh so loading loading grids can be specified at the entity level and the draw distances are driven by what we call LOD selectors and they're constrained by loading grids. So you have to have them matching otherwise you could have objects disappearing uh with the wrong setups. And uh so so beyond the LOD grids we have the fake entity grid that is um uh that contains

decimated assets and aggregated assets. So and can go up to 4 kilometers and above that we have what we call point cloud that is our mass imposter rendering and renders until 8 kilometers. And finally above that, beyond 8 kilometers, you'll only have the terrain vista with many details baked into it. So here you can see the naked terrain on of shadows with absolutely nothing. Then we add the LOD meshes. So the short main and long long range grid and finally the fake entities and point clouds. So in reverse order we have the fake entities that are not supposed to be seen that close. then the corresponding LD meshes. So I mentioned our point cloud renderer that is a mass quad renderer that we use to render trees up to 8 km away. This is without it. This is with it. So I will go back and forth because

with the fading can be hard to see but it's really everywhere. And this is a debug view to show the amount of imposters we're under. It's really like a lightning fast uh quadander. So GPUdriven pipelines have been the cornerstone of NV since Assassin's Creed Unity in 2014. Uh it shipped many games and went through multiple iterations. Uh from left to right, Unity introduced GPU cluster cutting and indirect D calls. In the middle, Vala pushed this even further with mass instancing and batching. And finally, in shadows, we added the support for virtualiz geometry. So in AC Unity the batch renderer as we call it uh was a DX1 class API GPU pipeline I said it the main goal of the batch renderer was to reduce the cost of expensive DX1 drawers but it relied on multidirect for that to render to render clust meshes and it in it introduced fine

culling at the instance cluster and triangle level. So the batch renderer had still a significant significant part of it still on the CPU. It didn't support Bless textures which limited the amount of batching we could do. It was only really per material batching and it also didn't support a sync compute as you can see in this capture. So it had a significant cost and was delaying the render of the meshes. So the second version of our GPU pipeline is called the GPU instance founder or GPU. It is designed for the RX 12 class APIs. It is fully manless and batching is not performed per shader instead of per material. And the GPU performs batching at load time and is designed to kill millions of instances per frame. As you can see, many more steps were moved from the CPU to the GPU with this new pipeline. And to achieve that the GPU relies on something we called database. So I know what you are

thinking where am I now talking about database but it's not what you think. H database is a container for data structures uh that can be shared between the CPU and the GPU and it follows data oriented design with a referencing mechanism. I think of it as a super structured buffer really uh rely on shed input groups or sig as we call it our internal compiler for shader bindings to generate all the helper functions and the binding code. So we augmented this uh sig language to also declare tables. uh so in the context of the GPUdriven pipeline database is used to share the the full scene description between the CPU and the GPU and we use database analogies it's where the name comes from to structure our data in this example for example a row my object is just the equivalent of a pointer to a my

object so this is an example of how we declare a table in our sig format in the top and now we access the members in C++ and HLSL all the interface again getters setters are generated so it's very easy to manipulate you don't have to write that uh one of the key features of database is the support of relations relations link uh different tables between each other so one to one relation is the database equivalent uh of a pointer really while a one to relation is an ext extension of this concept and it simulates a pointer with a size for an apple to point to multiple children. And now you see where I'm going. We are going to be able to have a full C++ like description of mesh relations and and properties. So different table instances on CPU and GPU storage and we support a different mode of data replication to ensure that the data is propagated from one instance to another and it's

generally from CPU to GPU but you can do more than that. Uh so we have the copy mode that is the most simple one and will copy the whole data from the CPU to the GPU into a bite address buffer. Then we have a dirty row and dirty pages update and and this maintains a dirty mask at the row or the page uh page level and replicates dirty rows or dirty pages following what you decided to do to the destination table. And finally, for very large tables uh that we don't want to store fully on the CPU because it would saturate the memory, we have a mode where we only store uh dirty um dirty rows on the CPU until they are flushed out to the GPU. So this is what our scene description looks like on the GPU and it's quite quite quite close to the CPU presentation. It's not identical because we don't need exactly the same parameters. So we optimize it, but you see it could could be the same. who could have a similar thing on the CPU. In a nutshell, K mesh has up to five

loads and each load has two LOD distances, one for the main view and one for shadows to allow customization for shadow LOD distances. And the association of a geometry with a PSO is made at the sub mesh level with a batch ash as we call it. And we make sure we only have one batch per PSO. So there are absolutely zero duplicate. And this is how we declare all that with tables in SIG. So it really looks like structured buffers really and each uh each uh table are linked by row and range relations. So if we zoom out even more, you see the previous slide is in the little square at the bottom right. Uh now we want to focus on leaf nodes and instances here. So we also see the world in is organized as a code tree and to achieve good calling performances it relies on a hierarchical structure made of entity groups and lift nodes and lift nodes really is a group of instances and

probably we should rename it in the engine. Uh so we gather instances that especially close to each other uh in the same lift node to minimize their burning volumes and to make core scing more efficient because this step is done on the CPU. In terms of code, this is roughly the life cycle of tables. So first we declare CPU tables and G GPU tables separately. Then we populate tables. In this case we create mesh entry in the K mesh table and we set various properties to to this mesh tree. Then eventually we will trigger an update of these tables to the GPU. So I don't show the underlying code behind that but it's an implementation of the replication policies. I described before and finally on the GPU we fetch this data and use it as we like. So the interface is very C++ like but it it also maintains maximum cache efficiency uh in data access patterns because um we

can we can use annotations with this language to decide if a table is a structure of arrays of or of structures and even do mix and match. So really database is a sort of super structured buffer that I would like to have in in the so finally this is what the entire GPU pipeline looks like and I will redirect you to the GPU3 articles if you want like to have all the details about that but I will still describe the most important ones because there are some steps that are very specific to our engines anyways like uh maintenance code. So on the CPU there is a course co course for some cunning to lift nodes basically group of instances as I described on the GPU if you look at the two next columns there is frame curling and per pass culling. So the frame culling instances that pass the CPU culling against all the view forms of all passes and then LED selection and blending logic is also

performed at this stage. The purpose culling performs pass specific form and occlusion culling and for example for sun shadows we do antiroom culling per cascade. It also prepares instance data that is required by the vertx shader and pixel shader to access geometry and metal data for a given instance because we work with unified buffers and for for vertices and also for materials and constants. So we need to maintain a form of description of all that to know where to access the data. And finally at render time we support optional cluster and triangle cuting before the final execut. So additionally to the GPU instance renderer we added support for micro polygon geometry pipeline is based on the night from Brian Caris. It was presented in 21. Um in this pip is a it's an evolution of discrete cluster mesh pipelines really in this pipeline a mesh is made of

cluster hierarchy with continuous level of details. So we stream clusters individually based on their visibility and polygons are either either hardware or software asterized based on their size. So micropoly is used for gbuffer and shadow asterization. And uh in this pipeline geometry takes roughly half the memory of the same geometry in GPU. It shares similarities with other techniques. For example, we use the same MEIS library uh to perform cluster partitioning. Uh we either use mesh shaders or softwareization depending on the triangle size. It also has two passarchical Zbufferion system. uh it renders different materials from a visit buffer to a GB buffer and also clusters are made of 128 triangles as opposed to 64 triangles for older GPU pipelines such as AC

Unity. So some differences though uh in our case for mesh simplification we still rely on simply to to generate the cluster group simplification. It does a good job at that. uh we support user custom code shader with a manual API to allow for custom shed code in the shed graph. Our pipeline is also fully bindless as it is with the GPU instance oner and during software asterization. One thing we did as well is to uh swap X and Y coordinates depending on the triangle configuration. So b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b b basically if it's horizontal or vertical uh to improve branch currency because this algorithm basically perform scan lines algorithm to rasterize the triangle in software and usually it's performed horizontally but if you end up with a vertical triangle flipping it will improve branch currency and workload balancing. So some stats um uh we currently only support micropolon for static opaque geometry and in this game

it's really the main city architecture and static geometry renderer and GPU instance renderer on the other side is our massive alpha tested vegetation renderer. Um so in Q2 we have around 28,000 micropolon instances for 34 million triangles rendered and as you can see in this precise case 90% of the triangles are software asterized but this of course depends on your geometry level and also the render resolution that will change the the the coverage of pixels you can have per polygon for the GP instance render in this scene it's mostly trees and we have around 9,000 instances spread across all passes for close to 2 million triangles with it. So um the reverse scenario now in a in a forest micropolygon is less solicit in this scene as you can imagine but still under some buildings and rocks here and there. I'm actually standing in a on a building here and it's really the GP

instance render doing doing the heavy lifting in this forest. Uh so we we have uh we call three million instances to finally render 30,000 of them and the scene before culling as 1.5 billion triangles and we end up rendering seven millions of them. So so really we had we had no pressure to to support alpha test with a microprogon even if it's definitely something we're going to do in the future. So now let's dive into my favorite topic global elimination. It's always been a first class citizen in Assess games and also in my heart. Uh in AC Unity on the left we developed a volutric open world GI. Um in AC origins in the middle we made it sparse to adapt it to very large open worlds like 16 by 16 km open world. And finally in shadows we added support for seasons and we also developed a new rate global nation pipeline. So for shadows, we targeted a scalable J plan from the beginning. We know

retracing is costly and uh we didn't want to sacrifice other parts of the game to ship it at all cost especially at 60 fps and moreover consider considering the hardware landscape it made sense to give more choice to the players and we can see generally well received to give this choice. Um our J pipeline is more of a spectrum going from fully baked diffuse and specular to rate traced diffus and specular again. So tracing can be either rendered in hardware and software with computer shaders and there is one special case in this game we have the hideout that is a fully dynamic kind of sim construction and it requires refreshing otherwise it will have no GI. So the first version of our baked volutric GI is from AC Unity. It was baked on the GPU using re bundles and I will refer you to the great talk of Joshson about indirect lighting in God of War. It's based on the work done in SEU and it describes very well the

pipeline. The main difference with Unity is the open world structure. So the system went through multiple iterations. Uh for example, we didn't support time of day at first. It was added later in the next game syndicate. So in AC Unity, the world was covered with a uniform map radiance volumes and mip maps acted as a level of detail at farther distances. It didn't support dynamic time of day as I said and it was faked with four fixed static ambiances. Later syndicate added dynamic time of day by blending between two fixed GI key frames. So, as you probably know, uh, Asen script pivoted to more RPG- like experiences with larger open world games since origins. And if we naively extrapolate numbers from Unity or Syndicate, we see that the PEGI technique would never have scaled to uh 16x6 open world games. And

you can see it would have made up to 500 gigs. If we push this even further, Shadows adds four season. And now we're close to two terabytes of GI and over 600 days of back time. And even if if we consider we have an infinite computing farm which we could have today with Google, Microsoft cloud and everything, we still have to make the space on the Blu-ray for this. So we had to do something about it. So the first thing we did was to shift to a CPU pastor back and for several reasons the the GPU baking requires a lot of VRAM and memory cliff was a real issue especially at the time of origins. uh there also less variations due to drivers and diverse GPUs on on the build farms and uh it makes uh our inputs deterministic so we can know exactly what to bake and what not to bake and so it's faster to dispatch because less work is performed on the local machine and we also introduce the concept of a density map and sparse prop

placement so in biggi solutions texl and pro density is everything it's really where the details come from. But it doesn't need to be uniform across the whole world. And this is the key uh to saving space. So we use density map to paint the GI resolutions where to ensure we have the right amount of details where it matters the most. And with this only a fraction of the map is at the resolution of older games such as Unity. Uh AC Unity had 50 cm everywhere in the world. Whereas in this map you can see it's only the blue areas many cities that are this resolution. But these areas are still larger than any areas of those games previous games. So the data is still too big. So what can we do to address this? So to further decrease the size of the big GI we distribute probes in a sparse sub tree and a node is subdivided only if it contains surface. Uh we also discard probes that are inside geometry to to save even more

space. And in average we end up with roughly 10% of the probes we would have with a uniform distribution. So to support time of day we store 11 key frames, one for local lights and 10 for different positions of the sun. Uh yeah I said it's a free data driven. So you could imagine you have more for your your needs. Uh we store the sun local and sky separately in a yg format. And so the luma value is the y value basically is directional and stored as harmonics while the chroma values are scalar. And for seasons we we we aimed at a minimal bake size overhead even though we have to bake the gi multiple times. Uh so we made two assumptions that we know are fundamentally wrong. uh first spring and summer would be the same season as far as interacting is concerned and to be honest it's not such an approximation because albidos are almost identical if not identical and the the the variation

of the seasons comes more more from asset changes and and uh fog and and and weather and second the luma remains the same for all seasons and is stored only once while we store separate comma values for per season and it's clearly an approximation. You if you want to break it, you can break it. Uh because there can be mismatch between human ka values. For example, if a geometry changes between seasons or doesn't exist in each season. But in practice, many vegetation is heavily affected by seasons and problematic geometries can be flagged out and skipped in the base. So a tech artist can just remove problematic objects. In practice, we didn't have many problems. So at one time sparse box cells are insert in inserted in a cascaded 3D volumes. So those cascades are uniform. We sample uniform cascades at one time. So it goes through a decompression um step uh to make the sparse data not

sparse and then it is inserted cascad. So we perform cross cascad blending and in place gi block lod blending when the block is loaded to avoid popping with new lods. So on the left we had the discrete GI grid of GI block as you can see in unit unity and the mid maps are load stream with the view distance so in a discrete fashion exactly as we do for a distance based 2D structure streaming. It's exactly the same thing but with 3D volumes and it worked well honestly the the game was beautiful. Uh on the right it's the sparse GI blocks that are inserted in a cascaded GI volumes to create a much more continuous level of detail. So for some numbers here in the back you can see val stage of the bake for AC shadows with top left the vau cells bottom left shows the the the

cascad grids density at run time and the final lighting at the bottom right. So as I said if we had followed an approach similar to a sensor unity we would have end up with uh two terabytes of J data. So it would really have blown up the space but in the end we end up with roughly 9 gigs uh many relying on the density map and the sparse grid. So for some results as as you can see we we didn't go for full exactitude or accuracy. It's a clear an approximation but in the end it gives a plausible and believable results and we are pretty happy with it considering the size. So for specular reflections we use in the following order screen space reflections relatable G4 maps and a dynamic Q map at the player position to act as a regional Q map as a fallback. So G4 maps are not something new. We pre alido normal depths and they

are dynamically reli at runtime. Shadows are also baked in these cube maps and to do so we store eight key frames in the 8 bit texture one bit per key and then at runtime we select two key frames closest to the current time of day and we blend them accordingly. It works very well. It's surprising. It's it's a nice trick and really because the cube maps resolution is not so high you you won't even see there is no uh no filtering for seasons we use what we call variation states and variation states allow for datadriven game specific logic at the entity level and we use it mainly for seasons in practice we store three variations per cube map and I will remind you spring and summer are considered the same for indirect lighting and in total we have 16,000 cube maps in the game So it's a city and I'm not counting the variations. So not all of them have a V variations. There are some interiors where we can skip I think for example a winter varation but most

do our dynamic map is written to be very very cheap to render it render fake entities. So basically decimated aggregated level of details and it uses lower resolution for shadows and as you can see a comparison on the left with the dynamic Q map on the right is the same local Q map the same position and they are pretty close. So finally just to show some variations of on of some local Q maps to see why there's a spectrum of what calendar is. So now as I said in the beginning we also support retracing it was a big development in our game and we're particularly inspired by the work done by the standrop folks at Ubisoft and uh we developed we decided to follow a similar design fabing in light tracing and supporting both hardware and software tracing. Uh but we are lacking data to back this up and we weren't sure where if it would fly or

not. So we wanted to keep our options open and for this we went for a complete abstraction of inline on inline software and hardware addressing. So our rate tracing stack is abstracted under hardware abraction layer that we call fusion that is shared at Ubisoft across multiple engines and it's developed following an insourcing model. So this library is basically in our internal GitLab and following the insourcing model we can contribute and make it better for for many projects at Ubisoft. So we developed a unified rateing API to abstract platform specific APIs which is fairly common I think but also hardware and software tracing probably less common and inline and on inline. So we wanted switching back and forth to require minimal effort. Uh and so on the right in the green square you can see our retracing loop with a callback interface and in more details uh this callback interface interface serves as a

inline and on non-inline abstraction when using inline retracing depending on the required state it will invoke the corresponding call back. So heat any heat miss and so it's the responsibility of the user code to write these callbacks as if there were they acted as the different shader stages we get in the non-inline retracing code. So in the case of non in line you can do the same thing you can just call the same call back in the corresponding shed stage. So for an apple in the m shedder you will just call the same function and and so on. But of course for each sheders you probably want to use sheder tables otherwise you wouldn't be using line. Uh but for us we think it's very practical in practice because switching back and forth between the two pipelines can help validating assumptions testing new functionalities and seeing what works best for your game. In terms of C C++ code, the switching between software and hardware is super

easy. Simply done with an enium and actually we even support CPU tracing uh the same way on some platforms. Uh and the difference to set up inline and online inline tracing is also minimal. I show in the bottom some code probably will read it at home. Uh it mainly comes to setting up the shader table really. So our ebid hybrid retracing GI combine two steps. First there is a per pixel rate tracing made of screen space rays and wall space rays and a DJI DDGI light probe cascades made of five cascades for a total of 10,000 probes. Uh and we update 1,000 probes per frame more or less. So our rate tracing its are stored as a set of PBR properties in a rate tracing gbuffer and then we write the its in a defer lighting pass. So this is what our rate tracing pipeline looks like. So we trace we

first trace second arrays in the screen space and this acts as an acceleration because everybody knows hardware arrays are very expensive. uh so we want to avoid using them if you can uh without any approximation resolve them in the screen space. Um so if we find a heat we store it in the it buffer and if we don't we keep this ray with the same t value and we fall back to workspace weight tracing that is basically tracing with other hardware arrays the BVH so continuing the t value and continuing step stepping in a world so all those hits end up in a heat buffer then we then uh relight in a heat lighting pass in deferred and if there is no heat for the given pixel then we'll trigger a miss event and sample a prop cache. So this gives us the first bounce basically and to get the subsequent bounce we sample the uh DDGI cascades to uh bring back the the rest. So then everything is

denoised and we can add an optional RTA term as I called it that is artistic driven and that is specular if it's available to produce the final image. So you will ask me why do we add AO at the end because it's supposed to be part of the light integration right. Uh so there are multiple good reasons for that. Uh first we we sum two systems with different frequencies with per pixel rays but also with retrace probes and so redress probes can attenuate some details especially in AO. uh we also trace rays at quarter resolution in screen space on most platforms to to to add performances. So we can lose high frequency details. And finally there can be some disparities between the raster world and the rate tracing world. For example, we don't have the grass in the right tracing world. Uh because again for reasons. So if we reapply uh someto at the end you can see I go back and forth.

It helps a lot the large objects like houses, trees and so on to run them to the frame. uh and we also still evaluate a very subtile subile SSO term to make up for the fact that we trace in quarter rays and bring back like centimeter level details and you can see when I add this term it helps a lot the crown and the leaves and so on. So this is the final image. So our acceleration structure works as follows. We support both binless textures or average color for metal textures for for example for very low end platforms. Uh we call me maches by size or type and we use a global LOD offset to determine what to put in the BVH and for micropoly uh we base the section on LOD error metric to output proxy meshes as was mentioned by Nvidia

yesterday for these techniques. It's the best way to do it currently without mega geometry or these kind of things. Uh so of course everything can be overridden per set or per material to address local issues or or if you want more details locally. So this is an example of our acceleration structure with at the top right what we shipped in shadows with binary structures and the bottom right the GPA for albido for comparison. So for alpha test we tried several things. We tried using the blue noise approach to approximate a trans transparency factor and stoastically text kill the right amount of textiles. Then later we added a bless alpha texture support uh with close to no overhead. Everything was already available because of our GPU pipeline. But we ended up with either one of these two issues. So if we want a precise result any heat is very costly with many overlaps. Uh so it's a bit of a deal breaker in the kind of forest we

have in the game that are sillish and if you try to limit number of its then you end up with way too much occlusion and we've tried things like trying to compensate for that only in the vegetation but we are not satisfied it's it's more of a hack. So in in the end we set up for another way that we think is um a bit we didn't believe in it but we tried it and it worked very well. So basically we scale triangles according to the average opacity as you can see in the bottom right image and in practice it's 30% faster to trace than any heat and the diffus looks actually quite close to the reference and it works very well in practice because heats are resolved in screen space and and this is really accurate. So you get the best of two worlds pixel pixel perfect result for screen space and you have the all the layer of consistency at the same time and as a bonus it's a very good approximation for specular tracing as well even though you are wondering

disconnected sets of triangles for leaves so because we use inline retracing rely on yuber shader approach for redress materials and to be honest I was a bit panicked a bit when I watched the capcom talk about remap ing shaders with the JSON files. So I was really hoping we wouldn't have to do that and we didn't because at Ubisoft we are very strict with the shed and management. We have a low master shed counter and uh it's mainly for GPUdriven pipeline reasons we want to reduce the number of executing direct we have to do uh in the frame. So in in the end it was relatively easy to write a one fit demo uber shader but of course there were some epic fails. they always are and uh but as I said it can be easily mitigated with overrides. We can override anything in the BVH LOD meshes uh pip material attribute shaders everything. So as I said before we use a

unified representation of materials. They are stored inside a material table that we access with the heat geometry ID at heat time and seasons are handled with a material version in this table to account for fallen leaves to not create too much occlusion and also the use of lookup tables for leaf colors. Deferred weather is fully evaluated for specular tracing but not for diffuse because it be too costly to evaluate it multiple times. It's probably something we're going to solve in the future. Uh static snow is baked in the terrain vista though, so it's still accessible for diffuse. And this is what our accession structure looks like for each season in the same scene. So in a typical urban scene, this is an Xbox we end up with roughly 2,000 blesses for 30,000 instances and a total of BVH data of 320 megs. is bigger on PC of course because there are more things that play with other

abstraction. Uh our software tracing stack is very similar to startups. Uh it's part of our low-level abstraction fusion. It is mainly uh aimed at lower end platform. So it's not something we shipped on consoles for example. Uh it's more like for very low PCs like 1070s and and so on. uh re uh so it it's structured as a three-level BVH that we also abstracted in our low-level API and relying on space partition to perform partial updates only when something has been loaded in a cell or an object is moving. So we still rely on indoor volumes to avoid leaking in interiors and they're made of of a list of planes planes and it's the same data we use for biggi but also for many other system such as deferred weather but even sound uh and they are used to classify probes. So at sample time at the sample position we find the eight surrounding probes and then we sample weighted probes with the

same classification. Our light probes are convoluted into a near radiance cache pretty much as a DDGI from the Maguire and also as a radiance cache that acts as a fallback. And to stabilize the signal the signal we trace shorter rays and fall back to the radiance cache which is temporarily stable. And as you can see in the video before and after uh in a in a very challenging scene because it's many indirectly lit and the openings are very narrow. Actually, the result with and without it is is pretty telling. So, the output is a bit less precise and it's clearly a trade-off between pixel perfect precision and stability. So, for translucency, we stoastically flip rays on translucent surfaces and also if a so it hits a translucent surface such as a sojid door. So, so door is a Japanese paper door. uh we sample probes on both sides of the surface and blend accordingly uh

using the translucency factor. So local lights are inserted in what we call an omnidirectional clustered lighting structure. Uh it's based on a regular cluster lighting code. So this is something that is usually in every modern engine and the main difference is that the cluster volume is mapped on a uniform grid around the camera. So you can see this video the difference between uh frog cells here and the omnidirectional structure that comes after. So the difference is in this structure we have a two-level hierarchy. We have I clusters and clusters and I clusters contains 16x 16x 16 clusters for a total of 260k clusters and we first perform a course cluster cutting and then a finer g cutting at the cluster level. So I discuss I discussed mainly retrace diffuse but we also support specular tracing. It was really a last minute effort at the end of the game after it

was delayed. So it's it's a bit crazy we managed to ship it honestly. Uh it follows roughly the same design as retragi. We mainly target iron platforms with it. So PS5 Pro and PC. It was a relatively easy development honestly leveraging most of the work we done uh for diffuse and we we suffered to get it right. Uh and we had to run mainly BVH quality issues that were not really noticeable with low frequency diffuse GI but also denising. So for deniser speaking of denoising we have three of them in the engine. We have an atro deninoiser that is sim similar to SVGF family class of algorithm. We also have energy reblur from Nvidia and we have MSD from Sundrop and this is what we shipped. So it is a special entropile filter based on a recurren blur approach exactly like blur. Uh but what we do we also support material masking for characters, vegetations and dynamic objects for

specific heristics to handle discclusion challenges and so on. And we also support a spherical harmonics denoising. And for this we store the signal in Y COCG with Y being a sparmonics. So this is the raw output we get from retrace diffuse. This is after denising. And this is with spalammonics denoising. So I go back and forth to show you how much of a difference it makes. Final lighting and the final image. So this is with big GI an entire scene. It's not too bad. This is with retrace diffuse. This is with retrace specular on top. So I will go back and forth even though the scene is not very reflective. You can see retra specular helps grounding the whole image. So you see the an outdoor with just retrace diffuse. And now with retra specular you

can see all everything that is out of room is still reflected properly. So for performances uh what are the cost of our pro problem? So basically it's 1 millisecond on most consoles except of pro PS5 pro that has a better rate tracing performances and hardware and for per pixel rate tracing we're roughly around four and five milliseconds again except the pro. So we arrive at a total of between five and six milliseconds for diffuse that is spread across the graphics Q and the ascq for specular uh it's another story it's quite expensive and the main reason for that is because we shipped it at al resolution so al divided by two but full hate and the idea was to ship it in alphres because it looked significantly better and we had the budget for it. We still managed to maintain our target resolution on the PS5 Pro because everything was a bit cheaper and also we divided the cost of diffuse retracing significantly. So now about weather and

seasons we adopt a systemic approach for weather and seasons with atmos simulation and an graph. Our rain and snow are rendered in deferred and we also introduced the concept of multi-state entities that I mentioned before. So at most would deserve a world talk by itself. It simulates and propagates atmospheric factors with a fleet simulation such as vapor, temperature, humidity and uh it it collides the wind using low resolution vel data at runtime around the player. These quantities are fed to v systems such as volutric clouds to form to to drive their formation and dissipation but also wind and rain. And on the right you can see some uh quantities that we uh simulate function of the uh attitude. So in previous games we relied mainly many on an manager system and we used it to drive time of day uh lighting features post effects and so on. It's

mostly made of curves and has limited logit logic. But for shadows we developed an graph system that is based on our nraph system and fully data driven. The ambulance graph will consume inputs from the engine from atmos and drive uh the whole weather incident stack logic. Uh and it's really like a tech driven. The tech can the techart can decide how we want everything to to behave. So our deferred raining is based on Sebastian great blog post series is very well known. We used to maintain our render wetness and pedal pedal level by darkening albidos and decreasing roughness based on wetness levels. This is an illustration of how it works. In the first row, you have the final render function of the wetness on pedals. In the middle is the the wetness mask that we store in a GBF. It's a single float. Uh and in red is the things many characters that are not handled by this system because characters have a dynamic

character layer system for this. So this is a step by step with the dry scene, then the wet scene and then with the puddles. So also snow is an important part of our game and for this we have first what we call deep snow. It relies on a datadriven stamper that we uh can stamp capsule capsules and textures and we can use it to deform the terrain or any map. So next we have deferred snow uh with a dynamic accumulation and uh it's like a defer rain really it works the same way and modify meta inputs and we have the concept of cold and warm zones driven by a static dynamic snow mask painted by a tech artist. So I said before the defa snow modifies matter inputs. So it laps alo toward the snow albido and generally it's white according to the current snow level and it does the same for roughness and it also lowers trans translucencies

mainly for vegetation uh the higher the snow level is. Then we mask it out with micro visibility and finally we threshold it with snow with based on normal orientation to uh simulate the fact the fact that the snow doesn't accumulate on vertical surfaces. So you see the the whole sequence of dynamic snow accumulation. What I don't show here is the that when the snowstorm dissipates it will the snow will slowly decay in the reverse of what's happening here and then turn into wetness and puddles and to finally melt and dry and the logic is really driven by the umbers graph. It's it's doing everything there. So to exclude defa snow and rain from interiors, we use the same indoor volumes again from beg and retragi and so on. We rasterize them in an indoor depth map and we use it to determine the occlusion factor when we play the weather. It's very cheap to render

indoor depth map is just planes basically for particles. We render the surrounding in a depth map oriented with the rain direction and use we use it to occlude the rain particles and so basically the the rain particles can penetrate through a window in intervals. So now we'll talk about how we approach scalability and performances. So mainly due to the fact that we ship different J systems in performance and quality our frame looks very different from either mode. So uh to manage all that we developed what we call the platform manager and it's how we implement datadriven performance settings per platform and context. It's mostly but not only used for engine and graphics systems and the UI is auto generated and everything is live editable. So profiles are basically performance performance mode quality modes and and so on. Uh profile settings require holding the world while profile boot settings require relaunching the

executable. And you can see some examples on the right. Uh so air memory budget. So there's J technique but actually because there it's a bit special in this game with the ID out it does not require restarting the game or loading the world. Uh context is triggered by game play and can be for when you go to menus or or photo mode or go back to the game world. And it's a way to fine-tune render features and performances for specific scenarios. You may want in photo mode to crank up your hair rendering. For example, uh modifiers are triggered by data and they can be painted or triggered by 3D volumes. It's mainly to address localized performance issues or specific needs. Imagine you are in a cav. It's very different from an open world. So you can maybe even disable some other systems. So to summarize, the platform manager is a first class citizen in Anvil. It's great for factoration and profiling and is very useful to iterate with vendors according to them. So I trust them. uh we can simulate console

settings in the editor even though it's is a bit limited you cannot really anticipate what you will get but uh you get the idea. So to track trans resources we use two internal tools. We have a high level view available to track uh resource lifetime and also find the memory peak uh in the frame. And we also have a low-level uh um tool to inspect our allocators and debug for example memory waste or memoralizing patterns that might be undesired. We also have a lot of telemetry in the game to part to track performance regression and all these contours are stored in a database and everything. So there are a few examples to track. For example, dynamic resolution factor at the bottom right given the the in every position possible in the world uh compared to builds on PS5 to track GPU radation on the top right and so on. So to conclude, AC shadows is really uh is the first

game we shipped with a monor repo and a shard engine is the largest AC game we've ever made and uses the most scalable version of Envil to date. So what would we improve? So we developed a lot of technology in this game and we had to anticipate many things like a poly budget uh and everything to avoid uh um to where to be a bit conservative to avoid uh having to rework all the assets in the end because we made a huge mistake. But in the future we really want to use Micropolo going even more support it in more pipelines increase geometric details. uh GI big GI and retro GI is hard. It's really hard. It's good for players but it's really difficult for our developers and so we want to make retracing more prevalent to simplify our life on our pipelines. But we think giving trust to players is always good and the game was insanely hard to QA with all these different time of day seasons and so on. And so the upscalers is the scale world is very

fragmented and makes everything even more complicated. So yeah, I wish we could do something a bit simpler here in the in the industry. Uh but I think it's time to rethink how we approach this complexity in general QA and also how we present it to the player. And finally, graph graph based seasons and weather uh made it very easy to prototype and find the right recipe for the game. But again, difficult to queue and issues and we think it's an idea that requires more iteration and maturity. uh but uh we definitely want to push frame modularization and customization further especially in the context of a monor repo and a shared engine. So for some bibliography and I will be taking some questions if we still have time [Applause]

Great talk and thank you. Hello. I saw that recently you also launched for the Mac OS and iOS platforms. Yeah. How different was to work on those compared to PCs and consoles? And basically what tire do you would you assign those devices? Um the the the so the the the scale of platforms is very large between M1 and M4 and low-end platforms were were a bit of a challenge. um uh but we we we managed to to run it by really uh you know compromising things that are not necessarily things that will break the game on those platforms. I think the main challenge was working on a Mac environment having to to work directly in a Xcode and this kind of things. It's always we are used to visual

studio also it we had software addressing because I think it's what we shipped on the lower end Mac but I would have to check to be sure because with the I spoke about the platform manager it's so dynamic that sometimes you you lose track on what you ship on what platform uh but I think it's what we used for lower and max thank you thank you h great talk uh thank you I would I'd like to ask um I think for nan night there's challenge is that um the foliage it is quite hard to compress and uh for the night for the trees uh uh do you have any um uh optimization to to use microprogon for trees actually uh as I said we don't support alpha test at the moment uh and really the main idea beyond that is But we don't have any pressure to do it because we have a super efficient GPUdriven

pipeline for alpha test and as you can see it renders millions of triangles with no problem. You see the forest we have. So we're like okay we're going to focus on opic geometry get it right and after we'll work on alpha test. So but we we know it's it's a challenge and uh uh we want to take time to get it right. Yeah. Cool. Cool. And my second question is that um you uh you have to support uh you know many uh platforms like 33 milliseconds and and 16 milliseconds. Yeah. Uh do you have have a plan for the frame time budget or or just optimize as uh yeah the main issue was a difference between frames that didn't have RTGI versus frames that had uh RTGI. So performance mode and quality modes at two two different frames and uh so we tried to keep GPU scheduling as close as possible to avoid very specific bugs in

each mode. It was a headache honestly and I think probably we won't escape having tailored scheduling per per mod unless we try to to bring back those frames closer. Yeah. And there's also also a challenge to work in a monor repo. So there are different games in the same codebase. So anyways we are going to need more customization for for this and we started uh having like datadriven uh frame graph scheduling and these kind of things but we were a bit scared to push it uh uh a lot for this game. Cool. Thank you. Okay, if that's all. Thank you. [Music]