Pondering Orbs: The Rendering and Art Tools of 'COCOON'

[Music] good afternoon [Music] everyone thank you for tipping the waiter before the meal uh good afternoon everyone my name is migas vinon about one console generation ago I worked on lighting water effects and the spoiler on play Dead's inside and even had the honor of doing a couple of talks about it today I get to represent geometric interactive and talk about our game cocoon and show you some lighting water fog effects and even warp into some orbs please silence your phones during the big selda birth of the wild talk in 2017 someone in the front center row had their phone blast off at full volume in the middle of the talk and that someone was me all right cocon development began with Y Carlson in 2017 and together with

fell plated alumni Jacob Schmidt they formed geometric interactive early on they met our talented art director herin Co who was brought on to solve some of the visual Mysteries of the game's Cosmic Journey then in late 2018 I joined my nerd Graphics to find solutions to the technical puzzles of the production after years of hard work with a talented team of artists designers Engineers the game was born and it looks a little something like this normally this is where I would switch from the opening shot of our trailer to my presentation slides but today I'm going to do something a little different still going to talk about graphics and stuff but instead of advancing slides we'll zoom out so here in this Canyon where the game begins is the perfect place to show you the lighting the indirect lighting is a baked light probes or volumetric light Maps or whatever you want to call spherical harmonics irradiant stored in boxes by themselves they look like this the game was shipped on un 2020 LT using

urp1 and our indirect lighting replaces the various Global illumination techniques found within the engine before implementing our custom light Baker the game looked like this the indirect was just two directional lights stored in a uniform uh spherical harmonic but we didn't want to use light Maps because of the lack of support for things for like transparent particles or fog or animated or skinned mhes uh but we still wanted higher Fidelity than just a spatially uniform directional ambient you know lighting thing so in our we have irradiance which is a fancy term for incoming light or light that can be seen from a given point this is then turned into radians by materials which is another fancy term for materials show the lighting they saw bounce into your eyes our spherics are first order meaning that they have two bands one of an ambient component and another band of three directional components we tried second order but it show diminishing returns on our mostly diffused surfaces uh each of these four components is full signed HDR color

which is a lot uh during baking it's stored as half Precision floating point which adds up to 192 bits per boxel that's a bit luckily lighting is always positive which means that uh well the ambient part at least can be stored as bc6 H which is a hardware texture compression for unsigned HDR color textures this saves about 40 bits but uh that brings us down to 152 bits per box still quite a bit uh directions however are signed and while bc6 supports it we can remov set sign by adding the ambient part to the directional part and use bc6 and the rest of them to get down to 32 bits per voxel uh this is what that looks like that's quite noisy and still quite jicky bc1 which is another Hardware texture compression technique used for normalized ranges would help if the directional component was fixed range instead of floating range you can do that just by dividing the directional part by the ambient part making it normalized as we store it and then multiplying again while we're reading it and that looks like this it gets rid of

most of the noise it's quite tolerable at this point and we're all the way down to 24 bits per voxel quite good so another reason that we use voxel is because it lets us avoid parameterizing our missiones we don't have to do light map UVS or anything like that uh but light Maps uh we like light Maps we also use charts and atlases so this is what one chart looks like it's just a binding box around a particular subscene and in the top right corner you see what the data of that looks like and as we zoom out you can see that the charts are then you know multiple charts are in each scene and they together make up a atlas and you see what one Atlas looks like on the right there now to bake one of these charts we need rate tracing but I didn't want to implement and maintain an API like Optics or dxr or something like that luckily we already kind of have a rate Tracer in the form of the rizer Because rization if you think about it is really just a rate Tracer that traces a grid of rays uh that share An Origin point in the perspective camera case or

they share a direction in the orthographic camera case so in order to do this rate tracing we simply render a camera uh from an isometric point of view uh with a texture 2D array and then our view and projection matrices are simply defined by our charts transform and bounding volume uh and then we just have it render out with as many instances as we have slices in our foxal texture then in the vertex Shader we simply just set the slice ID to be the instance ID and add the position sorry offset the position by the slice ID so that each slice starts at that particular point now this generates a rate Tracer that can just look forward so we need to be able to you know store a direction and stuff like that which is where our Ray Matrix comes in this lets us uh control the ray direction as well as an offset and a t- Min T Max which are basically just added to the near plane and far plane of this uh particular projection Matrix and yeah then we just uh set one of these Ray matrices with a random Direction a

random Jitter of about one boxel and also use a near plane offset of about one boxel size to avoid any self shadowing and that kind of problem that we don't really want to see and yeah we just multiply this Matrix as the last thing in our vertic Shader so instead of maintaining dxr or meta passes we just have to include this one macro in the vertex program uh to make it compatible with the baker uh all this is just you know so that we get things to be out of the box we don't need to have super complicated shs we can just use one Shader for uh everything avoid copy paste errors anyway finally we have a little extra detail you'll see maybe now yeah it looks kind of the projector uh I've removed our vertex occlusion so if you look it's sort of like the strip of U what's it called rock down just below the surface I'm going to add it back now so that's what that looks like this is just um made using regular physics ray rays that are just distributed randomly across a triangle with some random directions but then the way that they're added is of course this little trick so this is using a sper harmonic triple

product which is just a multiplication product and a do product that come together I think it's a neat way to like compose multiple sponic datas together now these little details about the vers execution just means that we can like heavily reduce the resolution of our voxels and get away with a lot less uh and we also allow them to be painted manually which is just gives a nice way to like darken the ambient if need be anyway let's shift gears [Music] all right then [Music]

so we now just jumped out of the world contained within this world and we've been given this very bright and ponderable orb it's a bit of a torch to light the way in our adventure and naturally it has a pointl in it let me remove it and uh build that effect up again so just turned off the point light and let's build up everything that makes the orb uh we're going to start with the sun directional light and I think we're going to have to zoom in a little bit just to see the details much better so first of course the orti has to cast shadows and that sounds trivial but yeah that looks like this now it can cast Shadows uh the only trick that's applied here is that we make sure that we only cast with the back face of the glass so that it doesn't self Shadow the interior and we get the interior nice to lit up then of course the diamic can cash outow there's no trick there and then comes the point light so the point light also has some Shadows from the diarama in it they're just made using a baked Shadow texture that's just particularly used

for this fog effect uh we don't want it to be casting on the environment around it that would be too noisy anyway then let's turn around because we got to look at some uh some feet all right so uh we still want the character to cast Shadows on the ground even though we don't have Point Light Shadows and that looks like this that's a little too subtle so let me just uh turn off the sunlight and try one more time pay attention to the to the to the feet there you go so now that's clear so this is just our take on capsu Shadows uh it's just two lines one line going from one toe To The Head and the other line going from the head from the other toe uh and then it's got some thickness to it and a little bit of Feathering to it so the way that this is done is we just project these lines from the light source down to the ground plane get the distance to these lines and then add this feather blur uh this looks much better than what shadow Maps could otherwise provide for us and uh since they weren't supported in this particular version of the engine that's nice I don't think have to sell anyone on the joy of a good capsule Shadow anyway so the m you find within the fog

is uh this noisy height and depth fog that just gives the interior of that or some volume to it so let's just try to like peel back those layers one more time and zoom in and now there's no fog all right so the height fog itself is just to like sync the diagam into it sort of you know hide where it stops and then the depth fog is to make it totally opaque which looks like this this is just because we don't want our orb to be actually transparent for performance reasons also uh so like that's just like yeah making sure that it's opaque the way back and then we add some noise texture to that and that's really it it's just a ray March effect the ray marching is simply done in you know a fixed number of steps and to account for the step length when you're integrating this you just need to take the alpha to the power of the step length in order to make sure that each step contributes a correct linear amount finally let's talk about the reflective glass so you know the drill B now I'm just going to peel away these effects and just build it up recipe step by recipe step that's that's my favorite way to go about it all right

zoom in and no glass all right so the first bit of it is just a specular highlight it's the same specular highlight you would expect on any other Shader throughout the game just uh works with any sort of light type nothing fancy then on the bottom if you notice Boop there's a tiny little bit of a little little Fel going on there uh these things just add to of this contribution so far kind of just milkies a little bit reduces the contrast makes it a bit less harsh uh and then we have one final component which is this uh fake screen space reflection so it's basically just wrapping the environment texture in over the orb we had real you know screen space Reflections at some point but this just uh this looked a lot the fake stuff looked a lot better than the real stuff anyway this is then the total contribution of that glass and uh that's how that looks on the orb again now all these bits just so that the orb fits into you know any environment despite it being a very glow glowing emissive based orb I think it's nice that it just fits into any environment regardless of lighting

situation now uh about that world within that world as we charge it up it milks up opens up we jump through and we're back all right and out and there's the orb so uh that effect has a few steps let's break that down stop all right so uh rendering a world within the world is you probably guessed it by now and in a camera and you probably also guess this one and out of camera and they're discomposed with yourb that's it nothing uh there is a few com performance considerations to just rendering two cameras so let's go about the checking out the ways to optimize this one all right so for the inner camera there's quite a lot going on uh as we zoom out there's quite a lot of draw calls and quite a lot of small geometry that we really want to be hygienic about so uh when we are just wandering around

the world at normal sum levels the density of the world is perfectly fine but out here this is a lot of geometry so um since we can't see behind the orbs edges it's very simple to just take the coling Matrix of the orb and or sorry of the camera and crop them down to the orb so we don't run anything outside so now we don't have any draw calls that we can't see now obviously you can't see those pixels either so you can just like um use a scissor rectangle to get rid of those as well now we don't have any fragments out there but but you still see these little spiky sides there well if we just take the orb and use that to render anything that's outside the orb's view to the near plane depth boom then we get a TR crop tight crop and we don't render any fragments that we don't need the same thing goes for the other camera we just render the orb early in our depth prepass so that there aren't any fragments that are shaded that don't need to be there and that looks like this and then of course uh these are just uh composed using you know when you render two cameras and you just don't clear but between the two of them that's

what's going on so that just gets rendered behind it we render the sub camera first then the out camera and they kind of come together with a single resolve or single post effect that saves post effects we like not doing post effects twice all right that is sort of the you know essential performance hygiene but there just a couple more things also I need to get all right so to get the player to render in both sides we used to have a third camera but the third camera is bad so what we did was simply have the player rendered in the main camera and then we take the transform that's relative between the two cameras to teleport it over to the other camera when we needed to be there then there is the trouble of Shadows at this distance that's an awful lot of Shadow draw calls and Shadow texels so specifically for this in any for environment we have this thing called the static Shadow map which is just um a very low Shadow map for the entire world that's stored as PC pc4 um and uh yeah we that doesn't change

over the course of the whole game it's just a regular sterilized acid and we get to use that for for example in the uh in these orbs here H and then we have this fisheye effect what do I mean by that well now it's off let me turn it on again so it's not just for looks it's not just for the effect it's also because it helps us get rid of some more fragments so if I remove it again you can see that this is this a lot of fragments we could actually get rid of since they get you know outside the fishe effect so what we do is you guessed it it's a sissor rectangle so now we don't need like half the fragments of any of it frankly at this point and we can just get away with rendering just this part and then stretching that out to fill the rest of the screen so that's another way we just like I don't know reduce the load on the GPU doing a a good warp finally we uh use 16bit depth during our depth prepass because some platforms that ran a lot faster uh this does mean that you would get a lot of artifacts with 16bit depth so we just move out the ne plane quite

far when we're warping and the same thing goes for the fog the fog is segue a fril solution so we need lots of set depth in the fog and that's also why we move out the near plane when we're rendering this bit here um and yeah I there go anyway so about that fog uh like I said the fog is a fral solution it's about uh 128x 64x 256 uh it is is doing single scattering in its shading so that means that at each voxel it is holding the color of the light and the material and the alpha of that given point and that's just a regular shading function that you would find in any sort of transparent Shader throughout the game there's nothing fany going on there the sort of gacha about this fog is that it is pre-integrated meaning that we run a compute Shader before we actually use it anything so that at any given point in this voxal volume you not only you don't have like what fog is at that point you have all

fog between that point and the eye this then means that when you're rendering some geometry either transparent or opaque you just sample the fog once and you get the entire fog contribution between the I and that point this then means that it's compatible with say msaa and all those types of things in sort of a naive implemented way which is kind of neat anyway it also receives Vox lighting this this is another one of the motivations behind using voxal based lighting is that it can be received by the fog so it really helps to like ground it to the environment and stuff let me just like turn it off to the wrong lighting that's what that looks like it's subtle but trust me it really matters anyway let's uh open this door real quick and uh look at some Shadows they're built-in water breaks for me this's talk [Music]

all right so sampling the Shadows without any tempor or spatial filtering yelds this sort of like shimmery boxal scop stuff that I don't really like looking at so the way that we filter our fog is not temporally there's no feedback effect going on whatsoever it's just pure spatial filtering which lets us do really like quick effects on the fog for gameplay reasons when we want to uh the spatial filtering is mid Maps so it's mid maps on Shadows uh which means uh you probably guess this it's nearest filtering because it's it's not moment Shadows or anything like that um and these uh mid maps are simply generated the same way that the classic sort of Killzone 2 style stable Cascades is done but at each resolution is also stable across the board and they're sampled with DDX ddy and ddz the DD set isn't a hardware thing you just got to like find that analytically anyway um across the bridge you find these uh some

crystals here let's not worry about those yet uh but as I'm a big fan of this voxel you know light on voxel fog let me just one more time like turn the turn the wrong lighting onto that one for a hot second if my there we go all right uh so now it's got some random ambient color on it I don't like it here it is with the right one anyway flying into the clouds up here with this cool crane we'll see the fog increase H all right so one more time I'm going to reiterate that uh if you look at the uh part of the fog that's sort of under the cliff over there it's currently like shadowed correctly by the voxal uh lighting and if I return to just a uniform lighting situation you get this sort of like bright you know typical distance fog effect that I'm not too fond of so I'm just really big old advocate of uh getting you know 3D spatial Lighting on your fog I think it's pretty good predictable lighting especially in this world where fog makes

up such a big part of the the photons hitting your eyes anyway volumes are defined by a regular box transform let me zoom out and take a look at what I mean so this is this particular boxal fog volume here just a bounding box that you set up and of course as we sort of zoomed up into it the uh the density increased but you can also do other things you can animate the color and you can add this like more turbulent noise to it make it all grimy and stuff and like the Shadows this is sampled with DDX ddy ddz for manual myip selection this just means that there's no texture aliasing being produced by this particular effect which is good now like I said it's a localized fog solution so we have to voxelize these fog bounding volumes somehow and as you know by now I'm a big fan of abusing the rasterizer for naughty things uh did you know that on top of being a great rate Tracer the rasterizer is also a really good box boxer so when we are drawing the fog what we do is um we render to a layer

buffer and uh we render out the boxes as quads instead and we want to do them as sort of cross-sections of a box essentially using these quads so uh like our lighting rate Tracer we assigned the slice ID to be the instance ID in the vers here that makes sense but now they sort of just draw us like full screen quads which is not what we want we want them to draw within the bounds of the box of course so to only draw within the lines so to speak uh what we do is is we trace a line we do a line Trace from the back to the of the box to the front of the box and we find where that particular line intersects the slice plane and then we set our position to that and that then yields being able to draw only inside the Left Right top bottom sides of the box and uh now that still leaves the back and front of box uh to be Spilled Out of so the way that we handle that is just to set the uh fragment set position to be the uh T value of that line trace and that then uses the fact that the Riser has built

in plane clipping for well the near plane and the far plane of a camera and that's what we're using to make sure that we only color in the four sides of the Box this is a pretty neat method of only generating fragment threads where we need them I think it's neat that the GPU is capable of just generating threads where we need them it also let us use the built-in Hardware blending which I think is quite neat um and yeah that's how we blend multiple fog volumes together uh and this sort of concludes the base of our rendering so let's shift to some more visual effects type stuff after each World orb you gain a new ability and the orb takes on a second role well at least if we had beaten the boss I'm skipping all gameplay spoilers for you know reasons anyway from the orb an asteroid belt appears and within it you see within its orbit this crystalline bridge is revealed and this isn't the first time we've seen this bridge through the gate and over the bridge we saw these large deposits of crystals that I told you not to worry about but anyway out here the

orb reveals it and within it you see these structures made up of cellular crystals it's a volumetric verono diagram which is made with a regular Point grid that regular Point grid is then just orbited around with a pseudo random value what we can do is of course we can find our nearest uh cell that we're inside and we can intersect the next cell that we would see along our viewray and then the interior position is just shaded using our regular lighting function and so is the exterior and then there's just a refraction and some blending going on between them to get this interior look and finally this is sort of like monotonous looking in terms of color so to add a little bit of a splash we use some irradiance that's just a scrolling texture along the Lampert and um frel products of the uh of the um of the dot products anyway that's just multiply on top of the specular and the diffuse color uh since we use the regular lighting function it fits nice nicely into the various worlds

and even even does a little bit of a cute reflection there and some cute shadow drawing on the ground there good use of uh transparent cutouts now the actual ability to reveal these Bridges is an alpha dissolve Circle that's driven by the asteroid belt and it's quantized to just be per Crystal so the way that this is handled is just you uh sample the alpha value just by intersecting the orbital plane and then you get the orbital distance to this orbital plane and then you smooth step it at a range but of course you make sure to sample this value with the center of a cell and that's really how that's done so that's our first orb and I think this trick of having interior rendering as an effect on Geometry is something that we enjoy using quite a bit uh it's liberally used on various emissive effects such as like just throughout the game but this is a particular like procedual example of it anyway here's our second orb

all right now we splashed in here so in this world is this bogish water that I just turned off and now it's back it adds this uh refraction and this reflection effect on just the whole world and also within this uh water you find this uh fog which I've just removed it's this Merc that you see below and above and the way it's made is just like with a regular fog volume that like like the regular fog effect there's a a bright one on top of the water and a dark one below and of course the water surface itself starts with blurry refraction which um it's just using a standard like grab or opaque pass texture thing uh which works naively with the fog because the fog is applied during like regular opaque or transparent passes so when that's when you grab a snapshot of that that has the fog built into it uh so this is the pure perfection without any fog on it and here it is with I think it's just neat that the water doesn't have its own like bespoke fog system because it means

that we uh get like the nice shadows and the nice indirect Shadows of the you know built-in fog system anyway so the actual normals of that refraction are using a little bit of a cute trick I wanted to do something to indicate the shore and uh help ground the water to the environment but I didn't want to use like Edge foam or something so it's got these Edge ripples which are made with something called a Shore map which is basically like an SDF to the nearest shore at as a vector anyway uh so you take the distance of that you can animate the distance with a sign and fade it with the distance of that that same SDF and then multiply the nearest Direction onto it and uh yeah the texture is just an RGB texture with um red and green for direction and in the blue Channel you find the fall off or distance it looks like this it's made with a offline tool that just generates it and compresses it it's rendered with just a giant camera above the entire world uh which then uses a replacement Shader uh geometry Shader that at each

intersection of a triangle it generates a line that intersects and you know expands a quad and renders out the distance and direction of that particular point now the actual like just wobbly normals on top of all this is just two scrolling normals it's kind of boring uh but with one little little trick what it does is it stretches out the normals using the shore Vector uh to create this like sort of an isotropic stretching effect along the shoreline which just slows down the scrolling along the the beach line of it and sort of helps it integrate into the actual rieles that then get layered on top so it doesn't just look like a linear blend between the two effects anyway uh I think this contribution of like procedural ripples on top of like textured um normals makes for a nice mix especially with the an isotropic stretching along the side there anyway uh now naturally we have Planet Reflections and they're pretty normal just a low res planer reflection thing with with another camera uh and it's

using a lot of msaa to hide the coarseness of this uh Planet reflection and uh like the warps it's also using static Shadows to make sure that we don't do any Shadow rendering in the reflections speaking of warps got to avoid this guy there he is so this diarama out here is also a plane of reflection it's handmade for each warp point with a little bit of diarama geometry this just made B spokely for this particular waro and this is what that looks like um to flatten the look it's rendered with a green Sky Box behind it a green light and some fog here they are without the light and the fog just adds a bit of milkiness so you can see the light there we go adds some colored milkiness and then the fog just sort of flattens it all out so like sort of not distract the image too much much like the warps the reflections use the same sort of tricks like pting in The cing Matrix a little bit so we don't draw anything that's outside that boundary as well as a scissor rectangle kind of makes sense

and of course a depth mask just to avoid any unnecessary Graphics work and and yeah this effect idea practically for this particular puzzle so that was a good that was a good use anyway uh let's see I think that was the only puzzle spoiler during this whole talk uh sorry about that let's walk through this [Music] door that is guess I'll worry about that later all right I'm going to quickly touch on just like the specular light because um we don't use a Skybox Sun uh instead we use us an analytic specular light which I just removed if you look

in the lower left corner you see a little bit of a shimmery shiny stuff that's now coming back there it is so the only kind of funny trick with this is that uh I didn't really want to sha sample the shadow texture uh I just wanted to sample something else uh for cheap se's and the trick here was just to sample the depth buffer of the reflection and use a sample comparison State on the far plane and yeah that gets your frankly correct shadows in that now ominously out here it's kind of started to rain which is Rippling out on the surface and I think it is so subtle that a mix of projector and you know impa compression has probably killed off no biggy let's isolate it so very similarly to the shore ripples uh it's um like this veroni diagram texture that I D the rain map uh where you can you know animate a distance with sign and then multiply Direction on top of it and then we've got just to randomize it all we've got an index that we can just you know scatter around that animation a

bunch much like the sh it's also just got like red and green for direction in the blue Channel you have the distance that the sign is animating using and that randomization index is in in the alpha Channel luckily we have four of those channels anyway so I kind of just thought it was fun that the crystalline Bridges veroni diagram and the SDF of the shore rbes kind of come together for this one effect now that box over there looks really interesting but I kind of noticed we're kind of skipping all of that all right so we now have a second orb ability and this one is kind of a a reversal of the last because instead of revealing crystalline Bridges this one is a Vanishing trick on these columns which we can now use to ascend so when cloaked as it's called the solid volumes turn into this transparent fog effect that you can just go in and out of it will and like the crystalline Brides

we've seen these things before we saw them just uh just past the door in the wet world and out here uh they uh sorry the shading is done in a similar way to the orbs uh like them it's a ray March defect but instead of Ray marching inside of a sphere this is Ray marching inside of a bounding box but with some little bit of distortion on it which then also uh needs to have the bounding box you know expanded a little bit to fit that Distortion the shading is sporting much like the fog if you use simplified uh shading model with the non-directional indirect lighting so we don't have to sample the directional stuff uh as well as just sunlight and of course the orb light that's the most important arguably speaking and as usual I'm a big fan of visual effects that use the proper lighting model because it helps things fit into the environment uh here you see that the indirect lighting being sampled correctly kind of like lets it fit into this you know sort of socket there and out here the sunlight helps it

really you know ground into this Cliff face here anyway back on track so one final point about the transition between these things uh give going from opaque to transparent comes in a few layers I'm just going to peel back one by one so let's go down again all right so now there's none of it and then we of course fade in that foggy column there we go The Fad in is simply just you have the height of this effect and do a smooth step kind of expected uh and then the itself is uh comes in with a clipping plane so this is a SV clip distance a pretty good way to do that but then to hide that seam that weird cut there uh we then fade in on top of that sort of like a glow that's using the same clip plane it's just rendering the same mesh again but now with a very simplified glowy Shader and then to get the other side of that thing we then render that again but with the opposite clip plane and fade that on top of it now we have sort of like this transitional state that we can use to go between the solid and the uh trans parent versions and yeah that's another orb I think this

idea of using like the clip plane is a nice way to like do transitions between transparent and opaque things I haven't really found any hardware that like cares that you put a clip plane in there uh as long as you're not using it you know unlike something like SV coverage or a discard or whatever which the hardware does care if you use those things anyway that is now time for our third or wonder what's in there so in this alien world covered in translucent skin membranes it's all about refraction beneath its fleshy surface you see this organic interior and the exterior is quite simple sporting at normal map and some oscillating emissive textures for bioluminescence uh but the important part really is the interior uh it's partially using the same opaque gra pass as the water when we need to see the player inside it but when we want to see the player on top of it and avoid

occlusion and dis disocclusion artifacts it's using an offscreen texture that offscreen texture looks like this it just has like interior parts rendered into just this one part and then the actual refraction is kind of standard you know you sample the refra the the offscreen texture with some uh refractive normals on top of it to sort of break it up a little bit and some uh directional blur and this is to hide the fact that this is a heav under or down sampled effect what I mean by that is that the offscreen texture is rendered at a lower resolution because well it's offscreen we don't really want to spend to many resources on that and the grab pass is then of course uh down sample just to give it like some initial blur but this will uh create some inevitable uh by the your artifacts if you don't like try to like just hide that with a little bit of a directional blur anyway um I quite like this trip down to our

[Music] down here is one of the few places where there's no sunlight in the game it's just this big glow at the center where we find the transporter and you see these organic interior membranes down below and the Interiors are even lit up I'm going to make them dark so to do this the membranes on the exterior just cast Shadows with their back faces is nothing fancy it's just important that the interior is not shadowed by the exterior and of course like above these things are things that the player can walk on top of so it's important that they use the offscreen texture to avoid any occlusion artifacts anyway I know we've just taken a lift but this one's real cool so we're going to take this one too [Music]

[Music] so we're out of the cave and back on the surface and oh what's this so foreshadowing the orbs ability uh just beyond the gate we have a bunch of well orbs uh I'm I'm going to show you one more refraction thing just to uh knock the point home so inside this refraction effect you also have this sort of like fog that's filling out this effect you probably guess that this is you know a depth range with some color on top and when you're sampling a depth range especially an under or down sample depth range down sampled with like mmax filter

uh to get the depth range you know you you'll normally do something with by the filtering here if you just want something like a depth mask you know obviously you'll use something like sample comparison state to like get a nicely anti-as and filtered mask for shadows and stuff but if you're doing something with a range you know typically you'll you know do like a linear sample of it and then do your smooth step or inverse step or whatever you want to do but this creates this Edge artifact around discontinues of objects which I'm not a fan of so uh the trick is instead to use gather red for those of you who are into that you probably guessed it already and then do your uh range or smooth step or whatever operation and then interpolate it this is sort of like I guess you call smooth PCF or something um and it's a tool that's come in handy numerous times now uh we use these depth masks quite a lot throughout the game in this case we're using it to ground Reflections and here you see an example where we're doing the naive implementation and you you'll see some sort of stair stepping artifacts around various places and I'm going to

do that to fix it so uh here we just do the move step before we do any interpolation the same thing counts for refraction you might see sort of uh some like fringes around objects that you know have a few pixel shimmers in them and by doing the operation before we interpolate they go away I think in a game where msaa is our only anti- eling it's important not to introduce any aling artifacts for depth sampling effects [Music] so for this ORS ability now the world is full of uh eggs uh which um contain our orb all of them do so this was what the foreshadowing from before was uh back outside you see around here are these weird stacks of creatures and uh affectionately our speedrunners call these guys blo but um the canonical name is or grab that's not a problem though there's lots of eggs around so we can get it back um so if I hadn't

skipped this bit this is where you would see these guys foreshadowed from just beyond the just before the door um this organic Thief like its mechanical cousin are members of this funny klept tomatic family uh who just really love our property so before I hadn't really shown them how they steal our or oh sorry there we go it has now encased our key in Amber uh uh but we can still see inside of it it's using the same offscreen texture as the other refraction effects I showed earlier um but instead of a fog inside what you see is inside there's this volume light that's filling out the interior and you can see it sort of spill into the surroundings a little bit uh this effect is Ray marched inside a sphere and has two parts to it one of course the cookie texture naturally and the other part is the inverse squared falloff and marching is done in you probably guess this a angular fashion invented Simon Brown meaning that rather than stepping linearly what we do is uh

well which would look like this it's pretty noisy pretty terrible uh what we do is step in angles along the line of course doing so steps linearly uh in the cookies angular domain and important samples the inverse squ fall off uh but I had a couple of texture and arithmetic optimizations to add to this so firstly the heterogenous part is affected by which is affected by fallof uh remains constant throughout the March so we can just move that out of the you know Loop and just uh integrate the cookie alone in the loop which is kind of nice the second part is that we want to avoid cooking noise even with our low step count uh by you know trer we could use TR but it's twice as expensive as bilinear so we just uh use bilinear with the worst case MIP and finally uh since it's a linear rotation we can actually avoid having any trigonometry in our Loop so we just you know move that out and do a linear rotation for each step which is kind kind of nice and yeah we use volume lights for orbs orb portals

Keys tons of other things where we want more like pixel sharp volume lighting than our fril fog would otherwise give us anyway out here they've been fracking and processing this Amber and storing it in these pools it's basically just water but without pler Reflections just pure refractions but with a lot more fog and acting as an emissive during light bck I might argue that light emissive has a lot to do with it so the this is not the first time we saw this either it's kind of endemic to this world and out here you see that they've you know turned it into these Amber pods out here which use the same uh Shader ASR Amper crystals that I just showed you with the volume light inside of them which we've also seen light the way across our adventure I'm especially fond of how it lights up the fog in this particular shot here in this cauldron of fog here anyway so far we've traveled to three in the worlds in our otherland what if that wasn't the

each world including the one we just came I love ANC which just jumped into the orb to uh fulfill their role and if I hadn't skipped them this wouldn't be the first time we seeing these uh these fokes during your adventure you encounter these illuminated and within each one you discover your world or and meet their let me break that let me break down the effect that that do affect that one by bit bit bit bit anyway so back to neutral and before Awakening these ancestors get this uh statue appearance on them which is done with the same kind of Shader trick as the uh what's it called yeah cloak effects but instead of a glowy green emissive color it's just using a dark

blue albo and as they prepare to jump in uh once they frozen in this material they dissolve into this cloud of particles originally this was done with two deadly Shader SS uh the turbulent dissolve was done using a geometry Shader and it was fed by a Tessellation Shader uh the tessellation simply ensured equal particle size uh I want to confess to this because it didn't uh it didn't ship uh I wrote a CPU preprocessor that tessellates the mesh to avoid our Hollen domain sins but what about the geometry well you can see that in the top of it it it's got this midpoint where it's like just averaging the three vertices of the Primitive and using that to drive the rest of the effect so basically what what I ended up doing was just in the pre-processor I write this um Vector to the mid into the tangent of the mesh which then even in the case of something like a skin mesh will be transformed correctly into World space so that we can just you know do this whole thing in

the vertic Shader without any geometry Shader needs because really we just needed the midpoint uh anyway first this uh effect scales down each uh what's it called triangle into a little smaller point then gives them a little bit of a fuzzy random spin around and then orbits them in some like turbulent Cloud kind of thing going on there and then just translates them down toward a point to like suck them into the OR of there and yeah that's really how that works so now where are we well we are out of the gray world and into the sun World which has this you know faint Warm Glow in its thick cold atmosphere uh it's a world that plays heavily with light and fog in a bigger role it's kind of like we're floating somewhere anyway throughout the world you see these orbs that eminate this energy of like either light or in this case these ripples from them um that we're now going to look into a little bit and each of these orbs also had a little bit of a secret inside of them their material uses the same Shader as the Amber we saw before with the uh

what's it called volume light but instead of it being the center the transform of that volume light is now a orb which I just thought was a kind of a neat recycling of an effect there anyway so the ripples on the ground here uh they're done similarly to the wall Shore it's an SDF but uh to the Spheres instead of to the shore so uh unlike the water surface this isn't a texture instead much like forward lights for example this is done using a uniform array in the Shader so this isn't the first time we've done it this way and it also won't be the last uh the waves emanating from the orb on this portal here are also done such in with such a uniform array and so the footsteps on the ground in my opinion this is a neat or nice alternative to something like an offscreen Ripple or normal texture anyway back out here the world is starting to heat up a little bit so let me just break down this lighting here that we got going on uh here it is without any fog or any bespoke lighting it's very dark and barren you add the fog here it's doing some of the heavy lifting separating the playground and

the foreground quite a bit and you add the lighting and it it's kind of doing the same task it's a baked emissive specifically for this world to get its hot underglow going and without the fog uh it's gets a little bit jarring so they're definitely set up in tandem these two things but uh you see the cold from above and the hot from below that sort of helps separate playground and uh foreground a little bit better and we use this Bak the missive quite a lot for lighting scenes in various places uh in this case we where we didn't have any sunlight uh it's used to light up this place quite a bit here it is without any of these baked emissive things it's quite jarring a little bit a little bit dark and here it this back again this is just two large soft boxes found in either like extreme side of this big cave here anyway back out here again once again warmer once again brighter temperatures increasing now we're close to the end so deep breaths for me um there's still one more big

component of the world's within world's puzzle and to show it to you we're going to have to go on a small Journey but it's my favorite part so we're just going to take this lift over there [Music] [Music] things are about to get a lot nether over here so here's this weird structure which our enemy Carl out the world much like course is made with two cameras the inner camera

looks like this you can see the duplicated environment to the other side they have a little bit sand stuff kind of leaking over there of course we've also you know called our Optics correctly fit them with a scissor rectangle and also mask out the pe's edge and the same thing on the other side we make sure to do a nice depth PR pass out here so that we don't render any fragments we don't want to render we this this game did ship on switch so anyway uh yeah uh before composing them together just natively like the warps as well anyway bringing the worlds together we have this uh shadowed fog that's coming through which is then of course by this light that's also coming through the light uh is because you see how now without the light the sun is being shadowed by the door and on the other side it's also shadowed this is so that the the one Spotlight is the only thing that's coming in the sort of intersection to sort of hide the the seams of the Sun or the seams of lighting I guess you can say anyway uh the light is simply duplicated on either

side uh we also have the door Shadow it here I removed the door Shadows the door Shadows is simply done by having uh there you go there's these two planes that sort of like stick to the door frame and yeah just make sure that the duplicated light doesn't shatter around the sides uh anyway let's walk through naturally as we pass through the portal uh plane at some point we actually teleport but you can't see Wich frame out here uh in the GR world is another boss that uh I skipped up ahead it's um it's a can of worms I don't want to talk right so let's uh let's get out of here we have one more portal to look at sorry if this [Music] transition all right out here once again a lot brighter a lot warmer none of our concern we're just here on a stop to the other one [Music]

back in the purple world and here's another door oh there's water on the other side too I hate this door anyway um on the other side yeah the other water introduces yet another recursive camera luckily the coloring croing all that masking really helps us a lot uh on the outside here we see these um refractive orbs and stuff just and ripples on the ground oh and yeah we got a crystalline bridge that we're walking through across well this this scene was really the the profiling Benchmark of this game uh and oh sure why not I mean we got the Misty columns there as well just everything expensive on one scene very nicely designed um as usually I won't spoil anything but uh over here is the final parle of the game and I promise you it rules it gets really really weird but let's wrap this up [Music]

our Cosmic journey is near another portal get good mile good mile child this back in the deser so it's quite foggy up here what I did was I really concentrated fog density around theum of the Shadows to heavily exaggerate that shadow orbs are giving us a little [Music] fin this diving board like my narration I'm just trying to pill out the silence

[Music] this is the I should just expore the warning up front really Carl once again knocking the out of the par whatever this is [Music] so if you've seen our reveal trailer

this shot here a world within a world and so on um this shot was actually born in our reveal trailer before it became our ending back then we didn't have a that world there abrupt anyway so as things unfold the sun's atmosphere is shadowed by the sun's architecture you have disabled that particular sh Shadow uh like the orbs it's just shadowed with a regular regular cooki texture of sorts but instead of uh shadowing some bespoke rayar def this is just the regular fog that you find throughout the [Music] game Sorry Jacob this is rude uh cover is taken off of these Planet representations made by Nikki uh to reveal the the spotlight effect that's going on here that Spotlight in particular so uh it's not a pointl it's a spotlight because uh we really wanted

these self shadows I'm going to remove them now they're back uh and of course if it was a point light even if they had Shadows at at this distance it' be like one degre so I think the resolution would be have to be 2 million pixels to get anything done there [Music] anyway uh so anyway the planet's atmosphere is this round fog volume that you find around the planet here it is without now it's back it's shadowed using a custom sort of blob Shadow function which uh now it's gone now it's back and uh because our fog doesn't have any you know temporal uh sampling thing going on it you know we can just animate the fog like this and it's it's it's not going to fall behind or anything like that it's not going to ghost [Music] won't he let that play now anyway to exaggerate this space fog a bit I added

a shallow wall of higher density fog right at the intersection of the planet here it is without now it's back the reason for this intersection is because then I know that it's going to be 100% shadowed on the left side and 100% not shadowed on the right side giving much higher contrast here's that fake blob Shadow gone again and now it's back again [Music] I will now that concludes our journey oh another orb I wonder what this ending

means some emotions on that story [Applause] I would like to thank my colleagues for supporting me making the most elaborate talk uh I would like to thank y for egging me on to do it and I would like to thank at JG does things for egging yea on to egg me on to do this thing uh thanks to Kissa and The Advisory board for trusting me with content and thank you to Tuki for letting me liberally liberally borrow her MacBook and also thank you foring saying yes on

Wednesday credits music really gets people clapping I thought that would work all right um I can see on the time that I have 2 minutes and 30 seconds so if anyone wants to be the brave soul to get the one question in otherwise I will be outside talking to y'all and so with the rest of the team over there thank you to coming one last housekeeping thing please rate my talk give it five stars you know thank you for coming have a great rest of your Friday [Applause] [Music]