Blowing from the West: Simulating Wind in 'Ghost of Tsushima'

[Music] thank you hi my name is Bill rockenbeck and I'm an engine programmer at Sucker Punch Productions in Bellevue Washington today I'm going to talk about various techniques that we used to add windblown motion to our game Ghost of tsushima ghost of tsushima is an open world game which takes place in 13th century Japan on the island of tsushima you play as Jin Sakai one of the few Samurai to survive an invasion by Mongol forces in ghost The Guiding wind is a central gameplay mechanic replacing the traditional heads up display or Compass to show the direction towards various gameplay waypoints this keeps the UI uncluttered and helps keep up the illusion that you're playing inside of a samurai film it goes without saying but wind itself is not actually visible wind is not something extra add in the cohesive rendering pass like draw all the stuff and then draw the wind the wind is of course motion built into many different

rendered objects in the world these include particles foliage grass and cloth the wind direction is controlled by gameplay mechanics and can change at any time it's not in general possible to bake the motion into a pre-rendered offline animation and we prefer procedural or simulated motion systems in addition to the gameplay aspects all the large and small secondary motions arising from the simulations make the game less static and make it seem more realistic and alive a couple words about this presentation I've been lucky enough to be with Sucker Punch for the whole PS4 era Infamous Second Son was one of the first games to be released in 2014 shortly after the PS4 launch and goes to tsushima is one of the last AAA games to be released before the Next Generation came along the techniques described here were based on the quality performance trade-offs that we needed to make on the PS4 as the Next Generation comes along we may need to revisit some of these decisions the techniques I'll describe today are not pushing the boundaries of accurate academic simulation technology like you

might find in a movie or the effects are all done offline or in a tech demo where a large fraction of the available computing power can be used to simulate a small scene or more in the realm of heuristics and hacks which allow us to blow millions of verts around using only a small fraction of the PS4's computational capacity for the purposes of the game these simulations don't need accuracy per se but they do need volume and For Better or Worse sucker punches a nearly entirely homegrown engine and this talk I'll describe some of the Nitty Gritty details of how we implemented things I'm not claiming that all the techniques we used were a shining example of the best way to do things but they are what we actually did and maybe that's a useful data point so the model we use for the wind velocity is quite simple for the most part it's just a single Vector pointing from the hero towards the current current goal or in cut scenes and other non-guiding wind situations there's just a single authored Vector which might vary with the weather and be longer when stormy we're not doing any real modeling of fluid dynamics

and the main wind Vector has a constant Direction but we vary the magnitude a bit from place to place um using time varying purlin noise this is visible for example on large fields of grass and you can see gusts of wind blow through particle systems will also often add in some higher frequency curl noise just sort of make things swirl around more particle systems also have access to a system we call verticals these are invisible wind generating particles which can be sampled by other particles they are potentially quite powerful but we use them only in a few sort of one-off places particles grass and foliage also have access to some inputs that let us model local disturbances caused by footfalls or the passage of characters these aren't win per se but they do move things around in a similar way so I'll talk about them as well probably the most obvious thing moved by the wind in the game is particles the air is usually full of leaves dust smoke fog pollen since the very earliest days of the PS4

Sucker Punch has been using a proprietary particle system which has served us very well since we introduced it with Infamous Second Son the core architecture has changed actually very little from the Second Son version I gave a GDC talk in 2014 which covered the system in detail so this talk is going to gloss over the implementation but here's a high level overview our particle system is entirely GPU based system where each particle is touched only by a compute shaders running an async compute so it's fairly high performance and its biggest Advantage is that it's extremely programmable instead of fixed function features sets you can just sort of turn switch on and off it's based around these complex text-based Expressions which just get compiled into optimized GPU code and this lets artists mix and match features and sometimes surprising in wonderful ways I'm mostly going to talk about new functions we've added to this framework and our lead artist Matt vaino has an excellent blog post back in January about the effects and ghosted from that which I suggest you look at

there are a lot of leaves and goes like a lot there are thick carpets of leaves coating every surface across large areas of the island obviously we're not modeling all billion leaves at once and then the distance The Leaf cover does just become a texture but most of the leaves in the immediate area around the character and the camera are in fact live and they do respond to wind and disturbances there's something like 10 000 active leaves in a typical scene here you can see how many of the leaves on the ground are actively simulating particles which react to the hero's movement and they're kicked up when he moves past them there are some complicated Expressions on the leaves which model them as discs so that as they hit the ground they rotate according to the torques and things and they start touch and spin and settle down we switched our main train implementation during the development of ghost to a height map based approach with a hierarchy of tiles containing various resolutions of Titanic height map that can get paged in and out we gave the particles access to this

height map via a Terrain pause function which just snaps the given particle down onto the ground we also use it for larger scale wind gusts as I mentioned we're not adjusting the wind direction in general to flow around mountains in a complex way and this is fine for most things but for it looks bad if wind carrying stuff is Flowing straight into the side of a mountain and the stuff is just kind of Disappearing into the ground so we are doing the most basic approximation of fluid dynamics for just this case by taking several height map samples out in front of each particle and sort of giving it an upwards velocity in advance so that it goes up over things nicely a Technique we use to add a bit more interactivity to the particle motion is what we call verticals these are special flavor of particle which don't actually draw themselves but are visible only through their effect on other particles they're basically like little spheres of wind they output a position and orientation and radius and also a vector which determines the direction of the

wind using a coordinate system which is attached to the surface of the sphere so it has like east west north south and up down components so just by pointing this Vector around in different directions you can model several interesting shapes and of wind including circular vortexes linear gusts and sort of outward facing blasts vertical emitters atomically append these into a single small array listing all the verticals in the world and then regular particle emitters can access this array through a new function which samples the accumulated vertical contribution at a point and it's doing this in a Brute Force way just running down the whole list and adding up all of the contributions and it's someone amazingly works even up to like hundreds of verticals this isn't an automatic thing particle emitters only opt into this in a voluntary mesh fashion for relatively few places where it kind of helps um and since it's another building block function emitters decide for themselves what to do with this information they

might just sort of in a straightforward way add the vertical wind under their acceleration vector and so to sort of get blown around by it like these leaves over a campfire where there's some upward uh facing verticals emitting from it but we can also use vertical spawned by sword strikes and character motion to say extinguish candle flame particles as you go near them or to scare away animals all right crabs they're an aeroshot it shoots a little vertical that makes them say hey I need to dig myself into the sand now during the game tsushima is overrun by Mongol armies and so there ought to be actual armies we have all these amazing beautifully sculpted and animated character models and rigging for regular gameplay but those are pretty expensive the CPU has to compute all the AI and animations render a hundred thousand polys with lots of fancy uh materials and we can afford to have a few tens of these heavyweight NPCs in the game at once but there are some scenes like the initial Invasion where you'd really like

to see a horde enter animated particles here's a scene from the very beginning of the game as the Mongols are overrunning the beach most of the ones you see are full NPCs but if I fly over here away from the main action sort of into the distance you can see that there's a whole crowd of Mongols there in green who are actually a particle system then all those green ones are super low cost GPU only particles here's another example this is the samurai sportification seen in the background at the very beginning of the game it's just kind of a little bit of set dressing all of the Samurai here are particles up close they're a far cry from the Real Models but at any distance at all they're remarkably complex for taking basically zero CPU we added this feature specifically for armies of people now turned out to be even more useful for all kinds of Critters like butterflies Birds crabs dragonflies this crane is a particle as are the seagulls up in the upper left and the whole flock of Starlings animations for an animated particle are

compiled into a fat texture containing the local skinning Matrix for each joint at each frame those can get large so it's best suited for simple objects and maybe only a few tens of joints each particle can individually specify which clip and which frame within that clip it should display at each time step since animation state is just another building block artists have great freedom to use it however they want are artists have done some amazing things with multiple animation Clips like frogs which float on the surface of the water until the player gets near to them and then they swim away and spawn a little water ripple particle as they go or if they're in land they hop away so animated particles were easy to implement and they gave a really big bang for the buck ghost is set in feudal Japan and most of what you see is natural objects especially trees and shrubs nearly every vertex of all of these is in constant motion influenced by the wind most of our trees are modeled and speed tree but

the runtime is entirely homegrown the motion of the trees is implemented by effectively skinning the geometry onto a simple skeleton just three levels chunk branch and sub-branch the actual drawn geometry is attached to the skeleton these aren't implemented as traditionally skin joints where a separate stage of computation builds up these joint matrices they're actually run entirely in the vertex Shader using only local information stored at each vertex so each vertex stores extra data in the vertex stream about the branches it's attached to basically just the beginning and end points about the branch and the sub Branch so the vertexes are thus fairly fat it does have the advantage that it supports an extremely large number of branches when you add wind the vertex Shader takes the wind Vector at the tree's location into account and rotates each branch around its origin away from the wind to get the whole tree to bend and we also add a little bit of sinusoidal sinusoidal motion uh from each branch should make it bounce back and forth and the phase of this motion is based on a hash of its position so that the additional adjacent branches

and adjacent trees don't move quite the same as one another this wind sway is running on pretty much every vert and all of the greenery you see in tsushima including most imposter level geometry for trees covering distant hillsides they also optionally take the displacement buffer into account using very similar code based on the same skeleton so the branches can get bent out of the way as Jim walks through them now we come to Grass which covers most of the ground on sushima we have a whole GPU pipeline for specifically grass blades we handle numerous different styles of grass with no limit to how many different styles can be drawn in the same frame a typical grass heavy scene like these will process a little more than a million separate potential grass blades and after we've pruned this number down using a variety of heuristics like visibility calling and distance limits we end up drawing around a hundred thousand blades each one of these reacts to the wind I should note here that the pampus flowers you see so many of here are not drawn through the grass pipeline they're actually more akin to foliage

and are drawn through the same GPU code path that we use to generate large instance draw calls of uh for trees and rocks and stuff like that the basic flow of a grass system is this of course it's all done in the GPU there's a compute Shader which generates each blade of grass each frame we dispatch one instance of the Shader for each terrain tile out in front of the camera with one thread per potential blade of grass each thread first doesn't look up into a grass kind map to figure out whether there should be grass here at all and uses the train height map to position it it prunes against the view frestrum an occlusion buffer and Fades with distance we have a bunch of different graphs in the game so there's a big array of constant data for each kind of grass so blades which survive this pruning get a bunch of parameters computed like their height width Bend Etc and are appended atomically onto a buffer and fast GPU memory along with a count so we can dispatch an indirect draw the draw shaders also consumed the grass parameters to place individual verts and

color the pixels this is arranged so that we can do most of the heavy lifting only once per blade of grass since there are hundreds of thousands of Blades of grass being drawn in many seams the blade list could get big so we dispatched the compute and draw shaders in a pipelined manner with a double buffered blade list so that one set of blades is being drawn well the next one is being set up the actual vertex Shader for the grass is straightforward and takes all the blade information from the per blade compute pass and adjusts subverts to bend and collar Etc of the blade the motion is much like the foliage with a sinusoidal sway added to the top of the general bias away from the wind it also considers several other factors like bending each blade away from the center of its Clump and bending them downhill uh wind includes both high and low frequency noise based on the position there's also a TD displacement buffer around the camera which we write into using a special kind of particle as a hero passes by the blades will bend and tilt accordingly my colleague Eric wallab gave a GTC talk this year uh it goes into much more detail about how all this works so I recommend you see that

being set in Old Japan meant that we ended up with a lot of cloth in the game armor kimonos or sashimoto badeners that the foot soldiers wear we started off with a CPU cloth implementation left over from second sun which could handle a few characters each with a few pieces of cloth as ghosts took form the number of cloth Sims kept growing there was a bit of an arms race at times with artists adding more cloth and kind of tanking the perf and then me hacking the code to try to make it run faster somehow and so on in what I suppose was a virtuous cycle uh in the final game we usually have several hundred cloth Sims running on screen sometimes as many as a thousand in this scene there are about 250 cloth Sims for clothing banners tent flaps for a total of around 70 000 simulated vertexes and it's taking about two percent of this PS4 GPU in addition to cloth and leather we use cloth Sims for a few things which strictly speaking

aren't cloth like strands of human and horse hair and even hanging Bodies In some cases our cloth Sim starts with a Sim mesh authored in Maya here it's just a nice rectangle but in general it's not quite so regular this mesh is used to generate basic damped spring Mass system with nodes at the verts and springs in between here are the stretch Springs and we also had springs crosswise for shear there are even further Springs to handle bending but we actually do this a little bit differently we also support one-dimensional chains or ropes since meshes and mayak are lists of faces you can't have a linear mesh with just vertexes and edges and we use this a lot for individual strands of hair the nodes store information about the base position which in general can be skinned onto a skeleton and they also have a Vertex painted maximum distance that they're allowed to move from this position most nodes get a big radius so that they can move freely around but some of them like the couple in the corners there uh have a distance of zero which makes them stick to the

underlying model and that's how you attach them to things so to draw a piece of cloth in the world we need to author two separate meshes in our system one is the Sim mesh which we saw before which describes how the clock moves and deforms then we have a separate draw mesh which also contains the shaded triangles to draw into the scene and the draw mesh is often similar to or identical to the Sim mesh but might have more detail provides a place to attach rendering specific stuff like vertex colors and UVS separate from the SIM the actual connection happens the kind of skinning each Sim vertex creates a joint which generates a full transformation matrix from the point of view of the draw mesh rendering code these are just like regular joints but under the hood they take a different code path than regular animated joints because there's just so many of them they're like hundreds of thousands of them in a scene so these are generated by the cloth Sim on the GPU and are never even touched by the CPU as models get further away from the camera we often switch out draw meshes for one to the simpler level of detail

in these cases we don't switch the cloth Sims generally we'll replace the draw mesh but keep the higher resin running uh with the same the new draw mess attached to the same sim so we don't get a pop when we switch from one sim to the other we do Fade Out cloth Sim motion entirely once they get further enough in the distance this depends on the size of the object so little tassels might fade away after a few meters but a large Banner that you can see in silhouette might never lied out we had to actually push out the Simla distance for horses really far because their Mains are actually modeled standing straight up so as they got to the law distance they'd get mohawks the basic data flow and organization of the GPU cloth is this each piece of cloth in the game gets a single compute Shader dispatch running on a single thread group in async compute it consumes some fixed data like the topology of the springs and since we're using standard fairlay integration it only needs to keep track of each node's current position and previous position from this it can compute the current velocity

the Shader computes the updated positions of the nodes for the next frame and writes them back to main memory this memory is overlapped with the previous node positions and will become the next frame's current they sort of ping pong back and forth between the two buffers it also computes the new joint matrices for each node and writes them into the appropriate place so that they can be picked up by the drawing code later in the frame during the solve it also uses some fast GPU local data store to store intermediate results this includes the base skin position and as well as updated position at various stages since we're using a single thread group we can sync data across threads between stages with a simple instruction without having to do multiple dispatches or complicated cache flushes this does mean that only 512 threads at a time are active per cloth Sim and that's too small to get good resource usage so we do rely on overlapping multiple separate cloth Sims with the graphics pipe so that we can keep up overall performance although we only run 512 threads we can

actually support more nodes than that by looping the Shader so that each thread updates multiple nodes the fundamental portions of simulating the cloth are reasonably straightforward uh there's usual fairlay integration to take little time steps and apply all the forces and solve where each node should be this Frame and taking all the constraints and forces into account in general reduces to solving a giant n by n Matrix which if you have enough processing power you can just do directly and For Better or Worse we're not doing that a big difficulty with the spring Mass model is it requires many small time steps for stiff material it's fairly easy using traditional methods to believably simulate a bunch of like bowling balls connected by Springs or a big sheet of heavy rubber we integrate through these little forces everything moves nicely but for real world materials like leather or ropes these things are like stiff and don't stretch much at all and the way to model this accurately is to use a really stiff Springs so that even

a small offset from the rest length generates a large restorative Force but that starts running into numerical difficulties uh we're doing this with numerical integration taking little time steps and applying the forces if the time steps are too large for the accelerations involved things go bad quickly you might start overshooting and you know he's getting further and getting this positive feedback thing and eventually things just go crazy you can solve this in general by taking really small time steps and taking a lot of them but that takes a lot of computation and we're going for fast and dirty here so we're using a giant hack instead of using hooke's law to calculate spring forces and integrating as usual we're applying Springs in a separate step and just restoring the spring some fraction towards its rest length so even with a big time step we never overshoot the rest position of the spring so it remains stable the big issue this causes is that we lose our time step in variance you can tune the restorative percentage to get things to be more or less stiff but if you change the number of iterations it

behaves subtly differently this is fine as long as the frame rate stays the same but it has caused us problems when we've changed the clock speed for cinematic effects or when running in backwards compatibility mode and PS5 where we double the frame rate uh to get the SIM running in a fully parallel way we want to compute many different Springs at once we could just run them all at once uh with one thread per spring but this does present a problem of how to combine the inputs from the different Springs and avoid multiple threads sort of stomping on each other's outputs the way we're dealing with this is to classify the Springs into a bunch of separate spring sets so that each no node is updated by more than one spring within each set so we might first apply here the Blue Spring set sort of one spring per thread up to each endpoint from a single thread and then black orange Etc and syncing in between each so that they can see each other's outputs uh for this regular Grid it's really easy to find a coloring which satisfies this requirement but most of our cloth isn't quite so simple this is an example of the NP complete graph coloring problem it's basically

just solving Sudoku a simple greedy algorithm does a pretty decent job but I did squeeze a few more percent out of it by writing a simple genetic algorithm to try to choose minimal sets the solution of computing all the nodes in parallel has some big problems for most among them is that each node is only really interacting with its immediate neighbors it takes many iterations for a change one end of the Sim to make its way all the way to the other side let's imagine the common case of a banner hanging horizontally from a bar when we integrate the motion of this all of the nodes move down by some amount according to the time step and the gravity except for the nodes and top which are attached to the pole this causes the attached strings to spreads and so as we take a step let's pull back towards their rest positions but only the ones attached directly to the stretched Springs feel this pull initially if you add more iterations the wave of spring tension moves through the cloth and eventually things will even out and things will contract as desired but for a detailed Sim with many Short Springs it's more iterations than we're willing to compute

this is particularly noticeable when you have cloth hanging from something the visible effect is that the cloth becomes super stretchy even though the individual Springs are stiff the whole sin is very much not with the parallel implementation if described so far Jim's robe looks like it's made of rubber it's especially noticeable in the white tassels at the corner of the building where they stretch unnaturally and they're several times longer than they were authored the hacky solution I used is to identify during compilation mirrored by anchor points these are the ones which have a maximum distance of Zero from their skin positions and so are fixed in place the top row here each node remembers the index of its nearest Anchor Point and the original distance to that point and at runtime once we've applied the offset from integration we enforce a distance limit so that the um the Anchor Point doesn't get farther than that from the Anchor Point this can be done entirely in parallel for each node the allowed distance to the Anchor might be slightly greater than the initial distance let's stretch a little bit in general it's pretty close to Unity in this way we can apply the impulse from the anchors directly out to each node

directly without having to go through all the intermediate Springs I think of it kind of like in addition to the wobbly Springs and masses there are these inelastic pieces of invisible fishing line connecting each node to its anchor they can't stretch any further than that but they are free to move around within those radii and that allows plenty of wind flapping and movement there are some common cases where single anchor doesn't work well enough and so we support two anchors and this handles like the case of a rope hanging suspended and it gives a nice kittenary here's the version using anchors the cloth still moves around quite a lot and looks loose and blowy but it doesn't stretch like even all the way to the ground and the tassels are short and stiff just like we want man here's a debug view showing where the anchors are attached there's some cases which are not well handled Bankers which I don't have time to describe in detail it does work quite well for the majority of cases though cloth doesn't exist in a vacuum and it needs to interact at least somewhat with the things around it in the important case of clothing it needs to drape around the body of the character we only support a very limited form of cloth

Collision it's entirely separate from our General Physics model and built up of just a couple kinds of simple shapes we support ellipsoids but most of our character cloth collisions are built up of a particular shape we call a sphere pair it's just a capsule consisting of a pair of spheres and the frustum connecting them we Collide only the nodes themselves against the Collision shapes which is a physician to be really accurate um both spheres are optionally skinned on animated joints so it's possible to attach these to the actual joints of a character and they can build up a halfway accurate approximation to the shape of a human they also support a single Collision plane which can be set up to automatically match the ground beneath the character here that Collision volumes used by jyn and his horse difficulty with collisions especially only checking the vertexes like we do is preventing sudden movements from completely penetrating the cloth this is especially problematic because since this is a video game for Combat gameplay reasons sometimes the hero can change poses at superhuman speeds his arms say can be animated outside of the current

cloth between one frame and the next and once this happens it's kind of difficult to know what to do each vertex can get pushed outside of the volume the collisions but some of them are now on the wrong side and the collisions are working to keep them there so we're Computing each vertex in parallel so it only has local information but this is kind of a global problem you need to solve to get one Mana uh manifold outside another we have a trick for maintaining cloth on the same side of occlusion volume for some collisions in addition to pushing the node outside the volume itself we look at the original skin position of the node before any simulation of has occurred we find the nearest point on that shape and construct a plane tangent to that point but didn't set it backwards by some amount and we can strain the Sim mode to stay on that side of that plane we want it to be able to deform and slide around some but if it somehow gets too far around we'll stop it this works pretty well don't be surprised to hear but now we're not doing full cloth versus cloth Collision uh doing that right is a big N squared problem but we do support

layering one piece of cloth over another in a very limited way we do this by making use of some Machinery we already have in place you remember that each node has a maximum distance it can move from its skin position now imagine we want to layer this light blue cloth Sim over the black cloth Sim we're going to use the same kind of Max distance radii but instead of centering them on the initial position we offset them outwards and skin them onto the base cloth layer this means we have to remember an extra position and skinning information at each layered point but we just have used that instead of the original radius when we're doing that Max distance check this allows each node to move around and the smears move with them and it only works if the top cloth layer doesn't move very far relative to the bottom and if the bottom cloth doesn't deform extremely but it's pretty effective for many common cases the kinds costume is a good one to see this layering in action the separate layers show quite a bit of movement relative to one another but they don't usually clip through one another uh even

like that chain things on his back are layered outside of this cape so that's it for my talk thank you for coming thank you for listening and I'll do my best to answer questions in the chat uh or in this email address thank you [Music]