The Dao of VFX Animation

foreign [Music] I've been a VFX artist in games for roughly 10 years I have a background in AAA Shooters like BioShock Infinite and Call of Duty Modern Warfare 3. in addition I've worked on a lot of smaller games and a contract capacity and released a few of my own games that I solo developed like Flippy cups and now I'm a senior VFX artist at Wright so what this talk is about is trying to drill into one of these Central tensions that we face in VFX animation and that's related to what our Target actually is you know are we going for realism are we trying to perfectly capture the surface properties of all the natural phenomenon like water or is our real Target more of a more vague

believability I see this debate as mapping cleanly onto is our Target detail or is it actually underlying energy this is something that we face all the time in specifically game VFX because we take things like this this puff of smoke or Cloud every point of detail in this image is implying a vector a directional Vector of how energy is moving and then we do this and we completely violate those Energies and our work tends to fall flat at least in my opinion so this talk is going to go into how I like to make fire from leaves but before we do that I want to talk about why I would choose to do that in the first place

and why it works and I see why it works as being directly related to ideas from philosophical Taoism taoism is a complex developed philosophy that originated in the Warring States period in classical China and kind of the foundational idea of taoism is that there are deep running relationships between very different things in the world there is in fact this organizing principle or process that we are all a part of that we all emerge from called the Dao so what this talk is going to attempt to do is Trace the relationship between these ideas from taoism and World art history and how I am going to argue that it can be applied to VFX animation and help us with our work

so the way this talk is organized we've already done the problem statement we're going to trace this problem through some examples in art history we're going to consider a mindset that would privilege energy over detail to the expensive detail Trace that mindset to taoism specifically and put this mindset to work in VFX shaders so I have kind of a weird mixed background I have a an academic background in comparative philosophy at a graduate level with a special focus on China and with a an even more detailed focus in taoism in particular however I do not speak Mandarin my pronunciation is awful to any native or learned speakers in the room I'm sorry so this tension between detail and

energy can be observed even in still art as an example of detail without energy observe this beautiful painting Watson and the shark by John Copley this is is it's a gorgeous painting I've always liked this painting the composition is incredible the atmosphere the drama the figures but the water I have some problems with as a VFX artist um it's incredibly detailed it's it's it's richly treated but it kind of looks like gelatin it's like a Jello mold and the boy and the shark are like fruit in the jello uh the water isn't really interacting with them and the water itself I have a hard time seeing an internal logic of energy at work so on the opposite side of the spectrum we

have energy without detail and I'm bad at PowerPoint yay all right so as an example of energy without detail This Is A Thousand Miles along the river yansa the author is unknown but it's dated to the Ming Dynasty and the approach here is is radically different than the John Copley painting there's no surface detail whatsoever to this water it's all energy um the only things telling you that this is water are the the way that the water is interacting with other things its own internal logic of how it flows the fact that there are Eddies forming in these little pockets the way you would expect water to form Eddies the artist here has issued all detail in favor of energy

and I find that very very compelling I was I was leaving all right we're back so to do a deep dive on this particular approach to depicting water we could look at myuan's Water album or variably a study on the properties of water from the Song Dynasty and you see a very similar logic to the approach here there is little to no surface detail it's it's every stroke on the page is rather focused on capturing a deep sense of the logic of how this stuff moves in the shapes that it creates when it becomes turbulent and contrasting those shapes when it is laminar

so okay why do I find these depictions so successful despite their Simplicity they don't have the beautiful surface detail in this Copley they don't have the amazing sense of dimensionality and drama and Lighting in this Turner and they certainly don't have this incredible sense of surface reflectivity from this Anderson this is a watercolor by the way it's one of my favorite pictures of water in the world now this speaks to me specifically as a VFX artist s so what I'd like to argue is that these artists were essentially drawing with energy in mind at the Forefront they had learned at a deep level how Water Works by observing it and then we're recreating that on the fly with their

brush so as we've seen this tension between surface detail and underlying energy can be found even in still images but Motion makes this infinitely worse of a problem uh in fact erroneous detail or Superfluous detail is far far worse for us than a lack of detail for myself in my own work criticizing my own work in the work of others I call this wrong shapes going back to our smoke puff example as we were talking about every point on this cloud is richly detailed cloud is implying its own Vector of motion its own Vector of energy and when we do this we completely violate that sense of energy this is uh fundamentally wrong and we to each other we we ask this question all the time like does it feel right that doesn't feel right and what we're really doing is asking ourselves like is the energy right here or are we

fundamentally violating a deep logic by doing that so we're left with a question okay what led the Chinese Inc wash Masters down this route of privileging energy over detail in their work and I would argue that bringing it back to Taoism that it was thousand so Taoism as a philosophy this is going to be a caricature of taoism like I don't have time here to go into a rigorous laying out of everything about Daoism but if you want to invite me to do a talk on that sometime I'd be happy to do it taoism Looks To Nature but not just nature the the natural the spontaneous the unspoiled it's kind of rusoian in that sense and there's this term zuran which means the self-sow to act with zuran means that you are so

unencumbered by falseness that you just overflow with genuine action um your your spirit bubbles forth directly into action so the foundational concept as we were looking at a little bit in the beginning of taoism is the Dao and this could be thought of as the way of all things as they hang together the process the generating function but there's also a a natural individual way of each thing individually its own path to establishing that that Spirit of Zordon and because we are all grounded in this process there is a radical sense of interconnectedness taken as a foundational assumption in Taoism this is referred to in the texts as boundlessness no boundaries between

things or my favorite intoxicating bottomlessness essentially that every object in the universe opens itself up to every other object in the universe and there is a radical sense in which I'm looking at the entire universe when I'm looking at this box of water so from data Jing chapter 62 way making or Dao is the flowing together of all things there is truly in the world a deep running sense of connection between very disparate things I actually think this is a more true way to think about the world Than to Assume radical separateness between things so from this Assumption of interconnectedness taoism develops a morality of non-interventionism um there's it's encapsulated beautifully in this phrase which means acting without acting or non-coercive action

it is essentially acting with the grain of the Dao along the grain of the Dow not forcing things not getting in the ways of other things this is put beautifully in Dada Jing chapter 64. those who would do things ruin them those who would control things lose them hence because the sages do things non-coercively with way they do not ruin them and because they do not try to control things they do not lose them so taking a richly detailed puff of smoke and then rotating it to the right is not acting with way and in the sense it weighs both moral and practical it is both what you should do because you don't want to harm other things you don't want to coerce other beings but it's also what you should do if you want to be successful

there's a sense in which acting with relies on being able to see these deep relationships between things between beings in the world and act in accordance with those relationships to not do violence to them and this is Illustrated beautifully in the juanza my favorite book in the world uh in this parable of cook Deng or butcher Deng so how this story goes is the ruler of some province is having a cow butchered for him and he is amazed at the skill the virtuosity of this butcher the butcher's knife just goes clean through as if through nothing and the cow is fully butchered within a matter of minutes and the ruler says how did you come across this skill how did you come by this and the butcher says it's not skill it's Dao when I first started butchering cows all I saw was the cow from the outside I saw

its surface but as I butchered more and more cows I stopped looking with my visual eyes and began to see the Deep structure of the cow and how it relates to the universe with my mind's eye and so he just follows the Dao through the cow and this is from a famous Illustrated uh book of the juanza which I highly recommend it's very popular in both Taiwan and China it's highly highly recommended so the butcher has learned to see the Deep relationships between things he has internalized the Dao in a way that helps him do his work in this case it's the Dao of cow but we can extrapolate this to other things we can extrapolate this to Aesthetics so if you want to be a Taoist artist what you would do is internalize the way

of your subject such that you can capture it with uwei without doing violence to what it is trying to do naturally on its own this is put beautifully in this book The Great image has no form by Francois julienne as being used by the universe to depict itself rather than trying to represent the universe from without and he didn't just make this up as some french guy uh he drew this encapsulation of a set of ideas from the writers themselves uh the early artists themselves from their writings such as guoshi soad put a pin on it if you create art with uwei letting a subject ring true with its own Zordon then the Dao itself can shine through in essence you're not getting in the way of your subject like with erroneous detail or Superfluous detail

and the artists that created these depictions of water that we're considering in this talk absolutely had these ideas in mind to drive home how ubiquitous these Concepts like Dao UE and zuran were in the culture consider an amazing phenomenon Scholars rocks so Scholars rocks are rocks that are thought to portray the Dao in a sense they're like a shadow of the Tao they are records of energies as they trace the course and then you can find them on desks in gardens there's even this painting called mifu bowing to a rock by gulshu which shows how How Deeply ingrained this idea of venerating the Dao as you find it in the world to be okay so what the hell does this have to do with the effects animation so

just like a Scholars Rock every static structure in the world is actually a record of energy flow leaves are a record of the energy as it flew or flowed through the growing leaf just like a Scholars Rock a leaf is a record of energies flowing through the Dow so how do we make use of this I personally see a deep connection between the shapes and leaves in the shapes in fire so stepping through this quickly removing some of the detail of a leaf gets us access to kind of a more General truth about how they grow it is essentially unlocking energies hidden

behind this behind the specific details and then we can build that detail back up with motion and with Logics internal Logics this is an offset internal logic of leaf and then we can overlay an internal logic of Fire which is really just a gradient of Heat and Fuel and then clamp it bring some interior detail back with multiply and apply Color and that's how I make fire from leaves this works because I truly believe there is a deep relationship between the forms and leaves in the forms and Fire and because static structures are records of energy flow we can gain access to those but we have to remove some of the detail that would otherwise be erroneous and Superfluous in what we're trying to create in this case fire

so a usefulness of this approach is getting intraparticle motion which is a type of detail that works much better in motion than taking static detail like in that cloud and moving it around like this however we also can't do that with this uh that's probably even worse so working in this way uh kind of pushes you into a different workflow it's less about Sprites rotating Sprites around and more about meshes as paths for the energy to move and the benefit of this is it's extremely flexible if you've taken the time to build up energy that's truthful to the to the subject then you can give it all sorts of paths to follow and it will feel right you can also change the scale

you might have to adjust but it kind of just works so the general technique is getting at the more General relationships between things by removing detail and then building that detail back up unlocking the energy kind of locked in these static structures in the world and then letting it roam on the paths that we can make for it and this doesn't just work by accident like this there really are deep relationships between things in the world like draft skin uh the cossic patterns on the bottom of a pool and cracked Earth other people have noticed this um I highly recommend the work of Phillip ball on this subject and we actually already do this all the time with noise textures noise textures are a general statement about how things in the world work and they don't work by accident they're actually related in deep ways to how the universe is

actually structured veronai for example can be thought of as like this General case that shows up in many different um objects as like a sort of tension that gives rise to boundaries or boundedness and you can see veronai at work in these three things that we've been considering Ryan Brooks has worked with this I highly recommend his blog post getting the most out of noise in ue4 where he takes veronoy and he builds it up into cracked Earth and caustic patterns and even marble but personally I like to use veroni for water in particular turbulence so building up here throw it on a cylinder add in some normals and mask it away and some roll position offset and some quick stylized color and you end up here

so I believe there is a fundamental affinity between the shapes that my one was trying to capture here and the essence of varoni noise and hopefully you can trace a direct lineage from my one to this water Shader so a takeaway here is that focusing on energy more than detail and then using those energies to build up detail that's more appropriate actually empowers us to work in a wider range of art directions with more consistent results and actually it would have been possible to build either the Fire or the water Shader with like a slider from realistic to stylized would have been cool I didn't do it ran out of time so in contrast relying on service detail like reflectivity and transparency is

far less flexible and had I chosen to rely on those things to make that water look like water I suspect that the underlying energy of the water would have been inferior and again this works because there really are deep connections between disparate things in the world because static structures are records of energy flow everything comes back to energy flow eventually Okay so if we are thinking about dialysis Aesthetics is essentially internalizing the Dao uh okay how do I do that um for myself uh my path has been getting just really interested in these patterns for example patterns that emerge from Wolfram cellular automata a cellular automata is basically a grid of um like on off switches that have rules for just like their neighbors and and

what they're going to do and on on rule four you can see that uh you know it's basically nothingness and then full Randomness is is just noise but with the grid of automata and a simple rule actually really interesting deeply Rich forms can emerge like rule 60. in these forms happen to pop up actually all over the place a shell is essentially a one-dimensional cellular automata and I've also been interested in mathematical objects like a serpinski triangle which is more or less a fractal it um it operates on this simple rule and it turns out that even a serpinsky triangle shows up on shells and this is actually mine I have a collection of these cellular automata shells this was from my desk a couple days ago

these are my Scholars rocks but I think the better strategy to internalize the Dao is to just go out into the world to train yourself to see the connections between things and a great way to do that is by taking your own reference this is Verona noise that I found in the wild very studiously on a beer glass and I hope you can see the fundamental similarity between these forms in this beer foam and these beautifully depicted forms from my Yuan so the takeaways I hope that I've left you with is that as effects artists if you have to choose between energy and detail the choice for me is clear every time choose energy it'll be far easier to add detail to something with good energy than try to add energy to something with too much detail also that there really are deep

connections between disparate things in the world like there's there are ideas from taoism that are fundamentally true and that we can make use of those General relationships between things to build up a sort of detail that works much better in motion than static detail so to leave you with one last idea from taoism there is this concept of Xiao Yao Yo which means free and easy wandering and it is essentially to willingly shed some of the responsibilities from from society some of the expectations and wander easily through the world through nature in particular so if you are finding yourself soon struggling on a problem and with your head in your hands at your desk under a deadline uh shake off the deadline go for a walk employ a little Xiao yo go out into the

world and maybe take some reference images while you're there these are some suggested readings the Dallas Classics down to Jing and juanza two books by Francois julienne that are directly related to Aesthetics the patterns in nature book by Philip ball that I mentioned and that awesome Ryan Brock's blog post getting the most out of noise in ue4 thank you I think I burned up all the time with that uh like video mishap but okay if anybody has questions I don't know maybe the cas will have an idea of what to do cool

oh do your evaluation please hey Jeremy um so I like to think about things pretty mathematically I was wondering if taoism and Mathematics uh talked to each other much at the time I guess that's a good question I don't know of anything in the source texts but within the intellectual history of China ideas from taoism played a strong role and interacted with other strains like Tian Thai Buddhism and Chan buddhisms and Buddhism and these schools tended to be more rigorous so there might be something there to look into but I I don't know off the top of my head okay thank you no problem cool seems it thank you so much everyone [Applause]