No Man's Sky: How I Learned to Love Procedural Art

[Music] Okay. So, my name is Grant Duncan and uh I'm the art director at Hello Games. And it's been a little bit strange for me today because I've been surrounded by like art directors from Naughty Dog, from Boware, and some really amazing people with amazing teams and really large teams. and I work with uh there's four artists working in our game, but it's a procedural game. So, actually, I've probably got hundreds of thousands of artists working with me. Unfortunately, they're procedural artists and they're all mindless idiots, which uh means it's a bit of a challenge. So, I'm working on a game called No Man's Sky, and uh it's a science fiction game. And uh it's a procedural game. And it's a game about

discovery, about exploring the unknown. And when I say science fiction, it's this kind of thing. And I think when you often say science fiction, people kind of picture a kind of uh a future war, a kind of they imagine these great ships and space marines and lots of gray and lots of black. And to me, that's not what science fiction is. When I think of science fiction, I think of like 70s book covers and artists like Chris Foss and John Harris. and they were kind of artists who became really famous in the ' 70s and they've kind of got a little bit lost recently and I think that's very sad and if you look at a lot of book covers in those days they were very experimental they were doing lots of different things and a lot of the art wasn't particularly great it was actually a lot of bad

artwork but they were all trying new things whereas now it feels like science fiction has kind of fallen into a a bit of a language in itself itself. So, you almost can't stray too far from that without people getting a little bit confused. So, No Man's Sky is a procedural game where you're in an infinite universe. That's what you're seeing here. You are here. You are fairly insignificant. And each one of these stars, there's millions and millions of stars. And when you visit them, each star has a solar system. And so, it has various satellites and planets. and you can seamlessly travel between them. And this was like a real big tech thing for us. It was one of the first things we started working on was like the terrain engine and the ability to planet hop from one planet to another. And as far as our sort of research that we did or the kind of concepts that we went through, we looked at the real world. It's very easy when you're

working on games to look at other games or to look at movies and to kind of find yourself going down these channels. But we really wanted to kind of keep a believability uh to create these worlds that feel like real places that you actually want to go and explore. And all these photos are from obviously from the real world, but if you look at them, any one of these could be on the cover of a sci-fi magazine or a sci-fi book. And it one of the things from research in the real world is you start to realize just what bonkers kind of crazy things you find like lava caves or these kind of limestone pools. It becomes like you don't quite understand why they exist, but when you start to delve into it, it all starts to make a little bit more sense. And these planets, they wouldn't be very interesting unless they had life. We wanted to have this feeling of kind of like when you were a kid and you went to the beach and you'd go and sort of go up to rock pools and you'd feel like

everything was very new and interesting. And one of the things we talk a lot about is things like The Journey to the Center of the Earth or The Lost World and those kind of old movies or a lot of Ray Harry Howen films where they'd find like a mysterious island and they'd find they'd head off into a cave and they'd find these strange creatures. And so we'd look at real world creatures and we try to and we're not going for the real world just because we wanted to stay away from alien things. But what we wanted to do was to find to keep that believability that we can create these rules and we can make believable creatures and then we can break those rules to make more alien feeling things. One of the things with procedural generation when we first started was I work with like a team of coders and they just casually talk about it and be like, "Oh, how many planets are we going to have?" And they'd be, "Oh, we're going to have 18 quintilion planets." And I'm not particularly into maths, but

I wrote down the number to try to work out how many zeros that was. And it's it's one of those numbers that's so big it becomes entirely meaningless. Like you can write it down. You can even try to think about it, but it's just it's a pointless number because it's just really big. But it's also not ridiculous because it's just maths. It's just 2 to the^ of 64. So it's just a case of having the length of a seed of a random planet. And when you put it in those terms, it stops being insane. And also because there's four artists working in the game, we only have to do 4.5 quintilion each and we'll be absolutely fine. So, when they first started talking to me about this procedural thing, because the way it worked was that Sean, one of our coders, he was dabbling around in his spare time with procedural technology, and I didn't really know what it meant. And actually, after kind of announcing No Man's Sky, and if you read any of the comments, it turns out

that nobody really seems to know what procedural generation actually means because it doesn't just mean random. You can have random elements and it's not different every time. It's essentially just maths. And what it actually is is just rather than things being hand created by artists, they're created by algorithms and rules and they're what dictate what you end up with at the end. And but I still didn't really understand procedural generation. Uh in fact, the guys, the coders, they must have tried to explain it to me hundreds of times. And Dave, he's a guy I've known for a long time. He took it upon himself to just try over and over and over again to really get me to understand what they were trying to force me to do because I was quite reluctant because I think a lot of artists are a little bit intimidated by procedural generation. It's it's very alien and it takes a lot of control away from us. But Dave explained it to me as a big box of maths. And that didn't make me feel any better because I'm not particularly good at maths. So um but the way he explained

it was you take all these random elements. So in our game we'll have like creatures, trees, rocks, props, terrain, skies, space and all those things and you funnel it all into a box of maths. You turn the box on and it applies all the rules of our universe and then it spits out a seed value and that value is the same every time. So if I give you the seed value and you've got the same box of maths, you'll get exactly the same planet or the same universe in this case. And that's not an unusual thing for procedural systems to be doing. So people use procedural for quite a lot of different things. Like this is a game called Minecraft. Yeah, you guys might have heard of it. And it uses procedural technology in a slightly different way in that you'll load up Minecraft and it'll be a different seed every time. So a different planet. And depending what biome you're loading, you'll get a different kind of terrain. And trees are placed differently. And that's just each

biome having a different set of rules that dictate how that world is made and how it is decorated. And so as you're flying along, like the Minecraft worlds, they're not stored anywhere. They are being generated as you're traveling across the terrain. People also use it to create cities and buildings. Like in the top left there, that's proc world. And uh you guys should look at that because it's really amazing. He's he's doing some really interesting things with ruins and buildings. And in the bottom right that's a city generator which is by Chris Delay from Introversion. He was just kind of dabbling around with it. And none of those things are stored anywhere. It's just you'll click a button and it'll randomly populate. And you can see in the left you can also use procedural generation with handmade assets. So you can construct a building and you can hand make like walls, windows, doors. And anyone who's worked with like tile sets or modular systems will be looking at that and thinking, "Yep, that's, you know, we we all do

that." And but what's clever is that you use what's called an L system, which is just a very simple scripting system to dictate how those elements get put together to create a building. So, it's a really basic one would be you have walls, you have a door, you have a window, and you have a roof, and then you've got a very simple little house. And obviously, the more rules you have, the more complicated you can create buildings. So, yeah, I was still a little bit skeptical uh for obvious reasons. So, I decided that I'd ask a load of my artist friends what they thought about procedural art. know whether I was insane for getting involved. And yeah, I got this response quite a lot. And not many people could actually explain to me why they felt this way, but I'm not sure whether it was kind of it was quite aggressive. Some of them as well. It felt like they were almost being affronted or being attacked.

But this seemed to explain it a little bit more. And it's kind of unfair that a lot of people were saying it's just a load of programmer art. It's just endless boring terrains, blobby looking creatures, and taking control away from artists was the number one thing that people seem to be scared of. And I think the reason that people are thinking this is that it's true like a lot of our sort of tech in games that you'll see it's very driven by programmers and a lot of like it's always like they are at the forefront and they are pushing the boundaries and it takes a while for artists to get involved to make something a bit more visually interesting. So they're not necessarily interested in the way it looks but they're very interested in the systems behind it. And often that's why you end up with sort of that realistic style because they want they're often doing simulations of the real world and they're trying to make believable terrain with realistic grass and that often ends up with quite stagnant

looking visuals and that's not very interesting to us as artists. Something else a lot of people would seem to think was that procedural technology is random. I think they'd imagine that you had like a button that you'd press and it would just randomly make a load of trees. It would randomly make a load of terrain and just plonk it all in. And whatever you ended up with at the end, you had to stay with. You had no control. And I think that's because the truth is we're actually all control freaks as artists. We're so used to having complete control of every single pixel. Like especially now as digital artists, we can go in Photoshop, we can zoom right in and be completely obsessed over something that no one else will ever care about. And procedural art is kind of the opposite of that, but it's not. And you have to like I really love procedural art now, but it's definitely required a bit of an attitude change.

Uh this isn't a very good analogy. I think I was actually hungry when I was thinking about how to explain this. But if you imagine like we have every prop in our game can be random and can be varied. So you can click on you could have like a tree and you could be just clicking a button and it' be constantly giving you a new tree. And you could do that for creatures, for props, for planets and all the different ecosystems and all that kind of thing. And if you put that all together without adding any kind of rules or boundaries, then you just end up with this ugly mess. It's kind of the same as when you're painting. If you mix all the colors together, you'll just get this ugly kind of brown sludge. So, we knew we had to kind of think of a way of stopping that from happening. So, we have our what we call our blueprint system, which is uh essentially like a template system, and it's applied to absolutely everything you see in the game except for like the

terrain. So, it's all like all the props and creatures and spaceships and those kind of things. And the way it works is that an artist creates like a base template and then our system will spit out variations. And the way it does this, it can apply things like bone scaling or apply different materials, different layered materials. And it means you can get lots of different looking objects without having to go in and hand make them all. So here's an example of it working. So on the left is what an artist would have made, probably blocked out in Zbrush. And then on the right, you see it gets morphed. Then it gets randomly picked accessories depending on the kind of climate it's part of. And then to the right, you can see it gets picked what kind of material, what kind of skin it has. And then it has markings. And the two on the right are the bone scaling system. and it just randomly picks

its shape to get to very different silhouettes. And you can see the same being applied to trees. So you have they're both exactly the same except that the same they come from the same trunk and they get different branches picked and different leaves and then different materials. And then you can see at the very end it gets the bone scaling and the different poses selected. And this is just one tree. Obviously, we can produce hundreds of thousands of different kinds. And it's all based on libraries. So, we have libraries of hundreds of kinds of bark textures and bark materials and moss and all these different rules. This is a view of it in game. And one of the things with the creatures in particular is that in researching the different skeletons because we're such a small team, we have to be really careful not to make too much work for ourselves. Uh, and one of those things is that we try to share as much as many of the rigs as possible. So there's like the antelopes would share

with like horses and cows. And the more we delve into it, the more you actually realize that a lot of creatures, they have pretty much the same skeleton. And the only difference being or the main difference being obviously you'll get subtle differences like number of vertebrae but if you have a system that can move bones around and can lengthen bones and if you have a good enough IK system then you can actually get hundreds of different kinds of creatures just based on one rig. So the way we do that is we have we create a base a base rig a base template and then we apply all these different things over the top. So, we have like different poses. So, you can have a creature that is maybe more cow-like, so it'll be hunched over. And we or maybe you want to have like a sort of stallion, so it'll be more proud and it'll be upright. And because we can add these over the top, it means we don't have to create hundreds of different walk cycles and run cycles. We do have a load of different ones depending on the creature. Because we have bone scaling, if you make a creature really fat, we can automatically make it start to lumber a

little bit more. But if you make it really thin, maybe you want to bounce a lot, bounce a lot like a little gazelle or something like that. And this shows our uh this is our shark rig. And that also gets used for like dolphins, whales, and some more prehistoric or alien looking creatures. And the same way, this would just be one little blueprint really quickly made by an artist. And because we have like a material library. So this guy will be using like probably underwater scales and we have different markings which are all procedurally generated and all those things like fins and little spikes. They're all we just have libraries of hundreds of these things and they can all be applied and it's pretty cool. In the same way the ships this is an example of the fighter class of ships. They're a little bit they work a bit more simple than uh the creatures in that we'll have like depending on the

type we'll have a few rules like the fire will have a cockpit, it'll have a nose, it'll have engines. And that way even though you're getting all these different looking ships, there's kind of consistency there. So, you know just from looking at the silhouette of what kind of ship it is you're chasing after. And when we're building the creatures, we do a lot of planning because based on that one little blueprint, you're going to get all these different variations. If you don't if you do that blueprint badly, you'll just end up with hundreds of thousands of really looking creatures, which you don't really want. And actually, that's one of the things about procedural generation is that it is incredibly good and incredibly fast at making really art. You would not believe how fast it is. So, uh, in this situation, you can sort of see on the left, even early on, we were thinking about what kind of creatures can we share rigs with. So, we'll have like lizards, chameleons, frogs, and duck build platypuses. And these are all like earthlike creatures.

But again, our thinking was or our logic was if we can learn the rules to create creatures that you see on Earth, then it's really easy to then bend those rules and to go to extremes to make a lot more alien looking variations. This is the uh some of the concepts when you're first starting out in the sharks and the whales. So yeah, at this stage we had a very ugly soup. So we have the trees. You can be clicking on view variants. You're getting hundreds of thousands of different looking trees. Lots of different looking creatures. Very varied. So, it's not just hue shifting and the actual silhouettes are changing a lot and the scales and you're getting I was really impressed at this stage, but I was still really scared and very worried because when you put it all together in a in a game, it's just a big ugly mess. So, everything's

colorful. Everything none of the shapes. There's no consistency. Luckily, I work with some very talented coders. So the way we do things is that I'd create a load of objects, we'd put them in game, and then I'd moan to the coders, and they'd fight back, and it would be a lot of back and forth, but it was a very useful process because it meant there's loads of things I would do as an artist, and if a lot of you guys have been environment artists or anything like that, you'll do loads of little things that you maybe don't realize you do just because you know it works. So things like you'll put a tree down and you'll add a little bit of grass at the base of it and you'll add a few little rocks and you'll put some bushes and you you'll just be picking out these silhouettes. And with a procedural system, none of that happens unless you tell it to happen. So what we do is we'd have like a tree would get placed. I would then paint over it and I'd talk to the coders about like how do

we make it so that this is more likely to happen and we'd slowly build up this kind of list of rules that we would apply to the procedural system to make it more likely to produce a results that I would get if I was doing it by hand if that makes sense. So this is an example of some of the rules that we put in place. So when a planet gets generated, it picks it has a climate and ecosystem and a lot of that kind of thing is dictated by how close to the sun it is or how close to how close to the moon it has and all this kind of thing. And in this case it's like a swamplike planet. So have a lot of fog. And what we'd want for that kind of planet is that for the props that get selected or generated and for the kind of creatures that you find in that planet for it to be consistent, for it to be just for you to accept it, for you to not think it's unusual. And so it kind of based

like lizards and amphibians. You can see they've got a waxy sort of damp skin and also things like the colors and the type of trees that have been picked. So we'll have like swampy sort of hanging bushes and trees and then some of the red from the leaves will be added to the creatures just as almost like a natural camouflage. We can also have the opposite of that where if we want a creature or an object to really stand out, we can have it pick like a complimentary color. So we could have like a a bright blue frog or that kind of thing. And actually, one of the one of the things we had the most back and forth on has been placement of objects. And it's not something you'd think would be that big a deal, but on procedural things, especially on a game that's all about terrain, it becomes this huge deal. And procedural systems b use like noise functions and that's how you place objects on the terrain. And those naturally mean you get this kind of random scattering which is never particularly pleasing. So,

anyone who's worked in environment art knows that you have this clumping that you'll do where you place large objects and you place medium objects around them and then smaller objects. It's just something that everyone who works in environment art knows and but a procedural system has no knowledge of that. So because our props are all based on like they have an understanding of what shape they are, what their volume is, it means our procedural system can it can pick an object of a certain size and it understands that it wants to place a slightly smaller version of a similar object nearby and it means you get this kind of clumping and it means you get a lot more pleasing silhouettes. Something else we have is our color theory system. And I say teaching color theory to a robot is because it's actually quite difficult to explain. And it means like even when you do it yourself, you often find yourself breaking your own rules. So it's been a real difficult thing is just to come up with this system. So for a start, there's no random colors

in the game. Nothing random is going on. We have custom pallets. So every material in the game, if it's like skin, fur, scales, rock, wood, I have custom made all those pallets. So it means there's a selection of colors that you know are going to get used. And then each of those pallets, I can dictate how much hue variation there is within them. So it doesn't have to stick to those colors rigidly. It can kind of go with a few uh variables. And it means uh like what color gets used in which situation. So with things like creatures like a base color will get picked and then it'll often have like a lighter underside or maybe you want markings and those markings will often be like a complimentary color so they'll kind of blend in nicely with the object. And you can see the same on some of the foliage. And when you see it in isolation here, it doesn't necessarily

make very much sense. It's just like yellow leaves and blue moss. But because these are all linked together, it means like when a planet gets generated, say it's got like a lot of blue minerals, so the ground will become this blue color. And when a tree gets placed there, it can get told to have the moss to pick up the color from the ground. And that all it kind of all flows through the system. And keeping track of all that kind of thing is really difficult because we have 18 quintilian planets as I mentioned earlier. So flying around each one of those planets and making sure the colors are nice becomes this impossible task. So, what we have is these little robot drones that we send out into the galaxy and they fly down to each planet and they record a little animated GIF and then they spit that GIF out and they write it to a simple little HTML file. So, I can open it up and then I can just have a quick look and I can very quickly see if there's like way too many red planets or trees look really ugly when they're in this color.

I can click on that and I can get the seed value and I can then fly down to that planet and I can figure out why it's happening and I can tweak the rules to try to make that less likely. And so I think especially when it comes to art direction, there's like the best games to me are games where you don't question anything. Things just feel like they belong together. And I think actually I actually think Nintendo are the best at this bizarrely. They never talk about it and you never see anything about their tech or anything like that. But they're absolutely amazing. Like everything just feels like it belongs and that's a very difficult thing to get with procedural stuff because it's being generated in a way that's you don't have complete control over. So we had to find a way of grouping things together. So when a prop when an object gets made, things like horns you can tag and you can sort of tag things of a certain shape and type. And we can What's a good way of

explaining this? You can have it so a creature that looks aggressive. So if it's got lots of teeth and lots of spikes, you can tag those as being like hostile. And it means when you're generating a hostile planet, it can generate more of that kind of thing. And it means you can get a lot more consistency. So things like the you can see in this example the kind of trees and the little bushes, they've all been tagged as like I imagine it's blobby in this case or glowy. Very highly technical terms like that. But it means that when it's populating a planet, it all feels like it belongs. And the same way like these creatures, you'll see their markings. They'll have been tagged as glowy as well. And it means that it all just feels like it belongs together. and it has this kind of believability about it. And here you can see some of the clumping. So you'll see on the left a lot of those complimentary bushes to the orange and the way the cave has been decorated. That none of that's by hand.

It all just happens because of various rules. the same way like placement things like trees, we can tell them how far above sea level certain types of trees are more likely to be seen and or what angle certain bushes are going to grow on. And those kind of rules have this massive knock on. It mean like if I change like some of those rules, it will have it's really scary. It'll change every single planet. So I could accidentally check something in and then the guys will suddenly load up a planet and they'll just be nothing but trees and he'll run about two frames per second. I think one of the things with procedure generation that I found really really exciting is that I've been working in art and games for a long time and tools are so amazing now but they still feel really arduous. It still feels like a real slog to get to where you want to get. You're still having to load up ZBrush, transfer maps, blah blah blah, all this

kind of thing. And it feels like there's more and more tools now that are starting to enable procedural technology like the quick suite, speed tree, and substance design. and they're kind of avoiding all these pointlessly long drawn out processes where we can just get we can get results without having to hand draw and jump through all these technical hoops. I think also like one of the reasons I like I'd like to first of all thank Andrew for inviting me here because it's a bit of an unusual to invite me to this kind of thing, but I'd really like some bigger studios to be getting involved in procedural stuff just because making all the tools ourselves is really hard work. And I know that, you know, if Boware were to start doing some tools, that would be great. You know, I'd like to dabble with those. But it doesn't have to be entirely generated. You can have hybrid systems. So you can have you can still hand make art and you can

still have procedural systems involved in that. Like our systems are essentially character creation tools. Like even our tree, like every prop in game is just a character creator. So you have all these different sliders. They all control attributes like leaf density. And it's all the same theory as things like modular character creation, which is something we've been doing for years in games. So, I think there's real scope for other studios to do that kind of thing. And I think it really is skipping all the boring stuff. It feels like with digital art, it's very easy to get caught up in a lot of detail, but it's really nice procedural stuff for prototyping. you can really quickly get to results. And it means it's not all about adding variation. Like when I'm making a creature, I can just throw a few shapes together, get it in game, and then hit view variations, and I can just

see, you know, fins and spikes and horns. And even if that was just a process for creating concept art, that's still a really nice system. It's a real fun thing to do that you're just farting about basically and making all these different looking shapes. And you could then take that and export it and bring it into Mayer or Zbrush and then go crazy. But for as far as getting to something you're happy with, I think it's a really powerful tool. So yeah, any questions? [Applause] question. Oh, no time. Okay. Yeah. If you'd like to ask me any

questions, I'll be in the wrap-up room. I don't think we've actually got time for questions, I'm afraid. But yeah, if you catch me outside, I'll be there. Thanks very much. [Applause]