The Witcher 3: Optimizing Content Pipelines for Open-World Games

My name is Martin Thorson True and I'm a senior technical artist and I've been in the industry for about 13 years. Um, the last game I helped out was Witcher 3. A small disclaimer though, I am no longer employed by CD Project Red. I moved on, but I was allowed to do this anyway. So, welcome to content optimization pipeline for an openw world game. I will start with a statement that artist they can do amazing things beautiful things is how we show this is in-game by the way. Uh but they can also do things like this and this is not to bash on any artist. It's just stuff that seeps through the cracks. Uh, I call this Mesh

Monster. I have no idea what it is, but it's in there. Um, they also like to do stuff like this. This one about 5% of the actual mesh is above ground. The rest is below. So, Titanic. Yeah. Also, artistic creativity as I call it. It's the same here. 30% is above ground, 70% is below ground. Uh, one famous asset we had at city project was the chicken bucket. And that one has 1 and a half million triangles. And you can imagine it was a big on the screen. Um, it was very expensive chickens.

So uh this presentation will be in four parts. Why we well you saw how we did the optimization, the outcome and what we learned. So why there there was a need to find something that was robust and scalable to the everchanging demands of an open world game production. We needed to find the assets um that needed the extra love and attention. Um, we did not have that many post or pre-export checks and we had approximately 500,000 entities placed. That doesn't mean they are unique. It just placed entities. We could also change the entities inside in the inside our editor like give them more collision shapes, merge meshes, add

extra loads, uh add materials. We could also author assets like materials, um particle effects, shaders. And inside the editor, we also have certain components like light components, sound components, and vegetation. All of these costs, all of these need to be optimized. So, how did we do it? We created this. I call it the database viewer. Um, I will go more into detail on this later, but it's a database back end and a small viewer on top of it. with the world. The the general layout of uh the Redkit engine is that we have a world, we have layers and world has properties as well. And layers have

properties. Inside layers, we have entities and they have also properties. And inside entities, we have components and they all have properties. Um, we needed something to iterate through the worlds and the layers to find certain properties that we were interested in like size, vert count, materials, uh, radius of a light and whatever you actually wanted. So, we needed something to iterate through. We did. We queried all those properties we needed and then every time we found something, we spit that out as a one string inside a text file. Like I said, we needed something fast to implement because we're in full production. Um, so this was the solution. I'm not saying it's the optimal solution. This was our solution. Uh so we everything was found was put into uh a text file that is looking like

this. So it's an SQL statement basically just saying insert into blah. So with this text file, it's not a database yet, but with this text file, we can iterate through this text file and uh create the database. This ended up being several files depending on where in the world you were. The files, some of them became huge, 2 GB of text. Um, but with those files, like I said, we used Python. We iterated over each line and created the database. And this is where all the fun and magic comes. Anyone could come to me or they could do it themselves, come up with what they wanted to know. And for example, give me all the meshes that has

no loads and the triangle count is larger than 2,000 and the outer height distance is greater than 50 m and we can find all the assets. It will kick out a list of where in the world it was, the name, the layer, the position, everything. Another example would be give me the position and the radius of the light to cast shadows. It's a feasible question. Uh last example, give me the vertex count and bounding box bounding box area of mesh components. So this tool did all that. Um just to see if I can there quick overview artist or anyone using this this was on the server so everybody had access to it. uh they could write a statement if they knew how to do SQL

statements, write it over here. They could execute, get the output on the right hand side. If they liked what they did, they could save the command and since it was on the server, everyone had access to their commands as well. And the commands went over here and you can double click it to use it. So you don't have to write it again, of course. Um, if you wanted to find it, you can see on the map exactly where it was. So, you can see this light with radius of 200 m or whatever it was there, you can find it. You see it on the map, you can click on this map and that will actually teleport you. If you had the editor open, it will move you over there directly. So, the artist could do an SQL statement, wear the light and directly drop you in at that position. Um, you could save the image so it's an artist can actually see all the broken meshes or loads or whatnot. They can get

the image and see all the dots on the screen. The image was saved in 4K for the people who wanted to zoom to see, oh, I'm in this corner of this house. So uh city project red also had uh Google solutions. So all the data went up to Google to Google docs and this is how we presented it for the people that actually needed to help out fixing it. We presented lists of names and asset and what was wrong and they went to Google Sheets. We um as you yeah just said that uh as you see we could also have different pre-defined images. I will show you two next slide where we call it the vertex density map, high foliage map, uh light radius map. We we could just add more of these images.

Um and this slide is already done. So this is the uh vertex density map and the high foliage count map. Just to show you a little bit what what we did by default on the left hand side we have vertex density map and basically what that was we took the vertex count uh and looked at the bounding box of that entity to just get a idea of how many vertices were in a certain area. With this, of course, you can see directly that there's certain areas that are a little bit more dense. Most likely that is the area we need to optimize, but it can also be used for the level designers to see, oh, this area hasn't been populated. Maybe we want to add more stuff over here. The um high foliage count map was just for to see that okay in these areas we need to start trimming off the trees or

grass since everything was uh uh statements that we can kick off every week. This is basically how it looked like. every week, every Monday, I kicked off the same SQL statements um so that we could do a burndown chart. So connected to Google, grabbed all the data that was uh there and as you can see we can have a nice burndown shot week to week. So they can work during the week. we can put in strike teams to one week we needed to fix all the LODs and the next week we can see if it was fixed or not and this just helped us to focus what we needed to fix during that week. So the outcome here are some amazing findings. We found a shadow casting

light that was 4,000 m in diameter. And I'm in America. I have no idea what that is in miles or in feet. Sorry. But it's huge. And the funny part is as well when I looked at the layer, I think the layer was cave. So it's underground. Another amazing find was we found, again, not bashing on any artist, stuff sips through the crack. It's everywhere. We found animated bows. These were about 200. They were all underground. The outer height distance was almost a kilometer. And because they were animated, they had skinning data and bones. Uh I also found test assets. And this is a couple of week before final, the first

final. Um, we found a quarter of a million vertices on one ship. We found over 200 meshes that had 2,000 triangles and no loads. This is an open world game. You need loads. A lot of them. And we found that in one place there was 105,000 grass entities in an area that was smaller than 32 square meters. And the funny part in this one is that this had nothing to do with the artist. This was actually an implementation error of our folio system. So the programmer loved this. He could find this area. He could quickly go there look at a code and yes he said awesome now he can fix it because he had a repro case. He saw where it was and the programmer could fix this problem because it was programming issue and not an art issue. So it's not just artists that use this tool and could use it even

programmers. So the results because of all this data or we could focus on certain areas and certain assets and we could put in strike teams. So this week you can look at only the LODs and they have all the list to do that. Uh it was very efficient in that way to know exactly what you're looking for with all the lists. It was accurate because this is taken from the build. So in theory, if we ship that day, this is the data that would go on disk. So it's accurate. You had all the paths to the assets like layers and entities, components, it was all there. So it's and so what after that we had what did we learn from this? First of what went wrong the huge text

files. So have anyone tried to open a 2 GB of text in Notepad or Notepad++? Basically it coughs and says no not at all. Um, so don't do that. That's horrible. And on top of that, because it's SQL statements line by line, if one line is wrong, the database will not be generated. So you have to rebuild it again and circumvent that. The next statement is trying to add everything into the database. And I will try to explain this much better. If we have house A over here and we have the same house A over here, different layers or maybe the same layer, but they are the same house, just different positions. So this is a design flow from my part, but we were in a hurry. Sorry.

Yeah. Um, what the script, the code did was it found this house, put that as one line of SQL statement saying, "Hey, I found house a position blah." it continued. It found house A again and wrote a complete new line with the same data but just a new position. So that's the reason the text files are two gigs. If I would do it again, I would probably try to just say here's house A as a table and then I would have positions of houses A as a table. houses and positions um would also significantly lower the actual size of the text file. Um there was no real-time update to the database. So if an artist fixed stuff, they had to wait for the next build to execute the SQL statements and from the tool. Um if I would do

everything again, I wouldn't do textile either. I would just trying to put the database directly into the editor. Um, the last point is my findings because we work with Google Docs. I could not find it could be me. So, no hot feelings Google. I'm just saying I could not get the Google API to create pie charts from the Google Excel sheets. So I had to use a separate lib to do that for example and that's just thing we had Google we did all things but I need to use other libs just to kick out the burndown charts I would love it to be one lib so what went right like I said it was useful for more than just the artist level designer can find areas they wanted to populate uh programmers can find errors um we found things that we didn't know exist. Uh the data was accurate because

it like I said it came from the build. It was fast to search through because SQL that's the reason I chose SQL. Um and by having this image, this is something very amazing. Um during my time uh I've learned that if you give an artist an image of a map and saying these 200 red dots are errors, they are more much more happier than having an Excel sheet with 200 of lines. They're saying these are the errors. Somehow having an image makes it real. So it's super useful. So uh I want to thank um a couple of people and I will butcher their names I say sorry already. Um Kristoff Trishin, Patrick Drosvski, Emil

Givvenski, Wukosski, Jose Tesera, Costas Mikolopoulos and CD Project Red. Thank [Applause] you. If you have questions, there are mics everywhere. Otherwise, you can uh grab me outside out there. Yes. Hello. So, first of all, thank you for the talk. It's really informative. Um I just wonder um so you said that uh you guys have the database and then you guys were able to use the uh queries SQR statements uh to to to find something that you want were was really impossible to find during development. How much did you guys involve testers into this process running this uh you know just

running this uh queries and then to find errors? These queries were actually basically me sitting on Mondays and Tuesdays and to create these sheets. But the artist when they saw the tool, they started using it themselves. So the artist just open up the tool and they started to find their own issues and they started to write some crazy SQL statements that was huge. Yeah. So that was super awesome. So we did not have testers. The artist directly went on. Okay. Thank you. Yeah. I was wondering if you could talk a little bit about uh time series data about did you design the database so that you could look at historical trends that way or did you do all of that by analyzing your Google Sheets X plus facto there was just Google sheets analyzing so during this time I also had to learn SQL so it's it's it was it was awesome

plus C++ and all that. Yeah. Thank you. Okay. Thank you. [Applause] [Music]