Gaming x AI: How Atlas makes models for 3D worlds 200X faster! (ft. CEO Ben James)

we recently did a postmortem with a game that we worked on WE generated for them 20,000 unique buildings it was a VR game developer and at the same time they put a team of people on generating some buildings of their own so the whole process including training the AI generating the results feedback loops and then ultimately the game ready assets was about 200 eggs faster so we did the 20,000 buildings in the time that their team was able to make a 100 all right hello everyone I am here with Ben James from Atlas and we are talking about AI Ben can you talk a little bit more about your company and what you guys are doing sure you know first thanks thanks for having us it's it's really exciting to talk about what we're building at Atlas what we're doing is we are trying to create Virtual Worlds from usually collections of 2D reference images so things like mood boards or concept art and the way that we've built

our AI Tech so we built our own Tech stack over the last few years and it's really oriented around looking at 2D images trying to disel dispel what are the underlying patterns you know Global assemblage local relationships parts to holes that it can distill and then create generators based around that and so from a from a set of reference images we're able to make very diverse interesting eclectic and you know game ready worlds so how does that actually kind of um manifest is it like a Unity or unreal Plugin or in in terms of like games are you focused on gaming as a primary vertical are you going for multiple verticals how does that work yeah we focus on gaming because gaming is a is a really unique industry where you have the need for both high levels of complexity and Aesthetics coupled with real-time design considerations and so since we want to be a tool and a platform that that's widely accessible

to game developers we're a standalone Tech stack but we do Integrations with unity with unreal also with 3D modeling software like blender or or Houdini on the procedural side so that ideally somebody can just access this Tech but but still remain in the pipelines and workflows that they're accustomed to in the industry okay and how did you identify this segment as one that's attractive for you guys obviously AI could be applied in many different ways and I've interviewed like some of the conversational AI folks there's other areas in AI that some people are excited about voices like 11 labs and so there's all these various ways that you can apply Ai and for you guys in terms of like the 3D environments and world and kind of going from 2D to the 3D how did you come across that what was the rationale behind choosing this it's really tied to my background so so my my background is as a computational designer as an architect actually and I've worked for the last prior to Star an outlas I worked for the last few years at the intersection of design and

Tech sometimes this meant designing for clients where data played a key role you know like Formula 1 for example sometimes this meant designing where data had an active system systematic role in responsive facade systems and where I started to develop an area of specialty Specialties was with Machine Vision and augmented reality and I started thinking as an architect maybe there's an interesting way to connect the work I'm doing with Machine Vision with machine learning to act as a design partner and so I'm a self talk coder I prototyped up an early version of this about three years ago and I thought architecture was the perfect starting point because essentially architecture drawings are two-dimensional pieces of information plans sections and elevations that are embedded with threedimensional data through line weights and annotations and and I started looking at could a Machine Vision system dispel that information and then could a machine Learning System understand the underlying relationships there then we we got to early on funding from the European Union actually to build up so this about two and a half years ago now we got funding from the EU

to build up our own AI technology in accordance with high level guidance from the EU for trustor the AI applications which is which is really nice because that means that one we've we've been in this space before the boom of geni and it's changed a little bit our framing for how we pre-chat GPT we we're pre-chat GPT and and it's framed what we've seen as a problem so I think you know for us I remember very early on you know we started we talked to Architects visual effects artists Urban planners even and game developers when we were trying to figure out what's the best use to deploy this technology and a game developer told us you know I design more buildings in a single game than an architecture office does in 10 years and I don't have training as an architect and and for us that was that was a bit of a moment where we realized look there is a huge industry here that is building out massive worlds you know kilometers by kilometers Miles by miles of of Worlds and they need they need assets they need buildings they need Landscapes they need props and this Tech technology that we've built is a very good fit for

that because our technology is less about one toone replication yeah we're not so focused that if you have an image of a tree and you want this exact same Tree in 3D that's not really where our Focus lies our Focus lies in looking at a collection of images and trying to understand what is the style you you know really a design thinking approach what is the style that one is trying to dispel from this and then can you generate cataloges of options that are that are stylistically consistent and can you do it in a way that you're really at the core trying to just augment the intuition of that Designer allow the level designer the technical artist the 3D modeler to look through more creative diverse options and then whatever you generate for them can they edit it can they integrate it into a pipeline can they take it further with their own intuition and touch and feel so that it becomes you know really their own asset and their own world in the end so maybe we could talk about that in terms of the pipeline so could you describe from

an artist that comes up with a 2d concept that's maybe drawn or just maybe even sketched out I I don't know the Fidelity of the input that's required in the 2D side to then generate the 3D but could you kind of Step by Step walk us through how this works what does the artist do then using your tool then how does it get into like a Unity or unreal and then when you want to modify those assets you have to export them out modify import it back in or however that works it'd be awesome to hear exactly how your system works yeah so the the first step is is you can sort of think about our technology is dealing with with two buckets so one would be the Aesthetics and the other would be the performance so on the Aesthetics side it's it's working with the reference inputs from the artist and this can be like you note it can be sketches it can also be photographs it can also be generated 2D generated imagery and then we feed this into the AI and the AI starts to try to learn what are those patterns and it starts generating models out that can be seen by the artist and

evaluated by the artist and the artist can start to give some feedback to the AI you know I like this model I don't like this model and you don't have to be explicit yeah in in explaining to the to the AI what you like or don't like about it but the AI can start to show results that are more consistent with what your preferences have have indicated then you get to a point where you start to consider what are the kind of output performance constraints that you need here so we would like as an art team for example or a Game Dev team to work with this tool directly within Unreal Engine and I think that's a that's a pretty that's a pretty good approach in general for us to see that that a lot of teams are now liking to stay within engine whether that's on real or Unity but to kind of continue to iterate and Design Within engine itself and so then you you have this as a plugin so you call the technology through through an API call but they actually generate the models in the engine itself and then you as an artist can now start to again work with this start to populate at a larger scale right so now maybe you're not looking at individual assets you're looking actually at sort of the entire world that's being generated you're giving

even indications of specific parts of the world that You' like regenerated so this area of the map looks pretty good but this area over here could improve yeah and when you improve something you do it through a variety of ways so one would just be self- selection another would be feeding new reference images back into the AI to suggest okay you know I I think the style here should be slightly different from the rest of the world and so in this area I would like to focus more on maybe futurism for example or or some new sketches that I've developed that that more in line with how I imagine the style looking here and then I think the the last part of this is that generate should be it should be as though someone manually modeled it it should be clean topology of the geometry it should be a clear semantic segmentation on the AI side but but it just means there should be a structure to that geometry you should be able to select an integrate the windows the base floors the grassy areas of the map the roads for example because if you have that

structure in the generated models it means that you can really go and either individually work on them so you can select specific Edge loops and everything or you can work on them systematically you can connect them into procedural pipelines and suggest that you would like all the windows to follow certain material settings for example yeah so just just for like a simplistic guy like me and you know just because I I kind of have a high level understanding how this works but maybe we could even just go back so maybe I missed it but how so when you start with the 2D what what is a 2d input that goes into your system again is is it a drawing that's scanned in is it how does that actually get into the AI it's a it's a collection so we we need multiple drawings not of the same object just multiple drawings or images in the same style that you want the world to be and these are these are uploaded into the AI and then the AI runs a Machine Vision a set of Machine Vision algorithms over it to try to

learn what it it believes is stylistically important about those images so let's say for example let's stick in the world of buildings let's say you want to create a City full of organic fluid forms and you give you give the AI a set of reference images sketches you've done of these types of buildings the AI will be looking to try and understand what are the patterns it can learn from that and so the patterns it would learn from a set of fluid and dynamic buildings will be very different than the set of patterns it will learn from let's say more boxy structures but what about specific parameters that you would want to establish for these models in in your game like let's say the the poly count the kind of shaders the optimization of those shaders and then do you also you know treat things like lighting like if there's like fog if there's animations or or is it just the 3D models at this point I'm I'm just trying to fully you know kind of

characterize yeah so so you're you're exactly moving from phase one which is that Aesthetics right what the what the AI is trying to learn about the style into phase two which is the performance so okay what are the poly counts that we need to be aware of Are there specific restrictions around everything from you mentioned to things like UV mappings right collider meshes things that need to be generated and right now we focus mostly just on the on the 3D assets themselves so the the world the the objects the props the nature assets the buildings things like fog and Atmospheric effects lighting of the scene is not is not something that we are generating but but we are generating everything that should fit into a pipeline where an environment artist would would get the results they expect let's say if you set up a nice scene with with global illumination the meshes that we generate are clean they have you know there are not holes in those meshes the topology the the way the mesh is structured is very orderly and therefore

you would expect all of the light Reflections all of the atmospherics that you've set up should work well with these assets right and can you give me a sense of like the timeline so let let's say if you were to compare an existing process in which let's say you're trying to establish a map like a level for a shooter game or an MMO space in an MMO game how long would it typically take in a traditional process where you have artists building the different models and then kind of placing them them in the world and kind of having to kind of establish that you know the the the 3D World in a traditional process versus where you've got some artists that have made your the collection of 2D sketches input it into your system it goes into Unity or unreal and then to then I I don't know how much hand tuning it requires from that point but can you give us a sense of like maybe the time and cost savings and then

how much today of is still is required where you're manually having to like fine-tune some of that yeah so we we recently did a postmortem with with a game that we worked on WE generated for them 20,000 unique buildings it was a VR game developer and at the same time they put a team of people on on generating some buildings of their own so the whole process including training the AI generating the results feedback loops and then ultimately the game ready assets was about 200xg faster so we did the 20,000 buildings in the time that their team was able to make a 100 and so I think there is substantial efficiency gains to be had here but I would say the other thing that I would like to highlight is there's kind of two sides of this coin so on the one hand understand that efficiency gains is is very powerful especially we work with some really big studios and that and that's that's very important but the other side of it is that you could make a game that's 200 times larger right you could make a map that's 200 times larger for example or you could make user

generated content be part of your game because you you'd be able to generate cataloges of unique models for those users to choose and so the two sides of it are one side you can have efficiency gains and you can and you can save time and cost you can you can go to market or develop that game more quickly but but I'm also really excited by the game developers that come to us and are interested in doing a game that would otherwise be impossible for them to do without access to the tech right and I actually I think the other just kind of thinking in terms of the way that I would potentially use it is maybe even for exploration right where let's say you've got a design team that's that's working with art trying to think about different art styles for a potential world and then if you could just make some 2D sketches and it's kind of there and you can kind of jump into the the game into the build and just kind of walk around a map or level based upon that concept and then you can kind of get a much more you know visceral feeling of how that would would look

like that's exactly that's exactly how we come into games you you've you've nailed it we come in at the early 3D concepting phase when you want to look through a huge diversity of styles you want to have an immersive experience while you're trying to understand whether the way you are thinking about the visual style and even some of the performance aspects of the game make sense and then your editing tools kind of Follow that that same logic so you could be walking around the world looking at part of the world and deciding I want to regenerate this part of the world I'm happy with this part but I would be interested to regenerate this other part maybe in a into a different style or in some sort of transition or interpolation between Styles okay and how much I don't know how much you can talk about pricing and E economics but how does that play out so if if you're 200 times faster how does that how does that translate into economics for a game Studio that may be looking to use your techn your your so at the moment you know we've been building this Tech up in

stealth mode for the last couple of years and and and really excited that we're now building in public that we are that we're are out there but I think you know right now we work with a lot of larger Studios that have like you know more substantial budgets and one of our goals really coming up is we'd like to start to open up this Tech to more small and mediumsized game developers we'd like to that's that's the idea behind we're calling it like our platform it's what we're getting ready to launch in the beginning of next year with some Alpha testers and to say okay you know this is the technology that's that's powering some of these larger AAA quality games but how can how can maybe you access it in a little bit less of a Enterprise scale a little bit less you know super bespoke customization but still you know benefit from the efficiency gains and and all of that and so we are we are still trying to figure out what is the best pricing model for that particular platform case I think what's important to us is just that it's value oriented right so that if somebody is able to really achieve aieve a greater diversity of assets or shorten

their game development time you know that we that we would share in the value upside in the value creation that we do and so that's that's what we're trying to think around and it will probably be some model like per generated design or or per seat is another alternative but I think that's that's the goal is just to make sure that it aligns with the value that we create okay and to the point where you you said that there are some bigger Studios using this your services now can you talk about so one exploration as as a as a specific application but is is that is that the main use case are they also using it to actually for like the actual production you know world and assets that that's going to go live or are they still you know for now based upon the current state of the technology still manually doing all those those those kind of assets yeah uh perfect so so some of them yes are moving this through so we typically comment at their early 3D concepting stage for a variety of

reasons one it's a great fit for the tech you can be very diverse in the worlds you create but the other is that if you're a game Studio after 3D concepting once you start going into production you're hesitant to start to introduce novelty in the process right you have now a very sort of structured schedule and timing and set of deliverables so even if something appears to offer benefits you'd rather wait till the next development cycle before you bring in the new tools so for us a stereotypical uh approach is we come in at the early 3D concepting get to early 3D Concepts then we go into the pre-production cycle with the game so what what's important for us is maintaining what in the industry is called a non-destructive workflow essentially that as you build further with our with our tools so with our technology you can always revert back to earlier points and see those designs see those changes reflected in updated results so an example would be we start in the early concepting we start to generate buildings for a city those buildings now in the pre-production phase start to be evaluated for various performance

criteria the polygon count the way the the maps are being applied some of the collider meshes Etc and then once you're in the production phase you realize oh it would be very useful to change the way for example the windows are being aesthetically represented because they are they can have a significant increase in performance and we've seen this through some of our early gameplay testing you can go back to an earlier part of your design process the AI knows through something called semantic segmentation but means basically a an underlying understanding of the parts the holes that it is generating what is a window for lack of a more technical sense and so if you want to make changes to the window that the AI are that the AI are generating those would be reflected then in your in your more final design state where you've added those levels of detail you always have the option as a designer because the mo the meshes that are being created are very clean and structured you always have the options to go and add your bespoke manual process on them but I think our goal is that if we can help

you get you know 80 90% of the way there especially for those hero assets that's usually game ready quality enough for the background assets and for the for the hero assets it depends on it depends on how bespoke and unique you want your your Aesthetics to to be got it and you know recently there was a pretty pretty famous interview with Elon Musk by the New York Times in which you know a lot of people have seen that but one of the things that Elon said is that he believes that the amount of time it will take for AI to be able to write a book as well as JK ring and things like that is going to be about three years where where like the AI can actually discover new physics and things like that and so when you're talking about you talked about where where you are in Phase One what's kind of next as far as the capabilities that you believe are that you got are going to be able to deliver and what what's the what's the end point

like how much can be done with AI eventually that game developers can leverage in terms of the 3D World and asset creation well I think there's there's there's two points to make so one is the the broader vision and some of the feature road map that we that we are building out but I think maybe just as a premise to that it would be nice to flag that I think at the end of the day as creative and exciting as AI is it is foundationally at least the the AI that we are building it is a tool to augment someone's creativity so the worlds that somebody creates with this are going to be only as creative as the individual behind the behind the tool behind the platform but what we'd like to do is is try to help them as much as possible there are a lot of aspects of a game developer workflow that are that are just not creative they they are fairly banale things like UV mappings or creating collider meshes and I think what what we want to do is help the AI improve at each state so right now for example it's very good at creating large

diverse virtual worlds of high quality but would be interesting as one of our next steps to add in certain level design mechanics there so there is even a thinking about the world not only as an asset or sorry as a as a set of interconnected meshes but as a as a driver of the game story line you know we had a project recently where we generate also Interiors of buildings when we when we generate the structures and we had a project that wanted to encourage close combat shooting yeah in in this game and so one of the things we realized is we could teach the AI that lines of sight is a is a good loss function to evaluate its its own generated designs before giving them to the client and what that meant was that every time the AI was generating uh a catalog of of options for the client to use and to work with they all encouraged close combat shooting and so that was that was something where you know there's kind of a a higher level Thinking Beyond just simply the the meshes themselves and so these are the things that we'd like to we'd like to add out and kind of help game developers

just work with a very creative almost design partner in a sense okay so how long do you think before it takes like your system or AI in general to be able to fully design levels or Maps just as well as a human designer well on a on a technical basis we are very close to that but but that's where I think that that's where I think the the creativity of the human designer can't be understated yeah it's it's not in our approach it's not an either or it's not either the AI has has designed this yeah it's it's that did you have somebody who's very creative using our technology to let's say more efficiently look through a lot of different options for this world and and iterate rapidly through various assets to make that world that looks that looks very exciting and and playable um yeah okay and actually I think I forgot to ask

more about your company itself so there there's you there's like I assume there's been some some funding announced how how big is your team and how big do you guys need to get to be we're we're 10 people at the moment we are we are heavy on the technical side as you can imagine especially the the last few years we've been building Tech and stealth so a lot of machine learning engineers and Technical artists with a with a big emphasis on the technical side of of that title we we have announced now funding we just announced million in funding and we are looking to grow the team we expect over the next few years we'll probably grow this up to closer to 25 people as we as we expand and as especially as we start to open up this Tech not just to some of the larger names in the industry but also to some of the small and medium siiz game developers because I think we see that platform as a as a way to start to scale up effectively right and where are you based Ben are you are you guys remote or are you guys on site how you guys operating based in based in Vienna Austria and uh and a we have a hub here so about half of the

team is located here and the other half are remote okay and what has been the biggest challenge for you in terms of getting your company off the ground and is it is it customer attraction is it awareness or is it is it just been more of a you know major technical challenges that you're you're facing until now it will have been technical challenges I mean we have done we have done a lot of work to get where we are now with our with our technology I think the biggest challenge going forward is less of a technology Challenge and more of you know how do we take what what we believe is a very strong technology that has shown a value proposition within the industry and how do we turn that into project and execution modes so going forward you know how are we not as reliant on Just Simply Having the best tech for pred degenerative AI but rather we have the right approach the right solutions for the industry the right functionalities the right Integrations the right part Partners to make that to make that a much more successful business going

forward right and in terms of like if you were to look at the competitive landscape I you know when it comes to other areas in AI there's usually one or two players are are you aware of any other companies in your space do you expect more to be jumping into you know obviously if if you feel like it's it's a very attractive Market there may be others that are trying to go into your space as well what what does the competitive landscape look like yeah you know certainly I mean I think that the 3D generative AI landscape over the last especially 12 months has has blown up and there there is everybody everything from startups to uh to large foundational models being created on the research side of things so for sure the the the space is is both noisy but also rapidly evolving I think we benefit from the fact that we started before the generative AI hype and so what we have seen as a very clear problem in the gaming industry which is how do you

generate stylistically consistent assets at scale and diversity which fit into pipelines which artists can use and and sort of further edit and build on top of I think that's something that's pretty unique about us at this moment um I think when we look at some of the other 3D generative AI especially some of the research which is incredibly interesting so there are things like neural Radiance field or Nerf models or 3D diffusion models something that's taken a similar approach to the 2D diffusion models that have done so success that have been so successful and trying to replicate that in the 3D space what we think is is quite unique to us is our editability and our ability to integrate into existing pipelines because you know one of the things that that is interesting about us is the Champions usually when we when we sell into a customer the Champions typically tend to be the artists and the game devs actually because they see it as a tool that they can work with and I think that one of the risks of some of the state-of-the-art research as it applies to the gaming industry is that you get this approach of you either take it or leave it yeah because the tools the the the results that are generated are not

as easy to edit and refine and so for us the the radar is always looking is are there editable 3D generative AI models coming out at the moment I think we're in a good position okay and you kind of touched upon as as you were kind of answering one of the questions about you know having having people work with the technology and not not having the AI kind of take over in terms of level design and things like that but maybe you were as you were answering that question maybe you are also trying to address this issue that seems to be coming up where there are some people who have like this fear of of AI right whether it's you know because they feel that too many AI services are using copyrighted materials or that there may be a danger of replacing artists or things like that so do do you want to address that specifically like how do you how should people be thinking about a Ai and services such such as yours in that context yeah of course I can

understand I can understand the thinking I I'm I'm much more of a of an optimist in this perspective I think that AI is is a tool to help somebody be more creative yeah even as we've built under that ethos you know the first funding we got from the EU actually was specifically around building trustor the AI systems which means there has there has been specific implementations in the tech that we've built around human agency and oversight of that process transparency including into where data was was collected and sourced diversity of of those data sources but the other thing I would say is that you know I'm I'm an optimist and I think that if you would Embrace this technology it's not so much about replacing what someone is doing but allowing you to make so much more interesting worlds and games and and you know you can look at more Alternatives than than was previously imaginable you're able to even make games that probably were previously imaginable one

of the one of the customers we work with right now they they are Consortium 9 and they're releasing a game called nor and they want that the map is being destroyed as as people play it and then From the Ashes of this of this destruction a new world emerges every you know every few months or even weeks or possibly days and that type of of gameplay would would really not be possible without without a techn techology like ours and I think that's really exciting because that's something new and Innovative that then can come to the then can come to the industry right and maybe people would feel more comfortable if they could actually see more of exactly how your service works like how open are you I don't know how much is visible to just people like the audience here if they wanted to see exactly how your service works but how much can they find out through the website or publicly demonstrable yeah we we are still pretty secretive

but we are going to launch the Alpha Testing in the in the beginning of next year and that's really going to be our first opening up of the tech to to people that are not customers that haven't seen it previously and I think that'll be a good a good way for people to interact with it understand that exactly as you point out this isn't something to be worried about this is something that could make you more creative or could generate kind of large cataloges of interesting usable assets at at scale okay well I'll tell you what Ben if you are open to it you know our my studio is working on a FPS shooter game for mobile if you are open to us kind of recording us using your Tech I'd be happy to do that and and publish it but you know up up to you when whenever you're ready yeah I mean sounds sounds great I I would say I would say let's let's go for it you know the the issue with with obviously all of our stuff is that we we are almost always under NDA because we are working with proprietary data right the the IP of the of the customers that's what trains an AI model and the AI model they get back is their own bespoke proprietary AI generator so

if if you're up for it we can certainly do a part's right all right so for for our audience Hang on we're gonna we're gonna come come at you with a with a demonstration with with with Ben here okay well I guess in closing do you have one last message that you know in terms of a question I haven't asked you or things that you believe the audience should know about your company or this the the service that you guys are deler are are offering I you know I think we've I think we've covered the Gambit I like that we had a chance to talk about how does the technology work and why is it not probably something to feel threatened by especially if you're if you're an artist I think what what excites us the most about the future is as we start to work with more developers right the the small and mediumsized guys as we start to launch this platform to see where the creative use cases coming out somebody told me a story recently that the first the first television shows do you know what the first television shows were I have no idea

they they were filming people reading the radio and and and the the the exciting thing is that that this is a new technology and nobody really knows quite how to to use it yet but I think we're optimistic that when you give creative people this new technology you're going to end up with incredibly interesting creative anesthetic games incredibly interesting creative aesthetic worlds and that's what that's what really we're really excited by gotcha oh actually I do have one last question sorry there's been I've been thinking a lot about a new game just completely enabled by AI so for example like one one example I I give is like Dungeons and Dragons where the AI is the dungeon master but when you if you were to think of a new genre of game that is completely where AI is completely core and critical to that game what what would you guess is is some kind of new game that's enabled by AI I would guess that it is it is Maps where it is

constantly changing and evolving based off of how a user is moving and interacting with that world okay well then I guess they probably have to use your technology to make that game all right cool well Ben thank you so much for your time and for our audience we will catch you next time thanks Ben it was a pleasure thank you very much [Music] [Music]