AI & Game Development: The Current State of the Art | BITKRAFT Summit 2024

all right thank you everybody for joining from bit craft for having me I'm Brendan Mulligan I'm the founder and CEO of another bitra portfolio company Lial but I'm actually just here to moderate today for some much more qualified and interesting individuals I'll introduce them one at a time you guys can feel free to come up and take a seat we've got Kieran bot who's the technical director for foundation models at ROBLOX which he joined after they acquired his startup Loom AI we've got Wes Kerr Riot games head of AI and we've got Jorge Deval the CEO and co-founder of latent [Applause] Technologies all right so the topic for today is about AI generally but um as you guys can imagine there are some many

different applications we could discuss so to start us off I'd like to just have each of you say from the perspectives of yourselves and your organizations what are kind of the big opportunities that are most exciting for you specifically start oh I get to go first um so at Riot games we build a lot of session based multiplayer games that have deep engagement High skill levels um and one of our biggest drivers that we focus on is retention so how do we get you into the game how do we keep you in the game as it goes um and there we've been Focus focusing a lot of time on really thinking about an AI that can play alongside you uh and who understands the game way better than you do so that they can gently guide you up through that ladder um and help you uh understand the game deeply and help you get better at every stage of the game um and then a lot of the other efforts that we have are really sort of around uh efficiency plays and sustaining plays and sort of helping us create more content and more experiences for our players that will Delight them

okay I can go uh well uh at Laton we are building a generative physics animation engine we want to make animation physical and emergence and I think the implications of this go a bit beyond uh mere efficiency increase um things like this suggest new paradigms and that's exactly what excites me uh Paradigm shifts I like when AI can enable things that were not previously possible before and can give Quantum lips in the way that video games are built and experienced hey um so uh Roblox is a is a gaming platform more than a game itself uh so many of the efforts that we have going on in AI uh fall under two buckets uh predominently AI for safety and AI for creativity so um to keep the platform safe at scale you know when you have text chat and voice chat and other

modalities with lots of younger players on the platform we use AI uh extensively uh to to Monitor and and keep our uh you know people within the terms of Services of the platform that's on the safety side and on the creativity side we have we we've rolled out a number of tools for uh creation of 3D content because you know Roblox is a ugc platform uh built by creators who are smaller teams two or three people teams typically and with with fairly limited 3D experience so a lot of our Focus has been to uh help these creators produce much higher quality content while still being performant on the platform and there's a lot of Animation in other areas as well yeah that's fantastic it's it's interesting to hear there's kind of a spectrum from incremental and sometimes defensive Technologies to the more kind of uh large huge bets especially from the startups willing to take these risks

I'm curious um specifically just for Wes and Karen to start your organizations essentially relative to most of us startups it feels like you have infinite resources um the I know it doesn't feel that way always internally but compared to Jorge and me making our hiring plan that's how it sometimes feels from the outside I'm curious from the perspective of the big company though what what are the unique barriers that you have what is what makes it challenging to achieve these things from your organizations that are clearly already bought in and have these resources I want you to go first because I want to be jealous of your ability to actually ship to players um so uh yeah in infinite resources is a very nice situation for everybody but um uh I think the the challenges of uh building anything and shipping anything at roblock scale uh is what makes this whole uh effort worthwhile and also extremely challenging um so you know because uh

Roblox runs on all kinds of Legacy devices uh and also has a very broad spectrum of users um we have to think primarily about edge cases and thinking about edge cases where you know maybe a very old device or you know like a uh a language which which uh which is not currently supported those those are really interesting juicy challenges for AI so uh even though we can build a feature quickly just deploying it at scale suddenly brings in a lot of challenges so you have to pull in huge amount of computational resources um and then the the sort of the team that can pull uh these features off so typically that balloons into a a larger group working together and you know how complex that can be yeah I think ours are similar but also slightly different so one of the things I've been watching With Envy of

Roblox over the past couple years is their ability to actually take an idea and ship it to players and let it impact them where so whether it be code completion uh in the interface or some of the other materials work that you all have done has been great um but that ability to iterate with the player and understand what's working and what's not it is can tend to be much harder for Riot when uh when it affects can affect our brand and our games and negatively if if perceived poorly uh so we spend a lot more time focusing on ensuring we've got the right core the right game the right skins Concepts all of that before we ever unleash it to players and so in an era where AI is rapidly evolving and rapidly changing what it is it means I get to ship less to players that is frankly not good but learning material for us um and that means we get less data from it we we sort of can iterate more slowly uh than a startup would in this case so even though we have the

infinite resources we can also lose too much brand recognition or tarnish our brand if we ship the bad stuff that's great thank you for sharing and yeah that points me to Jorge which is obviously we have a different set of constraints as startups but there does seem like in some of their answers There's an opportunity for startups with things that we can do that they can't do you want to talk about that a little bit um sure yeah uh we don't have infinite resources and that makes you think very very hard about what you want to do so in a startup everything changed you need to have a a spear and yeah like put it very tightly where you want uh what you what you want to transform right being very Str strong-headed in the vision and flexible in the execution so we can take more risks especially when you want to propose a new paradigm for something uh but especially for a product that is very research uh based is challenging to be focused in product right and a lot of startups that I'm seeing around uh get

this part a bit wrong they could be very very research based they can have very interesting results but being very divorced from the actual game development workflow that I think is a challenging for startups in the industry that we are trying to to get right um and yeah and of course well to your question about the challenges of big startups that big companies at startups could face uh the answer is just too broad there could be a company focused in a very small part uh trying to help Kieran and West for example with the edge cases and there could be others that try to make something a bit more broad uh so yeah the answer would be too long I guess that's fair no it's great I think you you raised another thing that I'm interested in which is this idea that we were actually just talking about in the back which is in AI which is such a research Focused Field there's so much you could explore there's so many researchers who want to invent the next quick thing but then productionizing that research and putting it into something that a company would actually

want to use is a major challenge at all sizes I guess for any of you would you tell me more about that I know that's something you've all faced we spend a lot of time thinking about what's differentiated for Riot uh versus um what the the industry will sell settle for us um this goes back to a conversation we were fortunate to come up and chat with uh open AI way back a couple years ago and Sam basically said look figure out what is going to make RI better focus on that leave all the rest to vendors to sort of solve for you and so I've been watching very closely image generation 3D Generation Um animation work um and because those are opportunities for us to partner and buy with what they have while quickly trying to figure out what are the things Riot needs to do to make our games more sticky more retention what where are we not going to let a vendor play as part of that core game experience and make sure that we're leveraging AI in that way um in order to help us adapt for the new the new

changes yeah K I'm interested your take on that as well the research versus product yeah I think uh um I I I I think I agree with uh the the Insight here um many of the big companies will want to protect and innovate in an area that they think is their secret sauce so if if if your secret sauce is the game engine yeah you're going to you're going to put all your energy innovating and you you're most likely not going to acquire uh a startup unless there is a very strategic fit and a fit to the road map you're building right so it's it's probably very obvious but but I think that's that's mostly how Acquisitions happen um but anything that is not core is right for acquisition and so so these two systems can coexist uh and you know and also Partnerships um but if you are developing something that that falls

straight within the road map of a big company and they are actively trying to innovate there as a startup then that's a really exciting space for you to be but the place where things go wrong as you you pointed out is is if the research teams and startups are not paying close attention to the end game you know the the actual product uh then suddenly it gets valued only as a research uh um you know uh institution which which may not carry as much weight in in some of the uh the gaming circles I want to ask you more about that concept of kind of the core road map especially because if the thinking of it is well here's the thing that we're already very good at so let's make sure we leverage that that can push you into a little bit of an innovator dilemma though right I mean I think if you're West and you're saying hey well we make these session based highly retentive games that rely on other people being present all the time so let's just do that but with AI you know how do you consider the opportunity that might be

adjacent to that that doesn't leverage your strengths um and are you worried about those things passing you by yeah absolutely yeah I think I think that's that's really what uh what I I mean the leaders at roadblox keep worrying about essentially don't don't fall into a trap and and that's one of the reasons why we started this Foundation AI team at at ROBLOX um where we are trying to build a the next generation of a 3D Foundation model similar to the the text models uh out there but but but with really a large quantity of 3D data that we may we may have on the platform so so there's always that fear that uh you're not looking at the problem correctly or you know there's this other startup on the side that's going to completely surprise you that's unlikely uh but does happen and that's that's why we we keep an keep an eye out on all the interesting startups um and when we do see tra signs of that then we we definitely go for it right so um but say it's it's a balance

because if you think of the advantages that some of these incumbents have like for example I talked about this voice safety uh system that we built I mean you could not have built it without the kinds of data sets that are on the platform so sometimes that's what is the mode that prevents the the startups from from doing this and the other thing with in AI is there's so much open source Innovation that's going on there so the architectures are not really the the big deal right now so so startups looking at this space should ideally look at saying hey well if I don't have the data mode if I don't have the architecture mode what is it that I'm bringing to the table that's a great framing thank you Wes you want to answer a little more on that yeah I was trying to think how to frame my answer I like kieran's answer a lot um the the bit that we add on that we've been racing towards is we recognize that we just are unlikely to keep out pumping pumping out session based multiplayer games that resonate for years

um but we need to figure out where that puck is moving in the next six eight years as we work on some more complex games and so we're spending an a very large amount of time racing to the bottom to figure out what the real Innovation and what what it will be what that disruption will look like um I've talked to many startups and and oftentimes it's sort of a surface level disruption so um I've seen many demos not calling out any names or or plac where it's you sit down and you chat with an avatar in in a game um but fundamentally as a player I still just want to go play the game I didn't log in to play for two hours talking at a campfire um and and helping startups sort of recognize that the game play still has to be amazing it still has to draw me in it still has to keep me there um and AI has to figure out how it's going to weave through that um and so I've chatted with a lot of them we've come up with lots of different ideas just trying to figure out what that new game play enablement will be from AI um

and sort of racing towards that to sort of make sure that we are recognizing what's coming I love that great answer thank you it's interesting is both of you are pointing to something that I've thought about a lot which I think is often times missed when we see a lot of the headlines coming out of open Ai and the things that it feels like the technical stuff is what's on the frontier and that's what matters but it seems like almost universally in our answers here what we're hearing is the technical thing is is not what's actually the most interesting like from a product perspective that's not where most of us should be playing we don't need to be thinking about architecture in that same way it's really a design challenge it's coming up with like what is a player problem that we can solve talk to me a little bit about how that shapes kind of how you build your team because if it's just a purely research and Technical problem that's one type of team if it's a product problem and a player problem that's a very different type of team I I I add one quick thing there I I would I wouldn't characterize it as technical only or product only I would say what is the uh what is your differentiator it

could be a technical differentiator where you are able to serve these models 10x cheaper than than what what a open a could or or some other thing some other player could so so that is also a huge deal so sometimes these infrastructure Innovations can be what allows you to deploy a product uh and and yes of course ultimately there's a product so but sorry please oh um what was the question yeah just you know you could build a team of just researchers doing all the craziest technical and and maybe compete with Sam and those guys um but at some point you have a got to connected to product how does that change the way you construct your teams so at Riot the research team is is actually quite small uh by by comparison um we have six people working on game AI six people working on generative AI not a huge research organization which means we have to be very focused on what we go after uh in order to make sure it's delivering the most value to do that we partner with our internal uh it's called

Riot it's RDS I forget what the acronym stands for we have a bunch of them I'm sure everyone does as well um the but this is our our basically our backlog our R&D team that's building the next generation of games and so we partner very closely with them to see what they're thinking about from a design side and their product leaders are telling us what they view the vision of gaming over the next 20 years to be and then it's like okay what is the research challenges from that product that we need to go after and solve now so that when that comes about we're in the right place to to go after it and that product lead has been super valuable for us it's been a great partner to come in and say yeah I think it's going to be these and starts calling some shots and that really guides our research although it makes it a lot harder for us to talk about it um and so we have to find other ways to keep our researchers happy in publishing yeah that that leads to another great point was how do you hire in this market um there's a lot of money floating around for AI Engineers I'm

sure that this is going to be especially interesting for you like with a different set of resources what does hiring look like and how do you differentiate yourself as an employer you in a startup you need to take a lot of care with the people that you hire and typically in the first stages you go and personally Source the people or talk to them or hire them and a lot of things need to Vive you cannot allow to make a lot of mistakes then the next question is how do you build your team your initial question right uh and this needs to reflect the vision and mentality of the startup uh in in our case half the compan is dedicated to coure technology and the other half is dedicated to prototyping and integration and this is going to be kept this way because it's really important that those two team are interacting together to build something that is actually productive and playable that's that's our goal right as you scale and uh yeah uh AI is expensive and a startup can have an a

startup has an asset that big companies don't have that much uh fundamentally and structurally which is passion and the ability to work in something that is risky I think when uh couple guys get together and say hey imagine what this could be right let's build it and then they go and take a huge risk if you bet a a huge company on a bet like that then that could be very risky by definition right so that passion is also an asset for the employees uh options in the company and imagining what it could be are other assets uh that bigger companies don't have uh and yeah and the ability to work on a startup like a lot of people that we hire in the end they come from Big corporations I come from Big corporations right and uh that could be something bad or something good and a lot of people they want to be in a dynamic environment where they could be versatile as well and say we tell them hey you're going to need to make a lot of stuff like need to ver the talent and

they a lot of people they love it because they feel like they're doing things in big corporations sometimes and I'm not generalizing uh the passion is a bit more spread out you wonder why you're doing why you're doing there are a lot of levels of hierarchy and it takes a lot to make something that actually is real sometimes it feels like negotiating with your own company with different divisions right uh so those all of those are assets that startup have and with the smaller team comes a greater agility as well it's harder to turn ship on a big company uh even if it's made with smaller teams as as I was in my past is it's still hard yeah met makes per sense what about us yeah I I want to challenge it just so in case people are looking to hire they come work for me um instead um so it is I I think hiring is really hard for us as well given that uh we can't compete with the salaries of open AI and so for top researchers just looking for

cash the cash play is to go to the big research Labs um like Jorge we offer an opportunity to come in and start to build and prototype your own research so our team is like half your company where we're building prototypes and iterating there and that similar draw I agree that that is huge for a scientist uh to to sort of lure them away um and then the other side that we've spent a lot more time looking at is how do we level up all the riers internally um and so originally when we first started talking about this it was like all right who is top talent in the world that we'd go after and Chase and try to buy uh with our our Deep Pockets um it and we slowly flipped it on its head and it said we got 5,000 Riders who are really great people how do we level up their capabilities in AI so that they're all contributing and firing uh on all cylinders there and so we spent a lot of time focusing on that in the last year um and then we're getting ready to start r amping up the other side of hiring as

well again Sharon yeah I I can give you a perspective uh from my days at a startup I was a founder of Lum AI uh that was in 2016 AI was just starting uh we were building real time AI base tracking on mobile devices so um and then subsequently at ROBLOX so I in my mind um the reason why somebody goes to a startup um is to work with a small group of really passionate people on a specific problem right so if the problem is really interesting to you um you get drawn to the startup and especially if the startup already has a few people who you admire uh or or you know about then I think it it's a draw and and I think that's a challenge for bigger companies because it's the company has a much bigger profile than the the founders of a startup or or a small team of startups so so I think there is a a natural draw for people to go to startups and do

risky things and startups also give you a lot of flexibility in terms of you know you can work from anywhere you know blah blah blah and there's a strong sense of camaraderie and team which all comes with a small team right so um in bigger companies um that's obviously a challenge right I mean I think every group is like a big startup so it's you you you lose a little bit of that but companies tend to do a a a good job of trying to preserve that culture but the bigger difference is especially with AI the bigger companies sometimes have the Deep Pockets to take large number of risks so they are not just betting on one idea panning out maybe they're betting on four ideas panning out and all of the four can and can fail and you're not on the street um so that is the advantage of of a big company but uh and some people like that you know that not everyone wants to uh work in a company which is is very very exciting and passionate for them but then they

know that it may not turn out and they'll have to go look for a job so um I'm not painting the big companies in a bad way I'm just saying that you know the the tradeoffs are are that you get to work on bigger problems bigger impact potentially strong teams but stable whereas the startup is much more exciting things can fall apart but that that's the fun absolutely I I think that's the thing Adventure is a big sell Point uh I started up with an adventure and I I think I told someone once like yeah do you want to do you want to go and work to uh to meta or do you want to take the ring to mortar because sometimes it feels like like that a lot of stuff is going to happen and there is also something very very special about building something almost from scratch or building features that you need because you have determined that you need them so you need to build them it's not just an add-on it's something like H guys we need to do this we need to do this fast we need to do like this that is super exciting and a lot of people are looking for excitement because maybe they worked

uh I don't know at Google for like 10 years and maybe the money is covered uh in that aspect and they want a little bit of Adventure and excitement and to build something and put a big part of themselves into a product that they can love if you could uh change the minds of big companies about any topic that you could work with what would you want them to be thinking differently about horay and I'm going to ask the same question uh in in Reverse of like what do you wish startups were thinking about better or knowing better because I know they come and Pitch you guys as well all the time so but um in a way I understand their concerns so it's not that I wish to change them I wish to work together to build something valuable I wouldn't say hey I wish that I wish that you would buy your stuff of course I would wish that yeah but uh if they have concerns is also for a reason that that makes a lot of sense so the important thing is to build an actual relationship

to say hey what do you guys want to do in the future and and it's a process of just you know thinking about the future and having conversations about visions that sometimes can turn into a project that that is very good I I well in general for the industry and not only for big companies I wish that for example in gaming we would stop obsessing so much about making the games of today a little bit better and start thinking a bit more on about how they can look tomorrow right and that kind of mindset I've had a lot of conversations that that was some level uh as well and that's my wish and that's what I want to prove as well uh as a startup you cannot change it magically you need to to actually prove it as well that's great I love it Wes what do you wish startups did differently or knew thought about differently when they came to talk to you I think I mentioned a little bit about it but I love chatting with startups I I take every call that I can

um we talk about product we talk about what they're building um I I think where I struggle the most often is getting them to think about it from the not only what the product is they're building but also what's the gameplay experience for the player um and the player and for Riot is the most important thing um we build games for those players and we think and so it's sort of ingrained in our DNA to start thinking about okay well what what is that player experience that you're driving and why is that better than what we had before um and so when they come I often it will take a sometimes with some vendors it'll take a couple hours to get them over that hump of of not just selling their product but like what new thing can we unlock uh with this capability for players that is going to excite and Delight them and keep them coming back for more yeah and from my perspective uh for startups go take the investor money try something super crazy um and but but keep an edge so be very clear on what

your Edge is and see if there's a product Market fit with a big company I'm curious uh as we're coming towards the end of maybe a fun question for you guys which is um Wes if you had to leave Riot to go work at a startup what opportunities are most exciting to you in the world right now that you would want to go after it's a loaded question because Brendon and I almost did a startup together W Wes was going to be my co-founder and I'm not I'm not in any way hurt by that still um uh so I think for me it still would be a gaming startup uh that would leverage small teams to build amazing experiences very quickly um iterate with their players um in order to understand what they need learn from that in order to race towards that that hockey Puck that we know is moving over there um but with

a very concrete idea of like why this product should be valuable um and why it's a missed opportunity that everyone in here hasn't already tackled makes sense Karen do you have an answer to that question um I think um 3D is unexplored in the in in AI uh compared to the other modalities like text and and images uh so I would say that a a any startup or or a big company that cracks that puzzle and makes it really useful is going to be to to usher in a new wave of content creation and and 3D in general go a little deeper on that for me what what makes 3D useful what are the constraints that are not good enough today why is that an area worth exploring I think it's it's really fundamentally very hard to to you know create in 3D right I mean I think the tools are not are are complex it requires a lot of training uh you can go to Mid journey and create a beautiful image I don't think you can do that in

3D so I'm just painting that as a as a picture of of one of the things that you can do if you had a a 3D Foundation model um and and it's not clear how to build it so so I think you know a combination of lack of data lack of understanding of how to actually go about it correctly the research and the engineering challenges and ultimately making it useful for for creators uh to to to build produ s and and make money is is what makes it exciting that sounds great and Jorge to flip it for you um if you could join any big company preferably with a huge amount of equity being sold to make it happen but if you're going to join any big company uh who would you want to work with huh W well I mean there are a lot of companies that I think it wouldn't be a company that is just trying to keep its place uh that just pays you a lot of money to be there uh it should be a company that

strives to something uh and something bigger and that also the structure in which they work allow for a bit of Freedom uh to give I don't know I mean not not to um give a lot of compliments I think Roblox for example is one of the most exciting companies out here damn it I'm I'm a no I think I'm I'm sure Kieran knows their Corp Dev team we can uh we can talk let me explain like uh it's it's well now I'm going to give more compliments it's a problem that is differentiated they they allowed uh kids to to make games in ways which are easier and the problem is very interesting how do you make games so easy to create that a kid can do that that's a vision worth pursuing and I think in the next five to 10 years I wouldn't be surprised if an indie team created a game like GTA I think that the cost of making a AAA game is going to go down from 150 million on average to 200 okay something I think that's going to happen and you know enabling that kind

of transformation is something very exciting it's something that we're trying to do at latent because animation and interactivity are completely fundamental for that and you cannot do that without transforming them but I don't know I think Roblox is also doing that I'm also a big fan of League of Legends I know they're Corp guy too we can make some and I like the research that you guys are doing and social experiences which are very Dynamic and produce a lot of uh yeah Community I think that's very nice that's wonderful gentlemen thank you so much for your time I'll make a final note that uh Laton hor haaste company is doing a demo today at 4 p.m. so please check that out and again tomorrow at 1:45 p.m. so you can see it in action all right thanks everybody