A New Way to Think About AI in Games - Wayne Lee, CEO & Founder, Bloxmith

so I'd like to welcome to the stage Wayne Lee from blacksmith to talk about a new way to think about AI in games uh wait come on up so um before we begin I'd like to do a quick survey how many of you here are in game development have done game development okay roughly half and how many of you have not done game development but have an idea for games like you've been playing games for a while you're like I got some ideas I can do games all right excellent well what I'm about to share with you and to kind of share on the stage is actually a lot of interesting ideas that will help both of you both people who actually have been in gaming for a long time like

myself or people who actually have never been in gaming and has aspirations to actually create some really cool games by themselves so with that said I'm gonna come in quickly like I said I've been in gaming for a long long time I started many many years ago almost 16 years ago with blizzard and I was the managing director with blizzard and so with them I developed as well as launched multiple games we have launch World of Warcraft we launched our Stone we launched Warcraft Starcraft Diablo and I would say and all my years of game development and game publishing I've never seen a bigger change than what we're about to see through AI that's how big this change is going to be and many of you may be thinking well yeah I

think it's going to be big but how big is it really going to be I'm about to share with you something which may open your eyes up to what is possible right so sorry I seem to uh I think so I think we need a AI Clicker how about let's do this I'll do a hand gesture you move to the next slide does that work the remote AI next slide okay excellent all right so most of you that have been in game development you're aware of the typical seven stages of development right I summarize into five if you were to think about it

which stages do you think will be most impacted or which stages would be most impacted by the AI Revolution what do you guys think oh yeah I think for sure all but stages two three and four based on our our experiences are the biggest ones that are going to be directly impacted and so um planning which is actually interesting enough it will eventually be impacted but the that's the part that's least impacted because so much of the planning really is I would say the intuitive as well as the creative side and that's the part that current LOM as well as current AI are not able to do very well so that's the part that maybe someday I think probably in 10 years will actually be caught up with AI

but for now that's the part that you know as you it's in your head exactly how do you think the game is going to flow exactly how do you think the game is going to be fun that's mostly in your head that planning that's your head but everything else in the stage two three and four will be impacted by AI and stage five and through seven I would say this part will also be impacted by AI but not to the same extent as I will be sharing with you so this is a very dense slide this is a very dense light I'll be sharing different topics on this slide a little bit later as you can see it basically covers almost every aspect of gaming what you'll see is the question is what can AI do in video game development and I think most of you are probably thinking Hmm well yeah what can I do I think the accurate question we should be asking is what can't it do what can it not do and

the answer is almost nothing it can do almost everything from storytelling to music to video to concept art to 3D modeling to animation to UI ux coding customer interaction I can do almost all of it the part that actually has not been done I'll be sharing a little bit later is actually Esports right and so I'm going to save the storytelling as well as concept art for a little bit later I'm just going to talk quickly about 3D modeling animation effects UI ux because we don't have enough time to go over each one of these in detail but if you guys are interested I can definitely share with you our experiences in in using AI for each one of these components all right so 3D modeling 3D modeling if

we think about 3D modeling most of you are thinking Hmm okay well that's the important part of games that actually do use 3D model and what's interesting is AI is able to kind of solve that problem in two ways one you can actually simply do text to video text to model so by simply saying well I want to create a character with red hair that looks like she's from the future and has a curvaceous body right the game the tools today can actually easily create these 3D models that's one type another type that we've seen actually that's actually pretty common is if you have a 2d picture for example if your concept artist already drew a 2d picture what they're able to do is actually convert that 2D picture into a 3D model so imagine how much time you can save like I'll just be honest in the past we

actually have a team currently about 30 people doing development and of the 30 half of them are artists and many of them are actually doing this conversion and that can also all be saved the moment we actually start switching to AI and then if we think about animation as well as UI ux UI ux is again actually interesting enough there are two types one type that you'll see here the wizard is actually more of the type where you actually describe okay I want to create a UI for something like this this kind of game and a Google template template base so recommend exactly how the UI will actually flow so that's one type and then the other type that we're seeing actually that's potentially better for actually very seasoned UI artist is Galileo this is in our system so what they do is actually they kind of as you're creating

UI they give you recommendations to give you tips they remind you oh maybe this is something which you need to kind of do so this is actually a little bit more of a customized approach so these two tools are both very handy depending on how experienced you are coding actually The Cutting Edge of coding just came out yesterday so um I just saw the latest version of of frvr forever that is able to do game development by simply speaking to it by simply telling them I want to create a game of this type and I want to customize it this way it will create a prototype based on your description and after the the Prototype is created then it can you can say oh you know what hey I don't I don't like how this looks by simply typing in how you want to change it it's able to change the game

on the Fly and then you can actually then accept say oh well you know what I want when the bullet shoots this target It Takes Two Shots to kill it they can do that too so these kind of customization are actually now possible and it can it can even customize it okay well I want to control exactly how the the uh the motion should be controlled to why I want to use joystick or do I use Touch right so they can actually customize the code based on your requirement they can even customize it you know how many players do you want to say mostly play simultaneously play this game do you want a leaderboard right by simply saying yes I do no I don't they can actually customize and create a game that actually meets your expectation so this potentially will help many players and many people who actually have interesting ideas for games to create games without writing a single

line code and of course there's also traditional chat gbt which actually they'll create Snippets of code which then you can import into Visual Basic uh edit the visual Studios which then by assembling the codes together you can actually pull together a game but what I just described actually requires no knowledge of programming you can actually kind of create a game without it and then of course customer interaction there potentially are different types of customer interaction and I would say based on what I heard on stage Loop earlier actually was sharing about um how do we manage Community how do we communicate with Community interesting thing is we've done with our game pretty much all formats like we do Twitter we do Facebook we do um Discord um and oftentimes we're wondering how can we actually manage that more effectively and AI can do that now

actually in fact the tool that Vista is actually I have at the bottom is actually great at doing this and the buffer is another tool which potentially helps you to create content without you having somebody who's actually really dedicated to creating these content yourself but there is actually a black hole a black hole meaning this is the part that has not been addressed effectively through Ai and so if you're looking for an idea to start some kind of AI company in in gaming this potentially can be an interesting opportunity an AI that's actually focused on Esports and I'll talk about what can you do actually in Esports a little bit later so this is a simple format of Storytelling so many of you probably have seen whether it's bar whether it's um chat gbt actually the latest Rule now I would say

um that we've seen that's actually pretty pretty decent um I'll share with you guys after because we're running a little bit late I can tell um and there's also music music creation and so Music Creation there's two two different types one type is I will call it the reference based so what it does is actually give you different music you can choose from and you say hey I like the way this sounds I want to customize it this way and what it does is actually customize your specification based on this reference music that you have selected another approach as I will call it description based where you simply describe the parameters you want to say you know this is more like typically how we've done it in the past with Outsourcing so at blizzard as well as Riot we have big enough team that we have very large group of people that's focused on creating music but most game

students don't so we typically Outsource and in those cases you kind of specify I'm looking for this kind of mood it looks kind of um type of music and this kind of instrument this kind of paste and this this duration right so this is typically how it's done and so except the difference is you don't actually have to talk to outsourced vendor AI can do this for you at almost no cost and so some examples will be sound draw and soundful for the description based and the reference music based is actually like Iva and pier and fear recently have been acquired by uh Shutterfly oh actually actually I do want to kind of share one example of how easy it is to create music let's play this one [Music]

I created this music in 10 seconds by simply saying okay I'm looking for a little bit more upbeat cheerful music that's actually sounding like they're gonna go on adventure something heroic and I used to you know have to kind of explain this to Outsource vendor for a while and they may or not get exactly what I'm looking for all right and then video generation there are typically two types too right there's one type which oftentimes people create for social media so you would use this to kind of explain different things or for PR so those are I would call it tools such as Pick 3 as well as Synthesia those are actually pretty decent for those kind of purposes but there's the other type which actually I find even more useful especially as you create games they're going to be bound to be

times where you want to create trailers and typically in the past trailers are pretty expensive to create I remember like with our own game I think we spent about a little bit under sixty thousand dollars to create a trailer but you know and back at blizzard as well as right we spend Millions creating trailers if not tens of millions to create trailers so now actually they can create trailers by some point kind of speaking to uh AIC I'm looking for this kind of trailer so the example here would be so I simply told it I want a video of a flyover view of the interior of the view of the futuristic Esports stadium and just created one all right so obviously the resolution is not quite great yet and I think it's not exactly the way I want it yet but um I see this technology is coming along really fast so I think trailers can be

created in the future through AI relatively easily and then this actually I don't know how many of you have seen this video I think it's actually pretty badass so I'm going to share with you I'm going to share with you why I think such badass so important to note in this video the person that's actually talking as a player what he is about to say is unscripted he just talks to okay hey Jen how are you unfortunately not so good how come I'm worried about the crime around here it's gotten bad lately my Ramen Shop got caught in the crossfire can I help

if you want to do something about this I have heard rumors that the powerful crime Lord Kumon ayoki is causing all sorts of Chaos in the city he may be the root of this violence I'll talk to him where can I find him I have heard he hangs out in the underground fight clubs on the city's East Side try there okay I'll go be careful guy all right how many of you have played RPG games before role-playing games all right I I'm sure many of you remember almost all role-playing games most of the dialogues are typically programmed they are only limited number of options you have and if you try to speak to it in a topic that's unrelated to what they were focused on

they give you junk but not this actually the I would actually create a character just like this one and you can actually give it personalities you can feed it background informations and you can give it the quest that's supposed to give out by this character and when it does when it knows exactly what it's supposed to do as able to actually act just like a real person communicate with the players tell the players basically hey uh you know my name is so-and-so and I'm actually interested in something Beyond just what you're asking me right this is actually feels real for a change and so I would say this is something which I think is actually quite exciting and this kind of takes us to the concept art tools so the concept art tools um most of you if you're actually familiar with concept art you'll

probably have heard of um stable diffusion have you heard of civil defusion mid-journey dolly as well as I would say the latest tool I've seen that's actually the best the best so far is Leonardo leonardo.ai and you know it's amazing what they can do and if I were to recommend I would say definitely go with a diffusion based model tools at this point and so between stable uh stable diffusion majority and Dali and Leonardo I think primarily depends on how realistic you want your art to be if you want the art to be more realistic I would say um I found that my journey is a little bit better and also if you're able to kind of give out more precise information exactly

what kind of art you're looking to create that some tools are able to kind of deal with that a lot better than some of the others right so sometimes you don't know exactly what you're looking for is you kind of give out a really vague commands I would say um if you're giving out very vague command mid-journey gives you crap they don't give out very good like uh art but if you're able to be very specific there it was going to give you much better things and then those are some of the art that we're going to create it for ourselves all right so talking about limitations I know I'm already out of time um so I think it is important to talk about implementation because you start thinking about well everything's possible right it is but there are still constraints today maybe we'll address them in the future but as of today it's unable to do three things one it doesn't innovate it

actually gives you different combinations permutations of existing ideas that doesn't come out with brand new ideas so exactly how you want a gameplay to flow if you have some really interesting idea that has never been done before well you're never going to get those those are not going to come from the llms or chat ubt so that's number one two it doesn't make good decisions for you actually it's pretty bad at making decisions so exactly choosing once they give you recommendations choosing which one you think actually is going to be the better one as you because they're not going to make really good decisions and last one which I think many of you have heard this term called hallucination example I'll give you an example what's hallucination so for those of you that haven't seen hallucination before I asked tragedy BT what was Tesla's annual profit in 2020 there's reason why I asked 2020. because your 2bt 3.5 is

only trained to 2021 to ask anything after that will be unfair so I asked 2020 but even 2020 it gave us results of 721 million which is a magnitude of by almost 10 times that's how far off this answer is right so sometimes it doesn't know the answer the question is can you differentiate between the two and that is tricky that's going to take us to where I think the opportunity lies I think really there are four opportunities here coding Esports game balance as well as the last one LM plus rule-based AI the last one actually really interesting because it can potentially address many of the problems that we're seeing today with hallucination right and the biggest problem with rule-based AI is that it's usually less

flexible as it doesn't come out with as creative answers but usually it's much more accurate it doesn't get things wrong whereas you know with lrms oftentimes give you an answer that they act like it's right but it's actually not right right so I would say those are the the biggest opportunities so I know I'm ready out of time I don't know we're gonna do any kind of q a sure does anyone have a question for uh for for Wayne we'll take a moment or two to ask questions if there are any there's one there I was going to say does anyone want to uh ask Bard if he's got any questions for me that's right hello good morning good morning I have a question about the uh video generator AI

so does it use something that you would make or does it uh just make things on its own so if I understand your question correctly you're saying from the text to video is that the video creation part yeah um so if I have some character can I include that in the video yes so in those cases I would say you would actually use the previous model oh can we um can we go back to the presentation no we can't okay all right so uh remember how in the earlier I shared there are two different types of video creation one type is actually more I would call it um used for creation of like say

bloggers that are used for description understanding of a game right those actually are more easily created using your own assets so what it does is you can feed it assets or can you place it with assets that you have because it's going to come up with recommendation based on their standard assets which you may not like you say you know what I want to use my own asset for this video you can so you you actually the challenge with that though is you'll actually have to do some manual work yourself it's not it's not going to be automatically done for you you all you have to load the the assets and you have to select the Assets in I want to use this asset for this purpose here so some manual work but no coding so that's the the good part right so it doesn't require coding is simply after it's kind of Select and drop the assets that you want to use but if you are lazy like sometimes I am the second option saves you more time

that makes sense thank you I know we've got we're probably loads of questions we are a little bit out of time though um I think did you have a question here you want to take one question here sorry guys you'll have to follow up on me too much afterwards very very quick one for the NPCs do you have any recommendation for this yeah so NPCs what the example that you just saw um with with convey was created with convey and convey I I play with it for a little bit and I would say that the the cool part about convey is you can set parameters you can set parameters both in terms of personalities character you can also set parameters around the objective the objective you want this character to play right what is it that this this player is supposed to reveal to players and well actually then automatically figure out how does it disseminate that information to the players so I think that's the the good thing about convey um I would say that it's still not as

flexible as I want it to be and the key is you really have to kind of load a lot of background information in order for the character to act a little bit more special as opposed to generic and the beauty of it is today you have multiple tools you can use you can actually create a whole bunch of background stories through another tool right so the other tool for example chat gbt and Bard you can actually say hey I want to create a background story about this character which I create and and then you can talk to it say you know so well typically what we've done is it create first background stories like I don't like it I want to change it this way in that way oh remember so okay then you say elaborate you have even bigger more more populated background which then you can feed into the convey tool then actually it starts to act like

a real person right so that's the trick I would I would recommend using unfortunately we have to move on we've got a few more AI speakers but let's have a round of applause for Mr Wayne Lee right thank you it's an absolutely fascinating topic