Get Real: AI at Work | BITKRAFT Summit 2024

the next panel that we have coming up is a panel like we needed to have a buzzword panel right buzzword AI um so this is the one um portfolio company of ours fun Side Story some fun side uh some fun anecdote on this the only deal that we ever did where it didn't take us forever no we're not that slow to typically but like this was a deal where we saw the company we met the team one single Zoom call and we had a like unanimous I see decision that we wanted to invest in the company provide them with a term sheet and Lead their seed round literally on that call only time in our life that's the level of conviction we had on what they were building that was about two and a half years ago now I think kylen if I remember right so welcome on stage kylin CEO of inor that's going to share a bit more about what inor is doing today we're going to like head over in a

discussion afterwards good to see you man yeah good to see you thank you so much all right um thank you all for having me today so today's presentation is largely going to be sharing some of our learnings of building for the last three years of AI for games um some of it will be not surprising to the folks here some of it might be surprising um I'm going to try and be as honest as possible about what we've learned uh and so let's get into it so we started about three years ago we've had the opportunity to work with a ton of different platform integrators game studios entertainment companies and we've learned a lot in that period so almost 3 years ago we produced a technic technical demo called in World Origins and the idea of it was to allow characters to speak and have very basic actions but it was really the first generation I'd say of introducing this new technology and it was very similar to a chatbot style experience so you know thinking about a chat GPT style thing with his preceded chat GPT but that General conversational flow and as we've evolved we've learned that NPCs

really aren't enough in terms of just adopting AI for games and there's a lot more that's needed and so now we actually have a great team in house as well of XA game devs who are constantly working with Partners but also producing demos and prototypes to kind of battle test our own technology and so the video I'm about to show is produced by one of our designers Colin and it shows a variety of the different building blocks that you can use to produce different AI experiences Zen is in position I have eyes on noad in position snowman what is your status characters like Nomad and Zenith are powered by n world's voice model for live dialogue and voice generation we power both the content and the delivery of VO creating more Dynamic and immersive gameplay single hostile approaching draw them into the tunnel discreetly Zenith can you take this guy out

affirmative but engagement would expose us now Zenith is tapped for non-scripted situational awareness and threat analysis hey man come here for a sec looking at the sound meter on the bottom right we can plug in volume logic to Emerald's character engine allowing AI characters to identify whether the player is speaking to their team or local enemies I Have Eyes on You snowman hostiles at your 10:00 get into position in position I have eyes on five targets snowman identify your targets I'll take the big guy on the platform and the guy in red need identify your target

I will take the two behind the crates okay I'll take the leader Snowman on your mark on my Mark 3 2 1 fire what you've seen here is a combination of n world's character engine game logic environmental awareness and multi-agent director Technologies at work inworld facilitates the awareness of game State and logic with player activity to generate both improvised and scripted gaming experiences awesome thank you so that's mainly showing just bringing together a few of the different building blocks that we have um and what I want to share now is some of the key learnings that we've had over the last few years of actually working with this so these are some of the things that are real challenges that we encounter with Partners who are either going through AI adoption or have already gone through some Ty of test of AI adoption and where they kind of found those those kind of

main weak points or areas of blockage so the first one is the reality that the AI Quality Bar for or the quality bar for AI is very high and it's also increasing so it's very easy I'm sure all of our LinkedIn feeds are filled with 3D models voice assets a lot of cool modles that are producing stuff but really it's the novelty that that kind of gets us because the AI generated content is generally lower than human generated content and we've realized that we can't accept lower quality content just because it's produced by AI meaning everything needs to meet that same quality standard also we need to have game awareness so generating great content whether that's voice dialogue or narrative content based on a lack of context means you just produce something that is missing the mark for the player so you need to actually ingest the game state to produce the AI generated responses and also in terms of core game Loop so we've had partners for example who have let's say uh types of types of purchases in their games that require impulsivity and so if you introduce sales negotiation with NPC into an

impulsive buy game loop it's going to break the entire system and so you can't introduce AI just everywhere you need to be very sensitive to exactly how your monetization Works how your game Loop work and actually how it facilitates versus kind of challenges those um and the two last points is that we've realized that you cannot really achieve the quality bar that you want using the same systems as anyone else so anybody in this room integrating AI into their game will likely use different models different systems different configurations and that's the only way that you can get it to work and it takes a lot of time and effort this is to say the chances that you're going to get in 1 to 3 months some kind of quality that you want is very very low and it actually requires a lot of model customization custom infrastructure serving options to actually get it working and so this means that there's a first mover advantage and those we've seen adopting AI earlier are now leagues ahead of anyone who's trialing with it today uh and so that kind of feeds into the second point which is that we also need AI systems to be able to actually

integrate with existing gam Stacks that means whatever engine you use whatever workflow tools that you use for design time those can't sort of require an additional piece of work for for developers and we need to bring AI into that kind of standard workflow um and the key thing here is around control so Studios can't lose control of their IP their data their core content their narrative loops and this means that generative AI needs to be controlled by large logical systems that are tied with the existing game engines rather than allowing them to be too free flowing um and finally it needs to be future proofed so every month or week it seems like now we're seeing new models come out and what easily happens is you might integrate some AI game Loop and it's super cool but when the next Generation model comes out the differentiation or uniqueness of your game Loop actually gets eroded and so the systems that we build need to get more powerful as the next generation of model come out rather than be eroded in their differentiation and then last the actual infrastructure needs to be there to actually ensure that this is done economically scalably

and cost efficient so that effectively means that we need options that allow us to make sure that costs don't scale linearly that potentially remove actually all server side costs altogether and ultimately introduce more reliability than we can see with public endpoint so we've had a lot of Partners launch experiences first using some public uh Foundation model endpoints and found that the latency performance was not up to Snuff for player experiences because players are relying on this for core mechanics and so ultimately they've ended up having to move to Dedicated servers with kind of core slas attached to that to ensure that this actually works meaning the infrastructure piece here is is critical and it actually means that we need a lot of custom support to actually set up that infrastructure for any specific Studio or game and out of the box stuff just barely ever works and so now I'll share the three things that kind of to meet these these problems we found we need so the first is we need game specific AI components that means that the kind of General models that we see on the market that are trained to serve every sing single use case available simply will

rarely if ever actually meet a quality standard that is needed for production level games meaning we need to actually build core models and systems that are specifically built for games not for the general AI space overall and that's something that I think is lacking on the market today the next thing is that these models actually need to be customized and optimized so you need to be able to take a model or system that we give you as sort of a you know generic base and easily adapt that to your specific game scenario or needs and to be able to set it up for different serving options uh which I'll get to later but basically the same system could potentially be run on the server or even now distill to be run locally and that opens up a lot of new consoles and surfaces for players now having these building blocks is great but we also need to be able to design flows with them that actually meet specific problems so we need to be able to strap together all of these components to build flexible systems I can tell you out of all the studios we've worked with every single one has required completely different pipelines and this means that the tooling requir needs to make it easy

to kind of actually take a specific solution or problem that you need and build your own uh you pipeline of AI components for that and then you need to be able to distill that pipeline down to be able to run locally or more efficiently um we've also found that there's kind of a very there's a few common use cases that need to be had and we're trying now to think about what are the templates that are kind of making it easier for game ST to get started rather than kind of building from scratch and the last point is actually one of the main problems with AI today is simply cost so variable cost you know if you're running a game and you don't know if you're going to be spending a million dollar a month $10 million a month $20 million a month it makes it extremely hard as a developer to budget that into your game and also to understand how that actually fits with your end economics so one of the big bets that we're making is on the future of on device endtoend AI systems for games and having those actually integrated with consoles PCS so that there's no serers side cost associated with the experience and this is I think one of those are is that is going to unlock the actual

potential of this for game studios and allow it to actually be integrated into new types of games it's a very hard problem but we think probably within a year we'll have something that is actually able to work end to end with the same functionality we have today on server um working locally as well and last is that you know we need to have sort of a real-time performance so what we see happening with AI today is kind of a increasing focus on the AGI you know future proof um version of the world but for games we need something that is working really fast efficiently uh and it's just a different requirement than for everything else and so this is to say the infrastructure that we need as well needs to be rethought compared to what is needed for other AI components so bringing this all together basically what we I believe need to actually make AI work for games is specific AI components that are built for games and real-time experiences we need to be able to bring those together into pipelines and then we need to be able to distill those pipelines down to run locally or on kind of custom different infrastructures so that they can work for the cost and performance that you need uh for your Studios this

is basically to say we need to make sure that the tools are flexible enough to allow everyone to adapt them to their needs rather than having a very biased view of actually how to build and also the reality that we shouldn't push people for example towards integrating talking MPCs if that doesn't fit with their game Loop these tools need to be able to adapt to the actual game loops and mechanics that players want and ultimately make it more fun so now to continue the conversation uh I'm going to welcome out Michael Stewart from M12 Microsoft's invent investment arm and yasmina Beni who strategic AI Partnerships in Nidia uh Yen will be joining us as well to conduct the panel uh so thank you all so [Applause] much all right thank you so much kylin um to to sort of give this a bit of additional framing in terms of how we look at sort of thei like AI breakthrough and advancements and how they factor into like our strategic view on investments in the gaming space we

basically look at it in three buckets bucket number one for us is how AI enabled game developers to get more efficient faster and have an increasing capability to like pivot and like like changes in road map and and and intersections so there's there's like a use of AI to give you that capability of lowering your cost in production and in life Services as well as like being faster at the same time that's bucket number one so we love investing in and looking at Studios that make good use of that bucket number two is the bucket of emerging game behavior that we have seen here now on stage in in an exciting way so how does AI like provide and bring forward new game experiences through its application and bucket number three so we love companies that like bring that forward and bring that out that we can potentially invest in the bucket number three is like tools and services and platforms that provide AI technology to companies to potentially accomplish number one or number two and that is a company any were like in world it's

Square in uh so that said um question to you first Michael uh you're on the board with us at inworld which we're very excited about what was your thesis like how did you look at inor when you made that investment decision yeah and um it's interesting because the time when I met the team it was a very weird moment when uh the thesis we were developing was really around the metaverse and uh I I as in the back of my mind also thinking about new Ai and new use cases for AI and it kind of collided in my mind but really the the spark for me came from when I was speaking with the inworld team about generative AI which I'd really been looking for some Innovative use cases for we had a lot of the technology going at the time gpt3 was in general availability and there had not really been any startups or any you know any visible demonstrations of the kind of enthusiasm that's all around

us now but it was their background their expertise and kind of bring it to production bringing that kind of AI to production and then I knew that these types of settings the the fun uh you know entertaining settings would be there for the technology when it's ready I think it's more the maturity and experience of knowing how to partner and work within large organizations and staying at The Cutting Edge of it that that help clinch the decision for me thank you yes minina um thank you for joining us here your company sort of is a games company in the first place right like that's where you were born um so we can proudly say that as gaming industry um so you see so much in your position of like companies approaching you how can we collaborate with Nvidia how can you Empower them what some of the things where like it makes you scratch your head like some of the things are really off where people have a misconception of how AI could po potentially like like like lead to business model it's like

lead lead to Great products yeah so this is great question I feel like uh there is a lot of misconceptions around AI Solutions currently that we're trying to you know help educate developer Community as well as users that are not necessarily familiar with AI some of them is they think of AI as a plug and play solution which is not true right uh in order to develop uh AI Solutions in production you need to upskill your teams you need to uh make sure that you know AI is needed in that specific workflow and it brings a a certain value and another thing uh that we notice uh currently in addition to thinking that it's a plug andplay solution is that a lot of people tend to think that AI is going to go after their jobs Etc whereas uh you still need human in the loop in order to train these models provide feedback uh make sure that they correspond to your needs fine tune them

Etc so I feel like these are the two biggest misconceptions that we're seeing as of now so question is probably a bit for the entire group uh K Kanan you you you touched on a very important point that is like you're you're starting to like offload or like like put like the compute consideration onto local device on premise right so but there is experiences particularly greater like multiplayer experiences or more open world experiences where that potentially conflicts because every player in concurrency has to see those characters and that they might even interact with each other right which brings forward totally new experiences now very compute intensive at the same time difficult to then take local compute to do that how do you expect that path to these experiences with and and and you bringing down cost there to to like look like in the next years yeah so the the way we see it happening is once we have everything working locally end to end you can introduce new hybrid Solutions so this means you might have certain parts so it's it's very similar actually if you have if you have a Unreal powered

multiplayer game the rendering and everything is happening locally unreal is running locally but it's receiving those commands from the server side which is basically managing let's say the multiplayer scenario it's sending back some triggers to local command it's updating the current game state so there's this kind of speaking between the two systems so the AI is operating locally you can kind of think about that like some local actor right so you have these local actors that each player is actually interacting with that is sort of a local instantiation of the AI then you have a special system that kind of lives on the server that is kind of speaking to those local um instances and so what that means is that lot of the hard compute around like L llm inference text to speech different systems end up actually being run locally and that makes it a lot cheaper while you still allow the kind of complex Computing that manages the multiplayer scenario or let's say in a mult a large open game sort of the uh uh coherence of the environment as things change within it you allow that to kind of operate in the server side and so we see this kind of new world being where you have the ability at least for a player run it locally and then we've you know there's

also options where let's say a job becomes too intensive it can actually send it to the cloud as an option and then you have this director system living and so it's kind of this multi-layered system where the goal is to basically make it as efficient as possible and reduce the overall cost well maintaining the maximum control and flexibility for the overall experience and and I think this is ultimately how it will evolve and and how we see AI working in games in a few years will be very similar to how existing manually built logic and commands work within game engines today any any other areas and any other companies you work with or collaborate with where you see similar approaches taking place in other like like Target audiences or or customer groups yeah I can Si one example So currently we're working on customer service agents or avatars and for the most part when we released our solution Ace Avatar Cloud engine it was mainly targeted for the cloud so at the initial stage and then what we noticed is when our partners are deploying these Solutions uh in

production for example let's say you have a kiosk in an airport you don't necessarily want to be hitting the cloud every time you need to provide a response so what we're providing now is a local kind of solution where you're able to run your animation we have small language model to provide answers or anything related to the knowledge about specific products Etc so this kind of uh decision about whether we should be on the cloud or we should be local depends on the use case you're trying to serve so this is what we're seeing now being able to serve all use cases is uh Paramount Michael we we we have a thesis at bcrafty in gaming are often times like driving technology forward first case in point Nvidia and and S of it then transcend to other Industries now is inw World a company where you feel like how they go about like building like virtual characters

identities like they might be in a position where they're like better or faster to Market in like serving other adjacent spaces Beyond gaming yeah absolutely I mean uh I to totally believe the first company with the technology to make whatever you want to call the intelligence and the cloud Humane easy to work with enjoyable like your friend or companion someone you you know trust it's actually a a a very it's a jump ball anyone that gets to this first will start building the audience faster it could be any kind of company including startups and I have full confidence that um in world will be right there at the at the front of the pack over to you kylin have any any concrete examples anything I don't know fmcg other spaces where you've been like already working with collaborating with where your Technologies applicable yeah there's a there's a I don't know the right term for it but what I'm seeing come up is this like what I call gamified Media so dual lingo is always

the great example of this so you have kind of apps whether they're education uh whether they're kind of training whatever these different areas where it's basically using Game mechanics but in different Industries and I think this is already where you see unreal being used for example in simulation so one area I see is simulation which can be used for you know public sector it can be used for lots of different things for economic simulation uh there's some groups that we're talking with at Stanford that are using it for social sciences and these kinds of areas um then there's I think this kind of just gamified media which frankly I think is everybody's learning that games work really well to engage with players and that will transcend into how do ads look in the future you know how do you interact with a with an experience that is actually a learning experience so those are those are the large ones that were coming up so basically like training education advertising brand representation all of those and thinking about actually how do we make those basically into games so it's it's almost like how do we just make the rest of the industries that exist into into a gamified experience and

therefore it it naturally transcends the technology yeah yeah I can add to that point so when talking with inward so they already solve the hardest problem in my opinion right which is for gaming then they can easily cater to other use cases and we've discussed this before let's say for instance you want to build an agent that can be you know a educator for kids right you can build that all you need is basically design the brain provide the knowledge base have an avatar that can speak to that knowledge and basically have it on a platform could be a website or anything right so they already they can serve this use case because they already solved a harder problem um so yeah let me ask a selfish question to you both uh because we have a lot of portfolio companies here um how is NVIDIA engaging with like the games industry at large other than being a close partner and enabler for inw world but like is there any practical way for for portfolio companies are building in gaming whether

that's platforms technology that they build or games to interact with you to like like work and build relationship with you sure I can add some color to that so basically what we do is uh we build ecosystems of Partners uh that can take the building blocks we provide so actually it's not just about Hardware it's about software too right so we actually have 75% of our Engineers are software focused not Hardware focused uh so a lot of these companies approach us because they have a need for a library or a specific framework Etc then we engage with them in that sense uh on the technical aspect and that leads to a stronger partnership on all the other layers so it always starts from a need that they have and sometimes it could be companies we reach out to for example like inward right you can read about a company and see that they have a very solid team they have an actual product that works and solve reward problem so we approach them in order to collaborate

together awesome continued investment in the AI space and AI gaming space Michael like what's your what's your like investment strategy looking forward like how are you looking at the space so we you know we are still uh likely going to be mainly infrastructure investors in gaming for the for the time being but we enjoy and and seek to to syndicate with um investors that have a broader view across and and expertise within the operational side of the sector like bit craft to to me um you know to your earlier question the the the conjunction and and kind of like merging of these game likee experiences versus casual everyday maybe even productivity experiences is is a very strong belief I have and for what we want to do as the fund partnering with Microsoft as a as an optional way of of taking it forward we have now a number of different strategies to employ one of them could be partnering with Xbox um for the distribution and again Studios

advantages that they have there's also you know the technology infrastructure itself and the partners that Microsoft has aced so I think games going forward going you know flashing forward to the future are going to be a completely different part of your life than they than they were when I was you know gaming geek growing up and to that point you know having windows with 1.6 billion client installs it's it's just like it's going to happen it's very difficult to see what kind of company it is what the experience looks like but looks like but that's definitely um it's a key pillar of the investment thesis thank you Michael so concluding this we're pretty much on time um I'm super excited because what I saw here earlier today like that emerging gaml that like arises out of the technology being applied is something that gets me incredibly excited like I what I saw this morning I can't stop thinking about what that

implies and what that will imply at scale once you also solve like the compute part of it in a more meaningful way which is another part when we invested in the company I really thought it's going to be more straightforward way more straightforward but the challenges we encountered and the complexity of what all you have to solve for for making that work it's just incredible so I'm super excited that that inw world is part of our portfolio thank you for sharing all these Insight Michael and Jasmina you're going to be around for a bit here so I I hope you make it of that and for now I think we'll just open like our our catering here for a bit of a lunch right thank you very much everyone thank you all so much thank you