AI & Talent in Games Engineering: Hype Meets Reality – Live at Pragma OSS

[Music] Welcome to the Navit Gaming Podcast, where it is our mission to explore the business and future of video games. We bring together the industry's brightest builders, investors, and thinkers to keep a pulse on current events, dissect emerging trends and games, share lessons learned, and have a great time. This podcast is also part of Navik's growing ecosystem which ranges from free and premium research to consulting and advisory services. For more information, visit www.navic.co. [Music] This episode is brought to you by Lightseed Venture Partners. Lightseed is a globally leading venture capital firm with over $29 billion in capital under management and more than 500 investments across the US, Europe, and Asia. With its dedicated gaming practice, Lightseed Gaming, the firm is investing from over $7 billion in early and growth stage

capital, making it by far the largest fund focused on gaming and interactive technology. Lightseed's team combines deep expertise in gaming with a global multi-stage investment platform and a culture that truly puts founders first. Selected investments include Epic Games, Snap, and Stability AI, as well as game designers and producers who led the creation of titles like Fortnite, Call of Duty, League of Legends, Valerant, Starcraft 2, and many more. For more information, simply go to gaming.lsvp.com or check out the details in the show notes. This episode is brought to you by Overwolf. With over 1,500 games supported, 178,000 creators, and 13 million monthly active users, Overwolf is the guild for in-game creators. Whether you're a gamer, creator, or game studio, Overwolf is the ultimate destination for integrating UGC into games. For game studios, Overwolf offers

Curse Forge for studios, a white label solution that lets game makers and publishers easily integrate mods safely and seamlessly into their games, both existing and new, at zero cost. It's battle tested by AAA studios and games including Craftton with Enzo, Studio Wildcard with Ark, Take Interactive, and others. Alongside this, Overwolf is the all-in-one platform that enables creators to build, distribute, and monetize in-game apps, mods, and game servers. In 2024, Overwolf paid over $240 million to in-game creators, proving that they truly value the talents and contributions of the gaming community. You can check out all Overwolf has to offer at overwolf.com or simply check out the link in the show notes. Now, let's jump into the episode. What is up everybody and uh welcome to the Navic Gaming podcast. I'm your host Alex Decay and this is the uh interview and insight segment. Um if we sound a little different today, it's because today's conversation is pretty special.

I'm here boots on the ground in Ply Vista in LA at the Pragma OSS Summit. For those that will be listening on air, this is Pragma's third annual online services summit. We at NAVIK are a proud sponsor of the summit alongside Unity, AWS, Curse Forge, First Look, Hora, ID3D, K, Rocket Science, and Salsa. And hopefully I miss none of you. Um, this is our second ever live recording um with an audience in Navix's history. And so we are very excited to be bringing the spice today. So crowd, how are we doing? Wave to us. Oh my god. Oh my gosh. Everybody is so happy here in LA. Um, you should come to New York City for a second. It'll darken you. Um, you guys are great. Um, but up on stage with me are, of course, as James already said, Serge. Um, I'm going to blow your last name. Um, Ken K can nostalis.
Got it.
Yes. Um, head of engineering at Gardens Interactive, previously head of engineering at Singularity 6, and Meer

Wasserman, um, head of engineering at 2K, previously of Amazon Games and Electronic Arts. Um, and I'm very excited to be up here. um as I was doing some obligatory LinkedIn stuff um in our newsletter which if you're not subscribed um you can subscribe to our novice digest to keep you up to date on the business side of the industry um you know the money people but um when I was on LinkedIn I realized that this summit is described as quote this invite only summit is built by engineers for engineers shaping the online game services and technology and so I am not an engineer um and I feel very lucky to be here thank you Eden Gideon James Serge Mer all of you in the crowd Um it's it's it's awesome. Um but actually with the power of cloud code uh which we're going to talk a little bit about today um I basically am an engineer. Um and so you'll see all my apps that won't scale beyond uh five users. But that brings us to the kickoff for today's panel. We're going to talk about two topics. So AI and engineering in games and engineering in the new economy

and both reinforce one another. And I'm very excited to get started. But before we do, um, Meer and Serge, I want to hear a little bit about the both of you. Serge, we already know that you work at Gardens, a ventureback video game studio. And Meer, you work at 2K, which doesn't need to be explained. But beyond that, succinctly, I would love for you guys to answer each of you three questions. Um, the first is in your roles as heads of engineering, what is the best part of your day and pitch me on why I should convert to being an engineer? Uh, two, what is something you miss about the old days, and you can define that however you want, of games engineering. And three, tell me about your most hilarious AI upset. When did an AI tool do something you totally weren't expecting? And this will lead into our first topic. Um, Serge, how about you kick us off? Okay. So, for me as head of engineering, what I get really excited about is when I just get to sit back and watch the engineers throw out ideas and collaborate with the artists and the designers and just seeing that synthesis of ideas and back

and forth. Some of what Mike was talking about earlier with this balance between product and technical uh debate like what do you want to be able to do and what can you do? Seeing that interaction is just that's what makes me really happy is seeing those dynamics. Um and pitching on being an engineer. Do you want to talk about making a game or do you want to make a game? Like I think that's like and I'm glad you're trying clo I think the more people
the more people that can get access to whether it's tools like that or like some really simple uh engines that make it accessible like anybody can be an engineer. It's not a binary decision. It's a spectrum. So yeah push your limits become an engineer.
All right.
Uh yeah first off uh thanks for hosting. Thanks Pragma and um my PIC Serge who I shared an Uber with yesterday and found out we had a mutual friend that we both adored and didn't know we knew each other. So I feel like uh it's a small industry and you just you know find that out in events like this. Uh best part of my day um is basically just learning and

teaching and I really like learning a lot and I find I do that almost every day. And then you have opportunities um some days where you get to teach someone uh something that's not maybe obvious. Maybe you're helping grow their mindset and those days are really great. It beats the paperwork that's for sure. Um why should you be an engineer? I don't know. Like if you like problem solving, if you like uh reducing uh problems down to their constituent parts and you know piecing piecing it back together, figuring it out, um if you like keeping up with constantly changing technology, um you should be an engineer. But I think that skill set applies to basically any any type of work. So you're not really stuck in engineering, even if you do like those things.
All right. What's one thing you miss about the old days of games engineering? For me, it's the wonder. Like, I remember as a player, the 8-bit sound or the 16- bit graphics and just like

anything that was made, it was just amazing. And nowadays, it's much harder to figure out what the player hasn't experienced before and how do I create that experience, that feeling. Um, so it's just we're kind of past that tech innovation phase where yeah, it was just amazing what you could get this this box of sand to do. So, I I I do miss those days.
That's a fantastic answer. Um, I like that one. I wish I had gone first. Uh,
sorry.
I like So, I cut my teeth making AAA games at EA, working on Madden. I was a gameplay engineer there. And, um, AAA games many, many years ago were the size of like indie teams now. And, um, I I missed that. Like I I missed that you could have in a AAA environment like end toend ownership over almost the entire codebase. It was really fun. I was filling out one of the bingo cards about a Easter egg I added to the game. I won't I won't tell the story here, but I I that's doesn't seem like that possible on AAA anymore. I'm sure it's rampant in indie and double A possibly, but like yeah, I I do miss that.

All right, MI AI question. Hilarious.
So, two months into Gardens, AI really embarrassed me. Uh this is about two years ago. Chat GPT was pretty new. Uh, I had showed up to gardens and we were trying to do internal play tests and nobody would knew what was in any build that we're doing each week. So I'm like great export in JSON what the perforce change list is. I will get chat to summarize it. It was amazing. I then gamified and I'm like what's the best commit and what is the funniest commit? And that worked a couple times and then the fourth week I did this I announced it and again I'm still kind of new at gardens and I read the best commit and who did it and it's complete hallucination. It was like it's like nobody had made that change. It was a person who didn't work there. It's like okay I'm I'm killing this project.
That's pretty good.
Yeah. I I find hallucinations uh not that funny. Um uh but I do have a story of a friend of mine uh a real friend. Not not a friend.

it's not me. Um, so when GPT switched to uh GPD4, um, he had put his LinkedIn uh into the the tool and said, "Hey, write me a cover letter." And um, and it spit out that he co-ounded Activision. And he's like, "That's not that's not right. I did not co-ound Activision. Let me like reformat and try again." And it came out again. And he's like, "Okay, so there's two possibilities here. One is there's an alternate reality of data that this was trained on where I did co-ound Activision. that's uh it's reflecting that. Or second, it it couldn't possibly believe someone with my resume didn't co-ound aaa game studio. Uh so that's a funny hallucination.
Those are really good actually. Um but that's basically that'll actually get us started for our first topic, the use of AI in games and engineering today. Um we're going to open with some use cases, then tackle some questions around concerns and innovations. Um, and in an article from Pocket Gamer right out of TGS, um, the TGS organizer claimed that 51% of firms in the region are using AI

or generative AI and the use cases for the tech included producing videos and images, stories and texts and programming support. Um, in our kickoff, uh, both of you told me that Gardens and 2K are actively using AI every day on their engineering teams. And so I want to hear from you guys. Uh, what are your team's top three use cases of AI so far? Um maybe Serge you can go first.
Okay, I'll go first again. Uh so the three major areas are community management, data science and developer productivity. So we use GGWP. It's uh old school AI, now new school AI to really track community sentiment. Connect that with like in-game player behavior and get a full picture for player support for uh the community management to see what's doing on Discord. What what is the community feeling? What are the bad actors? What do we need to moderate? Second one is data science. Uh we've evaluated data bricks and metabase and going with metabase now. They've got a AI beta tool

that really lets you take this big stream of data telemetry that's coming into our platform and making it easier like to me I want to get the designers as fast as possible able to validate whether their ideas are working or not. So the more I can remove steps in the process where you got to talk to the data engineer and then the data scientist and then get this implemented and this report looks good. The more I can just let them query and just try different ideas even if it's not perfect. That's can be really great. And the last is just developer productivity. I think a lot of us you're using cloud code. Uh we try lots of different
I'm not really using cloud code.
Um yeah. So that's like the the major category I think is pretty traditional.
Okay. What about you?
Yeah. I'll I'll just put out an industry observation which is to say I think uh the second thing Serge had mentioned is super cool which is like natural language quering of data sets. um access to data right now it's like a bit of arcana you know and if you're even a really talented software engineer or server engineer you may not be like

super good at uh getting at data and the data you want to understand your customer better and so I think like NLQ is is it like it's subject to all the same limitations that LLM have right it's can hallucinate it can give you the wrong stuff whatever um but I think it does give us everyone access to data and that will help us understand what's going on in our systems better and also like what our customers are doing and I'm I'm pretty excited about where that heads as the um as the LLMs mature.
All right. So in the third thing that you said developer tools um you know you mentioned GGWP old school AI or new AI um and then uh but in the third in the third category you know you mentioned cloud code tell me a little bit about like the actual AI I guess the code assistant AI suite that you actually are using and how are you figuring out like standardizing that across your team um and how do you guys work on that at gardens? So we're very much in a experimentation phase. So what I do is I

give people a budget to optionally try out these tools. So I I think I've got almost every product been used. So Kodium now Windsurf is one engineer is using someone's using cloud code, someone's using Copilot. Someone just uses ChatgBT. Uh most engineers use the Intelligj Suite. So they're using Jet Brains AI. What this does is it allows us to sort of evaluate many different tools and figure out what they're good and good at hallucinating at and figure out like ultimately what stage of the project it really is useful at. So I'm not really focused on standardization to this point and more just like crossraining and experimentation. There's so much innovation happens. There's so much that changes uh that I think just helping the team learn for those who are interested in doing it. It's really a powerful tool and it it helps with like discovery and climbing a learning curve to really understand code bases you haven't seen before or be able to quickly iterate especially for backend groups who are polygots working in many different programming languages. You haven't see updated that Helm chart in a while. So you got to like get a

quick version of that and then jump over to some Cotlin and then write that bash script. It's really helpful to kind of help somebody move around between all the different areas of tech they want. So it's uh but yeah, we're more focused on experimentation and just keeping up with the innovation at this point.
I see any responses, similarities
um in terms of consolidation or Yeah. So one thing I've noticed is um the way you used to build games or I think we still do is you know you kind of play with a different tool bunch of different tools and you lock it in and like make sure like your pipelines are working and and you can you can go from there and lock it down. uh AI tools don't really fall into that schema very well because the pace of change is incredibly fast. I think Mike was saying that earlier actually. It's it's just too it's too fast to say like we're going to lock down this tool now where 6 months from now whatever the tool does could be baked into the model, right? You wouldn't even need that tool anymore. It might just be like part of like whatever model the underlying model is happening

or um it's progressed so the the technology has progressed so fast that like that tool is not as effective as another one. And so that's like again like industrywide I see this as a kind of a problem is that we have to understand the AI tools don't it's harder to lock in and standardize because of the pace of change and just have to be open to that and thinking about how we develop games um not just like which tools we're using.
Yeah. Yeah. And we're definitely going to talk a little bit about stability like in this section and how you guys are managing and thinking about that. But you you mentioned a couple tools here um like cursor um cloud code um etc. Um, and I guess like one of the first questions and concerns is like context and sometimes there can be tens of hundreds of documents that you need to fully wrap your head around. Um, all the systems and information you need to know as an engineer. And so what are you guys doing um to ensure that your AI tools have effective context um for your team and how do you think about making sure that that's secure?
You're looking at me so I'm going to talk.
Okay. Uh, context matters for LLMs a

lot. And I know it's not the question you asked, but I just want to lead with this to say like the if you ask an LLM like write me a story, who knows what you get. And if you give it a lot more information, you'll get something that's closer to what you're hoping to get. And the same is true with uh that's like how the underlying like technology works. And the same is true with when it's helping you code or create art or whatever it's doing. Um, and so you have this uh uh incentive to provide a ton of context in order to get the best possible outcome. Some of that's like the the data it has access to to evaluate. Uh some of it's the prompt you use. Um there is there are it's not just diminishing returns. There is kind of a a graph where too much context reduces accuracy. Um but I don't think we're like generally have that problem. Um, part of the problem though is that when you provide all this data, you have to think about like what is the company that I'm providing this data to doing

with that data. And so, um, I have an example again, a friend, not me, not a friend, but a real friend who, uh, at a different company who, um, they they used some slush art. They had like extra art lying around for a project that never kicked off. and they were uh uh trying to see if they could get a LLM to help create like uh product kickoff ideas or like game game game uh game ideas. And so they put the art in and they had a bunch of text as well for the for the GDDs. And what came out of it was like they got it working and they actually liked it and and it went so well that at the company they're like, "Oh, we we actually kind of want to make this game like this this is a we we actually think this could be something." So that's that's a good outcome except that art that they used was now like given to the third party company and everyone had access to it. And so then it felt like well we can't really do that anymore. We can't make the game with this art anymore and we have to you know pivot even though we initially own

the art we used it to train the model and now like that's that's gone arry. And so there's like this balance. You have to think not just like how do I give it all the context but like how if it's like a um enterprise contract that's generally pretty safe like your your data segregated. But if it's not if it's just a tool you're using you have to understand like what's going to happen with the data you give to the tools so that you can know like how to protect your business or your game or your ideas as a consequence.
Yeah. Yeah. And I'd be curious like your reaction to that. Sitting at a different, you know, scale, I I presume that, you know, your guys' responsibilities to set boundaries kind of be between AI access and system integration, you know, proprietary information that you're giving to potentially a public engine. Has does that differ for you a little bit being on the startup side versus being at a large, you know, company like 2K? So less from a compliance or security concern. But uh I think what we're learning is past vibe coding there's agentic coding where you're creating much more structured roles for the AI bots and you're giving them much more

targeted context. So from there you're creating kind of sandboxes to limit how much they can hallucinate by giving them very accurate roles and very accurate context. So it's similar in that we really try to think about as as Meyer said, you can't give them too little and you can't give them too much, but giving them the really the right amount of information. I mean, it's kind of like developers. You give them some documentation, but not like every single bit of documentation and the 12 versions of it. You give them the right documentation. Uh, and the bots can achieve a lot that way. So we I I do think about it, but less from a more from just an efficiency and be able to achieve the right answer sooner. M has there ever been a point where you've kind of in your $100 fund where you've said no, you can't use this tool because it's not secure. You don't trust where it's going? It was definitely a concern two years ago when we got started where there weren't really clear terms in a lot of the licensing agreements. Now there's a much more standard version of it. So if somebody mentions they want to spend money on a tool that they don't

want, I mean, we have a 55 person studio. Like people just come to me and say, "I'm ready to spend on this tool." Um but I'm less concerned about that at this point given our stage and and the scale that we operate.
Okay. Um and moving to that stability point that um you know me that you've already mentioned you know games locking engines or tools so that you can build on a stable foundation towards a solid launch with AI tools changing every 3 to 6 months as you've said um how are you structuring your teams and your workflows to specifically avoid vendor lock in as you've stated and also b keep your team at pace with learning different technologies um so that because eventually everybody might have to actually like uh congregate into into learning one.
Yeah, I'm actually curious to hear Serge's answer on this, but I'll just add into what I said earlier, which is like um I think we have to change the paradigm of how we view our tool set. And it's I'm not it's way easier said than done because like you need to lock in to produce a game, right? You can't

be having like everything change underneath you. Um and so I feel like we're we're kind of at the beginning of that of figuring out how to integrate AI tools and have them be effective over the long term. Um yeah the pace of change on these tools is like dramatic. So it's uh it's it's exciting but it also also makes it very difficult to launch a completed product.
So the products I listed that we are using uh are third parties. So I can basically hold that group accountable to make sure that they're implementing it to to deal with the implementation and if they have to change LLMs or models whatever it is that's fine for us. Then on the developers we are it's like blue ocean try out every different idea. I don't know how long these different tools are going to last but we're not building a product that really is going to require lock in. So to me it's it's more from like a a strategic perspective. I'll pick a vendor who will deliver that AI capability and it's up to them to figure

out if they got to change the implementation and chat GPT7 comes out and they got to switch to that great you figure that out. But uh for us, we're much more in that experimentation and just like learning stage.
Okay. All right. So, we've talked a little bit about the tools that you guys are using and the use cases, but I'd love for you guys to actually like maybe tell me a story about like, you know, and maybe specifically on the game design side. Um what you're actually like, you know, you what are you actually using it for in a specific example case? Um so, like, you know, you were using something like a cursor and how did that change? um did you find that impact to be like pretty profound and has that actually meaningfully changed the velocity and the direction of the production pipeline um you know whether that is in the iteration phase because I know that you know for you guys your product is in kind of pre-production early production phases so I have always tried to balance to the force between the game engineers and the microser engineers they typically do not like to touch the other side and one

thing that AI tools are very powerful at is helping people get across that learning curve It kind of instills a sense of fearlessness. I had a one of my back engineers says he describes getting over to the game side as crossing the Mariana trench and he's just very dramatic but it's very hard for him to worry about C++ which is not his favorite language. How does he understand all of these mechanics and how do subsystems work? But he's coding a bunch of backend social features that need to touch the game side as well. and he's been able to use co-pilot to really feel more comfortable and get it to the point where it's ready to have a pull request reviewed by the game engineers and say, "Oh, that's that's actually pretty good. Maybe change the way that this implemented, which is getting people across that learning curve is just really powerful." And I think it allows me to just swap who can work on different projects and great more and then ownership is generally a great pattern that I've I've really struggled honestly uh to get people able to handle both Unreal and Cotlin frankly. So uh that's that's one thing that's been very

effective in terms of getting people so all my backend engineers are touching Unreal code. It's great.
That's amazing. Yeah, that's um I think that's right. Like getting past the learning curve is that it's really useful there. Like AI tools feel less impactful than the marketing would have you believe, but better than zero, right? Like they actually they help. They don't harm, but at the same time, it's not replacing us. Um and so I I I yeah, I think I like your observation. I would agree with it. It's like getting people uh to quickly get up to speed on like an unfamiliar area. It's very common to have to do as an engineer these days. is you don't just get to like specialize in one language and walk away. It's not how it works anymore. Um, seems like there's a lot of use there for AI tools.
All right, before we move on to our second topic, I want to get your guys' sobering take on the like the are we there yet question. Um, and to be clear, I don't actually believe this question, but tell us uh why all of us like product and business people are out of

their minds when we think that making a video game with AI is like right around the corner. You don't believe that?
I don't like believe it like believe it, but maybe you're telling me that it is true.
I'm telling you it's true. We're
Oh, I'm verifying it.
Any other questions? Uh I have a friend uh god, why is it always my friends?
This is a completely different friend. It's starting to starting to sound unbelievable. Yeah, too many friends.
Oh, I should.
Uh he has vibecoded He's not an engineer. He has vivec coded a platform that itself lets you vibe code video games at an NES or SNES level and it's super cool. It's also completely insecure. Uh but it is super cool and this guy cannot look at code and understand it. Like that's that's the level. But he's very tenacious. Um I've been playing with that and it's and I thought to myself like that's better than I thought it would be. Um,

uh, Mike, you say made an Asteroids clone. Really? I also did that. I just like said like, "Can you make me a game like this?" I described asteroids essentially and it took about five minutes and it spit it out and I was like, "All right, that's pretty that's pretty interesting." Um, it does a lot more than that. And so on that measure, we're there. But that's not really what I think we're talking about when we say, are we making games yet like self-sufficiently? Because those games are a dime a dozen and we're just copying like prior art essentially. So, I don't think we're there. I think I AI tools, the future of them are augmenting human effort into making new creative games and whatnot. But if we can go back and make like very simple games, like it's it's there today. It's 100% available right now.
So, some of when I look at the hype, I view it as product managers having the IKEA effect. You know, like I made that table, so I now value that table more. And I I think some of that's what happened is they're very excited they made an app and it doesn't really scale.

It doesn't really work. But on the flip side, a lot of what I do with the engineering team is to give the designers or the artists tools to be able to build their game. And if this is another tool to allow them to build something, I'm not shipping a game that's all in the blueprints that the designer made, I'm going to come through as the engineering team and really make this much more effective and performant. But if there's another way that they can do that without me having to spend a lot of time building tooling to allow that designer to come up with a new idea and demonstrate it so they're not having to talk through the engineer's fingers to get through there. That's really powerful. It's it's as Meer said, it's it's making games uh but it's maybe at the prototyping gray boxing phase, not the actual shipping phase.
Yeah. Yeah. And I guess maybe that's what I meant by I don't believe it. Um I think cuz it's that distinction I think maybe that I was referring to and yeah I think like one of the really interesting things that as someone who's I've I've vibe coded one app on replet and um I realized it had no persistence and I was

like this is not functioning as I expected. It's forgotten who I am. Um and what I wish that it had done was kind of ask me the questions that like my principal engineers at my studios would ask me. They're like well how do you want it to function? And I think engineers often do that for designers. are like what do you actually mean that what do you actually want this weapon to do how should the enemy behave right and I think that if we get to that level right then someone like me without an engineering background that doesn't know the right questions to ask um would be very effective and so I'm actually really hopeful for kind of programs like that that will prompt me as if a principal engineer would prompt me um because I can answer that I can be like oh well like yes I want it to remember me or like oh yeah like I want it to I want you to be able to log in um or things like that so really curious to get your takes on that. Um,
yeah, the the thing is it's not on Rails yet, but like AI is good at that, too. If you said to AI, ask me a bunch of questions that a principal engineer would ask when I was trying to build XYZ, it'll it'll knock that out. And like that's kind of cool. Like you can see all the building blocks are there, but again, it's not like it's not like a

it's not like a Unreal experience quite yet.
Yeah. All right. Well, grounded in that reality, I want to move on to our second topic around talent and teams. Um, and in our kickoff together, we had a long discussion debate about whether or not anybody in their right mind today would try to build an MMO. Um, given the cost structure, um, limited budgets of course impacting staffing and technology decisions. Um, and if you have to do more with less, do you think basically the engineering scope of a game is being shaved down? And I'd love to discuss that between the two of you if that feels different in AAA versus at a startup. I think engineering scope's getting shaven some, but I've never had enough engineering scope to accomplish everyone's ideas. So, I am used to a whiteboard with 50 ideas and we slash it down to four or five. And if today it's down to three, you know, we still cut 45 or 47 ideas off the board. So, to me, it's less about the number we're actually going to ship because at the end of those are

just player bets on what the experience is going to succeed at. So with tools like AI, with engine game engines out of the box like Unreal and Pragma that can just help us ship fast, get get ideas out faster, it allows us to see sooner whether those are good ideas and and then we're maybe not spending all of the money to ship those three or four ideas. So I view it as more intelligent use of it. But yeah, I I never felt like I've had enough engineering scope.
Yeah, I agree with that. Like it feels like there's no more iconic duo than the year calendar year increasing and doing more with less. Like that's just every year my entire career always the ask. It feels pretty normal. Um and I think engineering scope so like that feels like a problem that we constantly have to to solve. There's another aspect to it though which is uh the the expectations for gamers increases. Every time someone innovates that goes from novel to expected and the amount of and it's especially true of AAA like the

amount of the expectations are incredibly high um for a full price game because if someone else has done it why can't you? And so that level that like baseline just goes up and up and up and even if your team size didn't change year-over-year uh you the expectations did and what what you can do relative to those expectations is now less essentially because your team hasn't grown to meet those expectations. So it feels like a a common problem a very real problem um and I doubt it's going to change anytime soon.
Yeah. And this morning um there was a discussion around um using boring boring tech uh or tech that works right and do you think in the realm of this this question specifically a smaller team and a smaller budget do you think that's pushing you guys to towards that safer boring tech and contrasted with all of this new AI tech that's new? Um so do you feel like you're basically in this constrained environment? Do you feel like you're using new things more often

or relying on old things more often? Um, which of it or a blend maybe?
Yeah, it's um yes to all of that. It's uh game when you're making a game, you don't go and you know grow up and think I want to build a backend so that you know there's no server downtime when I patch. Like no one no one says that. They just want to make a game that's fun and it's fun to play and you know all that stuff. And so what we find is like the boring tech um is supposed to keep us from having to worry about the very real problems of building games that consume almost all of the time so that we can spend more time making the game fun or interesting or unique or anything like that. Um yeah, there's a push for boring tech. I think it's all rooted in just like we want to spend more time making a game fun and less time worrying about anything else. I mean the way I think about it is I want to build internally an engineering org that is specialized in what makes us

different. So I don't view it as much boring so much as solve problems. So I will turn externally to a backend engine uh a rendering system a build pipeline social communities anything like that where I don't need to build it and somebody's got a solution great. Anytime my engineer is like, I bet I could do what they're doing. No, like let's build what makes us really special and get really good at that. Uh, I don't need us copying and implementing slightly more efficiently what somebody else has already solved. Uh, so to me, it's more about I'll then use some of the innovative tech to help us specialize in social gameplay features that we're trying to build that nobody else has. So I I view it as more an application of boring.
Okay. Interesting. All right. And the last question um in this section is about people. Um and many people uh we discussed this a little bit in our kickoff as well. Uh consider Hollywood to be a bit of a lites industry. Um like

there's coalitions of film and movie people who are known to like actively dismiss technology and as not approved way to make the highbrow art form. And uh if you look at you know it took engineers at Pixar like for example years to show the promise of animation tech to tell stories. Um, gaming on the other hand has been generally full of technopiles and a breeding ground for the adoption of technology often before other consumer use cases. Um, and I personally feel that there's a bit of a divide in gaming right now. Um, there's an aversion especially maybe among legacy developers or specific disciplines to AI adoption. Um given that you are engineering leaders, how are you bringing the organization, the artists, the animators, SFX along with you in the engineering economy where you may be interested in using AI to do high to produce higher output um or be more efficient?
Good question. Um some of the aversion like the version is going to be bucketed I'm sure and but some of it is related to uh am I training my replacement

right? if I if this tool gets super good, am I out of here? And um that was I think initially led by the marketing hype in I'm going to say like 22 23 where it was like AI companies are like don't you're not going to need humans anymore. We got you. Uh, and a couple companies tried that, not gaming companies like Clara, for instance. They there was a very public story where they tried it, wan it back, never mind. Like that didn't work so hot. And I think we've reached a point now where it's easier to bring people along who have that specific fear because we can say these are augmentation tools. Like they're here to make us better. Like you wouldn't be an artist and say, "I refuse to use Photoshop." Okay, I'm sure some artists do say that, but like most of them probably wouldn't say that. They'd be like, "Yeah, of course. Like it helps me be better at what I do." It's like, great. Well, the same is true of these other tools and the same is true for engineers using the tools and all the other disciplines. Um, so I think that conversation because of the marketing hype initially were like very fear-based and now it's becoming much easier to get

people in line with like this is just how we're going to work and it's okay and uh it does it does improve our ability to create outputs. So a long time ago when I had a full head of hair, I was in the open source community and I I learned from someone that if you come across somebody you disagree with, you have to listen so hard so that you can make their argument for them. And the power of that is then you're able to find that middle ground. And I find that's the tool I use very often when I'm talking to artists or other people who have concerns about the ecological impact of LLMs. Like there's a lot of reasons that people may not want to adopt this technology. And if I can puts in their shoes, then we can find like a middle ground to be able to move forward. I think a lot of engineering culture kind of like well we'll fix the ethics and post like we'll we'll address those down the road once we figure things out. Uh and I just really try to meet them where they are and understand like what are their concerns and people are much more amendable to change once you start to to work with them and and validate their their fears. It it's makes people less fighting you and more

just like able to change.
Yeah, Maya, I really like the example um and yeah, kind of approaching it from their side with empathy. Um about about Photoshop. I was listening I listened to this podcast called AI and I if anybody else doesn't know it, it's awesome. They interview like AI leaders and um cognition CEO and then the box CEO did one the other day. I was just talking about like just the adoption curve of AI and um you know at the end in the early 2000s everyone was building like an internet company and like every company today is an internet company. Um every company in the future will be an AI company. It's just an underlying infrastructure that just is the fundamental process of how you work. Um and that I think really stood out to me as kind of a way that I think about it for myself as well. Um because I think it's it's impacting not only engineers but business and product and the Excel monkeys of the world. Um, but you know, I also don't derive very much value from putting numbers from the cash flow statement into an Excel spreadsheet. So, I'm pretty excited for that to um

potentially potentially go away. But, um, thank you guys so much. We're going to do a quick lightning round uh before we go to audience Q&A. And so, basically the exercise here is just five questions. Um, and you just answer first whatever comes top of mind. Got it.
Um, so who will win? Uh, ChatgPT, OpenAI, Gemini, Google, Claude, or neither.
Claude. That is such a nuanced answer.
You just have to say what comes top of mind.
I guess pedantically none of them because, you know, none.
All right. None. Um,
but there's more to say.
Okay. You want like one sentence?
No, no, no, that's fine. Let's move on.
Okay. Um, all right. Claude Code, GitHub, Copilot, or neither?
Cloud Code.
Cloud Code.
Oh, nice. Oh, Gur Team Claude over here. Um, okay. Sora, um, Google

V3 or dot dot dot, god forbid, meta vibes.
None of the above. Is that an option?
Sure.
Yeah. May maybe the Google one.
Okay. Um, what is the scariest thing you've seen AI do?
Um, probably Will Smith eating spaghetti. [Applause] [Music]
I stand by that.
I don't think I can top that. I I was going to say force multiplying the hate post and ranker among social media.
That's what I said. Yeah, that that one.
I like your answer better. I want to Um and number of years until we reach AGI undefined. I know.
Okay.
Yeah. I five five is a five.

I disagree with the question. I I feel like
we thought chess was really impressive. We thought Jeopardy was really impressive. We thought searching the internet knowledge was really impressive. I think it's a terrible term. Uh, and I don't think that we're actually gonna achieve that because we can't define it. So, we're never going to get there.
Let's dive into this one because like the exponential rate of improvement is like hard to for our linear brains to like process. So, like there's I think there's a number. I just don't think we it's like trying to think in four dimensions. Like we can't really quite do that.
I think we'll redefine what human intelligence is by once we realize what the computer can do. Yeah, maybe
I've once heard the definition that AGI is basically we've reached AGI when it is more efficient to have all of your AI systems and tools running all the time versus like turning them off. It's like always efficiently doing something. Um, but if that's a definition, I think we could probably get there. But yeah, undefined.

Yeah. Well, and the capabilities are getting there, but it's way more like um energy intensive than to run a human brain. And so like that I'm not convinced. Like I don't know where that finally crosses that threshold.
All right. All right. Thank you guys. What a way to close out. Um I'd love to move to some Q&A. Uh James, I don't know if you want to come grab the mics. Thanks everyone for listening. And if you're listening on air, um thank you so much. Uh you can always reach out to me at alexandavoc.co. So questions. [Applause] [Music] Hi. Uh, I'd like to ask how do you deal with the questionable ethicality, legality, etc. of AI training data input and the questionable copyright of its output.
So, I think this is an issue that multiple countries are going to have to figure out legally what's happening. I I

go back to when the internet first boommed. There was an unclear sense of what digital copyright meant. If you took a book and you scanned it, what were your rights to distribute? And it's ethically unclear. Uh US copyright law doesn't really give a good answer. At the same time, people are being sued for copying melodies of songs from decades ago. So I think like generally there's a lot of intellectual property rights that need more clarity from the courts and I also don't want to be just US- ccentric because certainly EU and China other countries will weigh in on what this is. So I think it is morally uncertain but ultimately I want to see how the legal system decides these cases which it's going to come. People are going to sue. I think there's already some core cases in the work and I welcome those. I'd love to get some clarity on how these are treated. Yeah, it's it's definitely not up to the game companies to like figure this out. The AI companies are doing it, but yeah, it seems like it's this great question.

Uh it's it's happening right now. Like those definitions are being created because of use or misuse depending on your point of view. Um and I'm also curious to see where that lands.
Minor followup, but do you believe that so like the times right now is suing OpenAI for a lot of um uh obviously the scraping of articles that are written by journalists. Do you think that uh it's on responsibility of maybe like an open AI for example if they're paying basically to with the for these huge data warehouses of billions of dollars to train the model they should carve off some of that and pay the people who they're taking and training the data off of.
Yes. I mean it's like such a it's such a broad it's a it's a good question. Sorry. I don't mean to diminish you. It's a great question but it's like hard to it's hard to wrap your head around like the implications of any answer, right? Um I I I think as training data be you know starts to become available to all of us through our prompts and the outputs it becomes clear that like something's wrong. It's not clear yet

like how to make that actually feasible. Um just from a business perspective and a humanity perspective and all that I think is effectively what Serge was getting at is like you have to think kind of worldwide about this problem. Although I'm sure sure the US and EU and um a couple other big areas will like lead the charge there.
Yeah. And I think the answer is yes, there should be. But what is the economic value? What's the threshold? How much of a corpus means it's now something you can charge for? That's are interesting questions.
I'll give you a chance to expand on your this is a very nuanced answer uh piece. You had said you'd given a one-word answer, none of them, and I wanted to hear the rest of it.
Was this who would win? What was the
who would win?
It's because uh maybe this won't be a nuanced answer because they're good at different things, right? like the models are trained to um there was a fantastic uh uh video I watched on YouTube um where

some of the cloud researchers were coming in and just saying like well the models are really good at X and not so they fail a lot on Y and Z but that's just because we haven't trained them on that yet and then if there's an incentive to train it we would and then now they would be super good at that and that list of things we could be training models on is practically infinite. It's it's very large and um and whatever whatever the the companies uh do to incentivize training whether it's like businesses asking or just you know for the betterment of humanity or whatever like that we're eating into that very very slowly and so I think how models or how companies will differentiate is just like where they're training and what things are good at and for us as users it becomes the models aren't undifferiated they're actually quite differentiated but you're expected to use multiple. You're not just supposed to be like a cloud stand or a GPT stand or something like that. You're expected to actually say I use this model for these things, this other model for these

other things. Okay, so then backing up, well, who's going to win? It's like I think all of them. I said none of the above, but I really meant like all of them in that respect. Talk about junior talent and how they're affected. I know that's a big issue of concern. I'm concerned if I'm wondering if you have answers or thoughts in that space. I feel like these are macroeconomic forces that are happening, not anything specific to AI. I again remember back in those open source days, we talked about how the growth of the internet was exponential and then mobile and all these other things, but the scaling of talent is linear and so there was going to be huge demand nowadays. And so what that led to was decades of getting a compsai degree or get any engineering degree, you're going to end up in compsai and because that was the market demand. Nowadays, I had a daughter graduated a couple months ago with a chemical engineering degree and the compsai engineers are not instantly getting jobs and there's many other

types of engineering jobs. The the world is not overflowing with engineers. They just may not be going into computer science the way they were. So that's largely how I view it is just it is a shift in what was a very big drought for a long time. Uh I still worry about them. I'd love my daughter to go into computer science, but she's likely going to stick in chemical engineering. So I I think it's just a question of what we've been used to is just changed. Not that it's a specific tool that's come along or or some innovation that's changing the dynamic. It's it it's going to go back to a hard job to get into.
Yeah. I that that sounds smart. I think that's right. I will say my initially my point of view was like well it was smart. It wasn't it sounds smart. My point of view was like oh the juniors are they're they're they're messed up. Like it's not going to go well for juniors anymore. And I and I think there's like this cliff that started in like 2002 uh sorry 2022 not 2002 2022.

there's this cliff where like no more no more seniors were going to no more future seniors were going to enter the job market or at least very few. And so then as this whoever was senior in 2022 started to like progress and age out um we would start to re recognize that yeah like there's a there's a problem here like we haven't replaced our old seniors with new seniors and that's because the junior pipeline was materially impacted by hiring practices due to AI. Um, that was my first take and I felt that for a couple years and then over time as I've read more on it and talked to people I think I've concluded it's more just like what Surge said like well the the nature of it has changed. We're adapting how we're hiring and creating and promoting juniors or bringing them in and then turning them into seniors. That is adapting. Yes, but it's not gone. Uh, and it's just it's just a different world now. It's it's we're we're working our way through it. And so I no longer feel that sense of dread that like there's no more future seniors out there. But I I do think like we have

to acknowledge that the pipeline we're all used to that is several decades old is no longer accurate.
Followup question to that like do you think that there would be more engineers or more people that would be interested in being an engineer given some of the changes that are happening? Um I I kind of had this uh I guess debate with one of my engineers recently where you know his concern is oh my gosh like people aren't going to be classically trained in engineering anymore. They won't really know what they're doing. And I was like do you know machine code? He was like no. And I was like, yeah, you you know C++ or Python or JavaScript. Like that's a layer of abstraction and that level of engineering that you used to have to do and with punch cards and stuff is gone away, right? And perhaps this is this is a a hypothesis that this is maybe the next level of what it means to be an engineer. It it's different. So this answer will come suspiciously close to like claiming I prefer the abacus over the calculator. And I do not I I do

not prefer the abacus over the calculator. But um I think it's important to understand how the thing works underneath the scenes cuz engineering isn't really about when things go right can you make it like vibe code the thing it's fine if it goes right. It's about when it goes wrong how long does it take you to fix it. And the more you understand the inner workings of the underlying technology the better that goes. That's like most of the job. It's the most frustrating part of the job but it's also the job effectively. Um, so yeah, like we've definitely changed who's coming in engineering. Like Vibe Coding is a cool name. So you're going to get people that do it. Like my friend, right, who made the who who made the uh the platform, he he's not an engineer and he's doing engineering stuff now. Um, but he he can't debug it and and there are bugs and like that has to be solved. Um, so yeah, I think we've we've broadened we've definitely broadened the inputs, but I don't know if it's it's increased the quality of the outputs. Mhm.
Again, I sort of come back to the similar answer where I I think there's

just huge engineering demand in different specialties now like what we understand with the human genome, what we're able to do with chemical uh structural analysis. There is so much work in like solid state uh products in pharma like it's you know when we were growing up uh it was a microprocessor 3G mobile devices and that just that was the big sucking sound of engineering talent but I and I think as much as the AI companies are all insanely valued I think that there's other areas of innovation that that uh investors could get ahead of.
Any more questions? I think we probably have time for one. Okay. Um you you talked a lot about uh how AI will improve and change game development. Um what are your thoughts on like game play experience like the consumer experience like what where are you seeing changes or AI changing the actual game and like beyond like NPCs that can have a conversation with you?

Oh yeah, there was a cool game that did that actually. the whole game was built on just like you're talking effectively talking to an LLM and it's like a it was an e game. I I forget the name. It was a couple years ago that was I thought pretty innovative. Um I think it's but to answer your question I think the answer is through democratizing access to data which I talked about earlier with natural language querying like I honestly think that we we we can't use data to like define the world like because it's a data is any data we collect is an incomplete model of the world but it does contextualize what we know to be true and so that's how I think like if we're talking about the player experience what are they experiencing a lot of that's collected in data and aggregated and sometimes hard to see. Um I think and and natural language quering gives us the ability to have people, you know, designers and producers and people who aren't super good at SQL or data querying um actually try their hand at that and start to ask questions and get get answers and then

talk with people who know more about it. I think that will actually massively accelerate how we deal with um how how we help games become more fun as a consequence of AI. I'm trying to be open-minded while skeptical. Some of that comes from the types of games I like. I like a game that is very intentional about the emotion that they're trying to create with me, which is a combination of art style, game mechanics, a bunch of different things all aligning in the same direction. And I just am skeptical AI will get there. But if I look at more UGC or open-ended types of plays where it's much more of a collaborative and you're giving people more tools with AI to be able to to do that more rapidly, maybe that can work. That's not my type of game as much. So I I I can't rule out, but just in terms of what I find, I'm I'm skeptical that it's going to come, but I got good friends trying to make that happen. So I wish them a lot of luck. I don't have as many friends as Myer. I was
I think that's it. No more questions. Yeah.

All right, give it up. Thank you so much, Serge, Alex, and Meyer. Awesome panel. [Applause] If you enjoyed today's episode, whether on YouTube or your favorite podcast app, make sure to like, subscribe, comment, or give a fivestar review. And if you want to reach out or provide feedback, shoot us a note at [email protected] or find us on Twitter and LinkedIn. Plus, if you want to learn more about what Navic has to offer, make sure to check out our website, www.navic.co. There you can sign up for the number one games industry newsletter, Navic Digest, or contact us to learn about our wide-ranging consulting and advisory services. Again, that is www.navic.co. [Music] Thanks for listening, and we'll catch you in the next episode.