Game Tech: The Hidden Force Behind Gaming’s Challenges and Opportunities

[Music] welcome to the novic gaming podcast where it is our mission to explore the business and future of video games we bring together the industry's brightest Builders investors and thinkers to keep a pulse on current events dissect emerging Trends and games share Lessons Learned and have a great time this podcast is also part of novic growing ecosystem which ranges from free and premium research to Consulting and AD advisory services for more information visit [Music] www.nic.edu game development capabilities including AWS services like Amazon gam lift as well as solutions from AWS Partners to help game developers build run and grow their games these Solutions simplify how you use AWS making it easier to select

the right tools for your desired use case this reduces the time and investment required for game developers to adopt the cloud leaving more time to innovate and Achieve faster time to Value AWS brings over 15 years of experience supporting transformation for game developers including Riot games scopely second dinner Behavior interactive and Nintendo with hundreds of millions playing games build built on AWS for games thousands of AWS partners and teams of AWS Solutions architects around the world your game is in good hands within the AWS Community for more information visit aws.amazon.com gamch or simply check out the link in the show notes now let's jump into the episode hi everyone and welcome to the novic gaming podcast I'm your host Aaron Bush and I'm really excited about today's episode episode as you probably

noticed our podcast is super focused on the business of video games especially when it comes to figuring out what's next however business is a reflection of all sorts of underlying Trends and products and Technologies and sometimes like when it comes to figuring out why the games industry is facing widespread challenges it's helpful to dig a layer deeper un suus out more technical root causes and solutions and that's what this episode today is all about I'm joined today by jeni General partner at rendered VC and we're going to dive into how technology and game Tech specifically has shaped and continues to shape how game companies work and what might be holding them back in some cases so it should be a fascinating conversation and gon welcome to the Pod hey thanks so much for having me yeah so this is going to be a really interesting conversation but before we

dive into the weeds um I think it would be great just for you to share some quick background so to start could you give our audience a quick taste of what you were up to before rendered VC yeah I come from a Consulting background I started a practice at deoe us uh on 3D Graphics so we did all the strategy work for the major 3D software players but the technical work for the customers too so we implemented game pipelines but also VFX pipelines and more and more outside of games as well taking unity and Unreal Engine to Automotive Aerospace fashion companies a lot of outside sectors using realtime 3D nice and I know you've been pretty on top of just new technology in general I don't know if you want me if you want me mentioning this but I did listen to your Ted talk as part of the preparation for this podcast which was about you know virtual reality and and such years ago um so really excited to dig into all sorts of um technical elements with you today but you know final question before

we really dig in what drove you to start rendered VC what was that two three years ago now yeah two years ago you know we we worked with game companies and non-game companies and there were two trends that really stood out one game Tech was aggressively being used outside of games but that opened our eyes to the second Trend which was there was a very very clear difference between how software teams worked outside of games and how they worked in games and the Tech World in games felt like we were closer to the 90s compared to how other software Orcs worked so that's what we've noticed in that Gap became very apparent that that there was both organizational processes that needed to change within the games world but there was also a technical Gap around what was needed the specifications of game development and so that's what led us to start our investing journey to try and Target that Gap awesome well as I alluded to in the intro over the past couple of years there's been a ton of discussion about

games industry challenges launch failures budgets soaring faster than sales the bloated content treadmills layoffs you name it um and usually the framing from the media and the companies themselves centers on business or economic reasons and a lot of times that's fair Apple's ATT policies the co boom bus zero interest rate phenomena like all of these things are are real and impactful um however you also think that there are deeper underlying technical reasons um for why these overarching challenges in the games industry we're seeing exist so we'll we'll dig into more depth uh and kind of peel back the onion here but could you to kick us off just elaborate on your thesis here how would you frame that for us yeah useful way to frame it is every software company has a dynamic

where development performance shapes business performance you want to ship a good product you win your customers you provide a consistent stable product experience you keep your customers you keep your product relevant adid competition you prevent them from leaving games are software products too and and gameplay and IP are in the driver's seat but development performance still shapes business performance and this has become especially clear with life service games it brings to imagine a stick figure um you have the head which is the executive management team the GMS the EPS you have the body which is the studio staff Dev teams testers artists and then the likes of the technology that the whole studio stands on so the stick figure has been walking its whole life it takes years to develop a game they have a walking pace and then management decides they want to try life service games because why not the money is great so the head leans forward to run and then the body follows the head because the body then says

let's build a production Cadence of every few months every few weeks the body leans forward but the legs never learned how to run and so the head leans forward the body leans forward but the legs can't keep up and the stick figure Falls over so the industry headlines are observations of stick figures falling over and exclaiming wow running must be hard but failing to knowledge that the head and the body are moving before the legs are ready to run yeah that makes sense and a lot of times you know when management teams they set expectations they set Financial guidance um you know sometimes maybe they're getting a bit ahead of themselves in strategy before all of the details have been sorted out and it makes a lot of sense that you know in these cases there there would be you know many more details than meets the eye that you would see in press releases um that you know is more like internal Warfare or like internal um

you know at least dissatisfaction in terms of like how decisions are being made and we can get to to more of the human element later but maybe could you expand more on like why are live Services you know the best example here maybe in like today's gaming environment of like why this is a widespread issue and like what that actually how it manifests itself in these these companies yeah I mean with most most game problems get swept under the rug because there's the ability to throw people at the problem and make them sleep under their desks before launching their crunch time and it stuff so much stuff gets swept under the rug but games were built to ship once it was a boxed software product and when they start to ship every single week or every single month every small problem is felt over and over and over it's like a grade of sand in the shoe you're not going to feel it if you just stand up and try the shoes on but once you start running a marathon in

them you're going to get blisters pretty fast and that's the dynamic that life Services had about bringing the inefficiencies and and Gap in game technology really front of mind to everyone who works in the industry right and as with any technology and any business the technology we use you know in turn shapes how we work so clearly in the games industry the game tech people use shapes their development it shapes their production processes um and so Jon based on the state of game Tech and games today like what are those issues that are popping up you know whether it is with live services or elsewhere could you yeah yeah let's see specific let's let's nail some technical principles first and once those make sense the correlation will become a lot more clear so let's take a a couple

technical topics one's codee architectures so in code architectures the first Paradigm is well it's called monolithic architectures a a monolithic architecture let imagine a skyscraper every floor is built on top of the other floor it's every floor depends on the floor before it and that that's kind of what we call tightly coupled architectures there's dependencies where one file structure relates to another file structure and it's easy to get started it's simple enough when you know that you're done after 20 floors but if you need to change something on the 10th floor you can't just take that floor out and put it back in you have to change something on the 10th floor and then verify or rebuild all the floors after it because they're all connected so that's a monolithic code architecture on the other end you have microservices like like more of a neighborhood instead of 20 floors you have 20 houses and the data moves between each house and if I need to change house number 10 that's quite easy because it's not connected to the other houses so it's a spectrum there's not

one or the other there varying levels of how the architecture goes from tightly coupled to Loosely coupled but architecture impacts performance um because most games sit towards the model at the camp again you shi a game once you didn't need to worry about it afterwards you wanted to maximize performance for The Limited compute on a box and keep things simple when you start a development so these tightly coupled game projects they can be 500 gigabytes in terms of a game build and you have all these developers working on the same project and when they're done making changes it needs to be tested so there's a process called compiling where the game engine produces a build that operations teams then take to the testers and they give that feedback back to the developers who then make their change but this modic code base has everything so tightly coupled with dependencies so the entire code base needs to be recompiled not just the stuff we changed yeah so across 500 gigabytes that can take five or 15 hours on a AAA before build is ready to test

so I don't want to test after every change and wait 14 hours after every mining change I bundled 20 changes together before we do a 14 hour build cycle to test so now that means I'm going to wait two weeks before I'm ready to test to get all the changes together to then get the build out to test so now I'm only testing every two weeks and I'm not driving enough testing and Bug catching into my development cycle so the choice of architecture shapes a long waterfall process with handoffs between siloed teams development handoff preparation of build handoff testing handoff continued development and that's how architecture shapes a very linear waterfall handoff driven process which is very different than if you use a different architecture that you see in SAS companies you know Agile development can band these types of practices right so that that's architecture it's a complicated one coding principles is a bit easier in a game that ships once the priority is to

get the game out the door and so since you don't have to worry about touching the game code after it ships devs take shortcuts they make messy fixes to keep the project moving instead of planning meticulous code and robust documentation so going back to the skyscraper example there's a ton of duct tap that gets used to hold that thing up but if I now have to add 10 stories and the guy who built floor five left or got laid off we don't know where he put all the duct tape he never wrote the instructions so now we have to go back and reinspect everything before we can create a safe plan to build because of the dependency problem as well that compounds over from the architecture so it's it's pretty all it's all interconnected about how these Tech foundations architectures code principles they start to form Tech debt problems they shape a studi development processes the usual Tech choices end up with waterfall process CES handoff Silo teams limited testing

slow production speed and long-term stability issues and that's that's as much of a nutshell as I think we can put it into yeah no I mean that's really well explained for how nuanced it is in reality I mean you're only as flexible and as efficient as your underlying architecture is and as you say it's easy for over over time especially when you're under time constraints to that Tech debt loads up and just makes it even harder in the future to to improve um on top of it unless you make some some radical changes um it sounds like um so I guess where are the biggest changes still needed on the tech side like if you look at all of those problems and like how you think about fixing them like what does good look like these days how do you start moving towards good yeah so so we did a study of more than 500 Global Studios to investigate developer efficiency and first of all

there's no standard framework for developer efficiency it like that topic itself isn't well understood and is an underlooked issue so we borrowed Dora the Dora framework from Google it measures things like how accurate your planning is your lead time the time it takes from design to ship or or the rework rates so the game industry average PL accuracy for example is 3% 3% of the time they're accurate to their estimates and their lead time is three or more month months a modern software company has more than 80% accuracy and less than a 42-hour lead time so so that's what we're dealing with it's it's night and day now mobile and MMOs are typically in the top quartile they look so much better 33% planning accuracy two-month lead times but the best-in-class games are platforms like like League or Roblox that have 75% or more accuracy and ship multiple Services a day what do they do differently Loosely coupled architectures a focus on

modularity automation automation Automation and devops maturity basically the more you break something apart the more complex the system becomes so the more systems you have to have in place to manage the complexity and then devops is a framework of operating and Technology you have to implement the right human processes the funny thing is that this is normal stuff for the software industry it's cuttingedge practice in games now a huge word of caution because there's no single best way to make a game because there's no single type of game different games need different Tech and good I would say looks like a decision-making framework and strategic thinking that helps technology meet the business requirements of the gning and there are trade-offs to every technical decision and the stuff we talked about might not make sense for every studio if the game has a goto Market that's highly dependent on live services like a free-to-play life service game loose coupling and

modularity are key to build scalability and the agility needed to perform as a business but there's still a lot of triaa single player games that this might not make sense for right um and so a couple quick follow-up questions on on this I mean the the inefficiency is not you know executing on on timelines in the games industry compared to you know the modern software industry how much of that do you think is a function of the games industry just being more of an art form and more of a sense of like figuring it out as you go sometimes you got to take a step back to take two steps forward or go in a different direction uh which is maybe different from you know just coding software as a as a service you know versus the like the actual underlying technology itself is more the limiting factor like how do you think through kind of the balance of issues there yeah yeah yeah

so that there's definitely enough technology and enough explored technological paradigms like architectures and whatnot that can meet the needs of what games need to be doing today I think it's more an underlying issue of the willingness to explore new ways of doing things and management teams giving their teams time a mandates a culture of change budget and schedule flexibility to make mistakes trying something new uh it's also a huge issue with AAA is trying to play it safe pulling IP franchises which then pull technical debt from the start so I think the best way to approach it is you have to leave enough room for creativity as you said but you have to understand that if your business is going to depend on a certain function working you have to address that function working yeah um actually my my friend Ben wiberly who's over VP of

opsive jagex he said at best in the talk he gave at a recent conference it's about shifting left this a strategy to of how do you shift left your problems how do you find your problems and encounter them early assess your project do risk-based Assessments in order to make the understanding of okay we think that this creative is fairly locked down this decision makes sense we're going to move forward with it there's limited ability for it to change now let's go ahead and do a refactor pay down the tech debt before we move forward I think blindly charging forth makes sense when you have a Finish Line that you know exists but when there's no Finish Line and you have to operate consistently your strategic planning process has to meet that and yeah you don't want to stifle creativity by pre-imposed end state looks like but you do want to reassess your risk profile of and what your goto Market tolerance is and develop against that as you move forward in the whole project yeah that makes sense to me um and and one final question on This research report um is it possible for

the like our listeners to to see that is that published anywhere yeah yeah we published it with Griffin actually back in January or something like that it's out there on the web open for downloads uh it's it's it's quite dense I think section three is mostly related to what we were talking about today but I encourage anyone to go take a look at at the framework and the data that we present okay cool yeah definitely go check it out I want to check that out in more more detail too but okay let's let's um step step back again we've been talking a lot about um just technical challenges but obviously technology does not exist in a vacuum you know there's also like a human element at play there's human management as at play you're when you're giving the the stick figure example you know you're talking about human management as the head um and you know at the end of the day what people do with technology and companies

is still a result of how teams are structured and managed and how decisions get made in terms of R&D you know what what to prioritize or or not and in technology and and otherwise um so I'm guessing that if tech and production change so much some management practices probably will will need to to change too would you agree or how would you kind of frame the human management element of this in more detail absolutely change is so hard and it takes time and I think this is where management teams are failing rather spectacularly because managing such a drastic change across people process and Technology takes top- down support to foster a culture of change not muzzling developers that say that they need to work differently it takes support for

Budget time and training it takes or structure changes infrastructure budget again when when you start to break apart a code architecture into more components you have to invest in Automation and Management Services across that that takes time and effort and it takes time and effort to also break apart your team structure because now developers aren't working all on the same project they're working on individual components that they're responsible for their own own development operating in ships so production schedules lengthens you have to pay technical debt down across a project rather than when something goes wrong you want to be proactive rather than reactive that takes a top down focus on going through that process and supporting the people process and Technology shifts that happen over a multi-year period and it would be almost impossible and it would definitely be too much to buy off all once to say okay let's jump from

point A to point B but let's use test automation as an example if a game company wants to start moving in this direction I would not recommend saying let's do it all at once but a a management team can say look for the next title we don't think we're ready to jump into a live service free to play but let's at least start to test out some of the principles that would be needed to succeed if we did do a live service free to play Let's Start building test automation into our into how we ship games into how we run our Dev processes and if that's successful across a limited number of test scenarios we feel more comfortable for doing a full roll out of test automation across more test scenarios on our next game which would be needed to be able to achieve the velocity needed for a two or two or three week release Cadence management teams need to be driving strategic decision-making that helps the teams change not waiting for the teams to figure it out themselves without properly resourcing them or in the case of technical debt you have engines that

stretch back a decade plus layoffs and Outsourcing are incredibly detrimental to the tribal knowledge of how that proprietary code base that was built 10 years ago Works Y and th this is where management good management strategic thinking recognizing the interplay between developer efficiency and business kpis is is absolutely essential yeah so so what's the underlying people problem here then like are the managers not seeing the issues are they ignoring them or like you know most leaders of these companies you know they are pretty smart people right like they got into those roles for a reason so is it structural reasons holding them back that we need to dissect more like what what's like the real root problem that like needs to get better to enact change here so the problem is systemic right there's

so many cascading impacts of how an architecture shapes build time build time shape testing testing shapes QA QA drives player experience player experience drives LTV and RP retention so because the problem is so systemic but orgs are very siloed very few people are looking and seeing the cross org systemic nature of the problem and so there's barely ever one person that looks across beyond their own domain or beyond their own or to see how the impact affects the whole org in the whole motion of the team so that's a pretty structural and slight element also most I think in our survey more than 80% of the people that work in games never worked outside of games so a lot of these problems they're just numb to it that's how life is here and they've never seen oh what does what does software development look out outside the industry over at roblox actually

before they went through their hyperscaling motion they brought on board some folks some back in Engineers from Google and then when they had the hyperscale during Co they said you're trying to do what with this and and Roblox did the painful transformation of going over to a microservices architecture and it's how they've been able to stay relevant and capture so much player value over this long of a time um so so managers also are very set in their ways and comfortable in their ways most people who work for them are collecting a paycheck why change why do more work so this then flips over to top Management's culture of change culture of achieving more rather than culture of top down pressure to ship on time minimize budgets and cuts are around the corner so you can quickly see how the ability to change and the freedom to explore a new method of working that introduces a bit more risk to a project that topic is very hard to approach and then top managers rarely know how to shape strategy around around that to manage it effectively so you end

up right back where you started on let's just do it the old way right and I think the Roblox example is a a great one and obviously that's like an incredibly complex platform with probably a lot of tech debt in a lot of places but as you were saying like especially in certain ways they made good hard decisions that probably were painful but set them up for you know another magnitude tier of of success in the future um not to put you on the spot but are there like a couple other examples around the games industry that you would point to of teams that have oh yeah done a good job working through this I think gring it in aoup examples would be interesting Riot is a great example to look at so Nick titly and Alish Bond did a great write up about Riot's back end it was a monolithic architecture when it started but as as League built out more functionality to keep players engaged like crafting the stor War challenges unlocks and they started to serve a million then tens of millions then over a 100 million M yeah

yeah they needed to evolve to be able to quickly and nimbly deliver a new player value across a number of geographic locations and in those different locations that Partners where Partners manage The Shard they had to introduce even more complexity to manage so they did a huge shift to break apart their monolith into a microservice architecture which is great at first they built out more than 15,000 services but then they realized that their QA and load testing environments were't built to support that scale so they did the transformation then they realized again it was it was a cascading series of effects of how interconnected everything can be so they needed to invest in more infrastructure and a novel environment architecture to redefine how the whole product worked it's it's phenomenal work that they did but they made the trial by fire transition into being able to operate in a modern and effective sense and going back to the the benchmarking we did before that's why these companies perform like and software companies because they're architected and they

have the systems and processes and Automation and control systems to perform like a software company because they're operating a software product in the market with high uptime and new value being added every day and that's critical yeah that makes a lot of sense it might be interesting to contrast that I mean so you have this successful platform moving and then you have something like war zone which has 3,000 developers working out it but what's interesting is their Dev Cycles can get up to a year plus from designing a feature to shipping a uh from designing that feature to shipping it so they run multiple teams that are staggered so that players feel like releases are are happening much faster than they really are because they're just doubling down to their PR to hide it by staggering team so the cases of updates is faster than the actual development process is and so that's how bloat happens that's how that how what we call sublinear efficiency

scaling happens so long-term Tech development on inefficient Tech drives something called sublinear efficiency with developers let's say you have 10 developers and in production all 10 are working on new gameplay features after one year and several releases three developers are needed to start dealing with all that duct tape and all the new bugs so all these seven are working on new gameplay value for the player so you hire three more deaths now you have devs year two several more releases now seven devs are needed to refactor and B dependencies so you hire four more still have 10 working on new player value so now it takes 17 developers to give the same developer velocity you had at launch these technical factors Drive sublinear efficiency scaling which Drive team siiz blow blow drives Opex Opex drives profitability and that drives share price Ubisoft is a great example of that blow in the article you wrote too like when you when you look at teams

that have this incredible blow it impacts share price and I think that's the final leap that's needed for this to become prioritized for executive management because if this impacts share price it's a problem worth looking at yeah and not every brand not every IP is Call of Duty right and can get away with it um because they have a super loyal massive audience that's going to you know pay them year after year and as probably you know when we see the rising discrepancy between how costs across the industry for AAA games have just risen so much faster than than Revenue can catch up I have a feeling that um much of what you're describing is a kind of a root cause for for some of this all right well let's step back to live Services again for a moment to me it seems like there's a pretty clear industrywide pullback going on with live service games right now uh teams like

PlayStation maybe like the most obvious example are significantly pulling back and I guess to go to to your example from earlier about the stick figure this is a perfect example of you know the PlayStation management the head moving forward faster than the body could it stumbles and and then the organization has to you know kind of pull itself up and reset its priorities and so we've seen you know this team cut multiple live service projects we recently saw Concord launch and become a a massive flop um and you know it's not just PlayStation 2 we're seeing you know many of the unexpected breakout hits right now be more boxed products again um and sometimes not even from you know these large Publishers but you know are you seeing a similar Trend with live Services too right now and so and if so to what extent is that problem related to all of these things that we''ve been talking about so far yeah yeah I mean

the the trend's definitely there and I think the problem still stems from the same place you can't put a game into a business model so Triple A's want to play it safe they take franchise IP put it into a business model so so Halo they've been running the BAM engine since before 2000 and been upgrading it for every single Halo game and and that's fine it's had a very consistent look and feel but then with Halo infranet you try to take bam which is a you know sci-fi sci-fi FPS engine and turn it into a open world live service free-to-play game that that's going to cause trouble and that's why infinite went through such difficulty at having to refactor the band engine to Spa slip space yeah so so most games will have a regular update Cadence in some way shape or form you know Age of Empires too has a regular update Cadence but the go to market that depends on it like a free-to-play FPS I think that's going to regress to be one type of strategy instead of being the kind of end all be

all industry norm and absolutely young small teams with less to lose can try new things and they have an incentive to try new things versus large established triaa with again franchise IP and consistent looking feel and a game engine they're using from 10 years ago they have an incentive not to try these things so these younger smaller teams can make a game and form a business model around it which I think is fundamentally better than taking a business model and putting a game into it right right makes sense so so where do you see this going from here then yeah yeah I I I I think it's really just going to look like go to markets that are fully dependent on on live services to drive player interest and player attention just being one tool in the toolbox but on the flip side I do think more and more teams are going to learn the muscle of how to keep updating games and see the value and and lengthening the title's lifespan through reg update cadences okay yeah that makes a lot of

sense um so I want to talk about investing um and kind of apply all of that you've said to just better understand like how you think about it through through the lens of an investor but before we do that is there anything else that you want to to hit on related to you know these industrywide technical challenges before we dive in yeah you know it's been interesting to talk with a number of of strategy officer or or the teams and the CSO function has typically been one of looking at content strategy Geo expansion but under the new market conditions there's a lot of pressure to investigate production efficiency production um sustainable production but I think they're starting to encounter the same issue of well no one person contains all the answer in easy to present format it's it's something that is very systemic city across a lot of silos and I think there's now an effort to understand how development shapes

business impact and I would want to point to I I want to point to some some logic that plays out rather simply it's the tech determines your your scalability your stability your speed which is going to shape your death performance that's a game's Dev performance shapes a game scope whether you can deliver on time within budget what update Cadence is going to look like the patch failure rates that you experience the recovery times you're going to have and those shape your player experience and your player experience is fundamentally what drives LTV retention and arpo your business kpis that train of thought is where I think Focus needs to be driven towards is is is Tech death performance player experience business kpis got it okay well let's shift gears and talk about investing and uh I guess

the first big question here is just how does this thesis this view you have on the market manifest itself in the way that you invest like what are you specifically looking for or specifically avoiding um and you know that maybe shapes like how you invest in the games industry in a unique way we like to say that we want to invest not in increment Mally better Tech but fundamentally different Tech we think that young teams that can run circles around incumbent teams will have the advantage in coming to Market with products that are interesting differentiated and and try to capture unmet player needs and do so with agility and velocity so the problem with fundamentally better Tech is that the customer needs to change how they work and change is hard so we talked about this with the AAA especially aaa's they

need a lot of change management support and that means that the companies we invested need to have a go to market based around customer success not sales because you're not selling to this customer for them to buy licenses on day one and then expect them to be successful they don't they've spent their entire lives actually not using your technology you have to help them be successful with it so that year two year three they're better customers than they were on year one and that's a very very different go to market mentality than anybody in well honestly the whole 3D industry has ever had which comes from a world of $9,000 you know licenses for a 3D DCC tool doublea are better customers but have a slower Revenue ramp and honestly this is where VCS have been making it rather difficult because fund fund sizes in this in this market they're getting so big in the space and there's a lot of generalist Mega funds flooding in which is throwing return expectations and Revenue grab expectations out of whack they're not allowing Tech to Target the

right customers doublea and who have a time to Market of two years and smaller acbs they're kind of putting pressure to go after triaa so we want to invest in teams that understand that goto Market reality very very very well and want to bring fundamentally different Tech that could help enable teams to outperform incumbents I think that is our mentality when we look at companies we do some content to a minority of our portfolio but the majorities around the tool they needed to help enable that yeah could you give a couple of examples of like companies that you've seen or that you've invested in that have fit that mold I have two Darlings that we'll talk about one's we we talked earlier how building a microservice architecture especially for a backend can introduce a lot of complexity a lot of additional investment needed in systems and that's why teams like Riot had to shift from one to the other because it made sense to Lun with the monolith and then they

needed to scale and change their teex stack that used to be the case that you had to either decide to overinvestment exist they've done the hard work they've built a microservice backend product that you can take and run with without having to do the work yourself that's a huge Boon that helps you drisk your project but also enable scale and the the other would be model AI I mean test automation is every game company in five years is going to be using test Automation in some way shape or form it's not an option uh there's a really good talk a GDC talk given by Robert M A couple years back but on one title almost all game titles you see bugs the number of known bugs climb until launch because you know that you're tracking bugs but you don't have time to fix all

of them because your Dev Cycles are so spread out apart and you only catch bugs after the development do has been released so I think on this chart there were 6,000 bugs that they had to quash before launch then they had SE of Thieves which had a very strong test automation framework where they never experienced more than 200 known bugs at a time because test automation can test a game a developer's work before it gets submitted to a build before the build reaches the test and I don't know if you know this by the way but bugs become exponentially more expensive over time the longer they live by the time you catch a bug after you've released a patch it's 640 times more expensive than if you caught it when the developer made the mistake so driving test automation as early as possible around what the developer does before it infects the rest of the game project is absolutely essential to keep game costs slow and efficiency high but that means developers have to learn how to test their own code and today there's a whole

notion of just tossing that over the fence to operations to give to testers so I think pragma has done a good job making themselves well known and saying look we're here and we're ready to support your project model AI is something that almost every game company wants but they need to start realizing how much effort it takes from the customer side as well to actually go about building uh building success with a with a tool like test automation but test automation is an Absol stepping stone to devops which is a crucial part of achieving efficiency in the market yeah and and I think we actually have interviewed um both of those teams on our podcast before we we we talked to to pragma a decent while back but I think um modal AI we had even just a a pretty recent podcast from from May about AI powered quality assurance so if you're listening and are interested more in the these topics just you know type in the companies and novic and you should uh and you should see more in-depth

conversations there um well I guess to to follow up on this quickly like on the technical side of gaming like with infrastructure Dev tools um where do you foresee the largest investment opportunities and outcomes will come from in in the future related to all of this yeah this this is going back to that fundamentally different that we're looking for so so we're still looking for technical breakthroughs before companies can even be formed like polygons are a very expensive way to represent 3D structures with materials and light calculations and I I truly believe that there is a better way to represent 3D data in space than polygons we're seeing Gosa Splats and Nerfs come around and and they're exciting but you know with 2D imagery you had pixels and and raster graphics and and a 4K image can still be a couple megabytes

a 4K video can be tens if not 100 gigabytes and the reason why we have tools like figma and CA that live on the cloud that allow Graphics editing to become so easy is because of vector graphics the representation of an image as a series of math rather than the color code and XYZ of what R RBG of of pixels on a grid so polygons are expensive I think there's te veloping around how to represent 3D differently in a cheaper way and that might have implications around how much compute budget we have on boxes or whether or not cloud gaming can truly be Cloud I mean C++ is tough as well and between mic architecture C++ code and polygon being expensive anyone that's talking about Cloud today in the in the 3D Space is really just taking a program that's running locally and tossing it on a virtual machine slapping it and saying great this is cloud it's not Cloud we're very far away from being Cloud native and I think that as the core underlying

technology of 3D Graphics can change we'll see more Cloud native technology come to Market we're not even really tapping into that and from there we're in some really exciting territory right um how does AI play into all of this um I mean it's the it's the topic of conversation it's you know we're at like the top of the the hype cycle for you know AI everything um but you know as someone who's like actually deeply in the weeds of the Technologies and the the technology processes that you know all these companies big and small are running through like from what you can tell and you know based on how you're thinking about investing like what here is actually real and exciting right now versus doesn't actually interest you that much well I'm I have a Crosshair my back whichever way I turn on the topic so I'll have to bite my bite my teeth or

grip my teeth and say my answer the tech is great the use cases I think have been very overrated in almost every case it's just an incremental change gameart accelerating game art sure but you're still doing it with the same old function of polygons and traditional 3D there's some interesting stuff coming around with neural rendering but for the most part what we've seen is just incremental changes to to dialogue art crafting stuff like that open world I'm I'm more interested in through in ai's use in the graphics pipeline to replace how heavy processing is I mean faster paced inferencing might be a viable alternative to Brute Force processing everything in a scene also AI as a user in 3D simulations I mean I don't know it's not that popular knowledge but for example one of the projects we worked on was for a car company building a private data center to run unreal in the in in their private Cloud over and over and over to create

training scenarios for self-driving vehicle programs because if you need to train a car how to drive in London you can recreate London but then if you have to train it how to drive in the rain you can't just wait for a rainy day to happen so we're creating parameters and and the scaling of this went from couple hundred couple thousand 15,000 20,000 100,000 now it has more concurrent players playing 24/7 than any game in the world because you had AI becoming the user of a 3D simulation in a real engine so I think there's a very exciting Horizon there that's again fundamentally different not just an incremental change on a feature right um and do you do you think that there's room for AI or you know is it realistic at all to think about it like playing a meaningful role and you know speeding up you know some of like how these teams solve their Tech debt or you know work to build more efficient

infrastructure like you know some of these like big bottlenecks that you've been talking about you know may maybe it'll take time for you know the for it to like not be iterative and to be transformative in some new way but even if it is iterative is it still you know worth thinking about right now like like how exciting is% 100% I mean even the issue of duct tape goes way really fast when you have yeah simply generating documentation as a elra goes along and that can help so many emic issues about developers you know leaving the company or decade plus games looking back and saying what on Earth was this bit of middleware that someone did a quick hack on well now if there's easy and available access to documentation and tracing back all the dependencies around that it's a huge boom and this interesting discussion on whether or not that will lead to less developer

discipline because I can handle a lot of that workload for them rather than more discipline needed which the current day situation for creating those types of developer Norms but I think either way you look at it it's it's has a lot of potential it still comes back down to management creating a coherent strategy of how they manage that change in that process to bring that into the fold and how they create a culture of change of people are open to the idea of using it and then we can start to see the Practical applications and there'll absolutely be great outcomes for the industry around just these basic steps right one one last investing related question before we begin to to wrap up um I know you think of yourself more as a 3D Graphics investor not even necessarily like a pure games industry investor and so this means that you're also thinking about are is that are adjacent um to games and I'm just

curious really quickly like like what has that been like what are those adjacent areas and is there know we kind of talked earlier about the games industry being a bit insular sometimes not paying attention to everything else that's going on uh with with these areas that you're focused on is there anything of note that the games industry should be paying attention to yeah yeah I mean 3D Graphics itself has been built around the constraints of finite compute power on a box everything runs locally and so if that changes it has implications for games and outside but outside of games already Enterprise growth of 3D has something has been something that's been very clearly felt there's been a couple companies that were sleepy 10year companies that were doing less than 10 million in in revenue and they've all broken a 100 million there been around a 5x growth of the use of 3D outside of games in the past 5 years all consistently as as businesses have digitized or virtual Iz their workflows so the Enterprise growth of 3D

and especially 3D as a gym for AI to either create data for AI to use in the synthetic data use case or to train with reinforcement learning autonomous systems are extremely extremely extremely exciting and there's a lot of frost pollination now that's happening I mean people that worked in games are being picked up by Nike Adidas uh lbmh these these guys need people with game Studio experience to create pipelines that are fundamentally game Studio pipelines to create virtual imagery of their products even Ikea and Target are going down the way and VFX has been a huge crossover and there's some great work in the industry being done around the notion of how to more effectively create products and offerings in a cross media play using IP where the same shared pipeline to develop a game those assets can go straight into creating animations whether long form or short form so I do think it's a very gray blurred boundary and I think there's a lot of cross-pollination outside of the

games industry and from investing at a core tech level as well as watching the the applications and use cases expand and finally the Talent start to cross-pollinate I think 3D is finally becoming an investable asset class and I really credit the games industry for being the incubator and the Nexus of it and hopefully the distributor of it as well yeah no that's super fascinating I'm far less in the Weeds on uh all things 3D than than you are but even things like nvidia's Omniverse and like the idea of like the digital twins like a a computer reality that mimics the real world and you know all of the things that you can do with that it's just it's just so fascinating fascinating and I can't wait to see where all of this goes um two final very quick wrapup questions for you one um outside of you know everything we talked about today and your your technical games industry investing Focus like what

else going on in gaming are you excited about you know maybe maybe like a on like a lighter note like what what games or other Trends are you are you make you happy right now I'm loving Remnant too my friend Curtis told me about it and I looked at it and I said I don't know if I have the energy for another Souls like at Elden ring I loved Elden ring but it it took something out of me to get through that but Remnant 2 is amazing I love The Dark Tower Vibes it gives it's very closely mirroring Stephen King's works and there's something very interesting about that gameplay um I'm sure we'll be seeing a lot more Co-op PVE third party Shooters with hell diers 2 and Space Marine 2 also Paving a lot of that way in the genre but I played my way through those I think Remnant 2 has really Str Accord with me interesting I'll have to check it out I didn't realize there was that stepen King Dark Tower connection of sorts too so so that's cool um and

then finally gon if anyone wants to to reach out um to you or you know to to rendered VC in general uh where should they go yeah linkedin's always fun or email I mean it's pretty easy it's just G10 incentive to try and see more of that so so very happy to take any inbound and and give value back to an industry that gives me great Friday Saturday nights on the Xbox that's awesome well let's go ahead and and wrap up here Jon um this conversation has been a real pleasure uh I've learned a lot from you today and I hope everyone listening has as well um and and to all of our listeners thank you as always for tuning in and we'll see see you next time if you enjoyed today's episode

whether on YouTube or your favorite podcast app make sure to like subscribe comment or give a fstar review and if you want to reach out or provide feedback shoot us a note at podcast nov. or find us on Twitter and Linkedin plus if you want to learn more about what navic has to offer make sure to check out our website www.nic.edu digest or contact us to learn about our wide- ranging Consulting and advisory Services again that is www.nic.edu [Music]