Finding Games Gold In Tough Times: Ideas, Players & Funding | BITKRAFT Summit 2024

so next up we're talking about um games more games how uh do you build a good Studio how do you find players how do you validate early how do you iterate how do you scale we have um Lulu from starfarm um vagar Boos from from Stoke games and Jesse who's jumping in for unfortunately Paul his CEO got sick but Jesse knows exactly what's up inside his company so I'm sure he can cover it well so beger why don't we start with you as we kick things off you've been in the very early days at Riot um seen a little bit of the new development there um you've been through the R&D ringer at pocket gems um you've been on the IP holder side with WB then launch on game

and uh now you're doing it again with your own startup um one thing to not is that all these companies were either able to raise pretty big serious a or are monetizing very well so there's some Street credit to to what they're having to say and um vagar why didn't you tell us a little bit about your journey of um finding the initial spark finding the initial spark um so yeah so I've had a number of different experiences so Riot pocket jems mobile WB kind of focused on IP um so I think there's a number of different approaches uh I think this is also a great panel because I think uh we're going to get a lot of different perspectives from different people I'm especially interested in hearing lose Counterpoint to what I'm going to say so I'm going to I'm going to represent probably the riot perspective the most and that's very uh player focused so early on we knew we were going for DOTA players very specific audience we knew who they were where they were I came from the

community I was a huge DOTA player um and so I was just talking to them we had a forum we were very active on we did weekly play tests and we would get as many players as we could onto a ventrillo server after the play test which I think had a cap of 50 or 100 and our producers stepen snow we just ask them hey what sucked what's terrible what do we need to work on and so we had a pretty good understanding of what they cared about um and that really drove development but I think we benefited from the clar of the audience we were going after which not every game gets to enjoy uh so I think when I went to pocket gems I had some culture shock because it was one much more mass market like who is the audience oh we don't know people that show up um and they were much more data driven almost to uh in my experience the sacrifice of you know their gut I was like well what do we believe it's like well I don't know well let's see what the market says let's see what the do people retain I was like well if we don't think they'll

retain should we be putting this forward and it I think they had developed enough experience of being wrong and like finding Success Through grinding um that that is how they found success which is a very different approach and then WB I'm probably going to spend the least time on because they when I joined they had just gone through a string of what I think they thought was the conservative approach of looking at a game that's working slapping an IP on it and then releasing that and that ended up actually being disastrous because then they enter our Market years late with something that's not differentiated and no one cares um so I would classify that as a non-working uh strategy um so I I think if I were to advocate for one it would be the right approach in that you should have a very clear audience that you're making the game for you should ideally be able to pinpoint a specific person I've talked to a lot of developers that will say well you know

let's say we're making a game for TCG players that also like turnbas tactics trading card games yes and uh you know then they'll run into one that didn't like the game and they're like well not that person because they also play like action games and we don't care about action games like well they played the other two so we should probably listen to them um and yeah just talk to your players um I'll let Lou speak to probably the the pitfalls of listening to mucher players uh and maybe have some followups after Lou let's let's go to you your first company you sold successfully to king um you've been starting it again you've been in Mobile for a long time how's the funnel differently for you in terms of validating finding that with players and um how maybe have things changed a little bit with the times okay so I'm going to address that question so we actually um we were building our game with players right from the start and we had several

hundred players playing a prototype and you can you can put things in front of players much earlier than you feel comfortable with and get a lot of valuable feedback um however you also have this risk that you're getting uh like a confirmation bias with the most hardcore players and and that's um kind of the thing that we ran into is that the group of players that stayed and were vocal loved exactly what we're doing and they were just really in love with it we had players putting we had I think at one point we had like done an analysis of the engagement and engagement is really what we're looking for time um in our case uh you know flying and uh we had players that had put 500 Plus hours into the Beta so we felt like yeah this really going in the right direction and when that finally became publicly available and it was available to players that didn't know the studio didn't know the Genesis of the ideas and the vision um we found that players were bouncing bouncing off

the game and it actually caused us to go back and say you know how how can we make this more accessible did we did we get some false signal from our internal players and some false confidence ultimately and we wind up running two versions of the game side by side um one that was designed for a broader audience and one that was designed ultimately uh kind of through iteration for a smaller audience and what we did is really alienate our core Community vocally they were very very angry with the changes but their actions were that they were playing the out of the new game and so seeing two concurrent graphs side by side was super validating I don't really recommend you plan for that kind of thing it would be better to learn uh you know in other ways faster but because we were putting stuff in front of players early uh we did have the time to iterate pivot the product get to a broader audience ultimately get live and get

data from millions of players prior to launching the game um and so I think there's like some place that you can exist happily between Riot and pocket gems um and I think that's actually what would be my answer I think it's knowing your player is really really key um and when it comes to running tests like I always tell the team we can't test everything so you got to start with your intuition and then figure out if you're wrong uh we probably get 50% of our split tests do not uh align with our intuition and uh we kind of think of it like if we're not running tests and learning from players uh it's almost like a core kpi it's like what what are the things that we're trying to learn about in terms of player reaction right this minute and you know try to have two or three of those running at all times knowing that we just will get it wrong a bunch on the way to getting it right terms of how Mobile's changed um Apple doesn't help you anymore

I mean that's kind of it in in a nutshell um there have always been a ton of games I think there's no there isn't like a meaningful change in the amount of noise in the market it's just that you know organic uh I should say like Channel sponsored Discovery just is gone and so you really are kind of pushing more into the how do I connect players and get them to create a narrative together that they're willing to share outside of the game and that's actually part of the um emphasis of all of our design uh right from the very beginning is let's bring people together have them create epic moments that they're willing to share outside the game and those same epic moments are going to be the things that bind them together so we want to help players connect help them start building relationships we also know that players will come together like part of our core um activity when we were building the beta group was let's get guilds entire Guilds of players from other games to

just come over in a big group and those groups would come over 30 40 players all at once and they would stick together and they' play the game and because it was Private they wrote up details they knew the game better than we did now we have players that um have actually spent like have actually played 28,000 matches of our game so we also know the game play really really had longevity and um that's one of the things that these like weekend play I see a lot of PC game uh developers doing just doesn't tease out of that's just not a learning that you can derive from those shortterm play tests um and I think Mobile coming from mobile you think about like we're going to put it into test Market we're going to let it sit there for months and months and months and that's exposing yourself and your Creative Vision to real honesty fast I think that's key so on the note of PC um and only running this weekend play tests

Jesse you your team has probably shipped well over 25 titles together at this point um you know you've worked together across like a seven person co-founding leadership team with then bringing the Borderlands billion dollar IP to life how do you go about validating for your game at Ruckus today yeah sure so uh it's certainly a continuous process right I mean we you know we're doing user testing in some form as early as you know with our first play of demo right even if that's just among the team um however and you will continue to do that like throughout the process right it'll go from internal tests to you know the smallest external tests all the way up to like some of these guys were talking about um potentially a million players or more um but uh I think maybe what I'm hearing here is a bit of a through line which is we are very much data informed and not data driven right I think that's particularly important in a creative industry where um you know if

if if you need to effectively it's about Aesthetics and preference to some degree right um so sure you can you can take in a lot of information from players but it's not always the right decision to just you know follow effectively follow that feedback um um sort of wholesale uh so we are you know for that reason we're pretty sensitive to the the sorts of tests we do as early as we are at Ruckus right we've just finished a series a which was effectively um you know that culminated with a a a demo that's relatively close to to a vertical slice and so uh like the way we think about it is we're not looking too closely at engagement metrics until we're much closer to a complete product right I mean we've got um maybe an hour's worth of gameplay in total a little more now we've gone past that demo but right now we're very much leaning on um you know creative Instinct our experience as

developers and we'll shift that toward um again towards uh engagement metrics and sort of more objective assessment um as we go and that's that's definitely uh the sort of the learning that that our founding group um uh you know gained through working at a lot of I should have mentioned most of us well you sort of covered it mean uh our founding group we've all worked at larger AAA companies for the most part where they have entire user research departments and such and so um yeah certainly we've learned a lot from them about out how to sort of move that needle through the development process I think the other through line here is um what's really interesting if you're on mobile you can test super early very quickly you probably should if you have a PVP game you can also you know have two teams play against each other the players are the content and if you're you know session based Co-op you have a lot of progression you need to build a lot more before you can actually get meaningful signals so there's no fully right answer and you have to think

about what is the minimal awesome thing basically that you can put in front of players so you actually get that signal so it's very interesting that kind of leads into you know the the elephant in the room these days fundraising you know how do you go about it how do you make sure you show the right things to investors that they want to see um so you project that the validation steps you see are great they test your product you might not have that much monetization yet if any at all how did how how did you all pull off your your raises ultimately you want to go you want to go first you're the one here who actually sold a company before um yeah so we launched our game in November of last year um we had been live in a handful of markets um and also running a PC closed beta uh alongside our mobile launch and then on top of that running a web store that we were working on uh building uh direct relationships with play players by the time we launched we had like

500,000 players in an email uh marketing relationship with the studio lots of that we learned along the way we just said we really want to leverage player relationships with the studio we know in our past a bunch of us uh worked at a prior company Z2 which we sold to King that became part of Activision now is part of uh Microsoft uh we really believed that that uh relationship with players really matters and we kind of always told this story um if I have a you know a game in a bag and you can't see what it is and you know it's a blizzard game would you pay 50 bucks for it and we kind of Imagine everybody's going to say yes at least to the old blizzard um and so we want to build that level of trust with players and that means you know continuously kind of understanding the relationship uh building that trust means they're willing to have a relationship with us is I guess what I mean to say um so going into the raise uh we had a lot of ability to connect with players

we um we had a lot of data we had a lot of Engagement data we showed that players were making this their new hobby and as we were looking at the data we actually came up with this our own internal phrasing of how we're think about it we just want to build players next new hobby and you know that means that we can see clearly where the hours are going to go and how much time players are going to spend in the game and that uh ultimately it never ends and that's kind of one of the key things that we're trying to build toward is a product that never ends and that takes time uh you hope that it goes viral the minute it comes out but the reality is that's not a strategy uh so you focus on engagement with players and really kind of making them happy um and that's a key part of I think what we had is a lot of data we it had we had a core thesis when we started we said there are million players around the globe that will want to collect all the content in this game and then there's everyone else that's kind of a tourist and we play for a while and that kind of thing and we said this is a multi hundred million doll

game and it's just a matter of how long does it take us to realize the full potential and so um that's kind of that thesis has held strong and we've continually built proof against it uh I think for us I had like a default starting point of not wanting to work for four years on something to then discover it doesn't work um so I was very naturally aligned with with VCS from the GetGo like no I don't want to do that either um so I spent years before pitching just thinking about the space researching it getting data talking to players joining uh player communi you've also played Four thousands of hours in that genre yeah many many hours tens of thousands of matches um so by the time I was pitching I had a very clear sense of you know some understanding but then we had some thesis about how we were going to win this space and that was unclear so we

tailored our road map to just getting to a play test as soon as possible what's the minimum amount of things we need to build uh for a player to be excited about what we're doing and so we were play testing uh 5 months in with external players at 8 months in we had some of the most hardcore players in our primary audience and they wanted to invite their friends and so we let them in and then the next month they played and they wanted to invite their friends and you know we started uh giving them friend invites and we started tracking that and then we kind of averaged it out realized this implies a k factor which is a measure of virality of 1.3 anything above one grows virally so every invite you send out 1.3 players come in they're like okay well that's promising um but then to follow up on the point of talking to players and how you should have a little bit of a cynical view on what they say um so after play test I would hang out with players in Discord

for hours just chatting about you know their questions talking about where we're going with the game um and that was fine but once again I'm not going to trust what they're telling me um because actually on League of Legends of all the games I've worked on League of Legends may have gotten the worst feedback from players uh which might sound crazy um but it's because they cared they cared enough to kind of in a hostile way tell you how the game needs to be better until you fix it and I've worked on games where people say oh yeah this is fine four four out of five five out of five and then you invite them back and they don't come back it's like okay well I understand what that actually meant you kind of want the people that are giving you a three out of five over and over and over again like okay you care enough to come back um so not content with just talking to players I started recording well who's coming back month over month who's inviting friends and at

the 5-month Mark uh I looked at of our first cohort what percent came back and it was 53% I was like okay well it's like a d150 readout kind of implied it's not the same Apples to Apples because it's a PC play test it's for a one and a half hour window you know once a month but also in the middle of the workday so who really has time for that but if 53% of those people showed up that feels like we're on to something um but then even then you know it's okay this is our primary audience what if this is some kind of selection skew so let me find an older demographic let me find a younger demographic let me find players that don't play these types of games at all let me find European players because this is North America and maybe they're weird um so just trying to find every possible way I could be wrong because I don't want to discover that two years from now I want to discover it now so I think by the time we were raising again it was pretty clear that we were you know going not a theoretical Market but

a real market and that we were also hitting a sweet spot yeah and so in our case um we you know what we were hearing is is that data would be important all right rounding uh around raising our series a but we didn't feel like we could both get to the point where we had enough content at high enough Fidelity to then also conduct play tests necessary to gather accurate engagement metrics so we chose instead to uh just go very hard into proving how quickly we could get a demo to the fertility level and content completeness um required so that you know investors would have the confidence that shortly after we raised we'd be able to you know pursue engagement metrics and such and the reality is you know we had a number of investors who effectively just said no to us because we didn't have uh those metrics it was a

it was it was a big risk but um you know there were enough that were willing to say yes based on the approach that we took uh that it worked for us and really it's as as simple as that I think so many different exciting ways and paths that can lead you to Rome um how are you thinking about ultimately investing the resources that you do have in the right tools in the right infrastructure in the right launch timelines maybe we're reserving some marketing budget um to ultimately make the best use and increase odds of of success I guess if the question is how do we allocate resources um for us we talk a lot about Focus like relentlessly maybe um to the annoyance of my team because I'm constantly just coming in and asking do we need to do this like oh hey the team has run into a roadblock and my first reaction is not hey let's pour some money on or let's hire someone it's actually do we actually need to do

this um I had you know as one example I designed a piece of content that I was very happy with players really enjoyed it was great my engineering director came to me and he said hey actually this doesn't work it was like a time dilation thing this doesn't work in porn Network conditions and we can totally fix it I know how to do it but it's going to take like a month I was like well is this piece of content absolutely core to what this game game's about and it's not and so instead I decided to redesign it and it had to work with the same art and kind of fit in the same place in the game for the whole experience to work uh which is not easy but I spent like uh 3 hours on it and I came up with a new idea that players actually loved even more everyone got super excited about it and so I made a trade-off of three hours of design for one month of engineering and you know I I think there's just a lot of decisions like that you hit along the way where the

team's natural response is well at my last AAA Studio we did it this way we had a whole team of 10 people working on this so we should hire these you know nine other people um and I don't know if that's the right solution for small teams I think there's a lot of creativity you can apply um so I think that would be my my go-to for you know thinking about tooling or Tech or Staffing or you know what is absolutely necess NE yeah I think in our case um we're pretty fortunate in that our founding group uh we've all worked together in some capacity in some way um and we've all worked with similar technology throughout the years we've been you know on Unreal Engine projects for 10 years or more um and so some of the infrastructural decisions we were making were almost a given right and that is good for us in that it frees up time uh to invest

into um newer Technologies like the way that AI of course is affecting the game development process right so because we and in addition keeping up with uh some of the um you know more Cutting Edge rendering technology and things um Lumen and I the advanced Unreal Engine stuff so um so so yeah again we're fortunate in that we we sort of have some space built in to pursue that stuff and it's work really well for us um we're you know I would say as a group we're like you know we're we're we're more on the liberal side of moderate in terms of the way we adopt technology and um and so what that looks like for us so far is the adoption of um a bunch of AI stuff in the development pipeline specifically versus you know um using that technology as um as in in the sort of the core of the product and that's worked really well for us of course because it's easy to experiment with and there's um

generally there's an easy way to fall back to the traditional approaches that those Technologies help you improve on um and and so for that reason yeah I mean it's for us it's been pretty straightforward in terms of um where to allocate resources um around development infrastructure and accelerating development overall our Situation's a little more complicated so uh big Focus for us is really globalizing figuring out how to effectively Market we have really good data that says for example um 40% of our opportunity is in Asia Pacific um and we've done a lot of localization we're doing more sort of culturalization work there um including things like split testing and just really refining the product uh the new user experience uh and then beyond that um we're going to go to steam later this year or early next year and so really kind of uh

growing on the PC platform and so uh a big part of our investment is in just improving the quality of experience we call it uh first class citizen we've been live on the epic game store for about a year but that started out more as a um like a um a an Arbitrage play so doing a lot of marketing on mobile and then incentivizing them to play the game on PC give us a bunch of feedback help us refine that experience even further um and also capture a better rev share than we would have on Steam uh while we're building the audience and learning about how they're playing crossplatform because our game is fully crossplay uh crossplatform we have players literally around the globe playing together on mobile and PC together and um that is a big part of the community building and Community experience that we're trying to hit and so the next stage of that is really scaling uh globally we're not live everywhere either that's the other

piece so on that last note on distribution and reach all of you kind of have crossplatform potential in your games what are your decision points on when to take a game to Beta into full launch how much budget to reserve to throw behind maybe even working with a publishing or distribution partner putting in the engineering resources to make it work in a harsher mobile environment or or PC environment how do you how do you guys think about you know points in time in the future to make that decision the decision to launch is probably the hardest one I want to comment on that because you know we were in a place where uh Runway was getting shorter and shorter and game was Far short of our hopes and dreams at that time and I think that is like always happens 100% of the time I've never seen a Dev team launch a game that thought was actually perfect um May maybe somebody else has and and so ultimately like you know we said okay these are the things that we cannot live without and

that's what we focused on always asking the question like what's really important and high impact it's literally built into our values so we can remind each other all the time think big and focus for impact um and then you know getting a bunch of players in the funnel will get you really focused on what matters yeah I think related it's tougher we're having a lot of these sorts of conversations now right and so part of it comes down to um you know whether it's beta whether it's early access the first time you put your game out there that like that first impression is so critical there's it feels like these days there's um players don't give you a lot of leeway it's got to be really good the first you know on first touch um and so conversely just like Lou was saying um you know we we also we want to get it out there as soon as possible so that uh you know we we get that feedback it it focuses our efforts um so maybe the actual answer to your

question is well just plain I don't know yet um but um but it's certainly a very important thing for us to be considering where we're at relatively early right versus maybe these other guys yeah I mean all Echo it's it's complicated one thing I do really miss from mobile because now we're PC first is just the ability to soft launch over and over and see how things go and kind of preserve that first impression I think for PC console the first impression matters so much and we have a current Zeitgeist of people really relishing being the first to declare a game Dead and so uh you know if you go out there and you you know Miss and people declare you dead that's really hard to undo don't want to be on deconstructor of fun next week exactly they're going to really relish that um Eric's watching us I know yeah he asked me what I was working on I was like don't worry about it you can track It 2 years from now smart answer

um so I think that you just need to really watch the market uh I think there's an element of understanding the current Ze guys the current sentiment if you have an opportunity to you know ride a wave or you know somehow um enjoy the benefits of some current zeist I think that's a huge benefit but you also need to feel ready to launch at that point so I think you need to be building confidence all along the way whether that's soft launching or close betas or whatever that is generating some metrics so that when the moment comes you feel ready to say okay we're going to seize on this opportunity um if you don't do that there's always the you know the fallback of just like very slow growth of we're starting with a very small bubble of a community they're going to invite their friends they're going to invite their friends and if you really built something hopefully you don't need to nudge it too much you can see where it goes uh but yeah in terms of the you know how do you allocate

marketing dollars I think it's you just got to really watch the market and try to uh Seas an opportunity I think what we hear throughout kind of try to find early evangelists even before you're out even if you're on PC try to do that you know get even full Clans from other games in find that social validation um even if you do it behind closed doors um that's a great note to to end on um thank you so much Lou vagar Jesse put your hands together for them