Session: Accelerating Decision Making From Ideation To Live Ops – A Data-Driven Approach With Game..

[Music] Allison uh from game analytics is going to talk to us about accelerating decisions making uh um sorry decision making even decision making uh from ideation to live Ops again a topic I love to hear so Allison go for it thank you great I'll just give a minute to transition here so if you have ever struggled with trying to understand what your players want through data you're not alone I think this is a unifying factor for everyone who's sitting in the room today it's either that or you're tired and need a place to sit either way I welcome

you um I've spent many years working with Studios to try to answer these types of questions and especially now I have the great privilege of working at game analytics I need to move that chair but our logo is also here um where we work with literally thousands of Studios using our our platform uh every day and I want to attest to you today that um using data really has the power to make your decisions easier and your games better I will never forget in my career as an analyst and working with game studios specifically these moments of transformation uh if you will where we were Flying Blind but then we took a look at some data an example of that could be bejweled Blitz where we realized that a small change to a progression map could really engage a disengaged group of players and it did we worked with the game studios they made uh the changes and it made a significant impact to the game so those moments where you're not Flying Blind

you're using some data to move along your decision- making can be really exciting um that said I understand that it's really difficult to get your data in order it's it's difficult to um to start essentially so today what I want to do is give you a little bit of a framework talking about three specific areas where you can use data and maybe give you some starting points the first stage uh in terms of game development uh and where some data can be applied is in ideation so when you're trying to decide what game to make in the first place I was talking with the VC about this earlier uh today um how they look at this in terms of trying to understand the space and what you're trying to do when you're coming up with your game Concepts and how they evaluate games coming in the door is around some Market validation so trying to understand the preferences of of players that you're going for looking for underserved niches or areas with Rising demand

um also looking at your competitors directly once you have a game concept that you're considering diving into those successful games to understand what's working and what's not working to give you a more concrete example I want to um dive into some Market IQ Data Market IQ is game analytics's new ad and competitive intelligence uh tool which I recommend you can come and talk to us at our booth about it but I wanted to dive into word games specifically this is a game that I love it's a physical game called snatch it is a fast-paced competitive word building game it exists digitally a little bit out there do you know it no it's a great game um and I bring it up today because maybe you will make this game for me but we'll take a look at a little bit of data and you can make your conclusion on your own so what I did in Market IQ is start looking at downloads just literally what are the top downloaded word games that are out there I found three things about

word games uh the primary mechanic is this word building uh which snatch would be part of the themes in terms of uh just looking at the titles of the games and and what they represented were kind of half Zen half competitive the competitive being where snatch would be part of and then interestingly very few were two player had two player modes maybe this means that there's not a market for two player modes but there were some juggernauts here in this space with Words With Friends it's a Zinga franchise if you don't know it and then Scrabble go which is by scopely so there's some big competitors in the word game Space when you layer on Revenue the story deepens if you will I wanted to see if this two-player mode gave these games a leg up in terms of Revenue these are the two start games here Scrabble go and and Words With Friends um the answer is no to that it doesn't give them a per

se in terms of Revenue they're in the same Fray here and then again these word building games are the red dots here as well so word building competitive games are highly downloaded and highly Revenue generating so there's a lot of competition out there I didn't look at this data over time I think something that's interesting these games have been in the market for some time so maybe you see some shifts where you want to enter into this space that would be another way to look at this data also when you're looking in the competitive space I recommend trying to understand how much promotion is happening around these games as well so just looking at the number of advertising or the the size of advertising that is happening both of these games are in the top 20 most promoted word games that are out there so they have very large UA budgets behind them interestingly the ratio of downloads to the amount of advertising that they're doing is quite low compared to competitors in the space

so to me that means that they're retargeting with the ads that they're doing again maybe meaning that this these games are long in the tooth or a little bit old and there could be some space to come in the other thing in looking at ads is uh Less on the data side but more on the qualitative side just understanding how these games are positioning themselves in the first place um Words With Friends for example it's two-player of course they show the two ladies playing against each other I think they're two different people um but then they also really hone in onto the different modes as well and so that's kind of their positioning and how they see themselves in the market whereas this other game of Scrabble go this is really all about that two-player mode all of their ads were talking about competing against another player so again this is the kind of data you can start to get before you even write a line of code uh with some Market IQ Data as an example there's other things you can do to start Gathering data in your ideation phase running some UA campaigns as a method uh you highly popular with

Hyper casual but I do recommend doing that to understand if you can have some marketability for the game Concepts that you have building Community maybe this is more relevant for your steam or console games but getting that feedback early on and then of course play testing uh specifically and this brings me into my next section here on iteration so the next phase of analysis that you might be doing is this where you are doing your testing you're learning and your optimizing games really are not a static entity at all there's a lot of interconnected Parts in terms of your game play you have your gameplay and your metag game that was just asked about you have your monetization your progression systems it's very difficult to get all of this right on the first try and these Dynamics are changing over and over again as you release new content and as your players just want something different within your games as well so this is something that you start before launch and then it continues

pretty much forever and ever there are two methodologies that I like to highlight in terms of this phase of testing and iterating um because I say testing but really what you're trying to do is gather some meaningful data and insights around first OBS observing the data uh and then testing after that I want to change this slide actually where observation would be maybe 90% of what your effort would be this is simply looking at the patterns the trends the relationships in the data to try to understand what's going on you can do that and you can make changes to your game and see if it changes that overall trend as opposed to AB testing which is where you are doing more controlled testing hypotheses the observation can help you create what those hypotheses should be but I really highly recommend not jumping straight into AB testing before you understand the full landscape of what you're looking at there was David

Nelson I believe he's former King now building up his own studio in Malmo who said that if you are running an AB test you should almost be scared of what the results are going to be because you're testing something that significant for your game so again this iteration phase starts before launch and continues pretty much forever um I do want to talk about just finding some balance with this because there is an analysis paralysis that you can get into and as you're approaching launch you need to be considering what's going on with your budget and of course time to Market as well and finding some balance with the viability signals that you're pushing for now viability signals follow around your acquisition your engagement and your monetization themes the next three slides I'm going to go over super quickly just because of time they have a lot of detail and I'm happy to share the slid so you can go into them afterward but what they do is give a little bit of some ideas for specific pieces of data

that you can be looking at and some other areas where you can optimize again around these three three themes so acquisition how you're getting players in in the first place engagement how you're holding on to them through your game playay engagement is very expansive the types of data you can be looking at so really just a starting point and then monetization as well so all about how you are bringing money in the door to give a maybe tighter example uh of using these types of of data and kind of the testing and and um making different operation optimizations to your game I want to talk about paradoxum paradoxum develops games for Roblox they launched Tower Defense simulator in 2019 as a school project they're young uh group of kids they've had 3.4 billion visits uh since the launch of it and they were really struggling as they had this

success on Roblox to continue having you know deep engagement and retention and monetization so they integrated game analytics to look at some of these types of data they were looking at win loss ratios their win and loss counts per user what they did with that is adjust the game modes that were too easy but also the game modes that were too hard to try to bring things to the middle another thing they did was some custom heat map tracking using Game analytics as well to help read designed their Maps around where players were placing Towers they pretty much found it was almost placing the towers in the same place over and over and over again so they realized that they needed to have some more strategic or diverse placement of towers to make the game more fun they also optimized their tutorial steps just looking at the drop off in their first time user funnel and it was less about the drop off and more about the time spent at each of the different stages that they wanted to optimize they found that they were the players were they were moving through

the funnel just fine but really they were spending too much time on steps within the funnel before getting into the core gaml so that's where they did their optimizations so the last phase here is liveops and there was some interesting commentary earlier around liveops like what is this thing anyway and the Nexon CEO says that liveops is a thousand little things and I I definitely believe that it's when you're in the live op stage it's all around sustaining your engagement maximizing your retention monetization and just continuing to adapt um others on the track today I think we'll talk more deeply about this as well and there are different angles on it but what my angle is on live offs is actually around the an analysis method of segmentation so what you want to be doing is targeting diving into your data to look at specific groups of users and then understanding the distribution of those and how they behave over time as as opposed to the

high level trend of your data when you are doing this segmentation it again kind falls into the two categories of Engagement and monetization so grouping your players based on signals around engagement things like they're spending a lot of time in your game they frequently return to your game they're engaging in particular parts of your game like collecting the daily bonus or playing on a guild to try to figure out what is my signal for a highly engaged or a low engaged player and what are those players doing in terms of the rest of my ecosystem within the game and then monetization groupings as well so where are players spending what are they buying how does what is their first purchase for example and then segmenting those groups and looking at their behavior over time at game analytics we're building a new tool called our user analysis tool that helps

do this um more programmatically where you don't have to be getting into SQL we can help you build these segments and and look at them it's also something where you don't have to build the segment in the first place so sometimes you don't know exactly what segments you want to be looking at you just want to be analyzing the data after it's coming in and we're building a new feature that I'd love to talk to you about if you're interested my last Point here is just around um not sleeping on value segments as well when I hear about liveop segmentation it's like find your high value segments and build for those high value segments but I also think it's important to look at your low value segments as well these were a couple of examples actually from way back at my time at Pop cap where we were building a um what was it a where a hidden object game and what we found is that in this game that we had which was game one that the non- monetizer were just not returning to to the game and you could

think of a non- monetizer as a low value segment but when you compared to other games they were not returning as much as they should be so it was kind of trying to figure out what's going on with these non- monetizer in that that's driving them not to come back essentially so it's some a group to keep in mind as well and same with inap purchase preferences you have your low in this game we had a lot of people making low purchases this doesn't show the scale of how many were low to high but a lot lot of players would make that first purchase of a starter pack but they wouldn't move forward in terms of spending more money in the game and what we realized is that there were no coins in the starter pack so we needed to inspire them to figure out what you could do with coins by putting it in the starter pack so that we could move them through so these are some of the things that you can figure out by looking at the low value segments as well so data really is a strategy that you can use in a lot of different ways I do think it starts early it's a muscle

that you build up over time uh beginning in your ideation phase through your launch is your testing and learning and kind of into your live Ops a direction as well particularly when your game has some success uh I hope today inspired you to get started with using data if you're not already if you are I hope it inspired you to add a new methodology uh I would love to speak to each of you I'm also on the panel later um we are having a webinar as well on this topic next week with a couple of Studios if you're interested in hearing a bit more from that angle as well Thomas who is here can't give you a flyer about it on your way out the door but I do welcome any questions as well thank you very much [Applause]