How to Reduce Churn and Keep Players Playing

Welcome to the Naavik Gaming Podcast, where it is our mission to explore the business and future of video games. We bring together the industry's brightest builders, investors, and thinkers to keep a pulse on current events, dissect emerging trends in games, share lessons learned, and have a great time. This podcast is also part of Naavik's growing ecosystem, which ranges from free and premium research to consulting and advisory services. For more information, visit www.naavik.co. This episode is brought to you by Overwolf. With over 1,500 games supported, 178,000 creators, and 113 million monthly active users, Overwolf is the guild for in-game creators. Whether you're a gamer, creator, or game studio, Overwolf is the ultimate destination for integrating UGC

into games. For game studios, Overwolf offers CurseForge for studios, a white-label solution that lets game makers and publishers easily integrate mods safely and seamlessly into their games, both existing and new, at zero cost. It's battle-tested by AAA studios and games, including Krafton with Inzone, Studio Wildcard with Ark, Take-Two Interactive, and others. Alongside this, Overwolf is the all-in-one platform that enables creators to build, distribute, and monetize in-game apps, mods, and game servers. In 2024, Overwolf paid over $240 million to in-game creators, proving that they truly value the talents and contributions of the gaming community. You can check out all Overwolf has to offer at overwolf.com or simply check out the link in the show notes. Now, let's jump into the episode.
Hello, everyone. I'm your host, Devin Becker, and today I'm delighted to be joined by Elad Levy, founder and CTO of Dive. Dive is a data and analytics

platform that provides game studios with business intelligence and live ops tools to help them make better understand their data and make better decisions, scale their games, and maximize monetization. And today, we're going to discuss tracking, identifying, and handling player churn. Now, you've been on the the Navic podcast here before, but could you just give a quick overview of yourself at Dive for those who hadn't already heard in those previous interviews?
Yes, of course. So, I've been working for the past 25 years as a CTO and the founder in the games industry. In my last company, a game called House of Fun, is we got to a huge scale and ended up selling it to Playtika, where I stayed working for a few more years and implementing our secret sauce of managing data and live ops and fun stuff like that.
Cool.
Yeah, I've had the luxury of living the churn problems from inside a large studio.

And that's that was a great insight because today, what we do with Dive, among other things, is help game studios with the with churn.
What does Dive do specifically with churn?
So, the company, um, first of all, the the company gives the game studio kind of like a let's call it enterprise-grade data and live ops capability without building the whole internal team in-house, which will take them years, you know. Yeah, I spent my career staring at player data and working with player data, so churn is a very, uh, misunderstood problem in this industry, which is, you know, what I want to we can have a very long talk about today. Most studios cannot diagnose churn

because their their data is fragmented. It's it's a very common problem actually outside the games industry as well. So sessions would be in one place and the crashes would be in another place and monetization would be in a third place. And what we do is we unify all those sources to a single place and then we start looking for correlation if there is like relationship between something that one something that happened in one place to the churn. And that's that's what we do. You cannot fix churn if you cannot see it, you know, and this is probably one of the biggest problem of the game studios. If their data is scattered all over the place and they cannot see it properly, though it's it's very hard for them to realize what is happening with the game.
How do you guys define churn exactly?
Games that are young or just started,

we would probably define churn like very early like in a matter of like 1 week or something like that. Never days because some players would play in the weekends obviously, you know, some players it's it's very common in Asia, you know, people work throughout the week, it's okay and they play on the weekend. So if someone plays in the week in the weekend and he didn't connect for a few days, it doesn't mean he churned, it just means that he is busy working. So um we find that pattern, okay, which is like let's call it a range, okay? And it can be I don't know, like 1 week of inactivity or 2 weeks of inactivity. But games that are more mature have a lot of reactivation campaigns for dormant players.

So, we would define churn there like in every 3 weeks or 1 month, something like that. It's very different from the one game to the other game. Also, the type of game, obviously. But it's something that we work with every game studio. What we like I'm I'm just going to add one small thing. I like looking at actually something we called LTC, which is likely to churn. That are players that are lapsing. Someone who behaved in a like a player that behaved in a specific way, like, I don't know, playing on the weekends or have long sessions and all of a sudden they have shorter sessions or they don't connect that often, then we would probably flag them, you know, as like a possible We call them LTC, likely to churn. Because they're not permanent. It's not a permanent churn. It's lapsing. Let's call it a lapsing churn.

I mean, churn is obviously something that you guys are looking at, but what makes it so important to actually have like things like likely to churn and things like that, especially in the current gaming environment?
Yeah, so the the current environment, and that's the biggest challenge, the user acquisition exploded. And the market is saturated. Plus the privacy changes of, you know, ATT and IDFA, they got it tracking. So, they killed retargeting. And studios lost the ability to track and retarget lapsed players across apps. So, winning people back is way, way more harder than it used to be, way more expensive. The logic is brutal, actually. Acquisition is expensive and you can barely retarget the ones that you lose, which makes not losing them in the first place the only profitable lever you've got left. So, retention is like the new growth as I see it because it used to be,

you know, many, many moons ago you could buy your way out of churn or, you know, retarget somehow or just acquire more, but it's not working anymore. Every one of those players is really valuable. Plus, if you identify them early and put them into buckets and you see that they're like highly engaged or paying players, those are the ones that you need to keep an eye on close, close so that they don't Uh so, you know, if they are starting to lapse or not connecting as often or not engaging as often, it's something you have to do like right away.
Well, besides like not connecting as much, like the or as often, what are some other like key indicators that players are likely to churn?
So, uh yeah, I mean the session's frequency is probably one of the biggest. If you see a decline in that, that's the biggest uh

uh indicator, I guess. If the session length, if there's a stalled progression, so for example, you're stuck on the same level for weeks and you used to have like uh All right. We There's a dashboard that we create that is um content consumption. Like we How many like levels, for example, or scenes or whatever you consume per day. And if that momentum starts slowing down, then that's a flag from our point of view. Also, reduced spending, obviously, if someone has a deposit habit, we call it the paying habit of I don't know, the average is I don't know, $5 a week or something like that, and then all of a sudden he stops spending. And then the best and really cool indicator that we can talk about later if you want is the last session. Which is like the last session before he started lapsing or churning.

Uh in our platform, we record everything that happened in that session. Like in every session, we record everything there. The pop-ups, the engagements, the clicks, the game behavior, the play behavior, the session length. So sometimes the last session tells what happened uh right before the player left. So all that all those signals are before they leave, which means that churn is quite often preventable if you're watching if you're watching the signals, you know.
Well, that sounds useful. Uh what are you What are usually the biggest causes you see for churn? Especially ones that developers can address like ahead of time uh to prevent that sort of thing.
I think the easiest one is anything related to onboarding and loading. So some players, they churn even before

the game starts because for example, there's you know uh we had a client that we pulled the churned cohort of the last session and we found that the huge percentage didn't even clear the loading screen. So that was probably, you know, loading too too many assets, it's too heavy. Uh it's a very easy optimization, by the way, because you can background load all that stuff. So, that's one of the biggest uh problems which affect obviously day one because it's you know, you just installed the game and you cannot start loading heavy assets in the beginning. You want to let the player get in and play as fast as possible. So, we had a client that we worked on that for a for a while. A- another client recently that we worked with, they had too many pop-ups.

So, they call it pop-up blindness. So, uh like you just installed the game and you get like a gazillion pop-ups with um you know, pay this and upsell that. And if you if you show it in an ad, it's even worse. So, this is a definitely something that we see that um uh those revenue boosters is something that it's okay, but you need to sprinkle them in a very subtle way. We did an experiment a few years back, I think, in Playtika or with another client that instead of using a lot of like aggressive pop-ups, we used slidings. You know, something that will just slide in from the bottom or from the side. It will show for a couple of seconds and it will disappear. It's more subtle. It's more gentle.

And it's less invasive. So, crashes are probably the biggest one and people do not understand the impact. The impact is huge, especially when you launch a game and it's not polished. So, you have a ton of crashes. So, we we connect with you know, Crashlytics or Bugsnag, Sentry, whatever tool you use for crash reporting. And then we start finding correlations between the crashes and the group that churned. And the correlation is quite high, so we we we we fixed those many times many clients. And I'll just summarize that the biggest causes are usually not the game content or the game itself. It's friction or technical problems or aggressive monetization, and they're all addressable if you just see them in the data, you know.

So, when when people churn like obviously there's all the reasons that are like more technical stuff. When players are churning for maybe behavioral reasons or whatever or like like someone that's been playing for a while and churning, are there some typical interventions that can prevent like say giving them some extra currency, making a level easier? What are the typical things you you could do to prevent those like longer-term players from churning?
The yeah, the easiest would be to smooth the onboarding loading funnels. That's a very high ROI fix and many ignore it, unfortunately. And then the more deeper analysis, which is related to the content content consumption and the progression systems, which is a great that's one of my favorite dashboards, progression systems. How long does it take for a player to reach a specific level, and we measure that in days, in sessions, and in playtime. So, sometimes the content gets consumed

super fast, especially if the game just started and they don't have enough content, and then people would ask me, "Hey, but we don't have a I know D14 is like drop." And I say, "Yes, of course, because by D8 all the content is totally consumed, so how why would someone stay in the game?" Um and you know, level-based it going to be or like milestone based, it can be economy walls, what we call economy walls as well, obviously. And those are places that you can inject even a temporary, you know, like a a temporary uh, you know, power up or something like that to help the player pass that obstacle. My my mom, I always give my example of my mom that still play Candy Crush. Uh, she's um uh, she figured out that when she cannot pass a level, she just stops

playing the game and after a few days it let it it let her pass. And it's fine because, you know, they they want there are a lot of like data people and monetization live ops guys that work behind the curtains and help you pass so that you you continue in being engaged with the game. Uh, so uh, that's another one. Uh, pacing monetization, that's a very important. So, don't don't hit new players with paywalls super fast. You know, just let them be, let them play and then you can start in a subtle way, you know. If they want, they can go to the coin store. Over time, you can maybe offer like a one-time deal. Uh, and then lower it over time because sometimes, you know, I like starting the coin store at let's say $5. But then you go on lowering it with like these one-time offers. No need to do it every day, but, you know, once a week or

something or the next time that the player connects, lower it to four or three and eventually to one to see if you can manage to convert them. So, the monetization pacing, I call it monetization pacing, that's another smart way to address that and prevent that churn. And the target re-engagement, I mean, you cannot do retargeting. You can probably on some networks, but it's not that popular as it used to be. Uh but, if you find those signals before time beforehand, you can, you know, give a well-timed reward or a notification or some sort of a content drop, like something new. And the the idea is, in an essence, to intervene as early as possible on the at-risk factor, you know, on the LTC, and not after they've left, because

after they've left, you lost you lost the player. It's really hard getting him them back, you know, you can if they have if you have their email, maybe you can send them an email, but if they deleted the game, there's no you can't even send a push notification. So, it it sucks, you know, you basically lost the the player. It's really hard getting them back. And if retargeting is not exactly working as it used to be, then it you cannot just, you know, show them a retarget ads in Facebook or something, because privacy killed retargeting. Privacy killed everything.
So, how do you see reacquisition then like in the modern days? It's just hope you have an email or push notification or like what are people doing to be able to try and get these players back?
Well, the like I said, the lip the lapsed players are the best target. Uh and they already know the game and they like the game, so winning them back is way easier than

acquiring cold. It's it's way, way simple to work with someone with a a player that, you know, played the game and got sick of it for some reason than to try and onboard a completely new player. You need to segment them, obviously, because maybe it's not worth chasing every single player. You know, like a one-session tourist is not really worth it, but someone that was really engaged and then stopped playing out of the blue, that's is like this is like 100% someone I would chase, you know? You can use data to track who to win back and what drove them away so you can create this win-back offer that can address the real reason, you know, we fixed the bug, here is the reward. Or, you know, give rewards even for free just to keep them engaged. And the reacquisition, let's call it

reacquisition for the sake of the definition even though it's not exactly retargeting. The reacquisition will work if you fix the problem. Because if the game crashes all the time, they will eventually go away, you know? I've had I've had games and I worked on games that uh we, you know, players were churning because of a technical problems then we fixed them and sent them an email or a push and they came back and they played and everything was fine.
Well, I imagine too, like paying players are a bigger target than uh ones that were just kind of free players that maybe not as valuable to bring back. Um so, I imagine paying players are ones where it's like okay, well, we'll give you really good offers, we'll do something that like put an effort, maybe even multiple attempts to bring them back.
Yeah, it's it's the the rules of the game change. You cannot you know, because privacy means that you

cannot use you cannot cheaply retarget lapsed players anymore. You have to win them back in a more smarter and more data-driven way, you know? It's It's a different reality today. You cannot afford for your players to churn, basically. It's just
Yeah, it's too expensive to get them to get them in the first place. Yeah.
Exactly. Exactly.
And do you ever Do you see a way for us to get back to be able to cheaply acquire players or is that that just pretty much over with?
You can use prediction churn models. That's That's like You know, if you use those signals and you test them over time, like over few years, you can find those signals and then intervene earlier to avoid even like getting into that problem in the first place. Um I think that real-time personalization, like real-time personalized intervention, is also a cool option that you can

um target someone if you can in that session see that something is going to happen. And the game will adapt per player, you know, the difficulty will be dynamic, the content and the offers will be personalized. And AI is kind of democratizing this in a way, you know, because historically big studios Only big studios could afford it. But today, it's more, you know, I mean, we do our thing, but you can if you have the patience, you can build everything in house, even though it will take a while. AI churn prediction That's the last thing that I want to say about it. AI churn prediction is only as good as the data underneath it, you know, because if you collect garbage, it's just garbage in, garbage out. Uh but the studios that win will be the ones that get their, you know, data house in order.

It affects everything. It's one of the first thing I I work with we work with every game studio. We explain them that the foundation accuracy of the data is the most important thing because everything depends on it. The you know, churn KPIs, ROI, ROAS, all those calculations, if you want them to be really super accurate, you must get every all the data accurate as well.
Well, tell me more about the the last session recording you mentioned earlier. That sounds like something really useful beyond just the AI for trying to maybe find trends and things like that. You mentioned the pop-ups being one thing you've seen. Like where where a lot of people getting those pop-ups causing churn. Just tell me more about what you've learned from that whole process and and how that last session recording works.
The SDK that we implemented the game when you implement it then you record what screen the does the user see in every given

moment. And the pop-ups that you see and if they close the if they click the X button or the blah blah button or whatever. Plus we track in every moment, even from the login, we track the exact level and the exact balance that they have. So, when you gather that group and we did that exercise a few times which makes our life easier. When you take those sessions and you take Claude for example and explain Claude the context of the game and say, "This group stopped playing." Then you can learn a lot from because because obviously it can analyze data faster than you. Instead of you going session by session, it starts finding things in common for these users. So, for example, their balance was too low. They were stuck at same level. Uh they all had a crash just before they left or

something like that or their last session was a crash, so they stopped playing after that. So, uh it we leverage AI a lot internally to help us accelerate analysis of churn and predicting churn.
Yeah, no, it's uh it sounds like a really useful feature. Um with that being something that you guys are doing that's like a a maybe a bit of a newer thing, where do you see kind of the future of some of this stuff going with player churn, uh reacquisition, like identifying play patterns that might lead to churn, that LTC stuff, like all that sort of stuff? Where do you see that going in the future maybe over the next 3 to 5 years?
The one was the predictive churn models. Uh AI predicting who will churn before the obvious signs, so you can intervene earlier. Uh another is the real-time personalized intervention, so the game will adapt the player in terms of difficulty or content or offers. AI is basically

democratizing this whole all this whole world because we used to use you do this like in big studios like Playtika and really big companies had the skill set and the money and the resources to run that, but today you can even run it on small game studios. With us or without or without assets.
You mentioned um Claude. Is is that the model that you find works the best for doing some of this analysis?
I I love Claude for analysis, like more um code writing code, analysis, stuff that is more GPT for me is more of like an AI assistant, like if I need to know plan a trip with the family and look for places to eat. But Claude is when it comes to work

I personally actually the developers in the data analyst in the company, they all work with Claude. They They prefer it. It's more direct. It's more I don't aggressive, more concise. Not a lot of blah blah blah blah blah, you know, doesn't repeat everything you say and then
It makes sense, yeah. It's definitely one of those things where it's a little more of on the technical like kind of brief side of things that's helpful. So, that's good.
Yeah, more efficient. More efficient and the technical people we always we're obsessed with efficiency, so it's just it's easier for us to if, you know, we we downsize the team actually because less people can manage more clients today. This is the the impact the huge impact that you're seeing all over the place.
Makes sense. Definitely something I consider like especially as the whole AI space evolves. Um just couple quick last questions I

think are, you know, just things that uh you know feel free to just sit you know, say whatever it could be controversial it's fine. Just keep it interesting. But what is the dumbest reason you've seen for players to quit games?
Loading screen takes too too long. Like a 3 seconds, you know, sometimes it's like a 1 second too long and they're like they Oh, this sucks and they just, you know, like will throw it away.
Wow. Like like loading or downloading?
Yeah, loading. Loading or down If it starts downloading it's even worse, but uh you know, it's very common for people to rage quit, you know, the game that they love because it's just it's a slow boot, you know, I I see my my son is also I see him playing in if something that new generation if something's take like half a millisecond more than it should we all they already lose their patience you know they don't they don't have time for that it's like oh that sucks that

and they you know they like judge it on the spot that's pure crap
It's too much there's other games
It's too much exactly exactly
when they're all free
exactly exactly it's a very tight competition yeah so
that seems like a an unfortunate thing where it's just like optimize and and make sure that that doesn't happen outside of like the optimization and preventing crashes and bugs and stuff like that what do you think is the most important thing games can do to make sure they retain players?
I think that the experts in this industry are probably hyper casual games in that sense because they get you to the core loop to the fun part as soon as possible and people will you know if the game is fun people will stay and will play it it's okay you know if you have enough content they will play it for many many days and months and weeks and years to come but

the the mechanic the core loop try to get them to that core loop as soon as possible you know without all the pop-ups and stuff like that so that they can understand what the game is all about definitely nothing of those you know lying ads that you know show you one game and then you get another game but yeah I think that that's what probably be the easiest thing you know just it's really easy it's a really easy fix as well you know just load it fast you know boom it loads fast you play it and that's it remember that the platform dictates your patience because if Fortnite takes a little bit longer to load, it's okay. You're sitting on a couch with the remote to like with the any with the PlayStation with the PS remote and everything is good. It's okay. But if you are in a subway

and the next stop, you know that you're going to lose signal or something like like the reception or you're in the doctor and he's about to call you in a minute. No, you don't have time. You wanted to like load, boom, get your doses and move on, you know?
Yeah, that makes sense. Not just that, but the core loop needs to be pretty pretty short then, too, right?
Exactly, yeah.
I mean, do you see games where like they get you into the game very fast to start with, but then after that the pace slows down and those are able to keep players or is it like does it need to always stay pretty quick and pretty pretty on it?
That's a great question. I mean that I think that once you get like really hooked, like really obsessed es- especially if you have that uh if you're that type of person then it's a form of art to drag you to the next milestone and the mi- next milestone, you know? I mean, I I used to play Idle Miner Tycoon a lot and at some point it started repeating itself and I got sick

of it, but then one day I wake up and I realize that they created like a whole new thing and this they create like a world. So, that that shaft, that mine that you were working on it's like just like a comma on the balance sheet, you know? It's just like a tiny dot in this whole world full of mines. And that's like, "Whoa!" They opened I don't know how they they s- managed to revive that like stretch that content further, so really cool. My kids play it a lot, actually.
Nice. Uh well, then besides that game, what game do you think does a a really good job of retaining players and preventing them from churning? I mean, you mentioned like Candy Crush, like just what's a good example game of like this game does a good job of keeping people around?
I would differentiate between fun games, casual fun games, and games that somehow have a deeper way

to engage or stretch the content or something like that. So, you know, like a casual fun game would be, I don't know, Pixel Flow, I guess, something like that. You know, uh that would be a more light and like and I need to burn time kind of game. And then all those infinite builders, you know, some some owners, you know, like like a Ice uh Ice Age, what was that game? It was a Century made that that's a amazing game. It's just endless, you know. Right. All those endless builders, obviously, super cell. You know, I used to play Boom Beach for like months.
Yeah, we see a trend of games like trying to kind of bait you in with that easy core loop you were talking about for the day one to day maybe day seven. And then they turn into a 4X game.

Exactly.
that's that's the way they're keeping you around. Is that 4X game loop like you were talking about with even with the the timers from the um idle tycoon ones where they're just kind of
Yeah.
it drags it out, but as long as you don't get fed up, then you stick around, right? And then they've got to try and get you back.
Well, think about it that way. Every game caters to a certain group of people. And the challenge in the games industry is finding whether you can scale that group of people or not. So, I don't know. There was this really old game called uh I think it was it was kind of a very like a blackjack RPG. Really weird really really strange game. Super fun. Like I think that I me and like another friend of mine we were the only one

playing it I think or something like that. But um that's why many games they use like a easy or known mechanic because this way they know that the mechanic already proved that it caters to many people.
Right.
Yeah, for example, I don't know. If you take Gardenscapes one or you know, any of those Playrix that has the game is not the match three. The game is the it's the story. You're unfolding a story. Uh so my mom I told her try give it a shot and she's like it's boring because she wants she likes she enjoys consuming content and playing levels. While someone else enjoys the story and uses that as an excuse to move on to the next level.
Right.
So, it's it's really different I guess. But uh yeah.
All right.

exactly what you mean with those ads because I fall in this those ads all the time. I install those dumb games practically every week. Seriously. And my kids make make make fun of me because I I love them, you know. I just I don't know. I I I never have the time and I need to like burn time waiting in the bank or at the doctor and then like
Right.
or stuck somewhere and you know, there's like a I don't know. There's a color blocker. I'm looking at my phone now. Color blocker blah blah something. Maze fill IQ boost. I don't know like a lot of stuff. Uh back to your question. We have one of our biggest clients is um It's a company called Blue Bat and we provide through them um we help with the BI and live ops for the Hard Rock Group. The Hard Rock Group in the in Florida. So they have a resorts

like hotels and you know stadiums. They have arenas and stadiums. And they have shows so they found that they can engage those players. And I don't know if you win a tournament you get I don't know $5 in the buffet or like a a concert tickets or something fun like that. Uh there was another company that they used to do that. I I I remember the CEO. I don't remember the company name. That they did that with the sports goods, you know, you can you know win sneakers or something like that. And that that's cool, you know, that's like that will definitely stretch people to play. I mean some people will play out of you know obsession because they really want to pass that level like a flapping bird kind of thing. And other people would you know they want to win something and sometimes

uh I know I see my kids playing and they will grind hours to just get you know like $1 or something like that, you know.
Wow.
They would just you know they watch all the ads and click all the rewards and do everything so that they can I guess this is why Benjamin's that app became so successful. Because lots of people are, uh I don't know, grinding. And I don't even want to talk. We had a huge client of Solitaire with money Solitaire as well. And there is a real money, you know, match three. And those games are really getting popular lately because people are like, hey, all right, you can win a little if you can win a few a few dollars playing, that sounds cool. Plus it gives you some sort of a rush.
Right.
Back in the days it was illegal, but somehow magically all of a sudden Apple and Google are now allowing it.
Yeah.
They don't call it gambling, they call

it RMG, real money gaming. It's kind of like it's own category, but
Right. Is that a good way to prevent journeys? Is essentially be paying people to stick around?
It's amazing, yeah. It is. It's great.
Yeah, it's I mean, it's kind of like you mentioned casino stuff, and it's like they obviously are kind of a master of keeping people around and spending money.
Yeah.
But they also like dip into that with like free play credits and like you said the buffet, doing things to like kind of throw out those carrots
Yeah.
Exactly. That's why Playtika was a great school for many monetization managers and live ops and data people because they do it all day long, you know, across all their brands. And, you know, there's when you have a big portfolio you can start doing cross marketing. So, oh, you got sick from this game, it's okay, we have another game for you.
Right.
And you don't need to acquire because you already acquired them. You just show them like a pop-up that moves them from one game to another game that is in your

same portfolio.
Yeah.
So, it's it's cool, you know. And back in the days Playtika did like a rewards program, it didn't work. I didn't see a lot of companies doing that, which kind of sucks. Um, like if you play our games, our portfolio games, you will get like loyalty points, you know, like like the airline companies that you have like SkyTeam would connect several airliners companies. So, you want to fly those companies as you're gaining a points. And you can redeem them for whatever. So,
Right.
I think that this is like a really big evolution happening in those last years. Make some games. Yeah.
I mean, it sounds to me like one of the things that I've noticed about a lot of the stuff you said is that finding that likely to churn indicator is really important because then there's all these things you could try and do
Exactly.
people are out while you still have

access to them. Even if it's just that, "Hey, move on to another one of our games if you're just not enjoying it."
Exactly the same exactly. If you smell if you can predict that as soon prediction, it's a very like you cannot do prediction with the data of it, you know, 2 weeks. I need months and sometimes years because when you run the prediction algorithm and calculate the scores and validate those scores, that's how you do it in data. You you need historical data because you have an assumption and you need to prove it.
Right.
And you basically take a group of people that churned last year. Okay, fine. Now, let's take all their I don't know. Behavior players, game data the year before that or 6 months before that and try to test assumptions. Was it crashes? Was it pop-ups? Was it the coins balance? Was

it a difficult level? Was it this? Was it that? And then out of the assumption that I checked, let's say I'm throwing 20 or 50 assumptions. How many actually churned and that will like what's their score rate? Then you test that on another group and another group and another group and if it works you have you can predict churn. It's not that complicated if you know what you're doing.
Do you ever try anything about contacting any of the players to try and find out like just as a sampler? Like a couple of people be like what what caused you to churn or some way of like finding out like specific answers from people as a sample?
I had the I had this in I had this actually in no in Playtika. Um we had this like amazing customer support a team that had like a live chat. And it worked really cool because when

when you one of the biggest important like things that you need to do with those with the players that you have is create a community. Discord, Facebook, I don't know whatever. Just create some sort of a community so that they can vent, you know, they can write oh this game sucks because of this because of that. And sometimes it's better than data because it gives you like this weapon like the guy that is using the this and that weapon kills everyone and it it ruins the game for everyone.
Right.
So you cannot take that you cannot take gather that from data. You need like another human being to that play the game for hours and tell you what's going on.
Do you just do like a churn analysis where you're seeing like a trend of churn and then try and look for that stuff where you're like hey it's not an individual player thing it's like a something's wrong with the the game more broadly sort of thing?
I think that most of the techniques that we use are mostly focusing on crashes, content

and sessions data that we gather. But like I said I started saying before that in Playtika, I had uh we had a funnel loading funnel. And that's that's one of the This is like challenges that big companies face. It's not you know, like uh loading funnels in a new game would probably be, I don't know, 80% whatever. Playtika, it's like 96 or 97. We had like a really really amazing. Like it was You start fighting for the 0.5%, you know?
Right.
Optimizing. But, when you have when your DAU is high that 0.something is thousands, all right?
So, we had this problem on the loading that we couldn't figure out what is causing it, and we saw players churning because of it.

And I was so frustrated that I asked customer support. I told them, "I think that we have a group of players that is churning because the game is not loading properly. So, every time someone contacts We put a live chat. That's one of the things I tell all game studios. Make it easy for the player to complain. Because this way you can talk to them and understand what's going on.
Right.
So, I asked them I asked the the customer support. I told her, "Listen, next time someone complains about loading, I don't care at what time of the day, offer them like a gazillion coins for free if they let me do a remote session with them and figure out what's going on. And obviously a lot of people thought

that, you know, what the I don't know know what I don't know what this guy is going to connect to my computer and start touching stuff, but one guy said yes. And I I Hi, my name is I'm the CTO of a I understand that you love our game, but we realized that it's not loading sometimes and I would like to help you. And he was actually really happy because, you know, all of a sudden like if Think about it from your point of as a user. If you use I don't know If you use Spotify or something and something doesn't work and someone will reach out to you actually to get your opinion. And I fill out those forms. I'm one of those nerds that when I find the bug in Spotify I will click the contact us because I know there's someone on the other side that cares about it and polish those bugs and when we fixed I mean, I took over his notebook. It was like, I don't know, they woke me up at like, I don't know, 2:00 a.m. or 3:00 a.m. in the middle of

the night because he was from, I don't know, PST time zone whatever. And we find it like, oh my god, it's like the dumbest thing. It's always the dumbest thing ever. And then the next day we patched, we released a hotfix to production and all of a sudden thousands of people started going through the funnel. It's like magic. You know? So, it's just it's amazing. It's really amazing and you move Imagine that if you move 1% 2% you know, like a a tiny bit in D1 it affects D7, it affects D14, it affects D28 like So, that onboarding is really precious.
Yeah, I've definitely done that analysis myself where you're like looking at Okay, like we're you know, we're acquiring new players, but they're not really sticking around that long. Where is everyone kind of stuck at or stopping at? And it's like sometimes you find it's a technical thing, sometimes you find it's a

There is really There is a reason for it. Trust me, there is a reason for everything. Most of the time you can get it from the data. Some of the time you won't, and if you have a channel, like a Discord channel or something like that, or somewhere that you can talk to the players, that would be even better because it will help you understand faster what's going on.
Do you look at like user reviews then for like the one stars and things like that?
Yes, of course. We used to have in Playtika, we have we had this uh process that every morning uh someone would manually gather all the one-star reviews from all the stores and create a document and send it to the product managers. And then the product managers would go one by one with the QA trying to figure out is it a bug? Is it a problem? Products are perfect because someone worked on them, you know?

Someone polished them. And the polish polishing a product to perfection is is art, like seriously. This is when something It's It's so funny, but most of us, we just want things to just work.
Right.
We don't We don't want more than that, you know? Sometimes we don't need more than that. We just want it to work.
Right.
You know, the like if I click on a song on Spotify, I want it to play. If you know, the the basic function that it was designed to do, you know? That's all I want. That polish takes time and takes time.
Well, it sounds like it's even more important in like the live ops world because you can break that polish down. Like you could have polished up the launch version of the game perfectly, but some piece of content 6 months in, a year in, or more could somehow break something about the game or change the way that it's balanced, things like that. And suddenly you've got to look at player churn as a fresh thing. So you can't really just say you're done. Okay,

we've we've optimized the game, the loading times are fine, all the crashes are fixed. We're going to keep updating the game and hope it stays that way. You have to keep looking for that, right? You have to kind of keep those as you mentioned dashboards.
If I can add one little bit, someone likes some people likes calling it the GAS, G A S, game as a service and it's so true because it never ends.
Yes.
It never ends, seriously. I mean uh Look at the stats of Candy Crush. The revenue is still going up. And you think they would saturate the audience and everything, but somehow they find new ways to you know to upsell and convert and optimize. And such an old game, it works amazing.
I guess that's the value of re- reinvesting your money that you're making in making sure you could keep making more.
Yeah, many many game studios, they tempt to like create another game and another game and if you

found something that clicks, just scale it, you know?
Right.
It's really hard find finding that something that clicks. Seriously, like really really hard.
Are you finding that people are using some of these tools that you're talking about in like the soft launch period a lot then as well? Like with the period when you're maybe more most concerned with polish and and scalability?
I think we had a podcast about that, you know, about soft launching.
Yeah.
But yeah, I mean crashes are definitely number one priority. And every game is different. For example, you know, um We had this game that they had like this PvP mechanism. And they launched, but there was not enough players to match with each with each other and the bots were kind of sucky, so players started churning. So, we created dashboards of like uh matches versus players like mark

matches versus human and matches versus bots. And then we started to check if matches versus bot leads to churn.
Right.
Because you realize that it's a bot, you know, you see some patterns repeat or you know, something that is really obvious, you should have done it and you don't do it or like what the you know, like
Yeah. I mean it sounds like there's some stuff that's obvious that the data is just going to tell you and then there's some stuff where you have to actually make the data tell you what you're looking for like instead of just expecting it to show up, right?
It's not my quote. Uh it's someone else made that quote from the industry, but games are part art, part science.
Right. So, dash dashboards are the science part, right?
Yeah, data, live ops, the tech parts, of course. Yeah, there's a lot of tech parts. It's one of the most challenging industries in terms of scale

and complexity.
Hopefully we can figure out some things that that work more um consistently outside of just uh improving the loading times and and maybe get back to being able to keep players around longer because it does seem like probably not fixing the player acquisition anytime soon and you got to got to keep people playing as long as we can especially with so many games out there now and then I
It's really easy to test. It's really really easy to test. So, in dive, sorry, I'm just going to finish with that. Uh at some point we realized that data is not enough and we started creating the whole live ops dashboard that we have on segmentation and we added AB testing, and we had the the campaign management. And I think that AB testing is probably the biggest thing because you get rid of pop-ups for I don't know, 10% of the population. You change the loading funnel for 10% of the population. You see what happens with

that group. You track it. Did they retain better? Did they not? Did their retention was better? Did they end up paying? Did their session length was higher? So, it's You can test everything through this. It's really It's really easy.
It also test your hypothesis, right? Like actually is that true?
Yeah, exactly. Exactly. You can make a gazillion changes, but you just test them. And if you cannot AB test them, then you can test them on like stores with less traffic. So, for example, in in Playtika, we used to use the Amazon because it has a very low traffic or some unknown store, and then we see what happens there, and then we apply it to Android and iOS. Over time, we developed AB testing system that helped us paint a population and track it better, but
I guess that gives you like an alternative way to look at AB testing if you don't have the setup for it.
Yes, of course. Or you filter by country. That's another

easy way to do it.
Well, cool. I appreciate uh you shared all this experience and and that you've had as well as the things that you're doing now. A lot of valuable, I think, insights for people because this is kind of If If you screw this up, it's kind of hard to go back. So, it seems like an important thing to be It's doing now.
It's more expensive to make mistakes these days.
Yeah.
Exactly. Well, cool. Well, thank thanks very much for joining us today. I appreciate all those out those insights. Definitely a lot of uh kind of a hot topic at the moment, I think, for people. So, hopefully it's something
Yeah, not only in games, by the way. I I work I personally work with other industries. So, and that's I'm just going to wrap up with this part. So, if you know, you can find us in dive.games or in my email. And a lot of dive.games, but I I work with other industries also in mobile apps and different industries that has nothing to do and those same patterns repeat.
Right.
Seriously, like
Products and customers, right?

Exactly. It's a B2C. In general, it's a whole B2C.
Right.
Challenge that But games are very sophisticated, I think, compared to other industries, seriously.
Lots lots of metrics involved, definitely, because you have data because you're actually seeing what people are doing as they're doing it, so.
Yeah, I mean there is If you compare it to like, you know, booking a flight is is done, and it's like, you know, you search for the flight, you finally click, or, you know, Amazon, you add item, add item, and check out. That's really like basic. Now, try to try to track what's going on in Boom Beach when you upgrade your headquarters and you get attacked and the barracks is like spitting out and then you have like the harbor, you have like someone attacking you from sea and at the same time you do upgrades of buildings and expanding to a new like part of the world and imagine all the moving parts that you have there versus add item, add item, check out.

Purchase now, purchase.
Yeah, yeah, exactly. It's a very subscription. It's a it's a way more simple than Yeah, games are are fascinating because of that and this is why I'm I'm always I need to always stay in games in some some way.
Keep learning.
Yeah, exactly.
Well, if we people that don't that don't know the the technical aspect of that as well could definitely leverage what Dive is doing to help us to get that out, so.
Of course, yeah, they can contact us. They can meet us in the conferences and GDC and Pocket Gamer. We're always present in those conferences as well. And I really enjoy talking to the new generation because it's it's fun. I don't know. They're like
Lots to learn. Well, thanks very much for joining us today and thanks uh to all of you out there as well. Hopefully you guys enjoyed this. Thanks and next uh we'll catch you guys on the next one.

Amazing. Take care, man. Thank you so much.
If you enjoyed today's episode, whether on YouTube or your favorite podcast app, make sure to like, subscribe, comment, or give a five-star review. And if you want to reach out or provide feedback, shoot us a note at [email protected] or find us on Twitter and LinkedIn. Plus, if you want to learn more about what Navic has to offer, make sure to check out our website www.navic.co. There, you can sign up for the number one games industry newsletter, Navic Digest, or contact us to learn about our wide-ranging consulting and advisory services. Again, that is www.navic.co. Thanks for listening and we'll catch you in the next episode.