Masters of Publishing 5: Scaling Isn’t a Milestone, It’s a Grind (Samantha Benjamin, Growth)

Samantha Benjamin, senior director of growth. Nice to have you in your own office.
Yes. Thank you for coming.
Yes. Well, thanks for having me. Um, let's start with the most interesting question like what goes into scaling a game.
Yeah, it's the most uh broad question in a way um because so many things go into it. But uh scaling a game in today's market means like navigating this everchanging landscape of um variables. You have IPM, you have ECPM, you have ECPI, um the creatives that you're putting into the campaigns, the channels that you're partnering with and optimizing. And none of those things are constant. They're constantly changing. And we need to be optimizing to achieve the the best possible results while looking at the cohort KPIs and the financial KPIs and making sure that all the dots are connecting.
So everything so you're solving a puzzle

where the uh the parameters are changing constantly.
Mhm.
I'm curious to understand like what is the first signals that you're looking because always when you're talking to a publisher it's like well the marketability has to be high. Is it the marketability meaning like IPM meaning impressions sorry installs per thousand impression or is it the rollass or return on ad spend like what is the the number one number that you're looking to even engage in scaling
right so I think the two most important uh indicators in a way um in one hand you have IPM like you said the impression the the IPM is basically indication of how well your creative is speaking to the audience and generating installs for your campaigns, right? And
it's kind of like the the main and most important K KPI when you're talking about marketing because IPM is in a way free. Like if you have a good creative, you're able to reach a very large number

of impressions or users and generate the scale that you're looking for without having to change your CPI, which is the other piece of it. Uh CPI is cost per install. So basically how much you're willing to pay for every install you generate. And both of those combined like IPM versus ECPI multiplied by ECPI is what brings you your eCPM. And your eCPM is the KPI that determines how competitive you are in the market. How
your campaign is able to you know reach the best audience and the large number the largest number of users. And um a tricky thing is that today ECPI is not something that you determine. It's more like a outcome. Yeah. So basically uh in today's world we're running campaigns that are very advanced.
They're called um you have from value optimization to row uh optimizations. And basically what you're doing as a growth manager or a UA manager is you're

determining what what is the rorowas that you want to see at a certain age. it could be to age seven or age 28 of the user in the like in the cycle of the cohort
and um the networks are doing all of the heavy lifting let's say so the CPI is more like a outcome and it can be very much influenced also by market changes that are not under our control so even though you can in theory let's say always increase your CPI to become more competitive it's not always something that you first of all it's not something that you want to do but it's also not something that you're fully on control right the market may do that even if you don't want your CPIs to go up
it may happen as a outcome and um in addition to that I think something that is um very important to look to understand is that the cohorts also change like a campaign that you're running and it's working really really well for you this month may not generate

the best performance in a month from now because of changes in the market because of a competitor that launched a killer ad that is exploding and taking away all of the impressions from a certain channel or um things may change because CPMs are changing due to holiday season. So there is so many factors that go into the performance of a campaign. It's like would be very easy if it was like a math equation and it was very exact. you know the the job will probably be much more boring uh for the UA managers but it would be much simpler uh and the reality is that it's not that you need to have that um strategic thinking you need to constantly be tracking the the numbers you need to be changing and adapting and evolving in the way that you optimize based on what's happening in the market what's happening in the competition and to be able the to understand the changes in the cohorts and how you can keep your sustaining your skill scale, right? Because scaling may happen on the

beginning, but the most uh difficult part is sustaining it because in order to to keep your campaign running at a high uh performance in a way, there's product changes that you need to do as well as creative and marketing optimization changes and there's always a balance between LTV, CPI, and how you can keep the ball uh rolling for a long time. Um but I do think like in a way like for hybrid which is what these sessions are focused on um marketability and scalability is like a very key factor in the success of a game
and we still need to be
pretty much very much looking into it in order to sustain the the the business model of a hybrid game
and yeah so so you mentioned so many different parts I always hear from new games talking about how important is to monetization early on because essentially that lives goes into the u
the longevity or even the marketability

of the game in a way because you're getting your return on ad spend seen so I'm curious to understand what is the typical return on ad spend especially on games that are only beginning to scale and how far that does that go like does it start at 30 days and then how far is it being pushed nowadays it's like because everything is changing you would assume that if everything changes so fast competitors are coming new campaigns the return on ad spend is relatively short, right?
Yeah. So, I'm guessing like I'm I'm not sure what is the return that you are familiar with from your experience, you know, but I would say that let's compare it to hyper casual just to have a sense of like a benchmark. So in hyper casual you're usually you know ROI like break even after two weeks even let's say like seven to 14 days let's say you break even and then you start making your profit because
the revenue after that is already very thin let's say
and in hybrid is really different so compared to hyper right so basically we're talking here about break even days

that can reach day 90 sometimes day 120 wow
so it's like a much much longer um cycle to return the investment but it's a much higher LTV right we're able to do that not because um you know- because mainly of the product and the retention of the users and the fact that basically you're able to monetize really well on the long term so you don't need to rush into quick returns like on the first week right we could do that but then we would be limiting the potential of the Yeah,
because what you're looking for when you're doing hybrid or long LTV games is you want the best quality users, right? In hyper casual, the business is pretty much like focused on scalability and quick results, you know, like basically you're looking to have a very good CPI to scale very fast. Once you see that day one, day seven retention that is

within the benchmarks, you're ready to go like and you can make out of that a business, right? And in hybrid, that's not the case. like you're looking at long-term metrics like retention of day 30 60 90
interesting
and that's where the monetization keeps coming al as as well because we do monetize with ads so as long as also with AP but I mean as long as players are playing you know you are making an income out of that um play time so basically um the early curve of the rorowas is significantly lower meaning the return that you make on the first week the the first two weeks even is way lower than what we are used to from the hyper casual and that makes the whole business of a scale scaling a game much more uh complex and risky right like so it's not as simple as a hyper casual that you plug and play and you can scale and just you know as long as your CPIs are low you keep it going here there is a lot more um risk involved therefore it demands a much more sophisticated team

around all of the sides marketing creative product in order to make sure that all of these investments that we are putting up front are going to pay out in the long term.
Understand? And how how quickly do you get through soft launches now? Because if you're looking at return on ad spend that is,
you know, 60 to 90 days, then that means that you have to be in soft launch now for three minimum three months and six and and more. or are you able to get them out sooner where you're just looking at the uh the shapes of the LTV curves where you can see that they keep on increasing and that means that
you're good to go and you can expect that the LTV curve continues to grow rather rather than
a stable off.
So it's a great question. I'll say that like depends what you call soft launch. said there's different types of soft launch, but
yeah, go back to that. Like, how do you Cuz we talked before about the soft launch, but is there a soft launch for hybrid casual or is it more like now the games are on the market and we're just

increasing the investment and maybe even pushing back the the rorowass to be able to invest more and seeing how it performs or is there there's no like a when somebody comes to Samantha make a call, does this pass soft launch or not? Or is there such a thing anymore? So there is such a thing like soft launch. Yeah. But uh basically it's a bit like I'm just saying there's different uh
ways of doing soft launch. So I'll just clarify like what we consider as a soft launch
because at the beginning uh stages when guy and his team are working on the product right they do need UA because at some point after you build like this core that you believe is strong enough you want to validate that this idea is really marketable and you want to expand a little bit the channels that you're buying from so that you don't have only one niche group of users from let's say Facebook and they don't represent the reality after you scale right So we start getting involved on the marketing side from a much much earlier stage when

come talking about a hybrid game
and basically our job in the beginning is mostly for product purposes right so we are trying to understand what's the KPI that they want to validate we're trying to bring them users so that they can do their tests and make sure the product is built for the right group mixed group of users okay so there we already are opening campaign s budgets are going out. I mean we're spending already but it's mostly for product purposes. Once we feel very comfortable by we I mean like the product team together with the marketing KPIs that we are seeing from this early stage. Once we feel comfortable about the KPIs that we're looking at and we think that the game is on a good enough stage for us to start real soft launch let's say then at that point the soft launch will be about expanding geos growing campaign types opening more channels and when we are at that stage of like slowly scaling let's say it's um we're not going back to your

first question we are not optimizing after three months of running a campaign we have basically let's say day seven day 14 data of like a weak cohort we are trying to understand how quickly like how strong the growth between dayto day is compared let's say day one to day seven what's the ratio of the rowas how fast it's growing we're looking at retention KPIs of later days which we have from building the product right and trying to understand what we call like a multiplier you know so we can predict what's going to happen in a few months from now without having to wait and once we have enough installs and enough data I would say like I don't know two three weeks of running a campaign already with a nice amount of installs per day we have a ROI prediction that our data science team build that is able to predict for you the LTV of each campaign and all of the different geos on much much further days right so we are then

once we have these LTV predictions we're optimizing based on ROI predictions we call And it's a much much more u long-term optimization let's say and also a lot riskier because as I said the cohort of today may be very different tomorrow right and even though the predictions were right for the month that you were buying in maybe something changed in the market and it's going to shift the way things actually are because in the end of the day it is a prediction. So, so we are always tracking actual results and trying to make sure we are aligning it with our predictive KPIs and uh optimizing accordingly to have the lowest risk possible. So at the beginning stage let's say we will look at predictions on day 30 day 60 we won't go to day 120 even if we have
and um after we have more validations of what's happening on day 30 we'll start going more aggressive and more aggressive
always keeping in mind the profit

margins that we want to achieve on the lifetime right
is there such a thing as golden cohort in hybrid casual
no not that like maybe what do you mean by by golden cohort
I mean like the first cohorts the first months when you're really pushing on scale, you're getting a really really good quality of a user base and then you can just see the quality of the cohorts detracting pretty quickly. Does that happen or is it more
stable?
Yes. No. Yeah. I wish it would be more stable. It would be much easier.
Yeah.
But it happens. I don't know. Um I won't say that it happens necessarily that way for every game. There are some games that improve with time, right? Like the cohorts get better somehow. um because we changed the product because there is a strong creative that came in because market competition changed or any other reason.
Um but basically we do see that the more we scale like I think that the more we open the mix of channels that we're buying from the harder it is to sustain

uh the high rowas that we were getting. So the more you buy
the more quantity you get. It's harder to keep the quality right. Quantity and quality don't go hand in hand.
Yeah. Unfortunately.
Yes.
Um, okay. Well, so we covered that there's so many different things that are constantly changing and even when you thought that this is quite stable, it can still change later on with an existing cohort.
Yet you are in a business of revenue share
and scale and revenue or scale and profitability. Now, let's put it that revenue is a vanity metric. Profitability is the real one. So scaling and profitability doesn't go hand in hand. How do you kind of bridge that gap between a publisher and developer and you having so many vectors that are constantly changing and you still need to predict and through that kind of make a business case for both parties?
Yeah, just by explaining it we can hear how complex
the world of marketing is especially as a publisher. Um, so it's a great

question and I think that it's one of the most uh uh sensitive things to explain and to be on top of when you're a publisher working with studios. What happens in reality is that studios are working in their game, right, for like months
and finally they're like launching it. They see installs are skyrocketing. They see that, you know, the game is performing well because we keep scaling and revenue is going up and everything looks really, really great,
but they don't see that represented in their bank account right away. And and it's very difficult for them to understand why. And the reality is that when you're buying uh today on user acquisition, let's say a very simplistic way of putting it out there, $1 spend that goes out today to buy you as a user to play my game.
I'm buying you at $1 because I'm predicting that throughout your whole time in my game, you're going to generate $1.3

depending on what's my profit goal.
And it's a future prediction of revenue. like it's a cost that goes out today but for a revenue that will come out during a lifetime of a user in a game right so the scale and the profitability they don't go hand in hand right so basically especially at the beginning stages of launching a game you'll see a lot of spend go out and it will not necessarily be represented in your profit financially but the the end goal is the same right we're all aiming for that uh margin that we're looking for and it will get there. It just takes like I would say depends on the game and the curve but it can take like a few weeks or a few months to get that break even point and start seeing the profit coming
and one of the things that we are doing in supersonic basically in the growth team is that we have like a very very close we have to have it yeah because of the complexity of it but we have a very close um relationship with the studios

that we are partnering with on the marketing side also so that we can present to them everything that is happening behind the scenes because there is a lot of uh trust
that needs to go into something like that right because in the end of the day
there's the profit might take time to come in so we are explaining to the partners we have through uh presentations data graphs predictions changes in eCPM we're like presenting to them all of the insights around what we're doing to educate them and therefore also build some sort of like trust relationship so they understand the why
the money is not in their bank account right now as they expected or not as much as they expected. So there are constant conversations around that and um and in the end of the day we all do reach that end goal right partners that succeeded and published very good games with supersonic made the millions of dollars and they're very happy but in order to reach that we need that trust

uh factor and another thing that supersonic does as well is basically we have I don't know if you talked to anyone about it but the retainers yes uh if people are already familiar with the word or not but basically we have these retainers which is like some sort of like cussion space for the studio, right? Because in the end of the day, they have to keep their company running. So, we don't want them to suffer any impact. And if we truly believe in the game, it's also something that we are easily willing to invest because we know the the profit is going to come afterwards, right? So we keep that going for as long as we need depending on the studio and the case in order to wait for the financial profit profitability to be there not to put all of these overhead cost on the studio side.
Yeah, that's a that's a really good approach where where you're essentially teaching the business model to the developers because there's so many things. But have you seen when you go in and and open up the books and and really try to explain the the complexity of of

the scaling, does that help the developers also to prioritize certain things that you need on the marketing side to be able to scale like the ability to retain payers for example or drive repeat purchases that again increase the LTV and allows you to to spend more? How how has that type of effect happened or is it more just you're sharing information and developers are understanding and able to focus on what they do.
No, so definitely um it's like a two-way street, right? Everything um like I think what we are sharing also benefits us in the end of the day.
I won't say like it depends on the setup because we have people work like game studios, let's say game teams that are working with studios in Supersonic, right? And sometimes we're doing these sort of like syncs that you're talking about even internally so they can make sure to align it in the road map right and developers it depends on how much we think it will be understood by them but basically yes what happens is that

in the end of the day there is a lot of uh KPIs that are product KPIs right you as a product owner you're looking at your retention your arpoo your arpow all of the metrics that you're constantly trying to optimize and improve and we might uh do something uh creative technique that will impact all of those KPIs and everything that you're doing you're it's now like refreshed let's say in terms of a benchmark right so UA can really especially when you're doing UA scale like in hybrid games where the monetization still comes in at the first few months and it's not like only for very highly retained users you're monetizing so the changes we're doing in marketing impact product KPIs and we are trying to a lot of times uh use the creatives first of all to explain so that they understand how to look at it look at the AB testing methodologies in the right way not make product changes based on marketing

tactics because they think retention is dropping or increasing you know you may be buying from a very high quality campaign let's say and bringing a lot of IP users and suddenly the product person is going to think they did something amazing that increased uh depositors and everything is great but if this campaign goes down it will change back. So we are trying to constantly update the team so they are aware of what we're doing and our impact on the product KPIs and on the other way around we also see creatives in marketing in a way as a bridge between product and growth right so we are doing a lot of times syncs trying to understand ideas that the product team have or bring them ideas from what we see the users want in the market and what's really working
and to combine it vice versa. So if we see a creative is scaling right generating a lot of installs very successful but it's not exactly speaking the language of the game so the post install metrics will be very low we will try to update the product team so they

can think about ways that these could be maybe implemented in the game so we can maximize that marketability aspect but still keep the post install metrics at a good pace right yeah and also if the product team has an idea let's say which all the time they have idea Yes, but uh it takes effort and time and development and it's you know it's a high investment in the end of the day developing something in a game. Um they can test these ideas through our creatives and we can give them feedback of how it speaks to the users and what's the sort of performance we're seeing, what's the markets that are picking up on it and they can validate if it's a good uh idea at least from the marketability aspect before they go and they put all of the investment in the studio development side. Um so creatives is kind of like a bridge that goes both way right and uh the connection between marketing and game studios is key in order to make things work especially in hybrid casual games where marketability is still so um

is such an important factor of keeping the business for like years and years.
Yeah.
Uh running.
What's easier to improve CPI or LTV? Okay. So, I would say that CPI like I think yeah like even though it's my bread and butter, let's say, but um
I think it's easier if you think about the whole thing, right? You think about the manpower efforts and the time that it takes to produce a creative versus a product change. And in the end of the day once you already scaled the product you probably did already a lot of the work of the core work and the ideas that you had for that product to be for that game to be good right so finding the additional things that will keep it import growing and growing they happen but they happen maybe in a few months right it's not like every month you will see a AB test that is bringing like a 10% 15% lift
maybe not during the whole lifetime of

the game
sometimes yeah it's like a very hard um like it's a very hard um work to get that uplift on product AB test and it requires a lot of headcount a lot of people to do it and time to measure it to understand it and to do the u finetunings that maybe the feature needs uh it works when it works it's amazing and it's last lasting forever you know and it's very ROI positive but it can take much more effort I think
and on the creative side and especially with recently with all the AIs that we have that are already really efficient on the creative world. I would say it's it's a little bit simpler. It's still not a uh equation that you can say this, do this, copy that and everything will be great because there are many things that go into account. But we do see more often a significant impact from a creatives that grow in terms of like

that scale your game thanks to a very good creative. Just recently we had something that I was sharing with the team like of a creative that doubled the amount of revenue we are making in a game which is like pretty insane. Yeah. It doesn't happen every day
but not doubling but like something that picks up and that become makes your campaign keep running and becoming more competitive depending on where is I think is a bit easier. But again it's all depending on market changes. If now there is a creative, I won't mention the name that is dominating the industry and everyone is seeing it and a lot of actors and people that are famous are running and they're taking all the impressions away from all the different networks. So we have a very hard work to even sustain what we have, you know. So it doesn't always depend on what we do.
Interesting. What are the three misconceptions that people have about scaling a hybrid casual game?
Yeah. So first one I think is a good the conception of like a good game is enough. I don't know if it's only for hybrid to be honest. I think that's for

any sort of a mobile game in the industry. Um, a good game is definitely important, right? Having good metrics, but I think that the other side of it is the marketability. Like, do people really want this type of game? Is it like something that can scale and turn to a business? And sometimes focusing only on the side of having a good game is what leads a lot of companies to uh fail in a way you know. So that's like one thing that we know because we came from a hyper casual is that early validation of your idea and the marketability and the scalability potential of a game is equally important to the good retention that you're looking for in you know when you're building a product. Yeah, the
other two.
Yeah,
that was a good one. I I'll take it. But can I have two more?
We'll cut that. But
we'll keep it.
Yeah. Okay. So, the other one is that

it's a misconception I think in life honestly, but those with the deepest pockets win.
Um basically in
it's a truth in life.
It's a misconception.
Okay. Um maybe but um basically the idea is that you can have a lot of budget right and maybe you got like a really nice investment as a gaming company and you can really invest on UA and do very aggressive CPIs celebrities
you know celebrity spends without even looking at post install metrics and potentially scale your game right because it's true if you pay a lot you will manage to get a lot of downloads and a lot of users but the misconception in it is that it won't be sustainable sustainable, right? So, it will be a peak, but then it will go down and it might take you a lifetime to recover, even break even from this investment
and doing it in a way that is sustainable requires efforts. Not just from like putting the money in the campaign but optimizing it,

understanding the post install metrics, knowing how to overcome the challenges that we have, which are many as we mentioned before, and having your product team constantly optimize and and improve the LTV so that you can keep the balance between CPI and LTV. So just throwing in a lot of money into a campaign might work on the short term, but definitely not on the long term. Definitely not if you're looking at profitability as an important KPI.
I actually have a question. I know you want to move to the third misconception regarding that.
Um, are you a fan of a big launch campaign or are you more of like a slow and steady wins the race? H it um it depends like I would say that if you want to go big just because it's the launch and you want to spend as much as you can and you're not careful about your KPIs and at the beginning it's a bit uncertain right like we said you can try to predict but it's not for sure so you might go big and end up in a big

debt right and not
you go big and you go home
yeah you go big and you go home exactly and um so basically I think it's like a balance, right? You do want because the market changes so often, you do want to take advantage of your time because what like
what's working right now in your creative in your game, the trend
may not work in two months from now as well as it's working today, right? So, you don't want to miss out on the timing of your scale if KPIs look very good, very attractive, and you see there is a lot of potential. One of the things that I would say is a good indication is how quickly your campaign is capping, right? So if you start with smaller budgets, you don't go big and you see your campaign is capping very very fast. So those are like good signals to show you that the market wants that that it's like a good campaign and that you should keep opening the budget slowly until you find your sweet spot and within a few days understand your post install metrics. Um, but if you just go right away from the get-go, very very big, you

may
lead you to a lot of um, not trash, but like very low quality, you know, users and not necessarily profitable.
So, would you would would you say that
a best strategy would be start slow and steady, scale in a couple of different markets and then go big
because you're certain that you've been able to scale. I would just say that the the slow or the the strategic balance yeah methodical approach is that it's good
but not too slow also right because you do want like to take advantage of it you also don't want to buy at a 200% you know ROI
if you could have bought more and made more actual profit because this 200 like this 100% margin is not necessarily more worth it than more spend and lower margins right so it depends You need to be really smart about it. You need to know how to read the KPIs. In marketing, everything is about data in the end of the day. So, if you know what you're

looking at and you know how to assess these numbers really well, you will find the right balance because at some point you may
it may be okay after a certain point for you to just go really big, right? ive KPIs are really promising and you see that you're already reaching for example 100% ROAS on day 14 and your game is uh you know game that can target day 210 then why not
all right
what's the third misconception
I covered it to be honest I was going to say like that scaling a game that if you drive a lot of installs that's like a good indication for
that a game is successful you know and I don't think that's true I think that there's still a lot of not a lot but there's still a few people that scale very aggressively and necessarily the business behind it is not the most profitable business and the
like installs don't always convert into like you know profitable businesses. So like if you see something that is exploding or a campaign that is taking over all of the impressions in all of the different networks

it may not just like it may not be ROI positive but you know we won't know about it and usually these investments like we discussed it depend like this is hybrid in casual I'm guessing it's even further the return
so you might think you have like this amazing thing and in 200 days from now find yourself in a big problem. So scale doesn't doesn't mean success in the mobile gaming world.
So I'll summarize everything about scaling into it depends because there's so many variables to be honest
that change at at the same time that you have to be on the ball and
you need a little bit of everything but you need it right.
Yeah. It's like a screw master
just one piece at a time.
Yes.
All right.
Exactly. Well, Samantha, thanks for opening up all the uh intricacies of scaling a game.
People will leave more confused than when they came in.
That's this is how it feels like when you give a presentation to the to the team. It's like, hey, I think we're scaling really nicely and the revenues are climbing. It's like, well, it

depends, guys. And then they leave. It's like, we learned a lot, but I don't know what just happened.
I'm doing a good job.
Yeah. It's like, okay, it depends.
That's the world of marketing.
Well, thank you, Samantha. It's a It's a complicated thing, but um good to understand all the all the elements of it.
Thank you for coming here.
Thank you.