Session: The Science Of Success – Driving Growth With Prediction, Incrementality, And Scalable....

[Music] right can we have a round of applause for our second speaker of the day hi everyone good morning I'm super excited to be here and thank you for joining me today I'm going to talk about driving grow with prediction incrementality and scalable strategies uh I have been working with gaming clients for six years and I was a user acquisition manager before and it's still fascinating to me how UA user acquisition uh has been evolved uh the way we are analyzing data the way we have access to data Everything Has Changed

so this is our agenda today uh we will discuss uh first of all uh I will be touching open the overcoming this key challenges what are the key challenges and how we can address it what's our new game plan and uh Predictive Analytics that's becoming uh essential part of our marketing strategy uh and finally we will talk about incrementality mm and how we can have this Blended uh approach so um dve uh gaming is becoming more competitive than ever uh that's the fact uh with this explosion of new titles platforms genres the market itself it's getting crowded and as a result of that we are seeing Rising user acquisition cost effective CPI for gaming nearly doubled in 2023 and we will continue seeing this increasing Trends we haven't published 2024 insights yet but stay tuned and we are seeing increasing pressure to Pro Roi uh we have this

clear measurable Roi as budgets are getting tightening and accountability is increasing and privacy regulations reshaping uh data access uh and scan on iOS side and it's around the corner Google privacy sandbox and ultimately we are discovering new channels for New Growth connected TV is one of them uh in 2024 they have seen almost 20% increas in at spent uh and rewarded platforms are becoming super popular and as you can also see yesterday there were some sessions about rewarded platforms uh and I believe you have also seen their boot outside uh there are lots of players in the market so um I want to share an example with you what's our new game plan how we can address this key challenges right uh the example is uh let's say we started working with this network a and

we see 500 installs and everything is going well uh we are seeing incredibly High roas uh LTV and you continue investing in that channel and in the midterm you are also um doing incrementality test as an always on uh project and this incrementality results show that this network a was cannibalizing organic traffic on Android in the United States and in the long term uh marketing mix modeling results recommended to defund network a entirely uh when optimizing for installs over the next 6 months so this is going to be our game plan this is where we are going towards right but speaking of Predictive Analytics um where is this Predictive Analytics falling into in this equation uh let's talk about the power

of Predictive Analytics Predictive Analytics is shaping uh the way we understand the future outcome of that specific campaign result and there are lots of advantages starting working with this uh predictive metrics um it will give us this ultimate competitive Advantage um since we can start optimizing this campaigns from day one uh towards this predictive metrics so it will help us h draw targets faster uh fixing measurement problems uh in ESC Network side uh we are working with limited data we have limited visibility over uh iOS campaigns and uh we are investing our marketing budget uh with this data back confidence with this predicted metrics and we can also segment our users based on this predicted data and segmentation and monetization it's also becoming super popular that's a hot topic now right with this

Predictive Analytics uh you can segment your users in a way that what is the possibility this user is going to trigger in a purchases if the chances are slim maybe you will increase the frequency of uh how many ads they are seeing so sky is limit and traditional scan measurements say traditional scan measurement works with conversion value mapping so it will be based on this conversion values we are setting in a way majority of gaming clients uh they are uh setting different Revenue buckets in order to estimate the total revenue generated and the reason why uh we are not working with actual revenue on iOS side as scan does not share actual Revenue we are trying to get as accurate as possible with this Revenue buckets and with this predictive anal analytics what we can do here we can

completely change we can completely change the game plan and we can work with conversion value mapping in the end it's going to deliver predictive LV let's say it's day 30 predictive LTV we are setting up conversion values in a way we are predicting this value and uh scan will be sharing this long-term user behavior in the end so that was the Predictive Analytics part uh and the question was where is this falling into in this equation I would say it's one step further from attribution this Predictive Analytics improving our optimization strategies but still there is also the path towards to incrementality how we are going to uncover this true campaign impact so in a nutshell incrementality is a counterfactual analysis right uh and it

answers million dooll question what would have happened if you haven't started that campaign if you have't increase your marketing budget and incrementality is becoming a gold standard nowadays to understand uh the true impact in a privacy uh uh first way and it's the blueprint for success to align our marketing Investments with sustainable growth and how do we do that and I want to share one use case with you that's the new campaign start uh and the campaign has been active for four weeks and the target event inep purchases so we are targeting inep purchases and the campaign result shows that 10% increase uh in in a purchases but the key questions are what would have happened if you uh didn't run this campaign in the first place would you have seen less than 10% or would you have hit 10% anyway or what would have

happened if you run this campaign uh for only two weeks instead of four weeks uh have you seen this 10% increase more or less so incrementality is building foundation for scale it's demonstrating um marketing strategy Effectiveness if they are bringing genuine Roi right and there are several different ways uh doing incrementality analysis the way we are doing this uh we are uh working with synthetic control groups which we believe it might be a better approach because control groups uh might not be feasible every time if you would like to exclude this control group from the entire campaign it might also increase your bit right uh but we can we can discuss later if you want to we do not have enough time to get into details but if you would like to have a conversation uh we have a boot um boot number is uh G 20 so if you want to have a conversation please stop by we are on

the second floor so uh here uh we are trying to find out the difference between actual data and uh predicted data and we're seeing 10% uplift according to this test and the confidence interval 95% which means we are 95% confident that the actual increase in this events in in a purchases uh for this campaign somewhere between 8% and 12% so sometimes uh one of the frequently asked questions for incrementality incrementality yes it's a counterfactual analysis but does that mean this campaign um budget increase or starting a new campaign does that also have an impact on other marketing channels is there is there this halo effect we don't know with incrementality test right and

now marketing mix modeling is coming into play to help us to find out that and mmm is the missing piece of the measurement puzzle right because it offers 360 marketing strategy by taking into account several different factors uh non-attributed uh channels influencer marketing which we believe sometimes it not it might not be the last engagement channel right right influencer marketing or connected TV and you can even incorporate offline at spent and we you we will be also considering all different external factors making an impact on the results like holiday seasonality competitive competitor Behavior how they are spending their marketing budget or Channel saturation sometimes increasing your marketing budget doesn't mean that you are going to scale up right uh what if the channel is saturated you are not going to get much

so combining marketing mix modeling with this existing attribution models it will give you this comprehensive overview because mmm provides this topown view of marketing performance uh but it can lock the ground lity this deterministic and problemistic data can provide and blending approach will be revealing this interdependencies between channels we are talking about this halo effect like for instance uh if you uh if you are increasing your YouTube spend does that boost Tik Tok Effectiveness so mm marketing mix modeling it will help you to understand this interdependencies between channels so uh to to wrap it up uh this aggregated anonymized and delayed data it's becoming the norm nowadays so we and it Demands a changed approach and this changed approach will be including three different components attribution

in the short term it will be your single source of Truth and helping you this optimizing your campaigns uh down to the most grer level to the creative level uh to understand the effectiveness of marketing strategy but in the M midterm we will be testing this new channels new markets new campaigns uh to find out if there is organic cannibalization or if there is a genuine Roi here there's an incremental uplift and uh mmm will be the perfect pairing for long-term strategic planning uh you can forecast your best budget allocation what is the ideal way to allocate your marketing budget including offline span so are we ready to embrace this uh feature of mobile measurement uh thank you for joining me today if you would like to have a conversation we will be here all day we would like to talk we will be happy to talk to you you can scan it from here we are on the SE floor thank you very much and if you have any

questions I believe we still have some time question than any questions question hi thank you um maybe a question related to incrementality measurement so what are the various approaches that um you know can be done to to do so and maybe what's the the best one that suits the you know the framework inry steps that you've uh that you've defined so just you know how to measure incrementality efficiently and with the fewest hassle so incrementality this is also something we are working on right so you can uh create a test and we can deliver results like in literally 10 minutes but to be make it a part of your marketing strategy it should have it it should be

an always on marketing strategy uh you because you will be making some changes and you would like to test it out if if they really working out right uh maybe you are increasing your marketing budget maybe you are decreasing your marketing budget and you want to understand more if there is any negative impact uh or maybe you are getting into new regions uh and you're activating uh new channels uh the way we analyze increment ality it also depends which marketing channels you are working with if we are talking about Apple search ATS or search searchs in general right if you're seeing organic cannibalization probably you are not going to stop that marketing campaign because it's the it's the risk you are take you're taking in order to uh save and secure your audience so that your competitors they cannot just uh Go activate the campaign and uh P your you users uh but how they are

taking actions and how you can create actionable items with this incrementality test depending on how much organic cannibalization you are seeing or how much incremental uplift you are seeing you can create new metrics new cper install metrics new cper purchase metrics you can calculate new R metrics if you're seeing this incremental uplift maybe you would like to give more credit for that channel if you are seeing organic kind of ization instead of giving full credit maybe you can give you can multiply with 0.6 0.7 depending on this organic canalization uh ratio so um if you want to discuss in detail uh maybe uh maybe later on uh you can stop by our books and we can discuss your use case hi there thanks for the session um my question is with regard to the models that are being used like predominantly

this is based on scan measurement and scan being the norm um but the paradigms are potentially shifting because we have probabilistic measurement becoming prevalent um and Google opening up uh the possibilities there with intially endorsing fingerprinting late last year um do these models work with a probabilistic setup yeah yeah definitely deterministic and problemistic attribution it's still part of our daily lives uh and it will give us this real time data we can continue optimizing towards um but it doesn't mean that we are not going to run incrementality test it's completely different it's a different use case so problemistic it was an example let's say we are seeing 500 installs doesn't matter if it's coming from deterministic or problemistic attribution uh but are they canalizing organic traffic or are they really bringing this uh value incrementality will be the answer

and marketing mix modeling in the long term it will be analyzing regardless of what attribution methodology uh we are relying on uh they are analyzing all different kind of channels all your marketing budget it doesn't matter where you spending and uh ultimately it will be providing you with what is the ideal budget allocation here depending on install or Target activity um so maybe in a way you are allocating your budget based on this incrementality test results uh but since mmm is considering different external factors like Market saturation or Channel saturation for instance so it might give you more insights and you can still uh do some AB tests um just to validate this mmm models marketing mixed modeling do you have one or two example advertisers who are employing this framework now today or you

think there are big gaming companies which I cannot disclose their names but yeah they started uh uh using uh this mmm incrementality as an allbas on Project thank you very much thank you for joining [Applause]