Measurement in a Post Privacy Era - Hadar Telem, INCRMNTAL

[Music] [Applause] [Music] thank you [Music] so next up we're going to talk about the measurement in a post privacy era and we've got Hana from incremental here kallo header so if I can get you to come up we're going to talk about like I say post privacy era attribution mmm incrementally uh and from Theory to practice I've had it over to you uh okay hi everyone happy to be here today um so first question to the audience a bit of a wake up how are you checking you're in um so no

yeah so how many of you uh have heard of mmm media mix modeling or incrementality okay nice uh how many of you have heard about it only in the last two years okay quite a few uh how many of you are actually using that in your our organization one of those at least okay so today I want to show you and kind of explain to you about those methods and show you why all of you should be kind of start using those methods in your advertising so a bit about me I'm hidal tell him I'm Director of Business product operations at incremental um I actually started my journey quite a while ago as an analyst in veronis then as a campaign manager at Ali tour then I was a business analyst in politica and then I moved to huge where I eventually

became director of marketing Analytics and I was in charge of everything related to privacy changes uh and kind of iOS 14 how to lead the organization and I was searching out for basically tools and methods that will help us mitigate through the changes of iOS 14 and all the changes in privacy and incremental was actually one of the tools that I brought in to huge in order to understand basically what is incremental and what is working for us and I eventually decided to join incremental because I do think it's kind of the future of measurement um a bit about incremental it's a platform that enables measuring your overall lift in performance it doesn't require user level data and it shows you basically your value of the spending change based on everything also including your organic traffic so it can measure anything from TV to iOS

campaigns okay now go with me kind of four years back back in the days where we only had kind of our basically kpis what is our CPI Target what is our Rose Target and we knew where we are at but right now it looks much more like this so there are a lot of other methods and we're looking at all like incrementality scan where are we let's touch media mixed modeling what should we do everyone is saying something else how do we set targets how do we know if we are on target incrementality says a attribution says b mmm says C what do I do so today I'm going to try to help you kind of understand basically the background for that and also um try to live with those new reality

and new measurement tools so a bit about history incrementality actually started in the 1950s it was kind of the first Performance Marketing tool um at that time it was done by actually checking any impact on your performance by turning spend on and off so you have no spend no advertising and then you increase it you see a lift you decrease it you see downlift you increase it so all of this increases are the incrementality and this was the method that was used for many years in order to kind of check performance on an aggregated level it wasn't detailed as we couldn't do today then mmm or wait 1960s also beatlemania um but also mmm started so this was basically a method that helped make

strategic goals while incrementality helped to make decision making in the day-to-day mmm was more kind of trying to make uh bigger scenario planning and understanding um what should we do in a quarterly point of view and plan ahead for the five years 10 years at the time you could do that because everything was kind of more simple so we imagine it was something like that but in reality it was 1960 so it was more something like that um then the years of 2000 user level data I guess most of us started working in the industry in this time those good old days so everything was trackable or more to say everyone was trackable we could have the information about basically everything our users are doing and how we got it was based on their device data

so device data idfa ID for Advertiser on iOS Gad on Android um the method that used us the most is how we call attribution is usually based on less touch or left click it's the same thing it's basically connecting between um clicks that we have from the user ID and from the Publishers and from the information about app opens that we have on users who actually installed your app so whoever is doing that is usually the mmps the media the Mobile Media partner like apps flyer just singular and they are kind of the barrier thing okay we know that this credit goes to iron Source because the user clicked on iron Source add last before he installed

okay good so now these days we're kind of in the years of mix and match we need to work with all those tools so if I need to look at the Timeline once again we have incrementality and mmm that used until the years of 2000 then everyone stopped working with that because we actually didn't need it we could measure everything and then we don't need to have strategic planning if we can aggregate everything go to the most granular and up again and this day is it's coming back also um we got here because we had several privacy changes so when I started in the mobile marketing industry it was 2016. and I lived through changes such as gdpr uh where which was the European law saying that if I'm a user and I want to revoke all my information from certain Advertiser I have the right to do it or

I have the right to know what is the information um companies hold about me and there was the udid deprecation before my time and there was the Apple limited ad tracking basically saying if I have an idfa on iOS I can go to my settings and remove the information or remove the ID so that time you had to go and change it yourself I think the most interesting thing here is basically iOS 14 ATT everyone's talking about it you know what it is but at the time nobody knew what IDF is until it was announced in 2020 and then this is the Google Trend check for idfa everyone started to wonder what the is idfa um so that was just the year they announced it and everyone were like losing it saying what should we do how do we go from here no IDF a no marketing what do we do and then it was only enrolled a year

later um it was gradually so at the beginning I think that advertisers were like okay nothing really changed let's continue but eventually idfa has been deprecated and scan is more in use probabilistic matching is more in use and now there is more room also for kind of hacking probabilistic matching so the landscape has changed um in 2022 Google announced also sandbox their solution that hasn't started yet should be Q4 of next year and they also made their limited ad tracking in quite a delay and this year Apple announced privacy manifests which is in short their way to say if you are an Advertiser and you are using an SDK that has fingerprinting in it for example if you have an ad partner or an MMP that is doing fingerprinting you are now accountable of reporting which SDK is doing that

so we can see the pattern consider privacy changes also from what was said in the previous lectures it's all over the place user level data is going away we can still have it in some cases but it's much much more limited also targeting has also changed if a few years back we would be able to Target specific users and ads looked a bit like that now ads look like more like that we're going back from knowing who our users are and what do they like to targeting more broader campaigns so here is another Google Trends you can see that the search for TV marketing and brand marketing is also on the rise in some of these privacy changes because now you have less ways to know who are your users and you need to approach

broader audiences so we know the history now let's talk about what evolved basically incrementality um at least of how we work with that in incremental is a way to check all your changes in spend like budget increase budget decrease bid change and see what this budget change has made on your overall traffic including your organic traffic so if you increased your spend in 10k what is 10K brought you that you would not have gained hadn't you have made this change so it is giving you the information about your recent change and it is working on a on a level with a prediction model that has is saying you the added value that you would not have gained that is above the prediction line this is your incrementality

and System since the model is working in kind of a simplistic way in terms of the input it gets and the output it provides it doesn't matter if it's an iOS campaign or if it's basically a TV campaign or anything of this such also mmm media mix modeling has also evolved now it contains more levels it can be updated more frequently like monthly updates sometimes weekly it's still much more granular than incrementality so it's less granular sorry so you have like until the media Source level not campaign or ad group um and it has kind of more modern productization so if I go into the strength of each method I have here less touch incrementality and media mix so less touch is really good at real-time optimization and it requires no

historical data and it's basically simplified and it's quite easy to understand what you're getting what you're looking at and how to work with it incrementality is kind of a value measurement you know the true value of the change that you are making it's not traffic measurement so it's it doesn't give you your full if your level of spend is I don't know 100K monthly you don't know about this 100K you know about the changes you make in spend so it's a bit different it's more granular as it gives you up to an ad group level of changes check and it doesn't require any user level data because it is based only on aggregated data and mmm is giving you a synthetic ground truth it's giving you the true value of your entire media mix it's also helping in budget planning and in scenario planning which is planning for five years ahead with taking all

sorts of parameters that can affect your traffic in the long term so in terms of the weaknesses less touch gives 100 credit to a single click it was also always a problem with attribution because user journey is more complicated than just looking or just taking the single click of the user it requires user level data obviously and it's helpful mostly let's say on digital media incrementality it's kind of downside it is that it has delayed reporting it takes time and from the day you make the change until you can measure its incrementality because we're building prediction models and we're trying to understand what is the true value across multiple changes and it requires you making a change so if you have a business as usual budget that you don't change at all you can't measure incrementality for it

mmm requires enormous amounts of data basically years back if possible month at least it is limited in reporting and it requires priors so when I'm saying it has a lot of data that it requires it means that if you're starting a new game or a new channel and you don't have historical information about it you can't really get a prediction for it and in terms of priors priors is basically some assumptions that you give to the model in order for it to kind of provide your results mmm requires that and a very common prior to provide is actually attribution data like I assume my iOS traffic behaves like that I assume my Android traffic behaves like that and if you don't entirely trust those priors and you put them in your model then you get results

that are somewhat problematic so now I'm going to talk about use cases for each one in order to work with this you know you have to know when to use each so basically less touch is very good at real time and optimization that is on lower levels such as creative audience device targeting and time of the day optimizations those cannot be achieved with any other of the measurement tools I've mentioned incrementality is really good in checking the changes in your media mix and it's super good in cross-platform measurement TV campaigns radio podcasts all of these this is what it is made of made for let's say it is helpful to see the point of diminishing returns because if you are measuring changes you measure

another increase similar measure another increase you can see basically the incremental value of each of those changes points to understand where is your point of diminishing returns I was increased in 10k I got incrementality it was good for me I increased in another 10K and it wasn't good for me I'm going back to the previous level and it's helping you check cannibalization if I increased my iOS spending from on let's say Facebook and it took something from let's say organic this is something that incrementality also helps mmm is good with budget allocation and scenario planning more strategic so I put this in basically a table I know it's quite a lot but here are some summary of those best practices and use cases for each method as I mentioned here and I do believe that those could be really helpful for you if you know where

to look at and which use cases to use it's really nice and good for you in order to implement it in your operations so this is the last part I want to kind of give you some tips on how to work with all of this and kind of try to make them work well together I know it's complex but even though I started with the good old days saying that it was very simple marketing always had multiple sources it never was really one single source of Truth could be once in a while kind of connecting between your spend places now you have more measurement methods it's more complex but this is the new reality we have to live with it but to make a decision you need to gain trust in your measurement methods you need to analyze you need to make sure that you understand the impact of each

decision that you make based on a specific method on your overall traffic you can do it when you just start looking or using a certain tool and then you can carry on after you gain Trust and then finally add methods to your tools and internal operations even if you're a small company and you don't have a complex bi systems it's still important that you know for which cases and which places you need to look at and make sure you incorporate this in your decision making on a weekly monthly or whatever basis that you work on and here's a tools like a tip for Budget allocation if I were to use this so I would use less touch on a daily basis my day Zero I just started something I want to know how my performance looks like if I need to go on a weekly or monthly I would look incremental and I would look on an mmm model to make basically longer

plan and longer term plans the idea is that once we use the right method for the right task future is brighter than ever thank you thank you very much that's great stuff so again it's it's great to sort of see the level of information gathering that we're doing and even though we're not able to sort of pinpoint people the way we once used to be probably for the best I think it's it's a better healthier climate and one of the things that always strikes me is that I feel like now is the time where we're focusing more on interest we're focusing more on kind of actually having to have good adverts yeah and brand building I mean I mean I'm an old school marketeer you know Kevin forbid brand building do you agree is that is that kind of the consequence of the fact that we now have to be a bit more kind of smart and intelligent about the way we tracked it yeah you need to be more intelligent and

you need to know basically who you're targeting and you know that sometimes you can't get to all of them you need to kind of approach also broader audiences I mean I actually think you genuinely have to care more about your audience as well it's not just a numbers game anymore yeah because we're actually dealing with people and we have to respect them somehow along the way sorry I'm hugging the questions I know we've got Academia about to go up I'm I've got time for one if anyone wants to ask Ace quickie and I've got a quick question I want to ask can ask me afterwards absolutely feel free to ask afterwards and on that note thank you so much that was great stuff thank you thank you again

thank you