two & a half gamers session #14 - How to read Ad mon data for beginners? ATT & player segmentation

welcome everybody this is no gaming podcasting gamer session number 14 we are discussing late news fun stuff and dropping knowledge but don't forget this is a 4am conference discussion live let's not take it too seriously uh hey guys how are you today welcome jakob and welcome felix we're nowhere near as free as you are my friend welcome my day yeah where are you dialing you from listeners now yeah thank you very much guys well i'm not looking from california from santa barbara so jakub you are next because felix already uh recorded a podcast from new york i think or vegas now i'm from yeah new york now i'm in santa barbara and then yeah we're next so hopefully you can you can cover

one podcast from uh well san francisco or helsinki or helsinki well but it's not us so yeah well my my internet is super shitty so hopefully we are going to make it until you get privilege of the u.s how yeah i don't feel like how american can uh can actually browse the internet with this this kind of speed yeah how are you all good [Music] pretty busy week what what is happening yeah well well it's like your second podcast already today so you're on fire that's awesome you know before you arrived matcha he actually promised for my birthday he'd actually do one podcast without his sunglasses by the until then he will go for uh eye surgery

i guess or no well everyone never mind so uh funny things are happening actually uh so what we found out very recently other podcasts are trying to copy our wipe and atmosphere well that's uh that's perfect but there's one one thing i need to mention guys you can't copy fun but you can try you can try that's perfect this is you know i'm allowed to use these uh jokes now but uh our last raw video session was actually very well received so thank you very much for all the positive feedback you gave us and um well we are going to continue with these kind of analysis every month so um stay tuned but also if you have any ideas about

companies we should talk about or look at let us know let us know and also one last thing i want to want to mention is we just crossed 100 subscribers on youtube it's not a huge number but we will get there thank you very much for all your support step by step we're underground we're underground oh yeah well very sure very true and uh what we are going to cover today obviously mark zuckerberg's metaverse plans and uh in the meta financial situation then rovio revenue growth also uh att news about one year uh about the last last year and then we're going to talk about the ubisoft potential uh m a and the activision blizzard sales

and also going to talk about the well warcraft mobile leaks well that's gonna be very interesting honestly and then uh our main topic about how to read ad monetization data for beginners and uh this is really interesting at least for me because i'm definitely a beginner on this field but next time we'll do the you the same thing for the ua all right so let's uh yeah let's kick this up guys okay um so other than the quarterly results that was done on 27th of april by meta or facebook uh reading into the news you can see some of the actual announcements they set so i want to kind of cover those a little bit one thing that investors or let's say shareholders are not really happy is the increasing burn rate of facebook's reality lapse division which is pretty much 17 000 people that are already working on the metaverse project which burned three billion last

quarter alone which if we compare it to the previous numbers they said that mark said that last year they burned altogether 10 billion so it's already 3 billion in first quarter which if they continue burning at the space will be 12 billion so we'll see and uh this is still it's going to be an even bigger number man yeah probably yeah it will be even bigger but yeah they're still kind of toning down the expectations that the return of this whole investment will come sometime 20 30 the soonest if ever let's see but uh yeah the stock of meta never evaporated pretty much the growth of last five years so it's back to that like 200 level and they even said that they expect that uh apple's at tracking changes have cost them around 10 billion in lost revenue which is also like a pretty big hit and then regarding the announcements they said that they're gonna have a browser-based version of horizon worlds

which is their supposed to be shared metaverse app for meta quest 2 headset will become available already this year that's the one where you don't have legs and just go around with the avatars yeah amazing amazing when the whole world is mobile you will just launch the browser yeah we'll see but i mean it's just uh you know connected to the web three and all the web browser yeah the thing is that for again for metaverse to work it needs to be accessible through everything so it doesn't matter where you know you just cannot dump it down to one platform so that that's that's the problem you know it's the first platform i think first person is the headset already it's there yeah but then browser and then it's going to be a mobile and then maybe other consoles and whatever else why not like there's really nothing we can say about this like my god like burning that much money having that many people

working on something that's just like it's just i i can't even i don't even know what to say like cool best of luck well you know we will see you in 2030 first the first number i'm not gonna hold any facebook stock that's for damn sure like like let them take this bets i'm not gonna be part of it yeah continuing on the announcements they also said that they're gonna release project cambria which is their high-end pc headset if i understood correctly this should be a pc headset they didn't explicitly said that but i'm thinking that's my guess it will be because later down the line they said that they're aiming there's a professional market like work work headset or something like you'll be working in the metaverse through this so and they said that they're gonna have eye tracking tech in it which seems to be super super demanding on the hardware so i'm guessing it will need a pc to run itself because eye tracking tech opens a lot of possibilities such as soviet rendering which means that you can cut down pretty much sixty

percent of graphical demand just by rendering the stuff we're currently looking at at maximum resolution yeah i bet you have you have eye tracking technology in your business for sure but but for for me like i would say the other big opinion that i have on this is that eye tracking tech could be really really good on the ad business itself because you can then track the user's eye movements all the time so you know all the time where they're looking at which ads they did so and which as they didn't so or how long and so on so forth so this brings a lot of other possibilities which i think are kind of crucial to that and then they said that the headset should be able to translate even the facial expressions into the those like big digital advertisers in their horizon app so i'm still guessing it will be a pc one but let's see so like on this i would just say like the big litmus test or like canary in the coal mine is like the king undisputedly in the world of office tech

is microsoft when microsoft announced they're working on something similar then you know it's going to work but like no like do you guys see it like i i can't even imagine it but maybe i'm just not imaginative enough or forward thinking enough but yeah this sounds like such a risk like i see microsoft developing something like this but facebook okay i'm guessing they're going still for the platform play that they want to be the first to create the platform and kind of colonize it let's say but for me it's still screamed ready player one like they are the one that want to take over the metaverse and then there you have guys like team sweeney and epic who wants to play build the stream at hours and then of course apple is building their own headset microsoft they have their hololens headset which is air based more on that but yeah let's see i'm still guessing that we're far far behind the deck that that still needed to be developed in order for these things to work because yeah headsets are there you can go into the store right now and buy the metaquest too but you know how like there's few

million users out there that's it so don't see that appealing now okay all right should we uh come down from olympus and uh yeah talk about our next story that's a bit more close to uh what we discussed last week um do you want to take this one month yeah yeah i think so we can we can do it so uh so robin sees 26 revenue growth against uh mobile free to play decline so last week we discussed how rovio is actually doing and how they could do even better but this was uh this news was based on their quarterly results i can't even speak properly results and they saw 26 percent year-on-year increase in terms of the revenue uh drovio ceo said that um against uh the third consecutive quarter and quarter decline in the mobile free

to play market so they're actually doing pretty well um against uh the other companies the revenue increased from 67 million in q1 2021 to 85 million in 2022 which is nice nice increase and uh what i would say now is like the day the eu across all the royal titles increased year over year from 4.2 million in q1 last year to 7.3 million in uh 2022. games man and this is yeah this is including the ruby games player base obviously but well they made an acquisition and they know this is how everything is going they already got like 5 million and 20 21 in the full report so it's actually down if i read that right if we take out the ruby uh acquisition but why why would you take that ruby like yeah

of course because it's a hyper casual thing but hey for the company yeah i was like no but we're talking about users that you have on one hand 20 or 10 cents per install and then you have your usual casual three dollar five dollar users and you're putting them into the same pack yes because it's the same company the same company they bought they bought ruby games now it's it's not right portfolio of course the stock market doesn't agree with you like rovio stock is at the highest point it's ever been since 2018. so [Music] yeah stock market seems to love it and if they buy it that's what matters in the end right exactly let's see what what let's see what let's see i'm just saying like a lot of these numbers kind of look good on paper but if you look into the bigger context they don't look that good but why do you think it's not looking that good still i mean yes hyper casual

players sure but that's the part of the the acquisition strategy they have so yeah i mean they're gonna be moving forward yeah buying more companies i don't know like as i said like it's okay i'm just saying that from bigger context it seems different but why [Music] you're not only the crypto skeptic but like the life skeptic i guess or like the gaming skeptic or i'm not sure like how we should call you from that one man but yeah okay let's see cool should we talk about opt-in for att yeah let's let's do that so appsflyer this last week actually released some very interesting data basically from one year on analysis on ios and it shows

uh that the att opt-in that is to say users to actually opt into tracking is a lot higher than first was feared i believe back when apple announced this people were predicting something like 10 to 20 opt-in rate as standard but it turns out like the truth is more that in the uk us and canada att consent in gaming is at 42 41 and 40 respectively so all around that 42 uh percent point and this is kind of the same you're seeing in non-gaming apps and this kind of starts like me thinking there's like i think that was always in top of my mind like this during this whole privacy conversation like the vocal minority that care about privacy are really loud but does the majority of people really care about the privacy no they don't everybody cares yeah no it's just uh i saw these these numbers

as like yeah so now what it's like yeah okay it's 40 percent but nobody cares about that february last year when this was introduced or something like that yeah they slowly rolled it out though yeah yeah before we had the apocalypse movement of like it will end everything and now apocalypse that even doesn't even get me started man i'll like apocalypse i mean the real the real winner of this is just apple search ads like [Music] how how is the real winner apple search it's like tell me basically in terms of the in terms of the gaming the game no no nobody knows the whole ecosystem they went from early like what was the last time they broke out their revenue they said they were making a billion a year and that was up from like a year earlier from like one billion i know it's like this apple doesn't matter because they make so much money but you know seven billion more in a year it's still a lot of money maybe not my money guys

yeah well uh i saw all those uh the numbers and the the growth uh figures for the episode chats but in terms of the games like it's just not performing at all in terms of apps maybe but games not at all how much time it will take for google to actually hope on this or they will take like two years three years until they actually get they say that it's gonna take two years and maybe three at least at least at least two years all right until until they are satisfied and uh the developers are satisfied with the [Applause] final result of the of the privacy sandbox so until that happens then uh it can take more than two years even we'll see but uh yeah well are there anything else in terms of the att because it's like

what do we what can we even do with these like 42 percent uh number like does that make our jobs uh easier it makes my job easier i earn more money from like opt-in users like they're paid a higher ecp so yeah that's what i don't matter for me because for me on the u.s side it's like uh not that significant i mean of course you can get more data from the adult users but anyway we don't use it anymore how about you yakup do you use these numbers it's better if you get more ad revenue of course but other than that like i don't see it that much okay but do you do you get any um any insights in terms of the um the ftv and anything basically on these players in the article actually that like some sometimes i think there was one genre

that said that uh you get more opt-in when you do it uh later actually sooner i think it was like a hardcore genre but compared to hypercasual ones or the other ones to later do with the lower optin rate you get so those are some kind of i mean for you for you as a game designer if you get uh you know opting higher opt-in rates does it mean that you will be able to you know show these players like uh personalized special offers or like how does that work do you know what i mean maybe no okay never mind especially specialized offers are based on their progression in the game not the offers consent themselves that's that's the thing okay don't confuse game progression with the privacy consent then uh explain because i i i'm confused

okay so usually how uh special offers works or like whatever kind of monetization that based on pure power and iep transactions it's based on players progression in the game content so you know you arrive at level 7 you upgrade this you upgrade that and so on so forth it doesn't have to do anything with the consent itself consent itself just gives you better ecpm for those people so felix can squeeze more money out of them identify these players because uh like there's a lot of discussion the players in general because you don't if you don't have the levels in the game and you don't have these level special offers for example or progression based special offers then uh you know um you know how do i segment players let's say how does that replace what uh basically have some kind of progression there needs to be one even time is a progression just like time in game okay okay because yeah that sorry for uh

confusing everybody it's just how do you segment players in the in those this most att world that was the question yeah there's numerous ways to do this like it depends on like the spending depth of your economy basically okay okay well that's that's what i wanted to know good let's move on to well several privately private equity firms might be interested in buying ubisoft well i i have no idea who are these but it sounds very interesting this is a rumor article i would say it was original i think in kotaku or some kind of other source this pc game pc gamer our our favorite pc gamer but people are pretty much saying that they expect ubisoft to get bought and that uh based on what basically the market cap is really small basically yeah compared to other gaming companies and they have a lot of ip maybe not the strongest ip anymore but

so the consensus is that the current situation isn't that good which means that this is an ideal time for an acquisition to take because you pretty much buying it at the discount if you look at the share price it's uh topped at 110 in july 2018 and now it's somewhere at 41 so it's like a really really big discount currently ubisoft went through a lot of troubles uh through the like hr harassment scandals department and like lots of people left they even said that they're bleeding a lot of talent already uh the pipeline for the portfolio is kind of slow so they still have far cry assassin's creed and ghost recon on their menu but they're far away from actually materializing as a launched game they did their nfte fiasco let's say let's not even talk about it anymore but yeah and uh yeah so it's not anymore seven ubisofts for the roblox situation that

we're building so like one or two right it's much less all right i'm gonna bet you guys each 10 euros that if it gets bought it's gonna be blacks blackstone if it gets bought it's gonna be blackstone that does it blackstone really yeah so blackstone for those who don't know yeah also own vungle and liftoff and basically they've made a lot of moves previously like in around our mobile space and it could be quite interesting to see what they would actually do with that i could see some surgeries right off the back there like that much like inventory it could be interesting but then again now when i'm speaking and realized that ubisoft are tiny on mobile so maybe i see exactly yeah but they don't have anything they have ketchup and that's it do they have anything else yeah a shark yeah assassin's creed and the shark what's it called hungry shark shark hungry yeah yeah yeah and some mighty magic as well so he's like what

what's that you gave this one like the bread and butter of the companies either farker or assassin's creed isn't it yeah yeah of course or or like watch dogs or like all these other like big triple a titles i don't think so that's that much kind of synchronicity with a mobile focus company well you guys just missed out on making 10 easy euros oh man okay yeah the other important thing is that it's still uh 15 is still owned by the gila moon family so whoever wants to go by ubisoft needs to go through them that's just saying and uh yeah i'm i'm still guessing that uh either they come out with something great or they're gonna be sold and that is also my kind of guess i i think that the analysis kind of stands there now that we are in this heavy period of consolidation is a great target like something similar than activision blizzard but also there kind of went down it's a big discount and easy prey for some kind of big tech company so why not but who is that big tech company could

be amazon's a question okay so would they be able to do how would they shoot themselves in the foot what what would they shoot themselves in the foot by acquiring ubisoft why why don't they why would they [Music] i'm not sure like if depends on if they want to be okay if they don't okay if they want to be big on pc and consoles maybe that that kind of works uh keep in mind that amazon is also the publisher of lost ark currently uh in western world so my guess is that i would not really take the mobile approach here as for like synergies with mobile whatever i would maybe take just the we want to have stronger and more robust gaming diversification so that could fill the angle okay but we'll see it makes sense yeah okay yeah activision blizzard is it bunker

bobby time banging on bunker bobby i'll let you do this one rima come on oh yeah yeah so this is not really a good time for activision the company-wide revenues were down 22 year-over-year to 1.7 billion um well net bookings led 29 to 1.4 billion net income for the period was down 36 to 395 million uh activision business segments of the greatest decline with revenue sliding nearly 50 to 453 million i guess spamming things with call of duty doesn't work forever we'll see and then uh activision also saw a decline for the quarter in their 100 million monthly active users which is down 33 year-over-year and it's their lowest total since 2019 the launch of call duty mobile and yeah and then we have blizzard also

having like a down revenue of 43 to 274 million for the first quarter with the drop blame on product cycle timing for the warcraft franchise hey like i'm gonna go set it bunker bobby is a genius like he must have known this and then flip the burger flip the burger before microsoft found out the meat is rotten to the core is they also announced that this is this is something actually they went against what they said that they will be launching diablo immortal the infamous mobile version of diablo on pc also in june 2nd of june and the game has already 30 million pre-registrations which i think is like a record or something i guess bandicoot had like 12 million pre-registers and that flopped like yeah yeah doesn't mean that much anyway uh they also said that they're gonna announce a mobile game a new one next tuesday may

3rd uh which according to the latest leaks by our famous or let's say famous uh journalist jason schreier who said that actually there were the rumor is that there were two mobile games in development one of them being uh kind of a similar version to pokemon go that's the one i guess we'll be talking about last few years and it seems that one get got cancelled and then the other rumor is that this new game gonna be similar to clash of clans so yeah really looking forward to it tuesday release or let's say tuesday announcement and then uh last part of the report says that uh pretty much the brightest spot for the whole quarter was king of course the candy crush guys which so revenues up 12 percent year over year to 682 million candy crush is just crushing it man like carrying

it so lots of interesting developments i would say yeah i'm i'm guessing like i don't really have a good good feeling about this mobile game in warcraft franchise to be honest also what the pokemon go like warcraft game seriously how would that make make sense i mean in which universe maybe they started developing it right after pokemon go so it was in development for like last six years yeah that looks like yeah that sounds like blizzard that's true [Laughter] oh but the clash of clans came that's going to be interesting like blizzard hardcore audience sees most probably well what can we do at least this is not done on blizzcon that's one thing they took a lesson out yeah of course

this is some other style of april fool's joke and we'll see what do you think by the way about diablo immortal like what's what's how that game's gonna perform i'm looking but i'm not sure if it's gonna perform in west rather than in the east right it looks i mean it's developed by nitis so i guess uh it's gonna be very close to that i'm just super curious about the monetization if they actually gonna pull the the korean lineage method which is the insurance trick man like i i've lost all hope in all diablo games like i love diablo 1 and diablo 2 with all my heart and then like my heart was ripped out of my chest for diablo 3. they just you know or something it was just oh i couldn't even play it i was so sad and upset oh at least now they're doing a decent thing of read like yeah like upgrading the the diablo 2 or like remastering it which is great but like

but that was not done by like internal blizzard studio was it not no it was that by vicarious visions there you go like i said heart ripped up no hope felix what do we have today uh data for beginners let's let's discuss that yeah so now we're moving on to the educational topic so what i thought that i actually wanted to talk about today i actually had some devs reach out to me after the rovio podcast yesterday uh last week and they were actually just asking me like how do i even look at the data so i realized that app loving an iron source which are the two big mediations at this moment that pretty much everyone is on one of the two they don't have a good guide for how to actually look at monetization data to know if you're actually performing well or not and i just wanted to yeah show you guys and teach you exactly how to look at the data because essentially it's not your

fault that people don't know this it's kind of by design so if you think about it like the perfect monetization client for app loving max or iron source is someone who plugs in max or iron source and then never does anything with it because when you plug in max like the two standard like uh the two standard networks that you always get is app loving exchange and app loving and if you don't have anything competing against that they set the price and then you get a lower price same thing on iron source the only network that you start off with is iron source so that's why like my theory is that that's why there are no good guides around actually how to look at the data because it's not directly in their interests right so like they're great platforms both of them and if you know how to use them they're extremely powerful so i just put together a short guide for how to look at the admon data on app

loving and i'll do iron source another time uh so this one is a two and a half gamers first as i have actually made a presentation so i'll try to narrate it a bit for the people who are actually listening but you know if you're really interested in this topic i would probably recommend that you slide over to youtube and take a look because it yeah a picture smash the subscribe button while you're at it so there's two topics uh that i want to cover that are like the very basics of how to do well with your game on app loving max and that's advanced reporting and how to run good a b test if you can master those two things you're already like kind of away from the beginner zone and kind of heading to the intermediate and then if we have time i want to cover one last topic that i kind of call a cya topic which is cover your ass one little thing that you

should also be good at so the first thing that you should do when you get at loving max like the most powerful tool like everyone always looks at the performance tab or the user activity tab so the performance tab shows you where like how much money how many impressions and your ecpm very interesting for like top level if you're a ceo to look at user activity shows the basic things like okay how many impressions are you getting for rewarded compared to interstitials your ad arp dial and stuff like that but none of those metrics actually like show you how well that you're doing and the first tool that you need to yeah get familiar with is advanced reporting and it sounds more daunting than it actually is but yeah you find it here right under user activity here is advanced reporting so this is what it looks like when you kind of scroll to it so how to use this tool is relatively simple once you know how to do it

so the first thing that you gotta like set is you need to set a date so what you want to do is you want to put yesterday's data minus three days so the reason behind that is that the data you're looking at you always get quite a lot of fluctuations because kind of the impressions that you're getting it's not always the same amount of demand it's like fluid so you know these networks that you're working with they don't always have yeah the same amount of demand for your inventory so what you want to do is you want to look at a short time period that basically shows how you're doing in yeah like the not for so foreseeable past so that's why you choose minus three days and yesterday and then you put in the application that you want to look at i had to blur about the actual application because you know stuff of course check the check the network button network placement and ad unit test and then basically you need

to leave that blank because we'll get to that later and then add type whichever ad type for the waterfall that you're actually using then what you do is you click run report and then you should come up with a bunch of data that's kind of below where you can see like the tables start below and you scroll down and you click the plus button next to control so what that actually does it it isolates the report you're actually running so it's kind of away from any a b test you're running so you're getting a clean data set then you can save this report and you don't have to actually put in all these inputs all the time and then boom that's how you use the advanced reporting so when you actually run the report and get the data uh yeah you can save here when you actually run it you get something like this below it looks a bit different so this is actually a yeah excel sheet of the data i had to blur about kind of some of the things that you actually see but this is essentially if you filter by ecpm

this is your entire yeah waterfall that you're essentially seeing mixed with placements and bitters and this is essentially how to look at the data so the main thing here that you're looking at is fill rate that's the most important thing that you should always be looking at so in this particular reason like this particular waterfall you can see that unity on top has 150 ecpm has a filler rate of 0.18 so this is pretty much everything i lost my notes all right here it is okay edit that out okay so you see here unit on top 150 dollar ecpm filling at 0.18 that's your top call so the main issue is when you hear a lot of

people saying like oh yeah you need to be careful with latency like what does that actually mean so there are a couple of golden rules that you need to know to actually look at this data so the first golden rule is there is a standard to make sure that you don't have to wait or your users have to wait for long for the ad to load which means that all auctions in your waterfall have to be done within 300 milliseconds and if they're not done within 300 milliseconds they get kicked out of the auction so what happens in app loving is that you get an ad request someone is requesting let's say a rewarded ad in your game what happens then is one of the bidders responds saying yes i received this ad request and that's the bidding so basically what you have then is bidders say what price they're willing to pay and then the auction from the highest price from the bidders in app loving max flows down through your placements to see if they can get a higher price so if you have too many placements in the waterfall what's going to happen is

they're not going to be able to get all these placements within the 300 milliseconds which means that you're going to get latency so there's going to skip over some of this so app loving max is built in the way that usually i find it that after 43 placements in a specific country that's when you start seeing latency without loving max and what happens in app loving max is it usually skips the torso so yeah you get the response on that kind of the top of the waterfall but then you kind of skip the middle of it and then you start on a lower price which means that if you have latency you as a developer earn less money so there's actually a couple do you need to have less placements exactly so you shouldn't have more than 43 placements per geo in your waterfall to avoid the latency you need to be within the 300 millisecond range pretty much exactly and usually that is around 43 placements on applobing it's a bit different on iron source but we can get to that in another video

so the golden rule here to make sure that you have a smooth running waterfall is no more than 43 placements so the kind of thing that at least applying have told me is that if you have a placement uh that fills that has a lower fill than one percent or is under one percent of your overall waterfall revenue you should remove that placement and that's kind of the rule that you should always work towards so you can see this waterfall that i've used an example is no nowhere near optimized so that's when you need to kind of start running a b tests to removing and adding placements to make sure it's under 43 placements and also to make sure that all the placements are earning at least over one percent of the revenue of the overall waterfall revenue and have more than one percent filled also another interesting thing that i found out is that the top 10 placements in app loving max it's not designed or they don't design

the waterfall like so if you have less than 0.0 fill a 0.03 fill in any of the top 10 placements that screws up the bidding somehow and then you get lower kind of yeah yeah lower performance so you as an admon person in app loving it's your like kind of job to make sure that you're like below these ranges to make sure that you don't skip any part of the waterfall another kind of housekeeping note that makes you make significantly more money is that you should never ever have like one network that has uninterrupted like bid flow so let's say if this if you had like a unity placement that starts at 150 and then another unity placement 140 and then another one at like yeah going all the way down to a hundred before you have an iron source uh placement what happens then is after about two weeks the unity bitter realizes that

it's getting kind of similar ad request on all the placements so then it will start pooling the fill where they're actually placing impressions at the bottom end of all the placements so you earn less money so what you should have is yeah always another network between the placements and that's how you kind of build the waterfall to ensure that you get maximum hamburger approach yep mate america is just being so good to you i love these keep them closed but this is this is all based on the thing that you have enough sdks implemented so you can stuff something in it that's the big example you always run yeah so the the like the for those who are just listening right now it's pretty much just an excel file that has not that much data in it except the fill rate so all the time when you work with ad networks they're begging you to send you exactly this excel sheet to see how they're kind of performing compared to

other networks so when you get you should never never share with them right you can share with them exactly the sheet that i have now but if i was sharing this with a network i would even take the network column that even has the name here of the networks yeah and i would call them network one two three four five yeah and the reason behind that is let's say i'm sending this to my unity rep and the placement that's filling really high above them is admob if they know it's admob they're not going to try to beat them because they know they can't beat the biggest network in the world simple so again mushroom rule keep them in the dark yeah keep them in the dark yeah so yeah so the next kind of most important thing on app loving is conducting a b tests and you run a b test to make sure exactly like you're staying within below these things that i outlined before but you kind of just need to get really

good at it it's not the first time you run it so how to run a b test on app loving is you go to the ad unit you find kind of the game that you're kind of managing and then you click on this little three dots at the top that's called yeah create a b test and what you do there is you click create a b test you set a name and then you have pretty much access to a exact replica of your your ad units and then what you should do here for the golden rules for a b testing is you should only test one network at a time and then you should also only ever remove placements at one like in one a b test or add them so you don't want anything to kind of screw up your data so you never run an a b test adding let's say placements on fiber and removing on unity because then you don't know if the increase or decrease in the a b test on

the money it comes from you removing unity calls and adding fiber calls so the way you run them is you either just yeah you remove unity calls in one a b test and then maybe you add fiber calls in the next ub a b test to kind of see if it works so one step at a time i beat this so can i like a bit of the floors floor prices exactly exactly yeah so that's what you're doing so pretty much if you're seeing that going back to what we said before if you're seeing that a placement is higher than one percent fill that should be an indication to you that that network is willing to pay a higher price or if it has lower than one percent fill on the placement that means the network is probably not willing to pay that price you should probably move down the placement and yeah or maybe get rid of it okay why should i yeah sorry why should i do the ap test and not just change it on the on the normal waterfall because you don't know if you're earning more money

or less money like you can't like isolate that you don't know so you need to keep testing these things and it's yeah it's a bit frustrating the whole industry is kind of moving away from this but it's kind of being a slow move like unity has their bidder right now there's a couple of other ones working on it but i don't see this going away for at least like another year or two how long does it take to get to see the results i mean is it like day or week month so it depends on that if really really screwed up i mean you can have some really massive gains like relatively quickly like i'd say a week or two um but if it's like only semi screwed up i'd say you see results in about a month or two so yeah it takes time to do it right but to do it right like that's when you see the best results over time so that's when you can kind of see the ad i'll gain some somewhere between eight to 17 percent i've seen it sometimes if it's really screwed up yeah i think the original question was that how much time it would take for

this a b test to run ah sorry it was actually the question no no i didn't answer the question but yeah yeah yeah i was actually going to get to that was a really good really good topic so how long do you run these tests so if you have a really big game like maybe gets like a million impressions these tests take about you know a couple of hours maybe a day but if you have a small game that maybe only has like 10 000 impressions a day like these a b tests are gonna take maybe a week so it's just about getting a significant amount of impressions on both the like the a and the b part of the test to actually make a statistically like sound judgment on which one is better and which one is not does these things have built in like statistical significance tests so it tells you like how long so when you're running the a b test what you look at is pretty much the revenue figure and that's what you look at so they're going to pull like 50 of the user base in one of the waterfalls and 50 in the other one so what you're going to pick is pretty much

when the av test is finished which everyone made more money so yeah okay make sense or did i lose you guys oh of course makes sense and you do this all the time pretty much if i understand pretty much all the time and the reason you do it all the time is what i said before is networks have seasonality like yeah they have more demand in december than they do in yeah towards the end of january because mache is buying yeah less ua traffic so yeah what machia does directly and like implies like impacts yeah my job so matty goes on holiday and traffic goes to yeah no no yeah i'm never on the holiday guys that's that's the thing you get he says dialing him from santa barbara well i'm dialing in am i so there you go all right so yeah that brings us to kind of the last topic on app plumbing which is a really powerful tool uh which is kind of yeah the cya thing that i always

tend to check at least once a month and that is the debug report that's called network comparison and what essentially this little debug report is is if you click into it you select yesterday to the minus seven days and what essentially this does is you take the network and the network placement box so when you actually click the run report here what you showed is all the placements that you have in a game but you also want to filter them by this revenue delta so what this will tell you is how much more off percentage of the price that you have told the network to bid at to what you're actually getting so if a placement is broken or if you have screwed up and setting the wrong price there would be like a very large revenue delta here and then you're actually not going to be paid what you're seeing in uploading so like a lot of like issues that have arisen is either like a placement that breaks

which doesn't have that happen that often or which probably every admon person has done at least once is they put go into like fiber and they set a price of 25 and then really they put 25 cents instead of 25 so in their head they think that and they put 25 dollars in app loving and you look like a hero and you're making you know much much more money and everyone's happy and praising you like a king and then you have to go and tell everyone that it's actually not real and you screwed up and this is the tool that helps you find like to find out very sad very sad indeed if yeah so this was actually uh all that i had uh in terms of education uh i don't know do you guys feel more educated now on the on the topic i definitely educate

you i think pretty good this is interesting but uh i still i'm pretty lost in all of this uh this so i need to you need to educate me even more like i i have been live for some time and and i i had to say like it's really important people grasp these like as you said that it's built pretty much against the user that you need to kind of be on stay on top of the system because you need to use it properly otherwise the system will use you in the end and uh you will pretty much enrich whoever kind of use you that's how it's built man it's it's not not done like like a normal thing usually that you accept like a third-party service being built so that's there but uh my question is how how often do you kind of test new networks in these or like the ones that are new on the market or like how how often do you look into some kind of new things you know because let's say i have my usual hamburger that i'm just rolling in a b testing doing how do you choose them

as well yeah yeah that that's fine so so the first filter process that i go through is if the subject heading of the email is a hundred thousand dollar like credit for implementing us then it's a direct no and you'd be surprised how many networks reach out to you saying like a limited time offer 100 000 integration bonus so that's directly in the garbage then actually like what it is is pretty much which i think is probably similar to you on ua right you hear gossip from other admon people like oh yeah this is working really well for us and this is not working really well for us and that's kind of how you go because the only people you can actually kind of trust is people who are actually using it so you also kind of know roughly which networks are big because whatever networks that machi or like any other ua person is using and spending a lot of money on they're going to have a lot of inventory that they want to buy so that's how you kind of go about choosing networks that you work with and how often do you edit them

when you can see like what if you want to have an efficient waterfall you look at kind of share of voice and share of revenue so kind of the unwritten rule that you should always aim towards is that you want to have network partners that are at least making over 10 of your overall share of revenue and if it's anything below that then basically you should lean on that ad network to see if they can perform better and over time if they can't that's probably a signal that you should remove them because what you don't have is like a game that has 15 sdks ad sdks that can crash the game and they're not bringing any value to kind of your bottom line so less than 10 that's usually a signal that the ad partner is not good and you should probably think about replacing them less than ten percent in what kind of time period like month or half yeah you can look over like a month but like usually what you do is if you look over in a month you can go like hey you guys haven't performed the last month and they're like yeah quarters a

bit slow so like you can only really squeeze it's been like three months where you've been below 10 share revenue like what's going on and then you say like oh yeah you have another two months to improve otherwise yeah i'm sorry but i need like over 10 and yeah that's how you usually frame those discussions okay so you need to think about how to like how many sdks you have in the game you need to fit them in the 43 placements and then every month or let's say every quarter just check how much of the you can squeeze out of them you can okay yeah yeah i would say like it sounds nice and dandy but you also have to keep in mind that behind every sdk there's a sdk there's a programmer that needs to actually implement it and as as you said the more you have in it the bigger the qa team will kind of scream at you that something's broken always during the update i've seen this myself yeah well we all see that

oh yeah what can you do you want to make more money then uh this is this is one way how to do it we need to find the sweet spot perfect well uh i think that's it right for uh yeah thanks man that's really that's really amazing next week uh hopefully ua i will prep something uh very similar uh oh interesting ooh yeah well you're for beginners why not i mean uh i've been seeing these like questions here and there what's what's ipm cpi ros all these like fun stuff but also like what kind of um ua basics everybody should know as well so i'm i will grab something for you guys yeah good thank you until next week yeah until next week see you lat right [Music]