The Art of Achieving Peak Performance with John Wright, Jennifer Donahoe & Di Weng

[Applause] how do you how do that go how how do that go we're still what I think we're still on yeah [Music] [Applause] [Music] okay should I do are you going to use the clicker or do you want me to do the clicker you [Music] hi everybody we'll give you a minute to to switch out this will be much more fun than privacy I promise no I'm just kidding he's still here so I'm not a fun person we we won't start with what the hell is dma this time yeah all right so um let's do introductions hi everybody I'm Jen uh I'm a footballer um so you'll learn a different fact about me every

one of these I'll try that John my name is Ste uh I'm the ours gaming performance lead at Google and I've been working on uac's app companes uh since the beginning uh 10 years ago so H over to Jen hi I'm Jen I'm a product manager at Google um working on app campaigns with a focus on gaming and I'm John wri I'm the vice president of mobile publishing at qu I don't know if you're on are did you guys hear him anyone no okay okay I can talk louder my not my natural tone yeah no you're just going to have to scream one second I just want to make sure cuz he's got good to say oh I cursed yes I was wondering when it was going to happen I am so not funny in IST this is great much more funny on the podcast technical difficulties so while he's doing that what we're going to do in this session is is talk about some of the different types of um acquisition features that

that Jen has been building over time we're going to talk a little bit about some key points oh yep okay yep back have to kill anym I was vamping for you um so we're going to cover different points in the life cycle and I think what's fun is we're going to dig into some specific kind of tips and tricks that you can use at different points in the life cycle so John why don't you introduce yourself again yeah back um yes we do it second times alwayss right not third time second time but yeah I'm John Ryan I'm the vice president of mobile publishing at quali and I lead the mobile strategy at the business awesome so actually we're going to start with you I think if you haven't had a chance to talk with John and what they're doing at qual it's really interesting the way that you think about how you bring games to market the life cycle of those games and the process is is really kind of differentiated from where I've been at the some of the bigger companies can you take us through how you approach that in the different life cycles yeah um it's not the shortest um sort of explanation but I'm

going to try and do it as concise as possible so we look at it very simply from the beginning of the life cycle to the end of the life cycle which is the hard launch we look at it as four key stages so the first part being the MVP the minimal viable product then the alpha which is a little bit bigger a bit more robust soft launch and hard launch which are self-explanatory but from a from an MVP perspective the idea of this particular stage is essentially trying to build out the core Loop of the game so what you're trying to do here and and Jamie berer said it very good U very well earlier which is like you're finding the fun right you're trying to find that aspect of the game which is going to draw users back time and time again and the idea is that you are you know building out this experience that again can create this stickiness and uh essentially yeah you're building something that people love um when you actually start to you know go through and and start looking at

the actual metrics around that and and you know depending on where you are if you're not before you start testing marketing or after but really what you're looking at is the session length so how long people are staying in the experience for they're looking at the day one retention and then when you start to talk and we'll talk about marketing too is you'll start looking at your CPI right so your your CAC your you know consumer cost you know you're essentially looking at how expensive it takes to acquire a user for this experience um and and yeah so so essentially you know in those early days what we're really testing or doing the same as most people here I think will be looking at Facebook um which you know traditionally in the industry has been the go-to sort of testing platform um for mobile games at least in my career and uh again you're looking at the rationale between Facebook is more because of the targeting options and the scalability that it has so a lot of the time you can get lower cpis so the idea is again from a marketability standpoint

you have uh sort of benchmarks that you set you know for each stage of the uh the experience and as long as you hit those benchmarks you move on to the next one so yeah we're looking at day one retention we're looking at you know um uh CPI and if we think the CPI and the retention are in the right bounce then we move on to the next part of the experience which is the alpha so the the main difference between Alpha and MVP is MVP is normally 1 hour of content whereas Alpha is anywhere between 4 to 6 hours of content the idea is that if you have an average session length of say 30 minutes um this would give you enough content for 8 days and at the alpha stage you're trying to look at more around the day three and day seven so you're trying to build out you know a larger experience more content trying to give people the ability to come back and normally you're looking at things like lcv curves and and projective modeling pltv all of that stuff so yeah once once you've got the game to that particular you know area this is where you start to do more

robust testing so you normally stick to again Facebook predominantly but you also add in like an SDK Network so normally what we would do is we would go a test on Unity or iron source and we're looking for a comparison of what does Facebook bring versus the sdks because at least for us 70 to 80% of all of our user acquisition will be done from sdks rather than other platforms um so yeah once we get to the game into the point where we think you know the day three and the day seven are are starting to you know Trend upward well Trend down but the idea is that it'll be a positive LTV curve something that looks quite long something that we believe can go to the next stage then this is when we start again going to the ne the the final stages which is around soft launch and hard launch so soft launch predominantly is just an area where you start to add more channels and you start to have a more robust view of what the users are and the quality of the users so this is where we add Google predominantly and uh and then we start to compare a lot of the you know

performance metrics not just you know how the retention or the arpo and the AR Dow cuz by this time we would have already implemented the monetization we're looking at ads we're looking at the ability to implement shop economies and balancing and then later we'll start to implement live Ops and and start to actually grow that out a bit further so yeah we we're every stage essentially has different benchmarks depending on the product side or the marketing side and yeah ultimately at the end of that if we add all of the networks and obviously with scale comes cost and cpis go up as you continue to scale so yeah at that point if the we add loads of networks we add Google we add uh dsps we add Tik Tok if we feel like all of those channels and we add you know both os's all the countries the transition from soft launch to hard launch is basically just a phrase It's just there is no difference anymore we're not

when I got into this industry we would talk about like burst campaigns and the ability to get something to the top of the charts really quickly it's not like that anymore like you can if you look at some of the great you know great games in market now like pocket Champs by madbox it's been live for 18 months and it's a steady curve like this and that's really what we want we don't want these short sharp experiences and and burning through them within three months we want to stagger it over a long time to ensure that the games are going to be Evergreen so I think I said a lot no that's great I think it's just interesting coming from my past where it was the shark fin it was the burst to the top and then you write it down because the Organics that you would get from being at the top of the chart mattered but now that there are no charts so like that doesn't matter quite as much in interestingly though we do still calculate K Factor so we have like a bunch of internal tools you know um pltv modeling uh a UA team who are here actually you know use tools to actually do predictive LT LTV based

on different UA sources and we use that in our data but K factor is still something we calculate and whilst you know Misha said earlier it's it's dead we still see maybe 15% you know for for K Factor so it's still something that we factor into the calculation um when we're talking about how much to spend yeah so AUA does Factor it in but you know 10 years ago Jen I remember seeing 40% K Factor 50% K Factor so yeah yeah I in uh at at Riot I had to teach them organic co-actor was the was a phrasing that made it more palatable for them and we saw around like 20% and it could go a little bit higher depending on on the game brand but one of the things that you didn't cover in in this in the life cycle was pre- um and so the one of the stories that I wanted to tell really quick um to give Jen a little bit of Kudos here is um at Riot launching TFT we utilized the pre-registration campaign from Google and it did had two

key benefits one was it allowed us to get an early indication of how our UA was performing which was actually really great like we were so surprised how low the cpis were um and so that was like a really good indication for launch the second thing is we were able to move people into the closed beta and we were able to do that in the regions where we needed to do that because TFT was a very Regional specific game we you know with the it was the first time an auto battler had really come out um on mobile in that way from and it was Riot's first ever mobile game launch we wanted to be really specific about the countries we picked to make sure that there were enough players playing the game because you have to have you know a bunch of like eight players playing in the game so um so Kudos five stars um that's my rating for you on that product thank you very much so what else um so we covered you know all of the different stages of the life cycle how do you kind of line up your products from Google against some of those yeah um you know Google offers a number of different products kind of across product areas to really

help um throughout a game life cycle from everything from pre-launch to launch and postlaunch um so I guess for starters we recommend integrating the Firebase SDK early on in your game development there's a number of different products that Firebase actually has that are quite useful um so Firebase app distribution for example um makes it really easy for developers to distribute pre-release versions of their games um to trusted developers and then they can get some valuable feedback um before wider release Firebase crash litics is also super super useful it gives you insight into stability issues um and bugs which is obviously quite important in the early game stages right before you launch you want to know what could go wrong right before you release something into the wild so um you know this Fire based crash crash litic product really allows you to figure that out um and make sure that you know gamers are going to have a really good experience um and then finally Google analytics for Firebase uh is really important just from a first-party data perspective um so it you know it really collects all

that first party data that you need to measure in up events um and really understand player Behavior beyond the install so um you know that from that perspective it's also very very helpful not only for like game development and understanding users but also once you start getting into user acquisition and need that data you know to feed Google app campaigns for example um Google Play obviously is a great place you know for game developers um and during the launch period you talked about the pre-registration product which you know is usually run within the first 3 months uh before a game launches really is used to build hype and get people excited particularly for hardcore games um captures early interest you know when a game launches um open testing is uh a product that you also spoke a little bit about closed testing open testing um which really allows developers to make a game available to a broader audience um who can really help test the playability and overall fun of a game before it actually launches um and at this stage kind of in the open testing stage you can also

start thinking about um you know how do you want to monetize your app how do you think about user acquisition for your app and things like that um so you know with that with those stages in Google Play we obviously on the Google ad side have complimentary products to help support through those different life cycle phases um so you know Jen you spoke about uh app campaigns for pre-registration which helps with that pre-registration burst to get people in early um app campaigns for installs supports both uh open testing apps as well as uh new apps so you can uh engage with users who are more likely to complete an install a post install engagement things like that um and then actually we do have a product for re-engagement so um once an app is launched you can run an app campaign for engagement uh to help advertisers or help re-engage with customers who have installed your app um so yeah those are the the Google ads products um and then finally obviously you know we have our UA products but would be remiss if I didn't mention um Google's offerings to

help developers maximize their monetization strategies so you know a lot of games historically were primarily iip focused I think we've seen a huge shift in the last couple years over to you know hyper casual and now hybrid casual and so admob um you know has been really really important um in helping you know developers set up kind of a more holistic monetization strategy um so yeah recommend using that as well lots of stuff yeah you use all those John very much I'm going to I'm going to look down to aish Center from my team and say just nod guys yes yeah we do great thanks thanks so let's zoom out a little bit John and um I I know you use all of the Google products but what else are you seeing across some of the other networks what are you using what are Best Practices yeah I know we spoke about this offline as well Jen but from my perspective I feel like a lot of the networks are predominantly sdks but some of the other smaller companies you know everyone's smaller than Google right so uh but I feel a lot of people have developed a lot of interesting sort of

Ros Optimizer based tools so you know like seven eight years ago I was the person behind the scenes that was doing like the bid level optimization on the source levels right so I was going to different countries going into the campaigns looking at the quality of that particular source which is an app and then I was making adjustments you know manually now you have all of these companies basically you just plug in your day three goal or your day seven goal and it it will optimize towards that um so you know um Unity iron source and app loving all have this particular technology and a bunch of other people do too I think interestingly I found out that um so opposite to what you're saying um Jen we're seeing more um from we were an ads first business which meant that you know hyper casual predominance which was doing 95 98% of Revenue through through ads so um we had really good modeling and LTV prediction based on ad Revenue data rather than inap purchase data and I think a lot of the people in this room will be learning

more casual sort of strategies that will allow us to do more inapp purchase but Unity told me last week they're doing an inapp purchase based roas Optimizer now rather than a you know trying to optimize towards paying users which ultimately for king and the big casual guys has always been the goal right percentage of paying users so that's that's definitely one thing um apart from that I guess you know it's it's more to do with the other sort of um the other parts of the industry that maybe uh getting a lot more um sort of focused right now we know Tik Tok has got a lot of uh mentions recently you know in terms of like especially from a virality element so if you're able to create a particular Trend or something that sticks you know Dumb Ways to Die made millions of users off the back of you know some some sort of smaller comp yeah exactly I mean you can't the thing is you can't plan for that virality is not something you have a strategy for it doesn't work but you should have it as part of your strategy and hope right

it's like a hit and hope strategy but yeah I mean apart from that U I have a friend at a company that rhymes with something flicks and they um had a verality um a moment with one of their games and he was telling me that he was being challenged to like repeat the reality and he's like that's like the definition of viral like you kind of if we could make it we would all make it like billion doll idea if someone could actually build something to do this but you know apart from that I would say apple search ads is potentially one of the fastest growing sort of elements um not the and CTV not the crap on Apple search as but um have you found them to be successful I I I yeah so this is this is It's Growing but I'm not seeing the results yet yeah it's the interesting thing about that platform is everyone's like spending on it but they never it but they feel grossed about it because they can't actually see the data

that shows and they're like well it's defensive and it's like well but is it really so yeah I don't want to get into the um yeah to the Crux of that to be honest with you because yeah I know we're at the competitors event but also I've Got Friends at Apple so we're being recorded and then the other one that um was interesting I was just at um a Snapchat event and it was it was cute so this quarter they're introducing roaz uh targeted campaigns for the first time and it was welc to 202 oh like welcome to this decade it was it was it was cute like I'm glad that they're getting there um because I do think having more networks and having competition is great so Jen from your view how are you seeing everything that's going on and did you like that we just on some other people was that fun for you no no no absolutely not I mean obviously you know we all need to be here to kind of make the the overall industry successful right so um you know I think in terms of what we're seeing in the industry you know John really hit the nail on the head value based bidding um you know

value or Ros optimization has really grown in popularity over the last couple years um particularly in the gaming space right because game developers are among the savviest and they really understand LTV they understand how to Value their different user cohorts and things like that so um you know we at Google obviously have had a tros product that D worked on you know many years ago and it it's growning success a lot the last couple years and I think most notably um you know the original t product was was built with I and mind um and so in the last couple years two years ago we launched um a tr- r for ad Revenue which allows you know advertisers to optimize for people more likely to engage with itap ads last year end of last year we launched tros for hybrid monetization which again gets to that kind of hybrid casual uh Trend that we're seeing where folks want to you know both capture users who are going to pay in a game as well as people who are going to engage with ads um so you know I think um you know things like that are are the future and you know I think D's going to talk later on I think a little

bit about pltv but you know just more and more different and diverse ways of of monetization and bidding so I think um we'll talk a little bit about creatives and it actually will go into the next session we'll get even deeper but from a creative standpoint what are some of the tools um that you have uh available for everyone that can help people really optimize their campaigns yeah so I think I'm going to keep this section short because Amit is speaking with you in the next session about creatives um but just a couple of key tips um you know there are only a few levers that we have in app campaigns uh for folks to to really move the needle and creatives are obviously one of those um so you know we recommend that you max out your asset mix for Optimal Performance um you know we have tools like ad strength and asset reporting that you know can help you see what kind of assets you're missing or how certain assets might be performing or you know how you should be thinking about switching them out um and you know we're just continuing to expand our different tools and Reporting um and testing capabilities to to help you know with creative optimization and again amit's

going to talk a lot more about that in the next session so please stick around for that cool thanks all right d it's your chance to shine in this panel last time you we threw dma at you so now let's get you into your traumatized yeah okay I think we're all traumatized from the so tell me about your products and what what you've got building it it's exciting to have kind of like Batman and Robin which one's Batman and which one's Robin and it depends on how you ask and we work together uh yeah Who Am I who am I in this situation Joker um you're the little I'm alred cuz I'm Bri I take offense to that yeah uh as Jen said right we've been working on really focus on roas for the past few years we launched the I roas around 2020 I believe uh then 2022 we launched as revenue roas and last end of last year we launched the hybrid roas uh which was actually very complicated because often

times right as a game developers if uh users certain making purchases you reduce out or complete outow right so actually turn out be very hard to model that relationship but we're able to do it so so th those are the various Investments we're making in TAS do you want to get nerdy about pltv I was bugging you guys earlier about I know I pltv earlier so yeah so I mean as John said uh you know right often times for game developers and just by the way this keeps surprising me how sophisticated our uh customers are often times like I have in in-house data science teams right and as you know as a UA marketing performance stuff you really I mean LTV means long or Lifetime right and he really wants to know how much those use are worth to you uh very early right so

a lot of shelters develop their own predictive LTV model so you can and and then now actually a lot of advertisers are successfully using that model with t right so instead of uh passing as real uh IP values uh if you're at three or seven days or whatever uh you can have a good pltv prediction you can pass those back to to Google as companies right and we're optimized for that uh so uh I one thing I would say is that it's very hard to actually have a very good pltv model so if you don't have a data science team uh don't try it but if you do if you're pretty confident in your model this actually can work really well then I think one of the main advantages I seen with pltv and I think it's true for why it was developed in the first place is that it dramatically shortens the window uh for your own measurement right so if you set a pltv and say 3 day or 7day window with app compains then within our window 3 or seven days you

can start seeing how well we perform right red to your goals and this allows you to tweak creatives bids budgets such that you will help you achieve your your uh monetization goals got it well thank you guys that's all the time that we have it was great to hear some of the new products and John congrats on everything you got going on at quality lots of great stuff thanks everybody thank you everyone good job good job team Batman or [Laughter] Robin thank you jff