Session: Can AI Keep Winning Ads Alive Longer? Zach Van Driel, Miri Growth.

[Music] [Music] Up next, I would like to invite Zach from Mary Growth who's going to be talking about whether AI can keep winning ads alive longer. So, take it away, Zach. Good afternoon, everyone. I'm here today to talk to you about whether AI can help us keep our winning ads alive longer. I'm Zach. I'm one of the co-founders of Mary. I should be joined here on stage by David, but unfortunately he was taken ill, so he wasn't able to make the trip out to Helsinki. Um but David David is the senior online marketing manager at Bytro, one of our clients we've been working with this year.

Part of the Stillfront group, and they've got a number of games and hit titles including Conflict of Nations. And we've been working with them this year to kind of find new winners, new creatives that are going to scale. But then there came the question of how can we make those top performers, those hero creatives last longer? A little bit about Mary Growth. So, we're a performance marketing agency. We've been founded in 2017. We're working with a number of really fortunately working with great partners a lot based out in here as well. We are a UA and creative studio, so we work a lot managing UA campaigns across paid social DSPs, but also producing and testing a lot of creatives. And for over these years, there's always been that same case of how can we find more winners, how can we make them last longer? So, together with David we had this goal, how can we double the lifespan of a creative? As when you're testing creatives, I think a lot of a lot of you may also be familiar

with the idea that lots of new ideas you have fail. Sometimes the ideas which you have the least faith in also turn out to work best. But the hero creatives you find, the ones which really scale up across your core UA channels can radically change the performance of your game or trajectory, but also your UA performance and profitability. And unfortunately these these concepts do not last forever. So, across the 100 millions of dollars we've been spending over the years, we've seen that creative fatigue can start to kick in around 4 to 8 weeks. Um for those hero creatives, it can be shorter, it can be longer, but on average it's 4 to 8 weeks for that effectiveness to really start to drop quite drastically. That'll be IPM starting to drop, but also ROAS starting to drop for those creatives. So, how can we go about doubling that lifespan so that we can be able to scale up faster, perform for longer, and ultimately grow the games we're working with? There are for us three ways that we can do this. One is we can combat auction fatigue, so that's really just refreshing the creative so Meta or

AppLovin or another channel thinks it's a fresh creative, gives gives it a fresh chance, and reaches a slightly different audience. Another thing is to make small changes that can have a big difference. So, that is combating people's fatigue to a concept, so whether they've seen the same ad multiple times, and they kind of know that they've they've seen it, the frequency goes a bit higher, and they kind of drop off a bit quicker. Third, if only we could just find more hero creatives, the more that you have, the more that spend will distribute across more creatives, and therefore the longer that each of them can last because the spend will be slightly further distributed. So, what we wanted to do was spend less time thinking about those first two which feel quite tactical, and spend more time, more of that human creative energy thinking of new concepts, understanding what's going on in the market, picking up on trends, and ultimately finding new hero creatives. So, we thought this is a really good opportunity to use generative AI to approach those first two problems.

So, to start, we thought about what are our options, and we pieced together a bit of a cheat sheet looking at the various ways that you can iterate on a hero creative. That may be with different video assets, with different audio, or different static assets that also in in that hero concept. And then we broke it down a bit further as to what can you do today with generative AI? And this here was something really practical that we were able to just use, look at a hero concept, and think about all those key variables that we've got to play with that can save us time by not having to execute so much on these on these versions, but then have a quick pace of finding new hit creatives. So, for example, a big one which we found which can have quite high impact is testing out that intro hook, which may be a different intro that could be more cinematic base. So, having something which is like showing more of a storyline, it could be related to the game but using more live action, which tends a little bit more to UGC. For example, you've got a shot out a train window. Best games to play on the train, you kind of jump into

launch into your game specifically with that hero concept which you know converts and engages. Other variables include iterating on the AI voiceover and script. So, you may not have a voiceover in your original one, you could add on a different voiceover, it could kind of play on a slightly different narrative to your original hero concept, and it can re-spark some some new engagement to kind of draw people through longer to stay in the ad, and therefore get them to convert later on as well. Within that, we also see like for example quite a key aspect is the sort of copy you use and play with. And I think within AI within sorry, within UA and the creative world for mobile gaming, we don't talk enough about how we can how we can use copy to our advantage just because often the copy is quite short. But there's lots of different tactics we can also use. All of these on the right for example, and this is ways that you can actually get a tool like ChatGPT to spit out different ideas for copy voiceover scripts for example that has a lot more variety. I think one of the things we

learned over the past six or so months is that if you're asking for the same sort of iterations or trying to think of different ways to do it, you've got really got to come up with a very new way of asking the same sort of question to get a different output. So, this was something we found to be very useful for us. So yeah, so this is overall like our cheat sheet. And then over time we put those into learnings. So, across different across those same key variables which you can iterate on, we've got different core concepts. So, that may be an animated concept which could be something like CGI, it could be gameplay based, it could be more of a narrative plus meta gameplay ad. And UGC is quite self-explanatory, but you've obviously got varying levels of UGC within concepts that can work, and static ads as well. So, we were able we were able to build out this matrix from our learning so far of those where you can focus to have the highest impact on increasing your IPM, recovering that IPM, your CPI, and ultimately the ROAS for those hero creatives by focusing on a few of these specific

variables. So, some examples. This was together with Bytro that we were able to to share some of these results, and we've got an example control concept here. So, it's this is Conflict of Nations, you've got a game where you can battle as players across different across different countries. This is one of the hero creatives, and we thought how can we refresh this because we were starting to fatigue. And obviously quite a big variable here is the map, the background. So, for some concepts that may be a smaller variable, for others it could be quite big, but here quite clearly there's a lot you can do with it. So, we were able to take this, and then iterate very quickly on the backgrounds, day, night, futuristic sort of map, historic type of map, something more sci-fi based. And then very quickly just trying out a number to go into testing. And this was the variable which had the highest impact on this creative. It drove a 36% improvement in IPM compared to the control ad, which obviously in

one aspect there's learnings that you can take from there and apply that same I mean in this case it was the night visual which really really stood out, and you can apply that to your new concepting. What this allowed us to do is we've seen our we've had a hero creative IPM start to drop off, we're starting to get to that baseline where it's not really helping you scale, not helping you kind of reach that next level which you're looking to get. And it helped us recover, keep it going for a bit longer, so we've got more time to find that next hero creative. Another example is sound effects. Let's see if this It's quite quiet. But just to show you the variety of sort of sound effects which you can generate nowadays is is is really impressive. And especially if you've got a concept where you've got sound which you can explore within that first 1 to 3 seconds, like that that really important hook. It can very it can very much just catch someone's different attention and paint a slightly different picture as you kind of go forward. So, these are net new sound effects. You can play with lots of different styles. Obviously sound doesn't have a visual, so we've got some lovely fonts here. But you can

play with something which is more humorous, more horror based, something which is a bit more classical as well. And all of these will just catch a slightly different piece of attention. Not only will it just like be a fresh creative to kind of start scaling, but also it can have a like you've got the same core concept which you're able to then work with moving forward. And in this case we saw that playing around with sound with the sound effects for this concept had a 17% improvement on CTR. We actually learned that the conversion rate even though we had slightly higher click through rates, the conversion was slightly lower, so the impact on IPM wasn't as strong. But combining these with a few variables allowed us to kind of keep going there. And last example here you would have seen sorry, the the core concept has these very pixelated here, maybe from further away you can see it, but they've got little vehicles here as well. Which for us within when we're building the UA creative, when we're actually testing, and we know we're putting time into every single individual asset, we're like, oh this is quite small, let's not iterate on it. But, we're able to generate a lots of

different versions quite quickly, so we thought, okay, well, if it's not going to take so much time, let's give it a test, let's give it an opportunity. We're able to remove that human bias. We actually saw that there was a 12% impact on IPM. So, something otherwise we would have ignored because we would have thought, is it not worth the time to be investing in refreshing that uh that asset? Actually turned out to have a lot more impact than we otherwise would have thought. Actually, lots of positives we can say from the experience with Byte Trade and what we've been testing and what we've also been exploring with other uh other gaming developers as well. Um but there's a lot of questions of what we should avoid and the things we should and shouldn't do as well. Um so, there's actually one extra um key thing that we've we've got as part of our process of uh of iterating uh on key concepts with AI is that we're never uploading those assets to a model to start to learn. We're never uploading those that your your own IP. It's not something I'd kind of recommend. I think it's actually like through careful prompting, you can actually generate assets which are quite similar to to

your art style that can really fit enough for a UA ad, not necessarily well, I think for your for a game as well. But when UA is so fast-paced, you're just really looking for something to to keep going for another few weeks, we feel playing around with in-game assets is is perfectly fine. Um another thing linked to that is we won't be wanting to steal IP. Um you wouldn't ask your artist, paint a Transformer, put that at the front of our ads and and use that IP within our concept. You also wouldn't ask uh ask an AI tool to to do that either. Um although it does look quite cool here. Um Another piece which we try to avoid is creating key IP. Um so, as you saw the kind of variables, even though they can be quite big in a creative, like the background, the map, the style, the art, um we don't want to be creating something which becomes very closely associated with the game and then you you set off expectations which aren't uh close to what people would expect. So, otherwise you're really stewing away from that core concept which you've which you've got. Um

And lastly, it's um what we want to avoid is lowering our bar. I think when it comes to having access to high volume, you can often say, oh, let's just try absolutely everything. And I gave you the example of vehicles where it was, oh, that's a variable we wouldn't try, but here we've got an opportunity to try it. But that doesn't mean we're just going to be spewing out lots of variants of things that we otherwise aren't happy with. Um And linked to that idea of um high volume of testing, some of the challenges we found. Um so, again, you've got access to a lot of volume, do you test everything? Um there's both the the visible cost of testing, so all of the ad spend which you need to put behind concepts is real ad spend. So, even if you're not spending a lot, if you're not spending it effectively, it's not a good use of investment. Um a way to avoid that is really by focusing on those already existing hero concepts, the success rate will be a lot higher and you can just really focus on the details. Um but there's another another piece is operational, just having a lot of volume, you've got to have uh people to

sign things off, you've got to agree to go into testing. Um and you've got to be rolling them out as well. So, automation can help with a lot of that. But ultimately, what we want what we're looking to to achieve in the longer term is to uh is to simplify and find exactly what those high impact things are, roll them out exactly the right time. So, it's less about experimentation and more about solving that problem of ad fatigue for a hero concept. And we haven't quite got there, but we're working on it. Um Another challenge we found which we've talked about a little bit is human bias. It's difficult to to remove completely. Um so, sometimes we'll we'll still be making that decision of that's not worth testing or let's prioritize this. Um which uh which is something that you should always try and challenge. That's how we're trying to overcome it is to make sure we don't only go with our assumptions. Test one like if we've got a new variable we want to try out, at least give it a chance once or twice, not do it 50 times immediately, but give it a bit of an opportunity to see how much impact it can have. Um and the third challenge we faced was

the limitations with gen AI. Um linked to not lowering the bar, like finding assets that really fit the style can take a bit of time and really working out ways of working with your your tools. Uh so, ChatGPT, Midjourney, uh Runway, Pika, um to name a few, these are some of the tools we're working with or we're connected with. Um Spending a bit of time to work out best practices on using them can really go a long way to actually helping you implement these iterative iterative uh process. What do we think's coming next? So, this is what I was kind of mentioning earlier on, we don't want to just have 50 or 100 iterations of every concept we've got. We really want to only focus on the ones that have the impact um because time, ad spend as well. Um so, what we're looking and what we think is to come as well is identifying those key variables for a specific concept without having to test it. So, got a new hero concept, we know that in 2 weeks it's going to start to

fatigue. Um and we also know that for uh over time we've learned that for example, generating realistic background with a serious storytelling voiceover can help us drive a 9% improvement in IPM. So, as soon as we see that 9% drop-off, we've got a queue, roll it out. Uh this is something which we're striving towards. Uh it's going to take us uh a bit more time to get there, but something we can we really think will help us uh simplify uh our our iterative process. Another thing, what's next? Um So, if you remember at the beginning I was talking about those three ways to uh to find new hero creatives or sorry, the three ways to double your lifespan of hero creatives, the third way was to find new hero creatives. Um We think that AI can uh play a part in this as well. So, um in taking and learning from what's going on in the market, what you see competitors are doing and other hero concepts outside of your specific game, um we can see that we can take that uh competitor concept, break out into frames, get AI to analyze those frames

and spit out the storyboards. That's quite there. Like that's already quite there. You can do that, human can do that, you can ask AI to do that as well. Um but then taking that and working out how to adapt it to your game is kind of the the golden key which we find as well. Um like copy-paste we don't see works. But if you take what works for a core concept, what's what's working and trending in the market, I think how can I really adapt this to fit to my game and make it relevant for my game? Um this is how we've we've uh we've over time seen that you can find those hit creatives. So, this is what I'm hoping to come will help with ideation. It'll be kind of a brainstorming tool as well. Uh it can help us tackle that third column which isn't yet uh at least on our side um something we've been able to tackle. I think I've got a bit more time just to wrap up on our key takeaways um that we've learned especially alongside Byte Trade. Sorry to David not being here. Um on focusing on those high-performing creatives already. So, we don't see

these AI iterations as a as a tool to use for new concept testing so much. Obviously, you can use generative AI to also help you produce new concepts, uh but we don't see like this hot fast-paced of uh iterations being for that that avenue of new. So, take those ones which are working well, uh revitalize them and help that give you the the space. Um And if you can do this systematically, this is what we see uh can help us combat ad fatigue. So, really being controlled and thinking about exactly what map you want to do, not kind of just again having so much volume there. Um and ultimately, this helps us uh on our side, it's like freed up our time to empower human creativity. Um so, really thinking through, what are those what are those big ideas that we can work on and what are the the the fresh new concepts, animations, narratives, storylines that we can use to engage new users uh and win across paid social. So, that's all I've got for you today. Thank you for your time. [Applause]

Thank you so much, Zach. Does anyone have one quick question? Can we get a mic over here, please? That's fine. Hey, super interesting. Which diffusion models are you using? Like low-level, flux, stability AI and which combo of UI? And the second question, do you fine-tune or train your models to be sure that you can reach exactly the output you want to reach? Good question. Maybe I should have talked about this a bit earlier as we were talking through the kind of tools that we use. So, for us, um we're not developing our own AI model. What we've been doing is using the tools already available, so it's something quite practical to kind of jump into yourself as well. Um and the sort of tools that we're using for like copy would be ChatGPT. Um for visual status, we've got Midjourney. We use Runway a lot for turning those into videos and bringing them life into animations. And then Eleven Labs for for audio, sound effects as well. Um so, I guess that kind of answers both of the questions in that

we're not we're not building our own model per se. It's getting super expensive because these tools are the the cost is just increasing. Yeah, I I I mean, maybe it's from our side it's a benefit of being in the agency in that it you can kind of collect you're working across a a number of games, but I think across each of those, they range between like 50 to 100 dollars. Like obviously, the moment you start connecting APIs and building them into automation, that is that's when those licenses do become a bit more expensive. But I think for kind of jumping in for for developers today, it's it's not too costly to use pre-existing tools. Um on our side, we've we discussed and I think we would love to to explore building building our own and that is a conversation we're kind of having at the moment. But it's also I feel like the cost investment to to build your own model uh at least to us at the moment, maybe really helpful if you've you've got the experience, you can tell us a bit more to it seems like that's going to be more costly than using what's available. And it's I guess we're approaching this in like proof of concept phase at the moment of like does this have impact? Does this answer our question? So far it seems to be yes, it's helping us extend the lifespan.

And then how you can make take that even further I think is the next step. Thank you. Brilliant. Thanks Zach. Can we give him one more round of applause please? Yeah, sorry sorry we just ran out of time for questions. [Music]