Fireside chat: How To Maximise Your Creative Win Rate - Zach van Driel & David Westerman.

[Music] up next I would like to welcome Zack and David to the stage and they're going to have a lovely chat about how to maximize your creative win rate so uh welcome to the stage good morning everyone um nice to be here today U myself and David were here to talk to you about how we maximize our creative win rate in UA brief introduction I'm Zach um the co-founder of M growth um I have a background being an internal UA and creative team um and about seven years ago co-founded and built up mer growth points markety which we'll talk about a

minute all right I'm David um I've been part of vitro for the last six years started as a junior make my way made my way up and yeah you can find me now in London talking to lovely Zach um little bit about Mary um we're a performance marketing agency uh we work a lot with mobile games to help them grow through user acquisition and creative strategy um and a big part of that is the creative strategy which is what we're here to to talk to you about today we work a lot with um game Publishers across a range of genres as well so we've collected a lot of learnings over the time um and a little bit about bitro yeah so um bitro was founded back in 2009 um it's been part of the silr gro for more than a decade uh based in Hamburg um and we specialize in LTS so you might be wondering what the hell is LTS so we like to coin this term a long-term strategy so if you think about a game

like Risk in real time up to 100 players in the same map uh that's more or less the idea these are the most notable titles so hopefully you've seen an ad maybe you clicked it maybe you have your game maybe you have one of the games installed uh so that is it nice um and to kind of set a bit of context uh for anyone who's managing UA anyone who's growing a game you know how critical creative is um critical in a sense that it can change a trajectory of a game's profitability in scale and growth um a lot of that we see comes through hero creatives these are creatives which redefine half your CPI maintain your ltbs and help you scale across a range of channels um these are hard to find um they are uh you have to fail a lot in order to find them and once you do find them they don't last forever so you're constantly fighting creative fatigue as you're trying to scale up your game and find new winners um so generally we see it can range a lot but we see around

four to8 weeks that a hero creative can last until it starts to fatigue and you need to find that new win so I'm not sure for you David how how that looks the ratio of your win rates and create fatigue I think that's probably yeah in in range and in line so I actually wanted to connect with a little bit of my presentation or introduction um my since I started as a junior also my job description changed over the years right and I find myself now started as a p p UA manager um I've dabbled with creatives and I've been focusing on the creative side a little bit more and this feels like my day-to-day so it's basically trying to delay this um fatigue as much as possible while you work uh to try to find that that that new hero so trying to uh make this period as long as possible while of the creative fatigue obviously um and try to shorten the the the period until you

find a new hero creative yeah absolutely I think an ideal scenario is you've got your backlog of Heroes um but you're not always in an ideal scenario in in the world of UA um so we're here today to talk to talk about a few strategies we've learned over times uh so uh we work we work together with bitro for about six months I think um across finding new creative winners um so we've got some long-term strategies uh which we've learned over the years across JRE as well um about adapting your creative strategy based off of the performance you've got in time so where you are is today how that dictates your next month and quarter of creative production um improving ugc so splitting out focusing on finding winning U ugc Concepts versus animation and gameplay Concepts um and then we've got some case studies looking forward and a bit more present on um trying to uh improve your win rate through AI iterations as well as predicting what can be a high performing ad before you go into production and then if we've got a bit of an open

discussion um so we'll start a little bit um on your production strategy based on web performances so if we're looking at where our win rate is um at the moment and our performance on the vertical axes where all of us want to be is in that top right stage where we've got got a strong win rate and performance is good where we see these cases and you're building out for scale um there's a there's a few things which we've learned over time which can help you stay in that quartile for as long as possible we've got a 4 60 ratio split between New Concepts and iterations you've got to maintain your safe bet if you're going for scale your creative fatigue is going to kick in faster so you've got to get some safe bets where you're iterating on your high performing Concepts um and then focusing on your New Concept exploration um to keep exploring for that new new winner um top left quadrant is more of an interesting one your win rate may not be good but your game's in a good place seasonality is performing well so you've got a bit of breathing room um this case we see

that focusing really on your New Concept Direction um can be can be what can help get you into that top left cortile um and therefore thinking a bit more outside the box so not necessarily focusing on what competitors are doing but maybe trying to break the mold a little bit instead of um fitting in the box of what works um and then these two bottom scenarios where we don't necessarily want to sit where performance is below Target or significantly below Target um one of our key strategies we explore is revisiting past winners this is obviously relevant for a game which has been around for a long period of time um sometimes these Heroes can really be brought to life again um through a new um execution style so you can go from ugc to gameplay to CGI style Concepts um and at the same time when you're in this danger zone therefore looking a little bit at what is working in the market so focusing on that competitor angle can be what can get you back into that positive Zone um and scaling up again um so yeah outside

of this mix of performance and your win rate is there anything else which dictates strategy for you I think in general this dictates strategy a lot I mean we don't have this I mean looking at this it's a it's a very cool Matrix and we kind of approach it this way as well U not necessarily um we have the the framework established and documented and everything but this is more or less uh what it is I would add a couple of things so one is seasonality something that is not reflected here some times you have to change this kind of strategy a little bit because we just went through Q5 hopefully it went well for you I mean it's it's appear that um all of us at least in marketers in in gaming we love it and hopefully wi invol for you guys but you have to take into account right and you can only not only um uh use this uh all the time and forget about seasonality that comes into play and might change um your strategy for that specific uh time of the year uh some things that also came to mind is um

a Black Friday where you might not be able to test as much um and those sort of things uh the other the other thing is channel mix so this might not be applicable to um every channel or you might have to follow the this strategy um depending on on what you're at with the with the channel it might be that um you have you are on a a different quadrant for Apple because you are very strong in EGC and you're covered in on the right for for meta and Tik Tok and such um so it can Al also be adapted by and differentiated by um the platforms that you're that are working and not working for you at the same time yeah I guess with last weekend with Tik tok's news we could have woken up today with a very different creative strategy yeah this was not applicable yesterday on Tik Tok for sure yeah but today yeah today you can apply it again exactly um so yeah so that's more of a overarching uh strategy which we devise over time um and looking at ugc so taking this example where you know that

ugc works for you you may have found some hero creative is how can you take things to the next level uh so between m and b along with David ugc is actually not a route which we're testing together so I think it's kind of interesting for us to also talk about it um and this is what we've learned over time of of trying to build out your your your constant refresh of finding new hero creators within the world of ugc starts off with with the Creator uh we see that the Creator itself themsel can have a really big impact um on perform perance may be quite stating the obvious U but having then a nice process for for making sure that you've tested out that like a new Creator which you're exploring there was always a bit of work going involved and onboarding a new Creator so you're starting off with sourcing production um testing out some sorry a little bit messed up uh testing out uh at least five Concepts or creatives with them including a few intro uh intro tests as well um and a key part of that is we see if like testing your your hero

concept with with a new Creator as well allowing you to categorize then into your winner standards which can be for that let's say scaling up for Q5 you may not want to only rely on one Creator um so you have a bit of a mix of uh of talent which you could work with um and then alongside that we've got this concept testing Matrix too so um we maybe focus on it too much maybe not but like this idea of you've got your test winners and your test results and then you've got the ba results as well so what actually scales and what actually drives performance unfortunately they're not always holding hands sometimes they're they're a bit of skew um so we have this Matrix built out of um of testing A New Concept so if it's a concept which um didn't win in testing but shows potential which for us we suggest uh or we we look at anything that formform within 80% of your kpi in most cases that kpi is IPM um if it doesn't have potential take those learnings away move on um if it does show some potential um this is where we

see that we can take some time to iterate it and kind of get out of that threshold into becoming a scaler um and then if it did win and it is a concept that then went on to scale we want to make the most of it um so we have this uh this framework we call the creative winner action plan where we'll tick through and and jump straight into iterations to then make sure that when fatigue kicks in uh we've got something else in the backlog to to to bump it up so a mix of hookit ations of the scene the motivation some copy voice over iterations background music length and so on so um bit of a framework of building out your your pipeline and having winners ready to to roll out all times um and if we have a winner that then doesn't scale it's thinking about how we can re-evaluate it based off of our learnings of what is scaling instead of uh Reinventing the concept completely it's how can we tweak it based off of maybe some your that single intro that already works at the moment to think become a scaler I would actually add a couple more to these so and talking

about EGC specifically um one of the things that you can add is localization so and I think that is one one important one that could be mentioned so if you have um a concept it makes sense and it is it is working well it makes sense to localize it to other to other languages so you can explore that in your usual channels um the second is playing a little bit with um subtitles with the graph with with the colors with the the fonts that is a small change that can also deliver some because if it's ugc right you're going to have someone talking so it's it's good to have subtitles and you can play with that as a little bit as well um the placement of the or how you show the the Creator could also be cool to tweak so we can have a green screen not green screen squared put it on top put it on the bottom uh those sort of things you can play with um and finally and this is a little bit specific to ours and and I'm giving but it can be applied to other other games as well um our game have the ability of um expressing story

lines or showing story lines right so if you have a concept that says hey this is the best country you can play in a certain game um you can take the same concept keep the same concept and but apply to all the different possibilities that you have in the game so if you have different confer you can pick hey let me take this and if I show the US let me now show Spain Portugal whatnot um and this can be applied to other games right um other Mechanics Tools characters that you can play uh so you can expend um a little bit on that as well do you have any learnings of like for example in the US you always going to be starting with the us or is it quite diverse in terms of the sort of when you've got that location as a variable yeah so actually this uh I'm going a little bit off topic but actually this um actually influenced the way that we uh change our tutorial map because we saw that a us um that us uh users were picking the US and they liked all the ad that were showing in the US and the idea spun out of creating

a tutorial map where you can play specific States in in that and then we connected that to the markeet and and brought it back so your your creative strategy your creative learnings can also then dictate your in game development yeah and increase the win rate with that which is the topic that we're talking about there but connecting a little bit with with the thing yeah um cool I feel like we're we're we've got a lot to maybe to still try and get through but um so another topic when when we're looking at gaml uh and animation Concepts as well um to maybe narrow down we've got two Roots finding those new hero Concepts so concept ideation which we'll get to in a moment and on the first uh the first side is improving your your your hit ratees of of iterations um we did this case study at the end of last year um building out um some learnings on AI iterations um so this is I'm not going to go through in detail but just a bit of a cheat sheet of how we see like starting with AI as a topic of how that can impact our creative strategy and our

creative production and testing as well like what are some tactics which we can use to then iterate on an already proven concept um so that's split across video assets audio assets and static uh as well um and I think we were having a bit of a discussion on this earlier as well of like we went in with like AI as the topic what can we wi like get out of that and it was kind of an interesting contrast of like you see AI as more of the tool and just something your disposal as opposed to it was to see that you approached it as a whole framework while we saw it hey okay we have the AI let's let's see what AI offers and what we can do with it and this most more focuses on AI is something that we can explore as a different thing so uh let's use it and build a framework to come up with different types of well everything that you can see you can see here so little bit of different approaches for sure yeah but um to see the differences yeah yeah it was very nice to to learn it together together so we did some tests

this has been this is like also another major we built out of those learnings over time of what drives High Impact versus low impact so naturally skewing your production to these higher impact uh iterations is what's going to drive up your your win rate overall as well um and then we have a few uh some results as well from the tests we did together so um we went through these are like human ideated ideas of what to iterate on a winning concept with AI execution um as well so this was taking here a concept datas on the backgrounds you can see there's a lot of variability you can get in it with um less of a lift compared to to building assets yourself actually in this case drove 36% IPM we'll move quickly through to show some other examples the sound effects you may not think that sound effects has a big impact I think by Channel it can you didn't get a lot of variety as well by Channel it can it can vary a lot as to what drives that sort of impact in this case I think we had 7 % increase in CTR the IPM was a bit less impacted so less

of like an immediate win uh but definitely a key learning to take away and I think this one maybe we were discussing as well of like you may not think that these like a small asset in the map for example the vehicles here can have a big impact on on overall performance on your IPM as well but we actually saw this this 12% increase in IPM sorry trying to Rattle through to make sure we got enough time to cover off the last topics um so yeah so this was something we did last year we wanted to do a bit of a recap we had shared this with some of those takeaways um we saw an impact um naturally you saw those results they definitely drove um they allowed us to find new winners um unfortunately this is actually something that negated away from our win rate because it was a lot of volume testing um I think one of the things you you were also mentioning is the cost of that right it it was exactly that so we're having actually this talk or this this chat because we were going for I mean there's an exponential um output of

possibilities that you have with AI and while we were trying to get more and more winners this actually ended up impacting our win rate because we had so much to test that we were just to producing stuff for the sake of producing and that had obviously a a budget Associated to it because you have to test every single creative but also time because um we're not a big team so everything has to be done manually I know that some of U managers probably here have fancy tools that everything gets uploaded and it's everything is automated we don't have that luxury um and it represented it also time that we could be dedicating to other things so that was a key takeaway a learning in fact of something if you want to pursue not to increase your win rate to help us find Winners um but it open up another topic of how can we then spin that around and actually um improve our win rate as well as finding those winners so the third takeaway here because these were human ideated Concepts or iterations too is that that human biased is difficult to remove which is why we

went down that route of okay let's go in volume testing so what came next was can we predict those winning Concepts so instead of producing all of them even though the lift of production was maybe not very high um the the cost of that was High um so our hypothesis was can we uh yeah I mean in short was can we predict that uh our process uh was um providing context so in this case uh we use an llm similar to chat apt um providing context of the the problem we're trying to solve um setting the challenge of I've got this hero concept this is what it looks like generate me a high volume um of assets to test and then refine it uh provide like tell me which which kpis is going to have the highest impact on CTR and CPI um to allow us to then fine-tune and only test five assets of iterations um on these hero Concepts um so what we saw we did this in two tests for for two two games with bitro

um taking this hero concept and um in this case for this hero concept the recommendation was test the intro this is what's going to drive the highest impact and what was really interesting is that we did we did see a lot of impact uh and we actually saw very high impact on ctrs so one concept actually almost doubled the CTR um compared to that that um that control ad with only an intro test natur you you can see there's quite a big difference in these intros which you can which you can do definitely recommend for anyone um to also explore what you can do with intro testing it's definitely uh kind of a nice entry point um but these didn't have the same sort of IPM impact overall do you have any ideas maybe as to to why I mean you can see the in is right and they're very very flashy um so in so to maybe to summarize it um it was not and you can see the ipms right the ipms are not where we want to be um but I wouldn't label as a fa a total fa

failure because CTR is actually impacted so there's maybe something here to be explored um we can uh talk about next steps which is already a good thing maybe we can talk about how the stores could look like um so it is a start I would I would say and connecting with with with the with something that we did uh the previous test um it's not a winner right it's not the next hero creative but um it is something that could help us extended period of the hero creative that we had previously um until we find the next hero creative y uh we are we are at time the other results for the other tests were were fairly similar um and just to to run through these takeaways um what we s is that these air iterations can drive CTR impact but the best combination from finding winners um was was this human and AI approach um and that the best thing that best outcome of this is that means we can

focus and spend less of our energy on iterating and fighting creative fatigue and spend more of that time focusing on A New Concept production more out of the box thinking um we probably don't have time for the discussion which not but uh we're going to be around here so if you want to have the discussion with us and give us your tips for the win rate and we can also have a discussion with you we'll be around yeah thanks for your time thank you [Applause]