We’re Only as Good as Our Last Creative: Creative Automation Can Transform Your Growth Strategy

now we're going to look at another area maybe you could say it fits perfectly in with growth because it's something that i'm hearing a lot about from growth marketers app marketers games marketers as well creative right is the one lever that marketers can really count on and it's a way to grow your audience and grow your revenues we're going to hear about this how to turn the creative into competitive advantage and our speaker our guest is adam stevens chief product officer co-founder luna labs a little bit background on you to kick off adam in addition to being at luna labs and going to tell us about how we can create meaning from our creative data a little bit about yourself i understand

and i see that 2020 was a big year for you over at luna you nearly tripled from 15 people to 40 people and that's quite an accomplishment and another accomplishment to note is that you have a track record working with app developers the past eight years at google admob before that technical program manager and product lead for facebook audience network sdk the title of the session we're only as good as our last creative how insights and creative automation can transform your growth strategy take it away adam so yeah i'm adam adam stevens i'm cpo of luna labs i'm super excited to be here today it's my first time presenting at a digital event so it's something new for me as well but hopefully there's a lot of you there who are signing in either today or tomorrow to learn something about creatives so my presentation is i'm going to be sharing with you how we can essentially use insights and ultimately automation to provide your team with a framework for success through creatives

and everything i say here today i've tried to simple simplify it so that it's it's something you can take away and have to think about after the session but obviously you know with doesn't matter about the size of your studio you can layer up all the kind of principles that i talk about to effectively grow your studio's creative abilities so yeah as a quick introduction to myself obviously i'm adam i am cpr of luna labs so luna is a creative technology company um and our goal really is around providing solutions for mobile game studios to be able to optimize manage create build all their playable ads and video ads from one centralized place and we do that through technology which is self-serve so that's really our our mission objective the company was founded in 2018 um and as peggy said we were 15 people at the start of 2020 and we ended it with just over 40 people and our headquarters in london and also we have offices in minsk um in croatia the netherlands israel and also the u.s before i get straight into the meat of the presentation though i think it'd be

interested to interesting to borrow 15 seconds of your time uh and just ask you to take a look at these two creatives and while you're doing it try and keep track of what you're thinking about whilst you're watching them so we'll go ahead with a countdown so from a user perspective i would imagine that you form some kind of i don't know emotional reaction to the videos maybe you have questions maybe you have expectations uh perhaps you didn't think about anything maybe you just watched them and they didn't really resonate with you but the key thing i want to kind of draw your attention to here is what do you think the user acquisition manager what do you think of the developer of this game is able to understand from what just happened much more than you were so essentially the developer is going to see some data around this the data could be in the

forms of impressions or clicks or maybe maybe their user also installed the app and then they'll have all that retention data and the the engagement data from that but essentially what i'm trying to point out here is that there's only so much that the developer and the user acquisition manager could even learn from this set of data and really that's what we're going to be honing in on in this talk and that's a that's a key problem uh it's a problem which every team will face at some some stage and something that we should strive to solve is like how can we bring meaning to this data so we want to try and turn this data into so much more than a zero or one in install or not and install a yes or a no and to do that we need to create a framework for our created production that allows us to add meaning to the data and so here's what a process can look like to build a creative process that allows you to do this we recommend that you control measure and understand your creators and we'll break this down throughout the presentation into those three points so point one is all about controlling the input this is having a firm understanding and creating

a framework around your creative production that allows you to then go on to stage two and measure it and with those two things combined you can actually create a level of understanding which is quite unique and gives you that unique advantage as a studio and obviously you want to be able to repeat this as well which is something we'll talk about quite a lot so before i go into the actual steps one two three i think it's interesting to talk about what are the goals of any creative process which we create as a team now first of all efficiency so we want to make sure that we're making the best use of people's times at the end of the day we only have a limited amount of resources as a studio or as a developer and you know it's really important that we use that time wisely so efficiency is key to any creative process that we're making also with regards to time right normally you'll find you're making these creatives with a time limit and so making sure that the process allows your team to move quickly is super important second one is kind of obvious right we need something to be effective we need it to work but ultimately we need it to be driving performance first as well so really

effective here can be interpreted in two ways the third point is about making it repeatable so with the process you'll find this is what we're trying to get away from you don't want to have these like one-hit creatives that you don't really understand why that happened but you know you're going to go again and potentially hopefully one day you'll find another hero creative the idea is that you make a process which ensures that time and time again you're able to repeat the performance at the end of the day we're making a business also we're making games right but it's a business and you need to make sure that you're able to hit your kpis and then the kind of the fourth factor which brings us all together is predictability again coming back to that business concept you're part of a team of a wider organization you need to have predictability to the monetization team to the games team about the amount of traffic that's come in you need to build a process that you can eventually rely on um so actually i switched it up a little bit i thought it'd be very interesting to start with a case study and i did this because it helps you visualize what i'm talking about throughout the presentation so this is a case study from one of our long-term partners at luna

they're elictus this is one of their games moneybuster which was top of the charts and if you follow these are actually four different playables which they run throughout the time of that kind of acquisition campaign and you'll see they started on the left hand side with a simple design with a dollar bill that you would shred in a shredder and then they stepped into this kind of process of iteration and again i'm going to talk about this in a moment but they were able to find that changing the dollar bill to a card changed the ctr by 8 in a pure split test now again they iterated further and found that changing the layout of the shredder gave them a further five percent and then finally changing the layout completely the shadow type which actually makes the game seem much more minimalistic and actually bring more focus to the tutorial gave them a further nine percent so actually from start to finish by following a process that they devised internally they're able to drive that ctr by 22 and that's just through one creative and it's keys and success stories like this which actually show you how the top developers are able to achieve success after success

um and we're hopefully going to give you some of those insights so stage one is to build a system that allows you to control your creative input this is like really the key point about how to formulate your ideas and how to structure them in a way that allows you to then create meaning out of them at the end of the process so generally i like to break down creative building into two segments so the first one is all about ideation and this is kind of where you start these are the blue sky thinking this is where you come up with a concept around what you want to showcase in your game so it could be winning level 13 in the jungle right or it could be failing the level again and again in the desert right these are remarkably different experiences that showcase your game but in a unique um in a unique way so once you have your idea uh generally what we recommend then is to drop down vertically and start creating your variations or your slight variances on that idea so it could be the character the camera angle it could be whether it's night

time or daytime these are subtle changes that don't really deviate too far from the main concept but they actually allow you to test different visual um aspects of the game different characters things that might actually draw in different emotions from the user um or while staying within that bracket of the general idea and we do this so we can keep track of the ideas so you'll see that we have let's say theme one and you could drop down and have five to ten different variations of that theme and when you're doing that you're able to track every single thing that you're changing within that theme and that's that's super important to remember and this is kind of the key idea around point one and hopefully you'll be able to you know look back to that money buster playable and start thinking about what were the the different variations they changed within that theme so once you've done that the next step is to measure the output now don't worry this is something we all do today so it's nothing new but just to call out how important is that you actually really measure meaningfully the results that you get from each one of these creators that you're testing on a given network also it's important to call out the fact around data reliability now you want to make sure that this test

is fair you know many of the networks today provide you with a b testing or split testing functionality that allow you to draw reliable conclusions from that data so once you've controlled the input and then measured the output the key thing here is to start understanding what the drivers in performance were and you can relate those from start to finish i'm going to talk about videos and playable ads slightly differently because they they each have you know a different approach that you would take but essentially we're going to be following the same thing the same thing so this is all about understanding now typically we refer to videos as black boxes you put them live you get some data at the end you make a subjective formulated opinion over why that video performed or didn't and really the skill set here is around understanding all the elements of that video and making that conclusion but what we'd be recommending is actually you can remove that kind of feeling of a black box and this is a very simple example but by breaking down all those variants into their subcomponents and analyzing the

data side by side you can start to kind of identify those key drivers so in this example and hopefully this is obvious but people love cowboys and they love them when they're in the snow um i'm sure that's obvious um but the key thing here is what you're doing is you're actually compartmentalizing the key drivers as i mentioned but you're taking them out as key takeaways so what you can do from this point onwards is use that knowledge as it's almost like training your internal muscle around the creative development process you're learning what works and you're using that for the next iteration and i'm going to talk about that a little bit later again this is quite simplified you can imagine that you know in a bigger a bigger production cycle you might have 50 things that you're testing over the course of a month but on each one of those tests you're actually able to find something new and something that you can use as a lever for the next iteration round so playable ads are interesting and in my opinion very exciting about what you can do with them because obviously you can control the input and you can decide what that user

flow looks like and what the you know the theme is and what the character is doing and all the visual stuff and obviously you can measure the output but actually we're seeing more and more and this is i think going to be a trend in 2021 the ability to add that third metric to your analysis and that's the engagement data so playables as the html5 content allow you to track different elements and different parts of the user journey with metrics so you could track how many users start the tutorial how many users end the tutorial whether the user wins whether the user loses how long it takes them to complete that kind of engagement session and really the key thing to think about here is that you can use this to almost drive your decision making so think about a video a video you might say the cowboy perform better and they perform better when they're in snow if you move that into the playable scenario then you say well actually i can see that maybe in the cowboy scenario the time to engage was much much lower now that actually gives me a little bit more information around understanding that

maybe the users actually are drawn into that game more because of the character and you wouldn't have been able to tell that from the video and obviously this has you know exponential effects when you start thinking about user journeys and you know the different trees that the user can go down whether the win or the loss has a different impact on the emotion of the user but just the core principle here is that you have all this information at your fingertips and it's extremely important that you start thinking about how to use it one thing i would say as well and um you know we can't really get away from it but in this idfa-less world that we're heading towards this is actually something tangible you can optimize for maybe you won't be able to optimize for the majority of your traffic for the install event but hey what about that 20 of traffic that you can optimize for can you get any learnings from that and say you know users who tend to win the level go on to install the app and can i actually start using that as a parallel to all of my other optimization tactics i think it's super interesting what what this will look like in 2021 so what you do next so generally now

let's say we've run our creatives we've had some learnings we know what works well what we want to do is use that as a checkpoint so what we're doing is we're locking in on that performance and we're saying the cowboy with the snow worked well or the bunny or the other the clown work better than the other one we're making these now data led decisions around what is actually driving the performance and we're using that for our next round of iterations and so what we're able to do is continually push ourselves up ratchet up the performance until eventually we find that hero creative that kind of end result of all of this testing that allows us to have a successful game launch the one thing i would call out here is that you know ideation when you're creating those brand new concepts that's where you're going to see the step changes in performance this process isn't going to allow you to change a color and see a difference between 20 ipm and 50 ipm right this is going to take you that 10 that 20 higher and the reality is in many cases that we've seen over the last couple of years that's what really matters you know that last 10 20 is what gets you over the line to make

it a positive margin a positive ros you know it's really these small details which which actually all add up to have quite a big meaningful impact so use your checkpoints and move through them quickly and then the kind of final sentiment here is that i hope that you can start to understand that everything that we do is basically bringing more data into our decision making and removing a level of subjectivity no longer do you have to say i think that video performed because or i think that playable ad worked well because we can actually like back that up with the data and what we've done is we've created a framework around that to enable enable the kind of decisions and there's a load you can automate in this if it's as simple as automating the data analysis or the tagging of your videos or the way that you uh interpret the analytics data or you know there's so much opportunity my recommendation here is to start small obviously build that process invest in that process but know that as you do

step up and become more experienced and also become more complex in what you're doing there's a load of opportunity to automate all these things and we see this as a big trend in technology providers um around trying to create this level of understanding around the creative in a very simplified manner so what are the kind of key takeaways here um first of all as peggy started i think creatives are the key lever for success in ua in 2021 especially considering the landscape changes my recommendation is to build your team and build a process around bringing meaning to the data so i think we live in a world where we don't need to be subjective anymore where we don't need to have opinions we can actually have data to back that up and so my recommendation is to go do that always id8 first and then iterate as i said you're not going to find step changes in performance from changing a color but you will from changing the complete look and feel of the the ad and the emotional ties that you can draw obviously you want to understand the

performance drivers so it might help you structure creatives in a way that allows you to do this and what we're seeing for example with video is often people create these um almost like small components of a video so they have three to five different outros three to five different intros a different set of music that they use these things are also things that you can use again and again so actually there's an added value there if you move towards that process in terms of your efficiency um obviously as i said a couple of times knowing when to iterate versus ida is uh is really important you know if you're on a 20 ipm and you need to get to a 40 don't waste your time on iterating look for that next step change go back to ideating hopefully some of the ideas that you drew from the iteration were kind of important and helped you get there but you know ideally you want to create a new concept in that case and then finally when it works automate it i think as i said you know we live in a world where this is becoming more and more easy and more prevalent and i think the kind of end goal of

everything we do here is to give your team more time to be creative more time to find those pockets of success and less time around this kind of manual data analysis or you know this variation testing so looks like i did it just in 20 minutes so hopefully that was uh on time but yeah i guess for me thank you very much um and i'm happy to take any questions or as as it says here you can always reach out to me on email or on twitter okay and you did it just in time just gotta get this off here i should have called out at the beginning that it's like locked down in london and i have two small children who luckily didn't run and grab me but they might have been screaming in the background at some stage that's okay i had i had to screw up i have a cat who pulled my camera down and that's i've been very careful with it because i can't stop to get down on the floor and play with the cable so did great job adam in a very interesting topic i mean

so timely because as you pointed out i won't force you to do it but it would be interesting to have you just elaborate a little bit on the place and importance and potential of all this all of what you're talking about in the ida f a less world which you touched upon is there some advice around that because if this is going to be the way to approach it if this is something tangible rather that you can measure is there something within your um presentation you can point out is some advice in that new world in that brave new world absolutely so i think um probably like everybody you've spoken to nobody really knows what's going to happen at the end of this there's a lot of very good articles very good content online about hypotheses and um opinions but the reality is we're gonna find out on day one pretty much um now obviously the consequences will be a bit of a ripple effect so what we've been thinking about a lot uh internally at luna is how

you know at the end of the day attribution is going to become more challenging potentially or it's going to be different at least so all we can do is rely on the things that we know um that we can um almost like stake stake the fork in the ground right so with creatives whether they're creative works or not is always going to be a question um the attribution is just a confirmation of that so what i'm trying to focus on with this presentation is how we can start answering that question internally and allow the results to speak themselves so let me give you a tangible example and we've done a couple of tests with this right and especially around playables so we'll run a playable ad and we'll quickly learn around that that you know potentially 15 percent of users win the level one and you know they can go on to win level two and actually that is the key behavior that we see within the playable that leads to high leads to high retention users um and i think what the point we're trying to make is that in this more difficult period to do the attribution it's much better for you not to have a download

than it is to have a download that instantly churns because essentially you paid for that right so that's affecting your margin so if we can draw conclusions from let's say android uh whilst we can still track there or from 20 of users who have opted into idfa to say well in general users who win the level tend to show the behavior that will then lead to a higher retention user that allows you to then optimize that experience instead of optimizing for the outcome so with playables we have quite a unique advantage so what you'll be able to do is say i'm now going to focus all my attention on making sure that as many users as possible can get through and win the level with the belief that actually that content is then going to be driving a better outcome for me in the end and what you'll be doing is you'll be monitoring the rawas monitoring your in-store rates monitoring your attention and obviously making sure that it doesn't deviate too much too far from your hypothesis but i think it gives something somebody something tangible to kind of aim towards and i think that that comes back to your point around i think people need something to hand their hat on right they need to be able to say okay even though everything else is a

bit up in the air right now i'm going to focus on this and i think creating a process like this and also using specifically playables is almost like this pre-first time user experience tutorial right it's almost like that gate to your game it's it's a way for you to qualify the user and then back that up with data to make those decisions what i thought was really interesting and i was live tweeting the whole time compartmentalize everything um that was very interesting to me because i wanted to understand just how far down that goes you know how granular can we get well you're answering that we do have uh we sparked some questions so answer that briefly we'll get to the to the people because they matter more than my question but i still want to know how how granular you're saying we should go yeah i mean it's it's the question we get asked a lot and i think the reality is it's going to be diminishing returns at the end of the day like if you've got a team that are optimizing for the font of an install button that shows for three seconds then

is it really worth the investment of your time to make that step change and actually are your users really going to care i think what you need to do is find a balance between how much time it takes your team to invest into making these tests uh to the point where actually you're still seeing positive returns so you know seeing sub 1 changes in ctr it's not really gonna you know move the needle but you know a lot of these tests changing the camera angle changing the character they can have five ten fifteen percent changes which is a simple simple switch because again coming back to that first slide where hopefully just stared at the ads for 15 seconds and one of them didn't do anything because you didn't interact with it and one of them was just a video you know you only have a small amount of time to capture that user and i think even i was surprised very on early on around how these subtle changes can make such a difference yeah and you showed in that case study i mean 12 ctr that's that's a big difference on that topic here what are some impactful creative automation you see publishers

are doing what should uh what should developers oh rather should developers build their own solutions versus third-party solutions yeah i mean it depends on on what you're trying to achieve obviously you know i don't want to act biased because luna does provide some automation technology where we can do some of this testing for you i think finding if you want to talk about it you know um objectively then i think look for a solution that fits into your workflow in the best way possible kind of coming back to those goals right efficiency effectiveness repeatability and predictability you want to find a partner or a technology solution that aligns with that if you want to go build it yourself and also you have to realize that you might have to input a lot of effort before you see any returns on that as well so generally i'd say start small as i kind of mentioned earlier but you know if you can figure out a way to use some tools mixed with your own kind of internal data systems you know

that's also learning for me you know we have a lot of this technology at luna but many of the big studios have a very complicated bi system internally that they use so for us as a technology provider it's all about how do we fit into that workflow um so i think it's really got to be a balance um okay but generally i would say just prove it to yourself at first if i'm honest like do a test it's not going to take you long couple of couple of hours a couple of days to set up a statistical test where you can go and actually see the value this provides and then with everything i think as soon as you see the value that's why you double down right and that's where you you can kind of um justify the investment whether it be in in time or technology but yeah i honestly think there's a lot of technology providers not just luna who are rising into this kind of area around you know augmented data on top of your creatives and generally as a game studio you've always got to make that trade-off between what do you do versus what you outsource um sometimes outsourcing is the best way if that means you can focus on being creative

which i think is still something which i kind of emphasize with our partners and with colleagues that it's one thing you can't change you know have you got a good game have you got a good creative and the rest is all just optimizing and of course you showed pays off pays dividends i mean it is worth the effort whether you do it by yourself or with a solutions provider one last question we have here are there any tips for testing creative variations specifically for multiplayer games what's the best way to emphasize the multiplayer aspect of casual games two questions for you yeah rapid fire yeah i think that's an interesting one i mean there's two there's two trains of thoughts right there's one that goes down the road of um people like playing multiplayer games because of a challenge you know coming back to the the previous talk around um you know the sense the feeling of winning that emotion sense they wanted to beat the other person i would say necessarily not necessarily

you'd have to showcase the multiplayer aspect of the game maybe you can like show it in a snippet or in a quick kind of brief section of a video for example but generally what we're seeing as a trend is trying to tap into that emotional driver as opposed to the actual direct hey this is a multiplayer game you want to play it right you know trying to find an element of the game using your games animations your games are something about the game that's compelling that gives that like kind of competitive itch to the user i think you know what we're seeing especially with more mid core casual games is that they'll do that kind of orthogonal gameplay experience that does do that whilst not having to say you know because you can't explain a multiplayer game in 15 seconds most of the time so my recommendation is definitely do that definitely look for those emotional drivers and try and find small fun engaging experience to kind of bring that emotion to light and then the second thing i'd say is experimenting so you know what you do want to have this kind of as i said this kind of parallel creative strategy that doesn't necessarily

represent 100 the gameplay i think it's important that you do try and break down that multiplayer gameplay game with a leaderboard and breaking that down into like a 10 second playable followed by a second section of the table that shows something slightly different so you're still kind of giving them that like that feeling but you're not making it overwhelming by making it like a full 30 second experience that ends up just feeling like a mini game and also like the handoff there and the experience there's this piece of what is really emotional really catchy going to appeal to me about the gameplay then there's another aspect of the creative and they sort of work together i'll take another question because that's what it's about here people want to learn um how much performance video how much a performance video has the chance to be the best app preview in the app store so how much of a chance i guess does a performance video have to be

the best app preview in the app store that's what i'm trying to understand here yeah how much performant video have the chance to be the best app preview in the app store maybe there's something around how much of a chance a good performing or highly performing creative uh video will have to be the best preview so maybe they can clarify but i can tell you one interesting thing against the app store videos is that you know we're seeing a general trend that the marketing team is now starting to view the app store as a you know a second dimension for them um because if you imagine you do all this work and send someone to the app store essentially that's your front page right that's your entry point and whether they convert or not typically would be seen as um you know that's almost in the hands of the gods obviously we have aso companies who do an amazing job around testing and proving the theory that actually these changes in screenshots descriptions names icons all have a

material impact and so what we're also seeing is people applying this creative strategy also to the app stores so testing those variations those changes and themes also within that environment is also like shouldn't be ignored like that's another arm to your creative team and also another way for you to maximize the output because they've gone to all this hard work of creating videos some of them which will never see the light of day there's a huge opportunity and another surface for you to use on the app store where again you're landing there after a click there's the creative match the creator that you just showed them you know is it a different character is it a different theme does that work does it not work something you'll have to answer for yourself but again i think going back to the efficiency point like it's definitely an opportunity for the creative teams to actually you know almost like their work to bleed into that aspect of optimization as well okay i'm tempted we do have one more question maybe do it really fast because we have another really amazing panel do's and don'ts of marketing in ua we

can't people can't can't keep people waiting for that one adam but okay one more how can i perform this type of testing at scale easily i understand your points by creating 100 yeah creating hundreds of playable ads to pull all this together it seems like it could be very time consuming definitely not true so how can you do it at scale yeah i mean is that where the automation comes in absolutely so now again to mention something that luna is doing right now is actually we're actually able to dynamically change the content of the playable on the fly so that actually you could create one playable ad with 100 variations held within the ad and that ad will dynamically optimize itself so this is something which i think is going to be an even bigger trend in 2021 especially when you can start thinking about i want to optimize the character choice for the number of people who reach the end of the level maybe that's like the perfect thing for you to optimize for you don't need idfa for that right you just need the engagement data

so actually we're seeing a lot of investment in in automation around this obviously i'm happy to talk about it because i work for a company that does that but you know there's a there's a number of people investing in that and i think that's definitely something i would recommend looking into regardless of of how you do it or who you do it with i'm trying to be you know unbiased here okay adam you've you've you've started avalanche questions i think you're gonna have to stick around and discord or go off in one of the rooms and stay there or um you know stick around for a couple questions are you planning to do that yeah yeah sure also you can email me i'm always happy to jump jump into an email or on a call i live i live this stuff so i'm always talking about it anyway okay because i'm gonna have to let it go for a bit adam it was very interesting very very informative otherwise you wouldn't have the last minute avalanche here people want to do this at scale they want to understand more about gaming well i wouldn't say gaming the app store did i say that no learn about succeeding in the app store with creatives um i'll let you go and move on to our next panel thanks so much for sharing all right all right okay bye

you