Ad Creatives Framework: Optimization Deep Dive (Matej Lancaric SuperScale)

you welcome everybody today we are talking ad creatives increasingly the biggest driver for alpha and user acquisition so a lot of people talk about creatives at a very high level but not today we're gonna go deep we're gonna go tactical and to do that we're here with Ma Tei LAN szerik from super scale who specializes in new way for their clients and has a lot of success in this area by the way Monty did I pronounce your last name correctly almost yeah very very good for for any any foreign people from not from Slovakia because it's in slow its come on come on think because everybody from us called me mata chuan Couric so can I can understand that but it's Michael I'm sorry so you got it thanks thanks man I really appreciate it well let's first start with impact and so for a lot of

PM's and executives out in our audience it'd be good to get a sense for just how impactful good versus bad creatives can be now some people are always gonna say a good product just markets itself but for mobile apps I think we've got a bit of a different situation so just to put some like high-level rough numbers against specific metrics can you give us a sense of you know the Delta the change we can expect from let's say you know a pretty good game with bad creatives and optimizing that with good creatives what can we expect yeah so first of all I mean let's let's do a little bit of definitions here like what do expect from like good or bad creative like how would you define that because I mean I hear a lot of lot of things from from other people that you know hey I saw they're really great at CGI effects really high production value and then I

mean it is it good or bad creating what would you say I mean I I don't know because you know we should we should different to look at some numbers and and kpi's but what do you think like how would you define the good or bad creative right so I mean you're asking me what I thought of course yeah of course because yeah well I mean I would I would assume you know there's the general discussion around IPM in terms of tapa funnel but then ultimately I would say that a good creative is one that optimizes your your rose yield yeah of course yeah okay okay good because again then we can we can talk about the creatives good performance and bad performance let's say so basically and also like not only looking at the Raza obviously that's the ultimate ultimate KPI we are optimizing our KPI scales but creative swartz we're also looking at the IPM the

CPI the CTR and also like conversion rates or click to innocence but then creative easily can you know break make or break your game or your business and this is something that that we saw also before for other companies I was working with let's say mix of Federation we were running the soft bunch for negative dimension on getting back to this game all the time because it was like my first soft logic global launch and we had pretty good success with that and we were running some some companions and we were getting like $4 CPI's and then then my colleague brought up some very old static image and we were just thinking about using it and adding it to the to the creative mix then we we saw that this static image is performing really well so then then we added also like very very easy animations into that and

then after like doing very small changes we were able to see like 50 cents CPI so I mean five-year-old screenshot me really bad looking everything and the seat and that all the numbers were so great so basically you know only these like comparison like $4 CP is in comparison to 50 cents that's super huge and that therefore like we were able to increase the global lunch budget from like 200k or something to like millions just great and then so the the good creatives a good performing creatives you need to grab the attention of the of the players you know immediately some sale it's three seconds but you know you're browsing the Facebook or Instagram and then BAM there is just one second and you need to grab the attention of the player right away and what we found out also like it's obviously true right but we were like running some some creative for in

social and for other games and we were checking the the metrics like on the on the Facebook dashboards like when when where when and where the people drop out from the query after like five seconds ten seconds and stuff like that and then we found out that like the first three or five seconds are like super boring so we just moved like the middle of the creative or the middle of the video with like a very important message to the right right from the start of that creative immediately so like huge improvements so like talking about like decreasing the CPI's and like from fourth point ace to like two point something it is great and CTR also improved so you have to you know think about these these things when you're optimizing or building the creator's got it and just talking about or so just thinking about

the types of creatives in and you know is there a continue maybe we could first provide like a high-level overview in terms of the type of creatives out there for our audience that may not be as familiar and then back to the former question in terms of like the kind of Delta that we can expect and you gave one specific example but is there a difference in terms of the performance against different types of creatives we can expect in terms of improvement for the different kinds mm-hmm okay so there are like I mean I will talk about Facebook creators and there is like plenty of plenty of types let's say like steel banners or still stills and then obviously videos with different resolutions from one scape to the square to portray yeah honestly playable but in there like a couple of others which is one is called Caruso which is like five or ten square images right next to each other newcomers really use this format well

because there's a show there for that but you can use this for showcasing your characters or you can use also video formats for these curves also you can use a couple of sequence from sequences from the game play this is pretty pretty good format and also like there is this format which is called canvas head format which is the instant experience and we use this quite a lot even before like there was there were playable and what this this is actually like a small landing page or short landing page mobile landing page where you can describe the game you can use different formats there and it it really it's really immersive experience for the player and it was kind of like hearts to build but then we thought about like okay how we can make it actually really easy so we used all the assets we had from like store actually and you can use text images and everything so we build a

small let's say store experience into that canvas ad and it's working really well so there are different and when I'm talking about like working really well I mean all the IP MS increase right away we all always looked at the draw us and like all the time was improved with all these experiments so basically moving from the static images to to videos can kind of really high impact and obviously I you know it purely depends on the game and on targeting on any other variables but then what we do are usually like to test new ideas just using the static banners and then like if we find a winner then obviously like moving to two videos which makes sense because it's it's not that expensive and we can quickly iterate off but then like we have to be careful about using only static images because we will see a high Frank

frequency for those for those type of creatives so people will see them quite often right after a first couple of days so that's why we actually mix like standing images and also videos so we have like healthy creative mix but obviously play I think playable as well and then in terms of performance like which if you were to stack rank order these different kinds of creatives could you do that for us and maybe also giving us a sense of like if you are allocating budget against these different types of breeders how does that usually you know I mean I mean I'm sure every game might be a little bit different but like roughly what could we expect yeah well obviously like focusing mostly on the video format the square one is the highest reach and the most you can level the most out of the inventory out there so basically like let's say like 70% of funny any budget or any resources go to

go to video the square woman yeah but I said not not forgetting about the lens convenien though it's like very small format nowadays but it's not it can perform really well and then moving to to portrait and then let's have like 10 or 15% for like testing other other types of creatives so statics there and playable there obviously like with play balls you can and there is the hypothesis that you will get the better quality players because they're already familiar with your game because they can you know they can play the game only they try try the game so you should expect better better numbers and it's not always the case and it tends to be pretty expensive in terms of the CPI's and it doesn't pay off on the LTV side so that's why I mean the playable are really great format and it needs some time to crack it so you can you can build it correctly but from

from my point it's definitely video obviously and in the different lengths you know not only using the 30 second video but also like 1250 eight even so it's all about this thing and actually you mentioned something that's a little bit that reminded me that so when you talked about like having a 10 to 15% experimental budget I noticed that that was a line item in a lot of you know with some of the previous teams that I've worked worked with having like a specific set budget for experimental types of formats and things like that could you talk about you know why people have that sort of separate budget for experimental stuff and how do you think about coming up with a budget for that yeah well it's I think why is it's like separate because you know when you were experimenting a lot this you will probably not see those money back because it's all about experimenting well let's face it it's it's it's that and that's why is like

very small portion of each budget but eventually in real life it's definitely more than like 10 or 15 percent I mean you have to you have to be crazy and you have to use different ideas for the creatives not only like testing different format but different concepts because nowadays you know like you have the Vino creatives but then you can iterate on a couple of times but then obviously it's not enough that creative will eventually die after certain period of time so that's why you need to focus on not only iterating the winners but also like trying to test as many new concept as possible right and then I think just kind of moving on to the next topic I actually wanted to touch upon terminology because notice that a lot of different companies that there is there are different kinds of terms used for the same thing so basically you know we're all talking about whether it's IPM

right which is what so we're talking essentially impressions two installs and then within that mix you know so some people call it IPM some people call it install rate some people call it CBR but then and then I think everyone uses so that equals CTR which everyone uses CPI everyone uses right so the CTR click-through rate and but then that app store conversion there's so many different terms for that so you know and basically what we're talking potentially is IPM or install rate equals CTR times your App Store conversion but what terminology do you use and what what can what kind of have you seen out there and then yeah yeah it's definitely confusing and a lot of different companies different terms so basically what we use is IPM obviously insert per mile and and the conversion rate for app stores is like click to install so I've heard that from Facebook

several times that they use this terminology and install rate I don't use that much because then I use always the CPI the IPM and is like you know I don't want to use like all the term all the terms for the same thing so it's like keeping it easy for me and it's the way to go so basically I using the IPM CPI CPR the CVR on this is the click to start right ratio or the conversion rate right from from the ad to actually install right and then yeah yeah as you said like different companies different terminologies but it's still the same thing same thing right right and then and just while we're talking about terminology so this is this basically represents a lot of the front-end top of funnel metrics and then on the downstream side I assume you're typically looking at you know bromas LTV and anything else that you

look at sort of downstream yeah it's connection it feel like you go as an LTV but also depends like what kind of life cycle or like what the stage the game is in but because I'm saying this because for example for soft launches when you vary in the retention stage I mean obviously like checking the monetization and LTV doesn't make that much sense so they're in that stage I'm looking at the retention numbers and I always like I PM's and everything but the retention numbers because I want to see like if this this creative resumes and also bringing the good quality players because the CPI can be can be but then you check the retention and you will see like okay so yeah see and they will see like lower retention numbers but if you don't check that you don't know so it's definitely definitely retention that I check also in the self in the software right so you're checking just just to be clear for our audience

you actually check you have attention against ad creatives and then how far down in terms of retention is it like d-17 30 or are you going further out oh it's d-17 mostly okay then like because yeah as I said like because in Softline is super important to check like what's going on also in the game and but also in our marketing side and at level KPIs in that stage that's definitely better to be retention numbers okay cool so I wanted to now talk about the creative team because when we look at creative teams at different Studios you know there's there's a lot of different things that we see so in terms of like the composition of the team could you talk about you know how would you structure a creative team for the typical game studio and then what are the key drivers like what do you think about to help you determine you know the what are the key

drivers are key variables that help you determine what that team should look like okay good very good question and I will just speak about how we build the team at super scale because the creative team as well because when I joined there's like five ûà ûà guys and no creatives so this was really really really so we started building this as ahead of you a and I was like in charge of this so I was thinking about like who should I who should I bring in and that was the key question and this is like something the like of philosophy to die I think it's really important to bring the right people on the bus and put it in in the right seats and basically I was thinking about my network and who would be the right could be the right right to bring it so basically I I had two two candidates for

like idea making and copywriting which is just like something that you definitely need because like you a people anyway guys obviously can and I do also think about the creatives all the time and talk with the creative team but then there needs to be a middleman some someone who can talk to the UI managers and also to can talk to the creative team so basically I I found two guys in my network with zero gaming experience but those were both of them pretty good gamers playing games all the time pretty good experience from are getting here in Slovakia because we actually we have pretty good and pretty talented people here so my also like my philosophy is that to bring like the local talent here and give them give them a chance to thrive so I picked up this this guy who was immediately hired after some some assignments obviously

and we gave him like very very easy assignment to do which which was to create some some ideas for games and then then he gave us like 50 pages PDF presentation as like oh okay this is the this is the attitude I really want to say this is the approach that I want to see in the team so basically right after he was hired then after one month he became the the creative lead and he's like the best asset I could I could ever imagine to have in the team and then we together sit down and started looking at our portfolio and our like games we manage games we could potentially manage I started thinking about like who should we bring also like motion designers 3d guys illustrators 2d motion designers and then we created created these requirements and then started hiring but also like we brought several guys with a

lot of experience from gaming here for Slovakia from other gaming companies here so all the senior guys but then obviously I want to have a mix of senior people and then junior junior guys who can learn and grow and that's why we also like hired these two two junior guys young couple actually illustrators and it was like one one a year ago and I could see the the progress and the so much improvement of what they'd done for that year but that was like because of the all the mix of the senior guys with the gaming experience but not only having those guys but also like giving a chance and giving the opportunity to the to the other talented people with the winning attitude winning approach to the to work to have this mixed so we can we can mentor them and see them grow and this is something that that I think it's it's

what I would do for any any other gaming company and then obviously like looking at the hard skills that's just something that we definitely need to need to need to check before before even getting those guys for the interview so basically getting them paid assignment because we we used all the time like the ideas that Martin created the the creative lead to to send them and then after they passed the obviously the assignment then we talked and this is like very important for me it's not it's not only by about the experience but also about their attitude and their approach and all the soft skills and you know how they will fit into the team because everything can be can be Cajun can be learned so that's really important thing is for me especially how the new guys can fit into the team and fit into the atmosphere and then so just

to kind of dig a little bit deeper into like how you figure out the structure so you've got like this creators lead person not the idea idea person and then you've got like you mentioned certain kinds of artists whether it's 2d 3d 2d motion 3d motion but how do you determine the size of those teams and maybe even an additional question would be then how do you also determine in-house versus outsource I mean do you kind of look at a game you you know some people will do it based on percent of budget some people will do it based upon you know like the creatives cadence and then depending on the types of creatives and but yeah yeah maybe I'll let you talk in terms of how you actually figure out the composition and the size of the team yes the size of the team we are at the moment like 25 people which is like 18 people from from creative team and others from UA so basically we we were growing like crazy

in the last in the last year well crazy for for our standards in the last in last year and we just it hit a good spot a sweet spot for like all the clients all the partners we have and we had before so we don't need like anybody else because again like different budgets and different games require different kinds of the new creatives so basically we were looking at like the again the spent and and the resources those those those partners our actually have so basically at the moment we are pretty much in you till eyes but then obviously if we got any any new clients or any new partners then we would need definitely some some more some more talent to bring in so basically like based on the number of span the number of clients and then like cadence they just came all together and that was the

final number at that point and this is like something that we are now okay with but then obviously as I said that getting getting any any new clients we have some buffer but then getting any new big clients then that would definitely mean that we should be looking out for we should be looking for any other new new hires and then I was also just trying to understand why like some games like a matching to mansion for example rumored they've got 30 plus people in house plus the use outsourcing companies just for creatives that's their creative team and then other Studios have like a few dedicated people so could you speak to like it maybe it's kind of a way of you know rephrasing the previous question but why would some games have so many people in some games not have so many people like how should you know how should ya know that okay so let's let's talk about loti for the magic convention so they have been house team pretty big

they're spending quite a bit of money from or worst I think we do in Johnny so you need you need like fresh creatives every week definitely and it's not a small number I mean if you are spending millions then you need a lot of new creatives because yeah they will they will trade off quite quite quickly and then what is really great also with like using outsourcing companies is that you will get like a new new ideas new pair of eyes and definite like different processes so this I you know a lot of big bigger companies divide these budgets like for like their resource isn't like in-house teams and then they hire a lot of outsourcing companies just to have more creative and different kind of concepts then they can just I think at the moment because you know after working on some some game for a longer period of time you you just quickly get to like this routine you're not seeing

any other concepts anymore and this that's where you know outsourcing company can bring a lot of value and then you said that some other companies have small teams well obviously if you don't have resources to build a bigger team it's hard to you know expand also how hard to use any outsourcing companies so yeah I mean the all the gaming companies use well the first thing to do is to build in the house team and then like go to outsourcing and then just going back to that question in terms of in-house versus outsource you know how do you make that decision between what to outsource and what to what to insource well now for me for my prospecting perspective is you know it's you need as I said before you need new ideas and you need actually at some point test the crazy ideas as well to see like what's

working out and what's resonating well with your target audience and that's that's where the outsourcing companies can help right away because if you see if you see that you are like you know running around and still using the very similar creative c-can't creative concept and probably there's the the best time to start looking for for any additional pair of ice but there are like giving you any percentages of or any any exact numbers it's kinda hard because it's all all depends on all the other variables so much I thought we could just kind of stop for a second here and then actually talk about you and super scale so maybe could we talk about like you your background your career just so people have a better sense of who you are and then you know it might be great to hear a little bit about super scale in terms of what you guys are doing right now and maybe some of the games that you guys have been able to successfully kind of launch sure yeah very very happy to speak about my

Southern and super scale well so again my name is mighty lion Tate I'm based in Slovakia and I'm from Slovakia and I've been in the industry for like seven years so far started during 2013 working at Pixar Federation which is the biggest gaming company here then after five years I just joined super scale of the way and overseeing all the you activities for all the partners then on on same time I also like workers like consultant for a couple of years already at the same time like trying to help other gaming companies to grow their games and businesses and basically like we have super scale which is well a lot of people say we are an agency or vendor or just not true or a publisher which is much way as well so thing is that we actually try and scale games and how we do that is that do the scalability assessments of all

the games out there and trying to look at different aspects of the game from game design and monetization data infrastructure obviously you a and creatives because the the gate the company started as a business intelligence company and then he forced into what we are what we are now so yeah we've been able to work with a couple of games from EA and Lego which I hope I could share not specifically specifically the the game game names but these are the biggest companies that we are starting to work from this year okay okay cool and so maybe now we could talk about this question around well so one of the things we talked about is having some type of overall framework and I know you have a specific framework that you use so can we talk about that now can you describe what do you mean by a

creatives framework so what is it and how is it structured yeah of course so basically we have these this process that we kind of developed through the time I mean it's a it's basically like there are two ways first is the like the regular creative framework which is like you know doing the creative analysis obviously like all the assessments for for the game a games that we manage and also like competitor analysis as well so basically based on those assessments we are like brainstorming the new ideas looking at actually target audience of the game as well so we know like what could potentially resonate well with that target audience and brainstorming the ideas obvious like choosing the ideas and like baking for with the client and then like choosing the first sketches and processing and they're like creative testing and objects like

evaluating in our internal tool but what is here like very important to mention in this frame framework before how I call it is that is they're actually testing or using those new ideas and how we test them on Facebook because I've heard a lot like a lot of other companies have different ways of testing and they're using either like a test account where they only test those creatives and then I move the mirrors to the account of the main and account other other people use like only app installs campaigns for like creative testing because they they say it's it's cheaper to get and much faster to get the winners which can be true but then you have to think about like how the actual Facebook work and if you if you use different optimizations like a pistol optimization or purchase optimize campaigns or value optimize the place you will get different results

with the same creative so this is very important I mean I'm not I don't say like the other methods or ways are wrong we just didn't didn't really work out too well for me so that's so basic like when I get the creative I'm using the same target audience and the same optimizations to have clear picture of how this creative resonate with resumes with my target audience and with that specific audience that I use in the campaign so basically using the campaign budget optimizations and then like creating the same at creating a new ad set with the same targeting same structure but then I think only new creatives so we have let's say going a little bit deeper here so we have we have the basic campaign structure with one out set running the creative they're buying the the purchase optimization and

then we see the creative fatigue you and we see that the the numbers are going slightly down and you see that the CTR is decreasing with the CPI rising after first couple of days so in here in this framework I use the new creatives but with the same audience with the same campaign just creating the the new ad so they're letting the came budget optimization spread the budget and if I after like first couple of days you can clearly see if the new creative is outperforming the old one or not and then if yes then always like pausing posing the adult creatives and then running the the whole budget into the new creatives so basically like D this would be this would be like the the testing methodology and the testing and creative framework I'm always talked about and this is something that proved to be really working for me and begins

manage and again I'm not saying like the other ways are not not the right or not correct but didn't work out that well for me as as this because this is like you know you see the the same creative with the same optimization method it's the same events and you know even though it's it's slightly expected slightly more expensive but then you have the clear winner and you can clearly make a decision based on these these numbers okay so maybe we could just take a step back in terms of the creative remark and so just for our audience and especially for those who are listening so you've got like the six steps one creative analysis two brainstorming three choosing the idea for first sketches and processing five creative testing and six evaluation so let's talk about one the creative analysis can you talk about so what would what are the types of you know analysis that you're doing in that first step

yeah so depends on the on the game stage again we we are jumping in I mean you always look at the the previous previous creatives and how they performed which is like the the first thing that everybody needs to do and I think everybody needs to do when study to work again if there is no number so there are no numbers to to look at and obviously we are trying to analyze the competitors and look at what worked for them these are the types of like the creative the first creative assessments and based on those you know looking at obvious like if we have any any numbers looking at their laws and the LTVs of those creatives and which performed well and trying to find out why and then and those learnings we are trying to bring into the brainstorming sessions so we know like what to talk about yeah so that I see four that you might use some tools like sensor tower

and sort of what what's work well you looking like let's say you've got a mid chord aim and you're looking at like mid core RPG game you're looking at like let's say afk arena you've got like those creatives with the mass hordes of zombies and stuff like okay these things seem to be resonating so so then you're just kind of looking at what's kind of happened in the market what's performed well there's stuff that you've worked on in the past and kind of try to figure that out then that takes us to step two which is your brainstorming phase and yep so in the brainstorming phase you are is there some kind of structure there or do you just like they okay what are all the ideas and just kind of write down all the ideas and then kind of think through that or a how what does that step look like yeah so basically although all the brainstorming sessions have the moderator there we like moderating although all the session and then obviously like writing down the old ideas obviously like after these many serving sessions well the the main point here is like not to bring everybody on

the sending sessions like as as you know a little people in there like let's keep it like three or four people maximum because like if you have like five six seven people like and not everybody can can brainstorming and it's like all whole but whole mess you just can't can work on that but yeah let's keep it a small small small group and what I would actually like not only like having these moderators the ideas I put to be successful and and good is to actually bring the game designers or PMS sessions to know like because it's it's very helpful to see and to hear their ideas and how they think about the game and also from their point they here like our perception of the game and then because a lot of time there are

like two different worlds but then in these you know in this case we're trying to breach like those two worlds and to work closely together and this proved to be very beneficial for like not only the brainstorming sessions but those like the the final creatives okay and then we go to step three which is choosing the idea so who actually like how do you choose the idea and who choose is it that creatively person or is it somebody else it's basically a combination of of our suggestions and our creative leads yeah a creative lead and then obviously like the partner based on our suggestions obviously because we have plenty of other experiences from other games that we have like more data points so we can make proper suggestions like what's which we think should work but then obviously like partner or client is the has define the world so then we just

speak I don't know like three or five depends on the on the cadence or the budget three to five ideas and then go for like the first sketches and then the processes right and then what is that like the first sketches and then is first sketches just for stills or like what about four four playable is what would first get just look like four different kinds the first yeah four four first sketches for playable that's definitely a storyboard so we know everybody can can see clearly see like what will be in that plug like every step because you know if we don't do this and then started building the playable it takes a lot of time and resources to build it and after you know just to hear from the client like yeah well this is not what I imagined well happens a lot bits that's why we just you know not only what we are but I think like a lot of other gaming companies do these storyboards so everybody is on the same page so this is

like this is very important step so everybody is on the same page so everybody has some expectations of the new creative so you know you avoid any any mistakes or any miscommunications all right so now step four and step 5 creative testing and you kind of touching upon this in in terms of the testing and valuation but maybe you could be a little bit more specific in terms of what happens during testing versus evaluation phase yeah so so then we have these like five five creatives and five new concepts so basically like running them immediately into the new ad suppose I as I as I mentioned because then like there's no no need for for anyway we just need to throw it out and see like the the first numbers so usually these like testing period is like depends on obviously pairs on the budget but we see like first my denote 5 to 7 base when we are spending like 16

vacantly higher numbers then obviously one or two days are enough I mean I can't say like any specific number of impressions because it's just you know I would be making this up but then you clearly see after first three days or first five days any trends where the new creative is going so then after we see the the winner there or or loser because not every creative even though you have all these processes not every every new creative is a winner then we need to go we need to take a step back and evaluate like why is this loser why is this a winner as well and then try to gain again go back to the brainstorming step and then create the new ideas based on those evaluations alone it's like not not deconstructing or only the the creative concept that also looking at the drawers numbers in this especially

in this step because that's the most important KPI okay and so this this face is a little bit iterative and then you're really trying to understand why are some of these creatives working why are some of these not and then doubling down on the stuff that works and and trying to try to push through that okay yeah cool in and you also miss I guess is part of this creative testing phase you're also looking at sort of the the you know downstream metrics or is that part of the evaluation phase well it's mostly like you know part of the analysis obviously like looking at the downstream attitudes but those but then it's part of the evaluation of course I mean obviously we can we can look at the the IPS but eventually like I mean the Duras or the LTV is the the true KPI that actually makes sense here right now is there and so once once you've got this

process down and you're coming up with a lot of great creatives like I would say that the next big question on my mind would be then how do you come up with that pipeline in terms of like you know we're gonna do this many creatives per week per two weeks per month like how do you come up with the cadence the number of variants and when to swap things out like how do you think about that so the first part of a question like how we come up with the different ideas so basically looking at the actual situation on the market and like doing this company the creative analysis as well as you mentioned like JFK Arena for example but then almost like looking at the trends in the hyper casual space because we find out that a lot of the creatives that use the hyper casual games we can use also for for any other games any other genre let's say we have a simulation game or farming game or whatever and we can show like the clear

progress there like the day one versus day 30 or day 60 or day 90 this is you know these showing the real progress in in the in the one creative this is resonating really well or like any any new vs. Pro variations for PvP games or any action games this is something that works really well but not only like taking a look at the hyper cash flows but we also look at the YouTube trends for example you know there is a couple of videos that people open chests in the games and those videos are like millions of views right so then you are like okay let's let's use this in the creative as well and actually perform pretty well as well because you know this is something that people love to watch and why not to use it in the creative as well if you have obviously if you have chests in the game but then not only YouTube trends but also memes

memes and any hype for example memes we use a lot of like Drake creatives and a lot of a lot of other other memes that we can use at the moment and then from these memes we just call from like static images to to videos because you know you can very easily do so but also speaking about the hype I mean so back then when the The Witcher series launched on Netflix we just you know we we saw that the witches is pretty similar to one of one of the characters in the almost hero game game from Spanish developer so you created the future because the guy from the game is called hilt and we just you know it was very clearly clear similarity between

the pitcher and the Hilger right and and the numbers just blew off immediately everybody was like commenting wow this is amazing draw us when immediately up it was really great so basically like using these types and like leveraging the the ideas and the characters you have in the game and you go a little bit crazy sometimes that can be like pretty pretty beneficial and and you know we use we have like very very strange sense of humor in our in our team so we tried to use that also in those creatives and obviously like it depends on the target audience of the game I mean we use like humor situation with thanks a lot that you know to target audience is male you know 25 plus they they love Family Guy or Regan Regan Maury so this is this is the human we we tried to replicate also in our creatives

but then and then also we're trying to look at the different like views on different aspects of like the characters for example we we did like interview with the villain from from almost to here as well like to to get the different different side of the game so he was like you know complaining about about how the characters are still you know every day is the same thing he was complaining about the characters how they attack him and you know he needs to go to the work and we saw like so many great comments and engagements on that creative people were like crazy Wow I like this I like this idea this is I'm gonna try this game just because of this hat because it looks legit I mean these are the types of things that we tried to do regularly and yeah this is something

that that pretty much work but you know it's all tied to the to the target audience this is really important to know who is your target audience like what do they like what resonates well I mean you see you so like the lilies garden creatives right I mean the time you're looking for lilies got it it's obviously like women and they definitely like these kind of like storylines and these kind these like not not like soap operas but anything like you know yeah dramas like this is this is so great and you can you can actually leverage these kind of things this is something that we used also for for one cooking game because you know we we just used a couple that came into the into the restaurant and you you could see the headline like hey your ex-boyfriend just walk walked in and then you have two two options poison him

or bring him the food and these are the types of creatives that present it very well with female tell you this because this is this is what they like I mean and this is having that you know depends on the target audience obviously like we can't use these looks for for my audience but then for male audience we try to do this humor situation for thanks a lot for example we did that with a couple of tanks and so yeah depends on the target audience that's like this how we can come up with different ideas and trying to monitor all the trends not only not only don't Facebook and also like memes and the actual situation on the market got it and then maybe maybe in terms of just closing the loop as far as metrics and evaluation maybe there's two questions I can ask you about this first I know we've talked offline about how some

genres like hyper casual are looking at slightly different metrics right so they're not as focused in terms of top of funnel front end on IPM they're more focused on CTR and so I wanted to like provide the audience your take on that yeah maybe we can start there and I'll ask you the second question as well later yeah so like for Hyper casuals I mean they are definitely looking at the IPM because that's like that's the like super important think but I think like that the question was more about different podcast you you were speaking about like the concept testing and the marketability of the of those games and I think there is like some question about why they why the hyper cage will focus on CTRs I mean obviously focus on other things as well but they've done so many tests so they have a lot of a lot of numbers to work with so they can clearly calculate all the

other metrics from those benchmarks they already have so basically like voodoo have a lot of CPC metrics and CTRs and then they know like if they're looking at like 11 eleven percent CTR and like two cents CPC the CPI will be like ten cents or maybe even less so this is how how they approach those metrics yeah this and just to provide the audience a little more context so I had an opportunity to speak with some of the you know a few top hyper casual companies and you know what they were telling me is that they don't even look like they don't even have a game right yeah of course I'm just testing a video they're testing this and then once the CTR is good then don't make the game so yeah that was the context on that of course because you know the video is actually the game yeah the video is like you can well you

clearly need to see like the game mechanic and this is that's it because you have like 10 second video and after you what you after you watch the video you can clearly see what's the game what the game is going to be about and if again as you said that if it's the CTR it's good and good is like 5% and above then obviously yeah why not to make the game right and then the other sort of evaluation question that I had for you was with respect you like the downstream metrics and we've kind of alluded to this before in terms of measuring rows and stuff like that but I think a lot of this came from the deceptive ads right and so there were what we saw were a lot of deceptive ads being used and based upon that did the front end metrics were very strong and then there was like this assumption in the industry by some game studios that great you know our front end metrics are a lot higher but then later on it's like oh wait we brought

all these players in expecting game a they got a game B and then the ultimate downstream metrics is good and so could you talk about you know sort of that phenomenon and you know is and in terms of the downstream metrics that we should measure is really the only thing and then are you measuring from a downstream perspective how what do you what do you actually measure in terms of you know DX for us and if it's DX row as then what what's the X is it 7 is it 30 how do you come up with that yeah so regarding the DX cause it's it's definitely like different from and dependent of the game but looking at the not only d7 because obviously b7 is not the only only metric and then like long-term metric they turn across for example but we need to know how does the curl to look like the LTV curve look like and compare that to actual create compare it to the creative level because

you know you see like from the from your analytics that how the curve ltb cook should look like and then you just go down to the creative level and this is this super important obviously so that's there you have like day 1 day 3 day 7 and day like so let's say day 28 and you compare the how the rest should look like with the creative and then make decisions but like back to the those deceptive ads I mean I can still see a lot of guidance caves and Hobbes caves as for for their games and and thingies and also like a Lily's garden I mean there's no almost nothing from the game but then it was the the creative was so interesting and I actually read that in some comments in the Lily's garden there's like well okay so the the creative got my attention I expect totally different game but then I just started to play the game and it's

actually fun and I was hooked immediately so thing is like if you if you get a lot of loads of players into the into the game with with low CPI's then there is a higher chance of success of like converting those those players into the actual players and then into the players eventually so and the thing is I'm I'm not sure if you remember but Kings of Avalon and also this disgrace really free the admin didn't look anything like the game but then nothing like the game squeeze but then I assume that it works really well okay okay but you know actually while we're talking about deceptive ads I know Google had some recent policy that

talked about banning these kinds of deceptive ads but do you think like well if that's true I don't know if you have any context in terms of what you know of that policy but what do you think will happen in terms of the industry assuming that it gets rolled out not only on you know Google but also on Facebook and elsewhere yeah so these deceptive ads I mean I'll policy from from Google I mean they they release this policy but then when it's actually going to happen nobody knows because it's it's always like this in Google I have good friends so sorry guys but still it takes so much so it takes so long to get like those deceptive ads out of the game and I've done the like the marketing game because they okay that's for Google I mean that's definitely something that's not gonna happen anytime soon but then I think like I'm worried about the Facebook because rights it's more

flooded with all these ads and it's starting to be annoying and and you know definitely need some some actions here and as soon as possible because you know we are we are trying to create experience and trying to set some expectations for the players but also on the other hand I think at the the metrics and the business side of those certifiers I mean actually I guess they work pretty well in the Ross perspective but they're you know we should be platter friendly here and trying to be trying to create the player friendly experience experiences in in those ads so make sure you like what they see and they and they're not like they'd they know what they're getting with with the creative and they know what to expect when they're actually into the game right let's try acting like this should happen

but you know when that's a that's a very good question and I don't think it's it's gonna be anytime soon unfortunately okay so maybe one of the other big topics from a creatives optimization perspective is this issue around creative fatigue and so in my creative fatigue for our audience it's when you know you've got the same creative that might be performing very highly but people have seen it over and over again and so the performance stop starts to drop and so from your perspective montane like how do you measure this and when do you know when you should retire a creative and then on the flip side you know when do you know when you can bring back an old creative that will perform very highly in them in the past and you know kind of resurrect it yeah so I don't think that there is like any specific point for for all the creatives so all all the creators like different creatives are different life a lifetime and but you can clearly see after like certain period of time let's

say some some creatives can can work for several years I was talking about the biggest adventure and we were running this one horrible arguably looking creative for three years we spent like I don't know like ten millions into that kriti still profitable still profitable and we were just you know that was just our evergreen creative and then just performing well for two years and I see I see those those guys still still like hovering around this this creative working with that with slightly better visuals but feel the credits still there and so after you know after two years then obviously this is the time you see that you launch the campaign with the same creative it's not performing well the CP ours are super low and you can clearly see that it stopped resonating well so you need to move on but it

doesn't mean you just can't like after half a year you can resurrect the you can't resurrect the credit and that's what we that's what we did a couple of times before so I mean there is not like you can resurrect any creative after half a year it's just you know we meet sometime there is no no silver bullet I think here but you know from time to time it's good to refresh those old evergreens and start start adding them to the to the campaigns and and regarding the creativity us as a mission it's like it's different from from other creative concepts so basically we can we can run a creative for months and it's super forming wealth but then some creatives have like a couple of days and then that's it and I wasn't able to like crack why is this happening but I it's like eventually you need to take a look at the the combination of the creative and the targeting and try to make any assumptions or like any any data take up

from from that combination because then after using the the creative with the different target audience with slightly different gears it can still work with but on that specific targeting and that specific combination of of the campaign structure it just just can't work anymore and it's all all about the evaluation you you will see that in that like point of the process that this is the time for like changing the creatives you know you can't speak about like any any specific numbers here because it depends on got it and so it's it's an art you're saying and you kind of need to use common sense and I can kind of understand that because there could be a lot of exogenous factors that impact the creative so like let's say you've got a super performant creative and then you're your lead competitor sees that and they copy it and then then that could lead to create a fatigue but are

there specific things that you watch for whether it's like a dip in kpi's of a certain level or whether you look at like the scale of spend relative to your the size of your target group or is is there anything that will help inform well maybe this is creative fatigue or not yeah so I'm yeah looking at like different different factors in Rio marabous it's definitely like the combination of potential of each of the of the target audience and how big is the target audience and also like it to spend as well and I'm looking at like any any dips in terms of the CTRs because always like when you launch the creative them and it's its convener it start performing really well from the beginning and we have these benchmarks of CTRs and CPI's and everything and then like if the creative dips under those benchmarks we just you know that's the red flag and we need to keep a close

eye on that so basically like for purchase campaigns like looking at the at the CTRs everything like under the 0.5% it's time to be starting to be a signal that we need to take a look at these these creatives and then obviously like when you see like the CPI trend is increasing day by day and you don't see the the stable CPI anymore that's also gives the signal when you need me to you know get attention like okay this is this is something we need to look at but they're like the CPI is rising can be also the fact that you are reaching the whole target audience but then again like as I mentioned before with this you can just swap out the creative and then you restart the process again so even with the same targeting on it and so last question for you Marty and it's actually a very critical question which

is tools so can you talk about some of the tools you are using for creatives optimization because I mean I've seen all sorts of stuff out there but it'd be great to understand from your perspective what are some of the critical tools you can use to really help you be as efficient and effective as possible in terms of creatives optimization yeah of course I mean there are plenty of tools and fancy tools you can you can use and I use like lucre tableau or absent or any any other tools for like any creative optimizations but then we built in-house create tool where we like try to evaluate all the campaigns on the creative level but basically if you are using FM pieces like Facebook marketing partners a lot of them have really good really good creative dashboards and if that's something that it's actually really efficient but then not every company as all those all those resources

and budgets to its work with with the tools so basically then if you're a small company then you can't afford these then you should be good with all the Facebook days before the all the Xbox games and apps layers but then it's really painful and a lot of a lot of time you need to be in those spreadsheets so what if you have resources then obviously like all those tools and now the second piece and creative dashboards can really help and that's that's why we actually build those creative tools in-house because we have these raw seeds where we like evaluate all the campaigns and then we just said like hey well let's go to the to the creative level so we can we can do the make decisions based on the real date actually ok and just to kind of take a little bit of a higher level of perspective in terms of tools so you mentioned things like looker and tableau and I'm assuming for that that's really

about your your data infrastructure in creating like dashboards in a way of visualizing like a bi perspective in terms of what's performing how you're doing the the sort of analysis piece and then when you talk about spreadsheets and you know this would be great for me to ask you because of my own experience when it comes to sort of managing archiving and putting specific metrics to krita's themselves what I've seen are just like these crazy spreadsheets with like like and literally you've got somebody spending two to three hours a day just like you know implements updating these spreadsheets now I know there are some companies like you know for example but all go comes to mind that help you like you know have a like a holistic view in terms of all your creatives and metrics to them and things like that but is there a better tour is this what you guys have been building in-house or yeah this is actually like what we build

in-house and I mean as as mentioned the Facebook marketing partners so when you when you are running or creating the campaigns with Big Al Gore or smartly or at Quan by Ken's to you have this creative - because because they they all have the creatives set up there and it's pretty easy but what is really important to to make the the capers and the creative is also like the naming conventions because then you clearly see what is inside the creative and what is there how long it is it is 40 X square or whatever so you just don't need to like see all creative but one by one so you just have to see like what is inside so this is like very important like to have proper naming conventions for those and if this is the spreadsheet and I am I'm not sure like how how long you could do you could be working with the spreadsheets because that's just they're

just too much that's too much so yeah I mean like all the although all the tools like mention to people going to no tools look at tableau that can you know be very time efficient definitely definitely by the way of doing creative analysis and evaluation then like looking into spreadsheets but then honestly again if you're a small company you usually it's like a yeah like a spreadsheet and then like some crazy complicated naming structure so you got your fuller directory system and it's like anyway alright cool well that was my last question but actually maybe one final bonus questions for for people who want to hang on here so there has been a lot like recently there's been a lot of talk in terms of the the changes

happening in terms of data gets that gets passed back to you amen from Facebook for example right so we're not where the fact that Facebook is going to stop sending user level data back to studios people generally say that that you know creatives all then becomes more important because I guess a couple things one there's less that you a people are going to be able to do and then Facebook and Google are just sucking up more and more of what you a people do so wondering if you could talk about you know what the impact on creatives is going to be based upon that data data change and then just your general thoughts in terms of that policy yeah so yeah I read a lot of catastrophic scenarios all the you a performance marketers lives are going to change I mean there is something that is going to happen definitely and then I'm really sure that the apples will change

the policy as well and we won't see like a lot of a lot of data points we seen nowadays I don't think that it's gonna be dead catastrophic because a lot of companies we've got will go bankrupt and I don't think like all those guys can afford that and I'm definitely sure that there will be some workarounds because there is all the time there is some workarounds and we fight we face these kind of like policy changes already couple of times before can we change when we moved from IBS and mature like which I these two idea phase there's oh there will be an I think and this is my opinion kind of not that what that negativities in this side and still think that there will be some so campaigns or any group to do find a workaround and there is always you know there's always a way how to because I read that you can't just create

look-alikes anymore because of the MMPs and everybody's like yeah well you can create Luca likes on Facebook platform and Facebook analytics and this is pretty crucial because there are a couple of ways how to create look-alikes and this is one of those one of that and the same same thing you have firebase you have audiences there they're connected to Google UAC campaigns I mean these changes will just you know make Facebook and Google bigger and obviously but what we can do so I don't see like that that much of the cutter software's there so I think the creative is already like super important because like all these like machine learning and all the automations activities that Facebook done in the Pisan and Google as well so yeah I still think that it's that bad but let's seem in couple of couple of in one six five is

growing or not thanks so much for your time Marty there you have a Mott say and also do you have two things do you have a final message for our audience and then secondly if people are wanting to contact you use your a good way for people to get in touch with you definitely I'm a LinkedIn and I do have my own personal website I wanted me I can share that after the podcast yeah okay so basically what I would like to share is you know the creative process is super important and not a lot of companies have that place so definitely think about how to how to you know after listening to this you should have a sense about how to create a new concept a new ideas but still the this framework works for me and I don't necessarily say that this will work for you right away and we have like we have spent a lot of

time perfecting this so be careful I mean there are a lot of insides I believe and I hope so but you need to be careful about how to implement that in to your actual framework because if you have something that already is working for you don't change that but try to experiment a little bit with what I've I said and you found useful to find a proper balance about experimenting new things and what works for you but also you know you just don't change your your actual framework because I said something something important because this this is something that I see also like a lot of a lot of times that someone just you know listen to a podcast or read the book and immediately think that this is gonna work for for that company well it can work but you know we need to be very careful what what - what you try and what what do you do next so yeah that's that's probably just from

a couple of comments from from myself okay great I totally agree with that and so thank you very much for your time on today and until next time see you later okay thank you very much for having me [Music] [Applause] [Music] [Music]