AI Creative Testing and the Problem of Testing Debt

That's far away not the the the most complicated version. That's the that's the pretty like I would say an average one. Uh and uh but basically for the small teams it doesn't need to be even even that complicated. You can start with just uh those parameters those variables that matters for you the most. And
hi everyone, welcome to another episode of Ry 101. I'm Roman from Tangent and today I'm joined by Mark. Hey Mark.
Hi everyone. Hi Roman. Uh thanks for having me today. I'm Mark VP of growth at Beat Matrix. Would love to talk about AI and creatives today.
Exactly. one of the hottest topics and uh creatives were always a hottest topic and now within like last two years for sure

and we would like to add AI to the mix to make it even more interesting uh as per our tradition our guest uh Mark prepared uh slides so we can just jump into it Mark if there is anything you want to say about yourself or bid matrix. Uh I think now is a good
yeah I just would like to add some contact uh context uh as at beat metrics we basically help mobile apps acquire and re-engage users and uh since we work with and we run like a thousand of ad campaigns for hundreds of advertisers every day and we work mostly with a more like complex models optimization models um which is called which are called performance-based such CPA for example um and uh we are optimizing towards um downstream events more complex events that is why we are really really

interested in what happens after an ad gets attention. So do the right people install uh they stay or they create value. So and that is why I would love to talk about um uh creatives today and how AI changes creative production and um basically spoilering the ma main idea uh why measurement becomes more important not less.
Super. Yeah, let's just jump into it.
Great. Uh sharing the screen now. Uh so here we go. So yeah, I hope you can see it clearly. Um so um so basically um as I already mentioned that um [clears throat] um AI changed creativity and and how the c creatives are being produced and um it basically um makes it easier makes

people makes teams easier to make ads but it doesn't make um them proof free. Right? So when I say creative, I simply mean the ad uh that people see and um AI can now make uh another hook, another voice over, another image, uh banner or video version very quickly. But um every version still needs budget, time and um enough precise data to judge it, right? So um the the hard part I would say has moved and um we are no longer short of ideas. Uh we are short of reliable decisions. And um for me creative accountability uh is that what I'm about to talk um means answering um three brain questions. So what did the ad promise? Who did it bring? And um did those

people stay and created any value? So um once everyone can make more ads uh the advantage moves to how well the team choose. So and um sorry about that. It's a bit sensitive. Um so um as I already mentioned everyone can create more right now nowadays because everyone has access to to the most advanced AI tools and um the new bottleneck is uh judging ads not making them. So before AI, so before AI teams often waited for designers, editors, video producers or um new concepts, right? So now they can create like tens, hundreds of options like in a day. So very quickly and um from the first look it sounds positive, but it can create testing depth, right? So more

ads than the team can properly test. So when the budget is pressed across too many small small variations um the results become noisy and uh there might be a lot of uh false winners appear and uh the best team is not the one with the biggest u folder of ads I'll say um but is the one that can make a clear guess test it understand um evaluate the result understand the result and um use that learning as an outcome come in the next round to make the decision.
Yeah.
So,
totally agree like you can generate unlimited amounts amount of creatives but you'll never have enough money like budgets to test them all. Right. So,
Exactly. Exactly. And even if you have enough money, um it doesn't mean that you can easily uh spend them out on on on all of those um thousands of of of

ads that been generated, right? You need to prove the real economical value. And um that's the I would say that's the key goal. It's still the same the key goal. And um there's like a simple analogy um is um so basically uh 10 meaningful different ideas can teach more than 100 versions that only change the background color or one word right so and and and when you have like a limited budget and everyone nowadays have limited budgets uh you don't have um u money and manpower to to test them all. So you need to make precise decisions, right? So and be careful. Always be careful.
What do we do then, Mark? We review.
Yeah, I I I'll try to elaborate on this and I will try to talk about uh what could be possibly done. Uh what uh do we do at beat matrix for our clients, advertisers? And uh yeah, we'll we'll

keep a talk about this. So and um we can start with like a simple uh survival loop for ads. Um we usually apply and uh here it is on the screen. Um I call it creative Darwinism loop. Um it might sounds technical but the idea is very straightforward and simple. Um ads have to earn the right to survive, right? So um many ideas enter but only ads that prove value should receive more budgets. So very straightforward, very simple. Um uh so let's talk a bit deeper about this. So first um you need to research what users care about. Then uh the next step is to generate a few clear ideas or hypothesis. Then um try to do a filtration. So filter out anything confusing, off-brand or misleading or

irrelevant to your product and um try to allocate a little controlled budget for a test and then uh validate whether the users stay and create any value for you and try to scale but carefully very careful carefully and then record as many as much that as you can and um try to um try to learn out of it, try to to learn from it and then make the conclusions to to scale to improve and uh to to to add some some more new ad campaigns, new ad ideas. AI can basically help uh at every step out of this loop at every single step and we'll talk about this but u AI should strengthen the loop rather than um replace it. So you can't just one one prompt at one shot uh the whole loop. You still need to take control over it

and uh you need to imply AI as a as a player not like uh not like covering the whole loop and the goal is not to make more files but to learn faster from the market. Where is it most important to have kind of let's let me call it manual input like where I do the job myself and not let the AI do it like is it a research because it's a very start or does it come like at the later stage
so um um I would say uh AI can enter every stage every step every single step um but um uh the most important I would say um um the manual input is the most important on the um uh testing phase on the testing step and validation um because when it comes when we talk about the real budgets the real money is that that exact moment when a human should

should definitely gets involved should take responsibility and should have the ownership. So and the test validate means uh start a campaign and assess the results.
Exactly. Exactly. So uh when you decide to test, you first need to um I mean the human first needs to take a look uh on the whole setup and and say yeah that's worth testing and here's the the budget for for the test. and a validation um it's it's it's more like AI friendly I would say because it's not it's it's decision less but a human still needs to um um take a look on on on the conclusion on that that they could propose because it's also um at especially at the very first stage and when the data is very limited if you you assigned a small budget the data will definitely be limited and the eye could have um not enough data to to draw a

very very precise conclusion. Uh that's that's the moment then a human needs to get involved as well.
Got it.
So and um um the easiest part of the ad to test first is uh usually the opening. So I'm going to talk about u the hooks. So it all starts with the hook and um at first AI doesn't need to to make the whole ad. Uh it can first improve the opening seconds for you. So the hook uh I would call is is one of the safest and the most useful places to start with AI and start to implement AI and to give AI um a certain level of ownership. Um, full ad can be complex, right? So, full ad creative can be very complex. Uh, especially for games, for example, when you're showing showing the game play and

and the concepts. Uh, but the opening has uh one very clear and straightforward job just to stop the viewer, right? So, the viewer needs to stop and needs to to to to catch you needs to catch the user's interest so it it could continue uh to view your advertisement. And AI can basically compare patterns such as failure, surprise, challenge or a certain transformation or reward for example for the games and then suggest new openings to test and you can have like bunch of really useful variations on how on the hooks on the openings that you can test further but the human still needs to check whether the hook fits the real product because uh to avoid the eye slop you can just simply you know take the the the the AI suggestion and apply it right away. Still need to validate but u it could be done in the hook but after the hook of course uh the body the proof

uh um u needs to prove the promise. So in hook you you give a promise and then the body needs to prove it and then of course the CTA the call to action needs to tell the viewer what to do next.
How does the process look like? Do I go maybe you can give a an example of a prompt or um what is it like where do we start?
So yeah so for for example uh like uh uh for example for a puzzle game uh let's let's imagine a puzzle game uh AI might create a several like fail uh then fix uh openings. So you can simply uh ask like claude or chpt you can first of all give him a context of of of the product that you'd like to promote that you're preparing the advertisement um for and uh ask him to create uh several failed then fix all things and it could suggest you ideas on how to to put it in that way so the users stop and and continue

to watch this and then can keep the team can can take a look on on on the on the variations on the options and just keep the strongness strongest openings. Uh but then it it's still in the body and the proof I I strongly suggest to show uh real gameplay in the body.
Yeah. Yeah. Yeah. Yeah. Makes sense. It it just like it kind of resonates with me. But even with this videos for for this channel, I'm doing this let's say 15 to 30 seconds previews at the start of the video. that that that's kind of a hook by itself, but uh what I've noticed I need to do it myself just because I know I know my audience and I know like what people care about whenever I give the maybe my prompts are not strong enough to be completely honest. So if I give it this video and say like oh pick 30 second

hook to start the video with it will pick some complete BS. So basically um I think it's uh it's very important to uh uh let the AI uh understand your goals right. So so what you would like to achieve with the hook and and then for example for for your case uh I would I would put it in this way. Um now I will first like uh uh pull pull pull pull the the whole transcript of the of the podcast and then uh I'll put it in the file and MD file for example. AI reads the G files very well and uh uh you can uh list down your goals like what you'd like to achieve uh uh with that hook with that 30 30 seconds preview for the video. So for example, you would like um um uh the people to the watchers uh the the the um the people who going to watch that podcast to continue watching or to to get interested. And then uh the logical

questions are who who is your audience, right? who who are those people who watching and uh how to get them interested and and and and those those questions um I mean I can I can continue and keep moving forward with those um clarifying questions but once you answer all of those you can even ask AI to to ask all of those clarifying questions. Uh when you answer all of those and and provide the the clear transcript uh it could help you to extract exact moments ex of the interview of the podcast uh uh on uh which worth to include in in in that in that preview in that hook and make make that pay basic hook and u and then you can even that take that conclusion and uh put it in the other uh uh uh AI and ask him to edit the video uh to put it in in before that video.
Let me try that. Uh I'll try it with you. I'll try it on.
Yeah. I'm I'm really interested to to to

take a look on the outcome.
Yeah. Yeah. Yeah. And I'll comment if this is a preview is going to be I in the comments. I'll comment if the preview is AI generated or not.
Okay. Okay. Deal. Yeah. Sounds interesting. I mean real real experiments. I love that.
Yeah. So um and moving forward um a hook usually works because um it creates the feeling right. So that's something that we talk about about the feelings and um not only because it shows a feature of an app or or something or but it um calls a feeling. So a user watching that and cause a certain feeling and um this is something that very important to to um keep in mind. Um I call it wipe marketing. We call it wipe marketing. Uh our marketing manager at Bid Biden really really loves that that term. Uh and it white marketing is simply a way to ask a more human question. uh like um

what should the person feel before the install, right? So people do not respond only to features. You can't simply show uh all the features of of your app of your of your game in an ad in a creative and uh and and and and think and and uh expect that the user will will go and download the app and stay with your app. Um users may respond to some feelings. Uh it could be relief like confidence, uh curiosity, uh progress or like alerts like urgency uh or a certain level of achievement or something like that. And AI is basically useful because it can help you to explore um bunch of different emotional angles and statements very very quickly. And um uh you can put the strategy in

the way um um to decide uh which emotion fits your product and your audience the most and then to to select and and to test them out. And that the useful useful creative uh brief can therefore have uh three parts. So basically what the app does uh what the users should feel and uh why they should act now the CTA the call to action. So those those three parts and um I have like a real life example over here uh complete anonymous but but yet like a fitness app um like the fitness app can sell like daily workout plans right um uh but they can't they can't elaborate on on the daily work plans in the creatives only about them they can't show those daily workout plans but um the stronger emotional promise that they can give might be for

example uh finally feel control uh of your of your routine. Right? So uh when that feeling is um implement implemented in in the creative production and you manage to achieve um and manage you managed to pull you managed to get that feeling from a user who's watching your ad. uh that could possibly um you have you have high chances to to to get this user converted and to move forward to click and then download the application. [clears throat]
Got it.
So and uh uh so yeah basically one straightforward line is like uh good ads connect a product feature to a filling and uh and that's could be turned out turned to a reason to act. So um uh and one job is clear when the job is clear the team can uh choose uh the right tool for that job right. So uh here here comes like uh maybe the the most

interesting part or the most uh applicable part um practical part I would say. Um you can guys uh screenshot and and try those those tools out. I tried to list it uh some of those um um I would we believe uh very useful in the creative productions um by job to be done but I would not try to like read every tool on this slide and explain all of them. I would just explain the jobs that needs to be done and can be done by those tools or with the help of those tools of those tools. So first um research uh research signals or research uh it is about finding competitor angles for example and user ang user language. Um second uh hooks and briefs. So this is something that we already uh mentioned in before is like turning that research into testable ideas into

testable hooks openings um creative scripts or whatever. Uh third is uh the production itself. It's about uh creating uh basically production, producing creatives, creating creatives, um short videos, UGC style ads, uh banners, product demos, whatever you you like the most. Uh fourth is about the presenter videos uh about talking hats, avatars and localization um translation uh which is very important when you're like trying to target uh multiple different countries and geographics. Uh and uh the fifth is u the analysis. So how you basically score uh advertisement and spot uh fatigue on some in this funnel. So how do you analyze the metrics? Uh what metrics do

you analyze where get the data from and etc and etc. So basically those those those five jobs are the key ones and um my personal uh pick I would like to to highlight here is uh uh Hixel. you you I I bet you heard about this a lot and I personally use that for my my uh for my personal goals and I love the way they they do uh the product and I love the way they expand um um expand the the user experience the with the new tools and there's something if you if you haven't tried this before uh yet I would recommend to start with uh those uh studios alike like uh sections like marketing studio or short studio when you can literally um design and fully produce your creative from scratch. And uh the one and new feature they implemented very recently is

supercomputer. Uh it is uh um it is basically uh a chat similar to what you have in in Chad GBT or cloud. Uh but it is uh directly connected to all of the Hixwell tools and you can basically talk to it and and and and go through the the whole process of creative productions by just you know talking to to the to the AI. Yeah. But uh there is no universal winner. just, you know, uh pick your your your slowest and messiest step and uh um try try and figure it out like uh which tools you like the most.
Yeah, man. I think this is going to be my hook for the video. I think this is such a good uh picture. And um also small comment uh about MCP since you've uh mentioned here like we've seen a lot of users just using MCP like for

everything like for reporting and even for the tasks um show me is a creative set like have an add fatigue and I should change and it will give you like connect to tangent nut and give you like whatever names you need to change. So like super convenient.
Yeah, I I I I love that a lot and we use it uh also a lot because uh we work with MMPs a We work with Tanzen and uh uh we we see that it's I mean for our operation team it it it's it literally like facilit facilitates um uh most of their routine a lot and um um I I think every modern uh measurement tool uh not only for the for the apps should have their own MCP and uh uh should should should build should build a bridge between um their data the they provide and the AI AI agents that could be

processing those data on the advertiser side on the traffic sources side.
Do you use any other uh MCPS outside of uh MMPs? Um I I wouldn't say that we we do a lot. Uh we do have our own uh platform and we do have our own MCP as well and uh but is like it's mostly like for for the internal use only um uh but yeah but I would say um uh for the marketers for the mobile marketers the MMPs MCPS are the most the most useful nowadays I would say.
Yeah. Yeah. Agree. Agree. Agree. Yeah. I think that's for me at least it's a trend 2026 like the
Yeah, that's that's that's all hype definitely. So uh um moving forward uh uh u tools like those tools that I I I tried to show here uh is uh they they

can make an ad look strong but um the question question is the key question is still is still there is whether it brings uh the right users. So um um so here you can uh oh no not not this one I'm sorry about yeah over here uh so um an app can be good at winning installs and at the same time can be bad and fighting the right people and I think everyone faced that that multiple times before and u it's usually not that easy to uh understand uh the reasoning to understand the reasons behind that and um basically this is like the key measurement slide and um uh CTR like the click-through rate and IPM like instars per mile per thousand impressions uh could tell us um whether the ad got

attention and installs but more like downstream um uh metrics like retention uh tells us whether the people were actually interested in the app. And uh the third bucket uh Ross and LTV tell us whether the campaign ma made any business sense. And I would say that's is something that interested your boss the most. And uh um if the real game is like for example um uh yeah I have like also like a real life example over here. uh um like um we've got a casual game um and uh it it got a creative uh with a misleading gameplay. Um and this is kind of quite a common uh use case for the for the gaming apps specifically. they're trying to hook to catch the attention of a user and try to to to to to make um him download an application

and and then uh when the user if there is a misleading gameplay um app can have a very very strong CTR and IPM right uh because the idea excites the user a lot and and it clicks and it downloads uh but uh when the real real game play turns to be different many users just you know simply churn and and and live uh the app and do not play it anymore. And even the retention of day one is is is completely uh uh awful. And um AI makes AI basically makes this this trap to happen easier. Um and because it can produce like a lot of um like dramatic like very very like thumbtopping uh scenarios like concepts for the creatives but um if the gameplay is different, if the product concept is

different uh you're going to receive uh the bad ch and uh uh your retention will will will not be there. and uh eventually your your unit economy won't be there and that is why a winner like should pass both an appeal check and and then the quality check.
Makes sense. Makes sense. Makes sense. So and um so yeah just to to take to to make it uh uh to give it a numbers related example let's imagine uh uh 1,000 people uh uh see your ad and for example 15 of them install it means that the IPM is 15 uh and out of those 15 uh who inst finally installed the app uh only a few or even like one of those users users return the next day. So the day one retention is is like uh is is around like 1% let's say. So uh that means that the ad attracted

attention but it attracted the attention of not the right audience. So uh that scenario happens a lot and uh when we compare appeal and user quality uh there are four clear outcomes I would like to talk about as well. Um so um yeah I I I called it like the misleading winner matrix. Uh you can also refer to to to that graph. uh it's it's clearly shows the correlation between um the insole efficiency the the IPM uh or a CTR it could be both uh and uh more like product related metrics such as retention and and and ROS. So the most dangerous ad is the one that looks like a winner but too early and uh the most dangerous decision is to bet on

on such an ad and uh lose your budget and receive zero in return. So this matrix basically gives uh four simple cases. Um high appeal and high quality true winner u no comments. Uh so if you receive high IPO and high quality it means the the IPM or CTR is high retention is high uh and eventually ROSS is is pretty stable and high. Uh you're allowed to scale carefully. Uh good luck with that. Uh if there is a low appeal and low quality uh just kill simply it needs to be stopped and reiterated. Um the very dangerous case is the one that I I I uh mentioned if you receive high appeal but low quality. uh it looks successful in the first trap word and you're about to scale but users leave and do not create

any value which means that you're about to waste your budget and that is a misleading winner and um and um and the the last one is uh when quality exists but uh the hook is weak it means that um CTR and IPM are low but uh the retention and and the ROS is pretty stable. LTV is pretty stable. Uh then you just simply need to repackage it. You simply need to um um go to any of those AI tools uh are good for creating hooks and openings and uh try to to play with your with your CTR and try to play with your IPM and try to to to create um better catching creatives. maybe maybe to play with some um misleading game play those gaming apps like a lot.
Yeah, I like this repackaging and um as simple as redoing the hook. Uh seems

really like promising and I guess easiest to do in the age of AI as you've also noticed earlier. Uh I have a I think at least one question here for the retention there is a lag uh basically you need time for users to spend. How do you solve this like do you predict uh LTV based on whatever your models or uh do you have some sort of uh best practices? So when when when we when it comes uh uh when when we're talking about the the the timeline and the the time limits uh in order to uh make early decisions uh you need to understand your your product uh very very deeply and um you definitely need to understand uh uh the connections between the the events inside your apps. So which causes which

event causes the payable event uh which event causes the event prior to the payable event and etc. So you need to to see and visualize those connections. And if you uh have a deadline on making a decision, you simply refer to those events that happened early earlier prior to the payable event try prior to the targeted event and try to make your judgment based on on on on those data.
Got you. And how do you allocate budget here? Like do you start with a static and like add a certain percentage once it moves let's say to next stage and you say oh that can be a true winner. Oh this is uh should be
yeah. So uh if we see like a true winner from first scratch uh it's definitely um the one that that that you need you need to try scaling. Uh but

uh yeah, except of the true winner, I wouldn't say that anyone else is worth scaling. It's worth investing.
But I guess in the real world, it's rarely a true winner. You're like looking at the performance, you're like, ah maybe yes, maybe no. Should I give it another day? Like just want
Yes, of course. And uh it's in in the real world is not that simple and straightforward unusual. And uh uh true winners can also vary, right? So you can have like a true true winner and you have like like a kind of a true winner. And uh so basically um the um when you have like both of uh the appeal and the quality in place at a certain level, let's say at a very low level, but acceptable, this this this something you're worth exploring before. It's worth scaling. But always carefully. Always carefully. Gotcha.
But yeah, and and uh I'm not assuming

like that just like um a very important disclaimer or a note I would like to add is [snorts] that I'm not assuming that every quality problem that is connected to retention and the ROS and LTV and more downstream metric metrics uh uh caused by by the ad itself, right? you can basically have uh like a a lot of a bunch of other issues that are not ad related uh related to the product to the on boarding to the traffic source to the to the audience targeting or whatever. So you still need to keep in mind uh everything at the same time.
Yeah, good point. Um so and even uh so yeah talking about the true winners uh as I said even a true winner could eventually like stop producing the same extra value and uh this is what uh they call fatigue and um u so it and basically [snorts] ad

fatigue is not only CTR decline at could be could become tired when uh like more exposure or spend uh stops producing enough extra value enough for your unit economy first of all. So um I also like uh put like a simple graph uh that we we got from one of our um clients uh like a lifestyle app. It's like a saturation example uh that we we've got and um uh you you you can see like the correlation between increment incremental trials um like the the free trials inside the application prior to the actual subscription payment and uh the daily spend uh and the cumulative uh uh daily spend uh in the dollars on on

the on the impressions. So you see that uh when we start spending when we when we first of all located the true winner and we start spending um with a 1k uh US dollars a day uh it it gots us like around a 100 trial and then we we we thought that this is the right true winner. we need to scale it and we did it like a five x five um five times scaling the budgets the ad budgets to 5k a day uh and it started to produce slower gains and uh it started to produce uh it didn't return us x5 of trials right so we hope we we we hope that it is going to produce like 500 trials but it didn't it produced like only 200 and then when we try to increase more because we still thought that it's a true winner. Um it it just became flat, right? So it it's tough to

produce any more trials along increasing the budgets. So um basically um yeah, an idea is that uh you you need to catch this as early as you can. And um people often describe fatigue as like a falling click rate uh when they look at the CTR. But um that is completely not the only one version of fatigue. Uh but that the chart shows uh the that pattern that we've got in the lifestyle app and it's it's it's irrelated with the uh uh basically the CTR but but still it is there. So um the key the key the key conclusion from that slide I would say would be that uh you always need to uh be cautious and and scale carefully and slowly uh even with the the true winners that you believe the

most.
Gotcha. Uh how often do you refresh your creatives? like of course depends but maybe there is a process
um with in the in the AI era uh [laughter] every day in the AI era every day but it's also like it differs completely different case to case because we we do work with like um um hundreds of different advertisers from different apps verticals uh those are far away not only the the gaming mobile gaming apps but u the the finance, the lifestyle, the e-commerce and and bunch of the others and um it totally depends on the product uh even on the market right so for example now it's like the world cup is is going on and uh of course those i gaming apps might might be having the the most of the attention and uh you you like depends on on on what happened during

the world cup um you might be changing the creatives a lot right so There are so so so many variables that are in that formula. So I don't have like the the one exact answer but with AI era yeah we are allowed to to create like uh tens tens of of different uh creatives a day and uh of course we're not we don't have enough u um human power in the budget to test them all and uh could to compare but yeah we still we still are able to do that.
Got it. So um and uh basically to monitor um um to monitor the the fatigue pattern and um the same ad must have the same identity, right? So, and this is might be a very boring slide, but uh u very important for especially in the in the AI era um

because you need to AI reads your uh data in the right way. So you need it to understand um um which creative uh is a true winner and which is is not and why. And uh in order to make uh that tradable for AI engines specifically uh I I would love you to create for your uh ad campaigns. Um, I would recommend to make the stable ids and uh the stable ids should connect uh uh a bunch of very important stuff and and and uh depends on how many of the creatives you're going to test on how big is your team and and and your application you you might have like a lot of a lot of uh uh ids over there. But that's like a simple example that I added here. Um

uh and um it's the idea is simpler than the co the code on on in that better bucket uh uh makes it look. So the main point is um the stable ID. So the same ad should have the same identity uh in the file name the ad platform uh for example Tanzen or another measurement system and the team's report. So that um then add a few like useful labels. Uh I I raised an example a few examples here like the hook the one that we we we talked about the promise um the format for example uh is on the example the ratio uh and uh the version. So basically uh the promise by the promise I would say uh promise match uh instead of truthfulness
[clears throat]
um because and I would recommend you to do so um because um for example is the

is this real gameplay or like styles real gameplay or a [snorts] meta game or a fictional idea or a more like higher risk mismatch. So um if we put this um promise and if we differentiate that that promise inside the um creative ID uh it will it will make a lot way easier for an AI AI agent to read the data and to understand and to highlight the possible true winners uh for you and explain the reasons and you can take that reasons afterwards and apply it to your future uh creative production. That's super super interesting. Uh let me try to read this. Uh so the CR I guess create this is a ID right?
Yes the ID creative four to eight.

Yeah. Uh AI probably AI that I
mid journey and clink is that's that's the the LM models for for the AI creation. Uh HK I guess that's a structure that I'm following. So first there is a fail and then there is a fix.
Yeah. HK is literally like a hook. HK a hook. Yeah. And the fail fix. Uh it's like the one that I raised in in the in the pasle game.
Uhhuh. Promog gameplay. This is clear UGC ratio lens. Cool. No, this is new. This is this is great. Um
that's far away not the the the most complicated version. That's the that's the pretty like I would say an average one. Uh and uh but basically for the small teams it doesn't need to be even even that complicated. you can start with just uh those parameters, those variables that matters for you the most

and uh you can keep expanding it um along along the the way along the the number of the creatives you're going to produce.
Maybe a silly question, but like usually the ads name are like inlined like how how do you do it like in this format where it's like most parameters as you said? Uh yeah, I mean it's it's just it's just in one line. Usually it's just one line.
So you already parsed it and uh
Yeah. Yeah. Yeah. For sure. For sure.
Gotcha. Gotcha. I thought maybe there is another parameter within a parameter that I
No, no, no, no. I don't know. Yeah. I already already like just make it readable for for for
Got it. Got it. Got it. And even with the first one, it was hard like for me when I was uh helping with UA to enforce this within the team. So they follow the same structure like how do you how do you do it even when it's even more complicated when we're talking

about like UA teams
so so that's the case uh people human humans they don't read it they don't read it manually uh that's that's that's the AI job and the AI really read it very very precisely and very carefully and uh that's that's what it's created for basically and uh even if you have like a more complex ones uh AI could could read it uh very well.
Gotcha.
Yeah. And um so yeah, basically with uh clean labels, clean um clear quality checks, uh team can improve uh the whole process with AI, I would say in one month. And uh that is like u um a 30-day uh creative AI creative reset um that um I've got here on the slide. It it completely doesn't need to be 30-day only. um is just for uh um for the

implication for for the understanding and the way uh I see it and um for the more experienced marketers it could be completely different but for those who are just starting to um play with AI around the uh creatives and around the ad campaigns uh it could look like this and uh it's pretty simple and straightforward. Uh so week one is about uh clean measurement. So stabilize your ids uh clear your uh inapp events. Um uh simplify uh views and styles. I mean the data that that you where you saving the data and where it comes from. Is it legit or not? uh make sure that the metrics uh are accountable and uh make sure that you have your strongest tenant MMP partners in place uh to see the retention uh to to track the revenue.

Week two is uh about ideas. So that's the creative week. So you can uh uh play with the AI to um to to to try to start with just a few uh ideas or hypothesis or guesses and try to figure it out um like what ideas you you will test test first week three uses uh two gates. So first you need to understand does the ad attract people and the second you need to understand are those people uh if basically if ad attracts some people are those people relevant and valuable. And then week four is uh you just need to analyze fatigue and uh simply kill the misleading pinners. simply kill those campaigns that are producing fatigue and uh [clears throat] and u doesn't bring any economical economic effect. So watch

fatigue, remove false winners, improve promising ideas and uh write the next briefs based on the conclusions that you draw. So yeah and um in the week three uh the phrase the IPM times retention uh um is a concept. It's not not not the universal formula. Uh but it simply means that um an ad should pass both gates before receiving any major budgets. It should pass both the appeal and the the users retention users amount engagement.
Got it. Got it. And uh yeah, you can set it up all in uh AI.
Yeah. Yeah. It can be it can be it can be simply done in a spreadsheet for a small team. You can you can run this in a in a within a spreadsheet and just give an access give an AI an access to the spreadsheet and and you can also have a uh all your team members access

to that spreadsheet and you can see the clear weekly reviews and do the do the reviews and uh only a few tests within that 30 days. But um once you improve the whole the whole workflow and you set up the the the whole pipeline you can start automation and that's is of course more complex but uh it's it's the next step. So this is just something you could start from.
Got you.
So yeah just a simple starting sequence that teams can adapt uh to their to their goals. So, and um um basically um to finish with uh one uh um simple idea, I would say um AI doesn't remove the need for creative strategy at all. It just simply makes good strategy and measurement more important. And um the ultimate goal is not to automate

taste or produce the largest numbers of ads. It's useless if you can process them in the right way and apply. Um and the goal is to learn faster uh attract the right users and uh make better budget decisions. Uh I would say um yeah that's that's those those three parts are the most important one and AI can help you with all of them a lot if you apply it the right way.
Super. And I'm seeing this uh QR code. What what is a AI diagnostic playbook? What?
Yeah. Basically, I I just decided to put it here for those who are uh interested in um more interested would like to to know more about the AI in um user acquisition overall like not only in in the creatives uh uh or creative production or creative testings but in the uh UA overall. Uh this is like um a

gift for the podcast listeners. uh the special one uh we prepared that material with our team uh prior and it's not like public publicly available. So here you go.
Sweet. Uh very nice of you, Mark. Thanks. Thanks a lot for this. Uh and uh thanks a lot for your insights. Um yeah, if you like this episode, please give us a like. Leave your comments. Uh we'll ask Mark to reply to them. this will help our video grow.
Uh it will. Thank you Ramen for hosting me and uh it's was always a pleasure and it's always a pleasure to to collab with Dungeon and uh as long as we are not only uh partners on on on on the ad campaigns perspective from the the traffic perspective but from from um learning perspective as well that's

really great
and uh I personally saw like a few episodes of yours and I liked it a lot um recommend to all of the listeners to to check the other episodes. There are so many useful tools for marketers. So, check them out. Thumbs up, likes, subscriptions, and all the stuff. Thank you.
Thanks, Mark. All right. Until next one. Cheers.
Yeah. Bye-bye.