☠️​ Marketability tests are DEAD! Do this instead by Matej Lancaric

It's still not the exact CPI. And this difference between the conversions and CPIs can make a very big difference in terms of the real CPI that you will get afterwards in the soft launch because there is like 1.5 or X difference between the estimated CPI and real CPI. It's 400 a.m. and we're rolling the dice [music] drops knowledge made of golden eyes. Felix with ads making those coins rise. Jack up designs worlds chasing the sky with the two and a half gamers [music] the midnight crew talking UA adverts and game design too. Mate Felix Shaku bringing the insight. We're rocking [music] those vibes till the early day master eyes on the prize tracking data through the cyerspace [music] skies. Felix like a wizard in disguise lift us to the high. Two and a half gamers talking [music] smack. Slow hockey stick got your back. Ads are beautiful, they light the way. Click it fast, don't delay. Uh-huh. Uh-huh. Uh-huh.

Uh-huh. [music] Hello everybody. This is Mate Lancharic. Welcome to another solo episode. We know you love this and we love doing this as well. So today I'm going to talk about one important thing that's kind of bothering me for a while because back in the days I was talking about marketability test a lot. I wrote about marketability test and how it can save your company millions a long time ago and since then I've [sighs] been thinking about rewriting this article for million times. Now I actually decided to talk about it. So prepare a presentation then talking about how to think about it, what to do before and all the fun around it and then I have a diagram and a few other things I want to show you and also we're going to talk about how to actually and not only talk about way I'm going to actually show you how to set up the campaign again because

loved it in the previous previous video where I took you through the campaign creation in the MVP UA template. If you didn't see it just go and watch it. So here we go. Let's just let's just do it like this. Share. Perfect. So here's a quick presentation about how you shouldn't do it. Honestly, you know how I going to uh name the video? Just marketability tests are dead. Don't do it. Do this instead. So okay. So I'm going to talk about how to define a goal like actually how people thinking how we're thinking about the marketability tests like defining the goal concept testing CPI testing CTR testing and all all the fun. So how to define a goal and what actually like how to actually think about it? You need to be able to say like what are you what are you even testing? There are still people that think that market marketability test is going to show you exactly what's going to work and what is going to have the

lowest possible CPIs and it's just a bulletproof test. So if you do a marketability test then you're golden. Well, you're not marketability test here or any marketability test just kind of show you the way which is the the lowest possible evil to go. So, you're kind of picking and choosing from different art styles, themes, settings, characters, whatever else. And you need to be able to say if you're looking at CTRs, CPIs, like what exactly is the KPI you're looking at? And even before going going to the kind of whole market abilities, people should always do a market research. But again, some companies just go really deep and go like this is the audience which you should absolutely absolutely agree. you should know who are you going to going to create the game for obviously but then you know like kind of going going into the interests and you're asking the different questions and behaviors and whatever else and then then obviously I

mean there's nothing wrong about just creating another data point in your decision- making so you go and list three five different competitors back in the days it was basically just going into the Facebook all these insights and then just kind of try to seek pick the the overlap between the game that you are trying to create and then some competitors. But now we have all the different tools like Quantic Foundry Solston you can ask different questions with Pikfu but also if you don't have enough resources and you need something for free just you can go and check out JGPT Gemini and whatever else just to help you identify different data points and competitors and whatever else. So here we have uh so we are going to do an idle battler and I want to know how the market looks like and this idle battler plus collection whatever I just made it up obviously. So 5 minute exercise you can you can do it easily and then obviously you need to know as I mentioned who is the target audience and people are just going crazy about

personas and if you are listening to me for some time which I believe you are I've been very much against not being against but personas creating a persona is quite misleading. It was a thing backing like a long time ago when you know you have a persona, you create a messaging and whatever else and you target these these people and these personas. But in in this era and landscape where you have five best performing concepts, freezing families, saving the king Richard Gates, you have whatever else AI stuff and like who cares about the persona? You can have 10 personas. Create 10 different messagings and then after all, you know what's going to work? Freezing families. If you have a a casual game, if you have a a Forex or something else, gates are going to do quite well. If you have a social casino these days, you put idle game in the onboarding. There you go. Nobody cares about personas. You should. You shouldn't as well. Anyway,

enough with the renting. You know, we go for the for the CTR tests, blah blah blah. I'm going to talk about this a little bit later. It's very important to know what are like how to interpret the results because you run the CTR tests, you have five different themes or art styles of whatever and you're going to, you know, monitoring the CTR and usually there's like almost no winner because usually uh let's say I'm having five different concepts tested. every concept like 3% CTR or 3.5 or 3.2 and these things happen more often than they're not. So it's very important to have a clear winner. You should be aiming for 5% plus if you're doing this these tests obviously 5% plus. How did I came up did I come up with this this number is like from the 10 millions of tests that I've done before. And as soon as you have 5% and

more there is an interesting uh interesting winner and something is you can something you can work with 10 plus even better but don't really don't really try to dream about this these are the results for hyper casual games 10 plus what else so clear concepts what are you doing afterwards so if if there is a concept is 5% it doesn't mean this is the golden ticket for for you making millions this means this art style has potential potentially the best chance to get the lowest possible CPI. But CPI as you all know is only one part of the equation, right? Because then you have the LTV on the other hand. So you need to be very careful about basically anything around the marketability test because it can be very misleading. So what if there is basically no clear winner? You can do a lot of different things. You can go and iterate and you can just test everything that you you want characters different background all of it nothing nothing nothing matters

that much. What I think you should do instead of this marketability CTR anyway tests are the CPI testing because if you are already kind of running and thinking about this and you should definitely test at least something before you go and just start 6 months production. If you're running CTR tests, it can be misling. Why? Because you're getting clicks and you're measuring clicks. So, you're me you're basically showing your ads to people that are clicking on this. It's no other audience, just clickers. That's why you need CPIs because this is simulating at least the CP the real CPI flow video or static image to the store to the install whatever is wrapped inside the APK. At least this is the real flow because even though even use if you use landing page and and different conversions and I'm going to get to it and show you how how you can set it up, it's still not the exact CPI.

The problem with my and this is the same the setup is the same. You have the the four three four five different static games idle videos that can kind of simulate the the real gameplay and you optimize for for the installs period. That's it. But then you you go for the CPI, IPMs and all the all the different KPIs that actually matters in terms of the install flow and make decisions based on this. So like this is kind of like this is the test that you you want to you want to go for because it simulates the the whole flow the best. Okay. So let me just kind of switch this tab instead which is obviously the the same thing but it's just the flow. So again clarifying the questions which geos whatever arts is the concept even worth testing. If you kind of uh answer these questions great let's go the CTRs are the test that we already talked about but again here CPI test is real or placeholder APK nobody cares you need

the C the CPI to get uh you needed the the APK to get the real CPIs. Why? Because even if you go and and try to send people to your landing page and optimize for the button that you have on the landing page which says play now or download or whatever and you test you you get that as a conversion point and you send it back to Facebook and you optimize for f for this event. It's better obviously than the click campaigns, but it's still not the exact CPI and this difference between the conversions and CPIs can make a very big difference in terms of the real CPI that you will get afterwards in the soft launch. Also, there is a lot of different elements like store conversion and whatever else that has an impact on CPI. But if you do this, go for the CP, go for the APK and the CPI because then what I want to say there is like 1.5 or X difference between the estimated CPI and real CPI. So you will be very much surprised that oh my god I had 70 cents in the testing and then suddenly you

have two or 1.5. It's a very big difference, right? So watch out. So again let's let's get back to this. You don't have the APK fine. You go to landing page and set up a test again. And I actually made a I made the landing page for you. It's five minutes. Gemini, thank you very much. So, this is our landing page. If if you don't if you have resources, go to Geek Lab or Friends of the podcast. You can set it up as a fake store page and you don't really need to do anything like this. It just cost you money. If you don't have the money, this is the way to go. What you do is just upload like this. This can be a landing page. You go, you have a landing page, implement Facebook pixel, you track every click on play now and send that as a conversion back to Facebook. Simple as that. We'll create conversion campaign. Beautiful world. If you are already here, you know what you can do? Wrap this landing page and the play now button in an APK. There you go. I mean,

this it can be that simple. The APK doesn't really need to have anything else. No other tracking, nothing. It's just this is it. I mean if you go in this and then you can just have a Facebook SDK in there. Easy as that. So you go there and then simulate the CPI campaigns. So that's the minimum flow. You go prepare creatives. I have here three to six ads. Ideal images. Well, ideal videos, but you can go with images and then different v variations. You always go to the US. That's the most important part. No UK, no other stuff that I'm talking about in the soft launch video which you can check out here or here whatever wherever the soft launch video you go to US. That's the market test. It's the the that's your target audience. Nothing else matters. I have here like 5 to 10K which is usually a long monthly test for for like market testing. But I think you can do better. You can go with 1,000 2,000 depends on the actual C like the CPI that you can get. You need

something like 500 installs in here and go with Facebook again. You can do CPI campaign. But if you go with the wrapped APK and you can actually send that event back to Facebook even that's going to be kind of simulation of the appevent optimized campaign which is way closer to purchase oriented campaigns which you will run in the future. So if you can do that that's even better and I this is kind of getting a little bit more complicated than I I want to explain but still CPI first if you're sending that sign up event that's even better. So results we have different teams different results. Look we are looking at the CPIs and just going for the the signups and whatever. Also very important comment from highend. Every time you you test anything a low poly poly art style always wins. Thank you very much hyper casual era. This is such a broad appeal. It's going to always have the lowest CPI possible. And then it's up to you if you want to work with

it or not. But still I I like 10 out of 10 tests in the past showed exactly same result. So again we're kind of trying to evaluate the results are good bad borderline whatever else. If it's good enough you go and prototype and go to soft launch or good enough for you. If it's really bad just kill it with fire right away. If it's every if everything is the same and all the all all the themes are showing the same kind of go to your team and look at your capabilities. If you're good in 3D, go for 3D. If you have only 2D artists, go for 2D. Kind of weigh in your own team capabilities and then try to combine these two data points with CPIs. Don't really go like, hey, but this is 10% better. So we need to go this because this is the this is the the result. Try to use your your raw your brain and just nonsense or just you know someone said or mate said that this is

we should definitely go with this. Try to interpret your own data. But anyway the also very important part is if you're doing a test and you're going in softline 6 9 10 months the data might not be that accurate anymore anymore. So think about that as well. So look that here is actually a short version of the of that uh of the diagrams. You can you can go and just check how [snorts] it works. But I actually created four different four different images and static images for for our test. So let's go to Facebook and let's try to actually create a campaign. So we go here. Whoops. We go here. Create a campaign. We can do leads or sales because we are doing a conversion. If there is no CPI, if there's CPI and APK, it's it's app install campaign that we've done before. If it's conversion only, then we go here leads. Thank you very much. No AI. Go to this. We have everything here is the

budget. Let's do 150 or 250 in this case just from the start. Depends on your kind of money in your bank account. Let's say we do 250. We want to go to conversions on the website. Maximize number of conversion. Of course, value of conversion doesn't matter. Let's say this is my pixel tracking obviously because I already set it up for you. So, let's say it's complete registration. And then everything is standard daily budget. We have everything in Slovakia. Not really. Thank you very much. United States. United States. It is. And we go down and not suggestions. Yes, this is what we want. Okay. Fine. We can go broad or we can just go for my case here I'm just going to go broad and all genders creative is going to going to go for the whole target audience but here this is very important we go only mobile no desktop and ideally either Android or

iOS I go for Android usually S4 but it's up to you can go you can do iOS as well and then 12 plus or maybe even even the latest OS version so we we kind of skip the the low we skip the low-end devices. Okay, so here we go. So let's say this low poly and then we actually go to adset level but so because we want to control we want to control the traffic for all four adsets. So we had 250 so let's say 75 well let's say we go for 50 per day which is going to be fine. So then we go okay is my personal nobody cares fine at least this is your is your URL which is now mine and I don't really want carousel image I want single image thank you much that's cool get rid of all this AI stuff okay so we

have image I already have this image so I have low poly there so fine thank you very much this is it this is it and this is it. Okay, are you ready? Then play game. Did this. Okay, so here we have the the first the first outset. We don't really want any of this here. Done. Okay, thank you. So we have low poly then. Let just create a marketability test recording to five gamers. Very good. Okay, so we have low poly. Then we create another one. Okay, we actually create three other three others because then we have let's say the styiz whatever then you go here change the creative see the landing page stays the same I mean whatever doesn't matter just for the sake of testing this thank you okay so then here then let's say this is

we change the creative here as well everything has the $550 per day budget so Don't forget about this. So, okay, this is the variant again. Thank you. Okay, so we have this. Okay, let me just change the text to two. And then the last, let's say it's variant three. I think you can do a little bit better uh naming convention here so you know what it actually is when you look at it at first site. This is just for testing purposes. And we have basically all of this done and set up. So we have marketability test one, two, three, four different ad groups or adsets with each of these their own budget. So we have equal chance of sending traffic to all four. This is what we run for let's say 3 days and then I mean I would look at it even tomorrow just to see like how it actually works and then increase budget or just stay the same because here we might get enough data for what we want

in let's say 4 days. we might not and we need to increase the budget. So then you hit publish and then thank you very much. See you in a few days. So this is basically how I would set up the whole kind of marketability test. So don't forget this setup was the conversion which is the landing page and force the CTRs and whatever else CPI is better. So I can go here and then just create this. So again market marketability tests are dead. Do this instead. Thank you very much for listening guys. If you have any questions, please comment under the video and join the Slack channel and see you next time. Thank you very much. Bye. Green.