How AI Is Transforming User Acquisition

[music] Welcome to the Navit Gaming Podcast, where it is our mission to explore the business and future of video games. We bring together the industry's brightest builders, investors, and thinkers to keep a pulse on current events, dissect emerging trends and games, share lessons learned, and have a great time. This podcast is also part of Navik's growing ecosystem which ranges from free and premium research to consulting and advisory services. For more information, visit www.navic.co. This episode is brought to you by metal.tv. Built by gamers for gamers and used by over 15 million gamers per month. Metal is the fastest way to clip, save, and share gameplay on PC. Millions of clips get shared every week, turning your game's moments into word of mouth at internet scale. With Discovery being one of the biggest challenges in the games

industry today, Metal is building a platform where publishers, studios, marketers, or anyone looking to reach gamers can find the right players across multiple genres and geographies. For more information, head to metal.tv or check out the link in the show notes. This episode is brought to you by Heroic Labs. Picture this. You're a diehard fan who paid the VIP package to play a preview mode of your favorite game. And once you finish, there's a full progress carryover from preview straight into a live launch. No wipe, no restart, just pick up where the preview left off. That's what MadFinger, a mobile game studio that expanded onto PC, built for Greyzone Warfare, powered by Heroic Labs. And the result on launch day every core KPI increased concurrent users daily and monthly active users retention and session length all increased. Heroic Labs worked closely with Madfinger's

engineering team under a tight deadline to shift the backend that made that carryover possible. Want the full story behind how Heroic Labs helped Madfinger turn a risky preview mode into their biggest launch yet? Find the Greyzone case study on heroiclabs blog by going to heroiclabs.com or simply check out the link in the show notes. Now, let's jump into today's episode. Hello everyone. I'm your host Devin Becker and today I'm delighted to be joined by Amit Carmy, co-founder and CEO of Set. Set optimizes user acquisition with AI agents, including creating and managing UA creatives for games and apps. And today we're going to explore the topic of using AI for UA creatives. So just to start things off, could you give a quick overview of the origins of set?
Yes. Uh first of all, thank you for hosting me. It's great to be here. Yes. So the origin of set started with myself and Yonyi, my co-founder. We have a big passion for tech and huge passion for games. H the real dream was to open our

own studio. H we didn't h really aim for mobile, but if you would ask me what is our biggest dream is to have like a PC or like an open world MMO RPG kind of game. But we understood that we don't have a real advantage there as founders. So we took the B2B approach and uh we came to one of the I don't know biggest uh spaces of issues which is user acquisition for mobile games and basically we have a goal to optimize that via owning the entire workflow of anything besides the media buying. So including data analysis of the creatives campaigns, creative idea generation, the gener the generation itself of the creatives including like high diversity of new concept and maintaining the high quality of that deployment to the campaigns and managing the campaign. So the goal is to really take control of the entire loop with just to optimize the performance like in order to improve ROS we believe that agents can help at each step of the process to reach the

final goal. So, uh 70 people now, most of them based in Tel Aiv raised approximately $60 million from great investors and great partners. Yeah, we have a lot of goals. We're only halfway through the current vision in terms of product.
Nice. Well, I'm sure we'll get to talking about some of the other half there. Uh but what does a set actually do uh for a UA team on like a day-to-day basis? What does it look like?
So, imagine that you're in a UA team. So either usually we work with the UA managers or the creative strategist or creative managers depends on the company and the structure. Usually we don't work with the marketing artists or motion designers. H what is happening? Basically we have a few agentic system uh one that is kind of the brain of the generation that analyzes a data from your competitors your campaigns the bells analytics and h general an like continued analysis of your target

audience. So you as a UA manager/ as a creative strategist you have a way to impact the brain of what our system is going to create whether with specific briefs or most likely defining the strategy and kind of directions to guide the agents to go forward to and then we have the part of the generation itself which we have our own internal tools that I would say in terms of vagentic harness and capabilities I believe that probably the best tools in the world for generation of video and playables. uh we have our own internal team that are working with the tools in order to complete what AI is not doing well uh well enough and then this is how we train the model of to optimize the generation and at the end as a UI manager or a creative strategist you get from us pretty high scale of video ads and playable ads at the moment high diversity meaning we're only focusing on new concepts and to diversify as much as possible because we believe that what is impacting the campaigns are winners that

are new concept and not necessarily iteration. We also do iteration but it's the not main thing but you're getting a lot of fresh new concept of video playables in considerably very high pace of course lower prices than the market and in the future we will include more UA u automation. We just started with first design partners a month ago our own platform that we also give access to uh for you to generate not only us h but yeah a lot of uh things to come but at the moment is think about it kind of a datadriven creatives uh flow for your team.
Nice. Well, appreciate the extra detail on that there since there's definitely a lot to that. Um one of your big things about your your pitch is uh beating creative fatigue. So how would you define that and how fast do you think that's kind currently hitting? you talked about like looking at new generation versus iterative and things like that. Does that factor into that creative fatigue?
Yeah. So first of all you know it's one of the biggest question and you know the

real answer about how to find the winner and how the deep learning algorithm of the networks you know everything from meta uplavin Google and etc is working. No one have this issue. It's a bigger uh let's call it area of research that is currently happening in deep learning that no one knows how the systems that are so deep learned are making decisions on you know milliseconds and based of billions of billions of every second signals like data signals. So there is no real answer for that but I would say that you usually h identify the fatigue itself. you're seeing like either topline metrics like upper funnel or the rush itself getting hit in your campaigns. the usual let's call it the medicine for that is starting to iterate on the concept that is already working but it's kind of a race to the bottom because you might in a short term impact the performance of the campaign in a good way like maybe you lift it up back to your original KPIs but in the mid/l long

term uh which a tough creative fatigue from the data we gather and the patterns we work with can like a tough one that is kind of a loophole can be I would say month and a half or maybe two months without a real really really real new concept winner. And what usually revive the campaign is like a strong new concept winner. It can be both video and playable. You know if you have both of them is great but even one of them should be enough. And today because of the targeting capabilities of the networks is down. So the creative fatigue is actually faster than h like significantly faster than the past. I would say that from few months uh I don't I heard partners that 5 years ago had the same winner for like 6 months today at high scale depends on the network it can be like few weeks h so you consistently need to fresh your creatives and to find a new winner that engage with your uh with your target audience and the users themselves. So I

would say the way to beat it is consistently feed uh the algorithm with new concept of video playables h that of course match somehow your target audience uh like uh let's call it I don't know passion or mids h but you never know what is going to stick h so you just need to try to try to try and this is kind of a consistent fight I would say that every month month and a half on average you will face a real fatigue and this is like how you need to tackle it like every month, month and a half to find a new winner and for that you need hundreds of hundreds of new concept if you're like in a very high scale.
Yeah, it definitely sounds like that's a big part of like using this technology right is being able to kind of keep up with that scale and if the fatigue gets shorter as you're saying it's trending shorter that becomes even more difficult for humans to keep up with. Um, in general though, where do you think AI creative is still kind of falling short

or what what can't it make or do like where are you seeing kind of the edge at the moment? So, I would say that in general LLM has a like pretty big big issues that are not actually getting better. You have a lot a lot of stuff that LM is getting better because inherently how it's being trained. uh for example code generation like we see it in every new model code generation is like insane and it's pretty reasonable to understand why because you just see you can just imagine the the number of the data you have on code generation is like insane it's a community that has been open sourced you know h inherently in the user base so much so you have huge huge huge amount of data and this is why LLM is getting better I would say that where AI is currently an LLM them are getting let's call it more stuck and not improving as fast as h as the other stuff is everything around special intelligence and understanding

2D and 3D environment of course 3D is hardel everything there is a big issue like from the asset generation itself but mostly the compositions and to fine-tune and make it perfect like pixel perfect is very very very far away so I would say 3D is still a huge issue that a lot of the big tech companies are trying to solve And a lot of research labs are trying to you know train the best model based on actually games a lot of open world like 3D games but it's not really getting significantly better and this is like where we do most of the hness of the agents and most of our manual team that is working with our system this is where it spends the most time on. So I would say assets composition of 3D scenes h gameplay in general is not very high quality. it's getting a lot of issues. So, it still needs a lot of work around everything uh in these areas everything around innovation and novelity of even ideas and I I'm sure

that you know it as well. It's not it's not a game issue. LLMs are it's a statistical model. Again, inherently it's not something that going to be innovative. It's statistic. So you see it for example with idea generation if you're taking it to the creative world you're just getting more of the same kind of and to to extract real novelty from LLM today it's it's very very hard we actually have we have a pretty big research in like over 10 people like PhDs that are working of two of these issues one is the generation capabilities but the other one is the ideation actually itself which is I didn't know how big of a problem it's going to be like to extract innovation and novelty from LLMs and yeah it's like an open research topic and a lot of stuff that we are working on in our internal lab but yeah I would say innovation special intelligence h like under that everything around the 3D world is still

not good enough and you know at the end you want the users that watch the ads and the creatives they need such a high quality h everything should be perfect like unless it's a video you know you have this very cool intro cinematic stuff, but at the end you you always go to the game plays like usually and the gameplay generation, you know, it's so high quality. It's so high quality. It's so on details. Everything is pixel perfect. And this is sorry for my luggage, but this is where AI usually the deal. It just, you know, it's not 100%. I would say it's like between 60 to 80% depends on the game. And the real hard part is the 20 or the last 30%. And this part is not really getting better and better and better.
Well, that makes sense. It I I kind of wonder then, so if you're talking about wanting to be pixel perfect and uh needing to kind of work on these these rough areas, do you have a process then where you're kind of like throwing away videos or or checking them like with some automation? Or is it just more manual like this doesn't work, throw it out, do it again, that sort of process?

Like how are you sort of doing the quality checking of these videos for that perfection? Man, we we have a manual teams that are doing that. Areas that AI is not good enough, we have people that are doing that because think about it at the end, our partners, they need results, right? They don't care if we have a huge lab of AI, a huge number of people. We in this area we have both of the like both of them like we have big number of people that are validating the process all of the time and teaching the model and we have of course a lot of AI initiatives and products internally. uh but we have teams that are validating everything and we are kind of trying to I don't know if this is a real world but world but agentifying them over the time for example we have a creative producers team that are reviewing every playable in the playable specific process we have every playable that goes out to the of the system you have a creative producer that watches plays it give its feedback to the system so we are trying with all of the data we accumulate from the a

creative producers. We are building our own creative producer agent h and then you have the evils of how to measure quality which is again it's a huge huge huge huge research topic like I don't know how to measure quality you know it's such a subjective area it's basically impossible so you're trying to do evils for that for agents that are trying to to kind of hit high quality but it's a we're still in the people phase and I would say really that I believe that we're probably I don't know the top three advanced companies in creative marketing AI space and man we still have a lot of people and AI is very far from uh this uh I don't know this trend or hype you see like AI is going to replace us all and no no it's it's still not there like you still have huge amount of work
well on that topic then uh you mentioned playables a few times uh I got to imagine those are a bit more difficult

to create like consist consistently than the video. What stuff are you seeing that's working well with creating playables and what's not? Especially as playables are kind of becoming I think a little bit more in demand than just straight videos. Uh yes. So it is a bit harder. It's still you know significantly if you are a studio and you will work with agency like the cost per level and the cost per video is like you have a huge difference. So P level is still very costly and the demand for that in terms of the network side is like it's insane at this moment. the number is they require you it's it's really insane and if you work let's call it with traditional agencies and you pay like three to four to 7k USDs for one new concept and you need 20 new concept a month and uh so you're reaching numbers that you can't really support I would say that the first shot generation of playable ads is like getting better because it's mostly around the code generation it's of course it's depends on the complexity

of the game and the environment and the gameplay and the assets and what stuff you have and what stuff you need AI to complement. But considering you have huge amount of assets and you know how to structure the folders correctly, it's a lot of small fine tunes in tons of different areas. But considering you do all that, I would say the first shot generation of level is nice to reach to the 60 70%. The other part, you know, it's a nightmare. you still even maybe you experienced that with your own CL code or I don't know what specifically use but sometimes you can tell an agent something so easy to say like let's take make the text larger and you even you know let's say you have a a plead you have a text hello and you want to make it a bit larger can even sometimes say make it 20% larger like you give it the exact number and it just the output is like I don't I don't I don't know what the he's doing. So, you control the input process but not

the output process. So, the 70% the 60 to 70% it's pretty good mostly for 2D games. 3D still far away. uh the last 30% is very very very tough and you can get to a lot of loopholes of uh like input output misunderstanding and agent hallucination that it's getting better again but and again to be honest it's like I'm still shocked in a bad way what mistakes agents are still making even for me personally it's like inventing stuff so if you try to build a system that is generating player bells that are hitting the campaigns like this is the real goal not to generate level to find new concept that are high quality that move the needle in the campaign. It's far from being fully automated and the easy part is the beginning and the fine tune is the hard part is like the real really real hard part and everything around you know asset management even the organization of the

code so because you need to control a lot about the size of the file because it need to be under 5 megabyte and again if you have huge envir 3D environments you need to fine-tune every assets so yeah so I would say the beginning is getting easier and easier, but it's reaching to a cap that it haven't been able to break yet.
Well, hopefully the process you're going through helps improve that just because I imagine you're you're working that out, figuring that stuff out by actually having to do it on a day-to-day basis. Uh I'm kind of curious on something what you mentioned around the the having all the assets. Are the assets provided by the game developers? Are you generating your own assets? Like how does the whole asset side of that? Does playables obviously need assets in a different way than video does as you as you touched on a little bit.
So both of them of course need we need assets. The question is like you said like what is the quality of the assets and then where do we get it from? H you have some IP kind of games that we work with for example. So you can generate stuff with AI. So putting that aside. So

they provide everything and if we need something specifically we can buy it us as a company. we will confirm it with the customer and this is the process with AP usually we get a big let's call it a chunk of assets from the customers but then it's mostly h like our own models that are generating assets in 2D it's like again way easier 3D it's still complex and if you will do like let's call it idle arcade all of the player ads and you know the complex 3D ones with a lot of progression mechanism and a lot of animations, a lot of different assets, a lot of things that are happening at the same time. Everything 3D and beautiful and on point. This is the area where AI is still not there. So, we have two options as said one, we get the assets from you. The second one, we we have people that are actually creating them. And again, I'm

speaking about the perfect perfect stuff. We utilize AI in the most customized way possible for each of our customers and games. H but sometimes again you reach to a limit that you must have this final h or a lot of not only final but like a lot of manual work. So we support it with manual theme and like getting the assets from the customers like animation and stuff like that is still again far far far away from being solved mostly like in 3D characters to being solved with AI or mixo like general marketplaces for animations is it's good but imagine that it's so easy today like everyone is saying so easy to create ads right and the market is kind of flooded with tons of numbers of games and so the competition of each user is getting harder and if you will do an ad that is just okay in terms of quality because it's easy to create. So it's not going to last because your competitors are

understanding that you must h getting the bar of quality and power bar of concept and ideations and the level design way way way way higher and the only way to do it today is with real processes and real validation of people that are still in you know subjective judgment of what is good quality and what is not because agents and general LLMs are still not Well, then I guess that brings up the next question then in the sense of like why it makes sense for all the processes you have being like I don't want to say boutique because you're not boutique necessarily but like being a little more hands-on a little less of a giant enterprise sort of thing right now compared to Meta, Google and App1 that all ship their own AI creative tools now. So I I kind of figure where where do third parties fit in? And it sounds like some of it's having that that sort of human supervision and sort of client by client handling. Like what do you see in general in terms of that the overall market growing with you've got the the

super easy ones shoved into every ad product and then you've got the ones that are like hey we understand games we understand how to do all this. It's a good question. I would say and again this is a you know I don't have the truth right so I can I believe that we will win this competition based on few assumptions that I have but of course I I mostly hope that and we're working towards differentiate ourselves but I would say that my assumption is that the networks themselves h the meta the Google of the world the uploving in the world it's very very very uh smart from them to generate their own generation tool right I would say that I know how big of a challenge it is to reach to the really high quality of the ads and I think for them and we're in deep touch with them. So I also know this is the reality they are focusing on the 90% of the studios or the apps that are kind of in charge of the 10% of the revenue. They want to lower the bar for

entrance for their network. So they want to come to a studio, very small one, an indie one maybe that has a new game or an app, you know, they're just outside and say, "Hey, open a campaign with us. Let's do a quick AI interactive playable, not not complex, not 3D. You don't need this stuff for the beginning." So I would say they're lowering the threshold for interest. And this is where AI is is kind of completed. Let's say it like that. It's easier to ship tools that are AI generation generated videos or playable ads that are just enabling you to be part of the game. But to win the game or to to uh to service the 10% of the studios that are in charge of the 90% of the revenue that the competition between them, you know, is fierce. There you need the real tools, the real supervision, the real validation, the high high quality. they don't uh you know they they don't compromise anything must be perfect and this is where I

believe the networks won't h focus on because it's too big of a focus it's I know again it's a huge technical issue to solve it's not something that you put an engineer on or 20 engineers and you solve it you need to put the male engineers you need to put AI researchers you it's it's just a lot of stuff and it's not in my opinion the main b the biggest business value they can get because they can get more value from more uh studios/ companies that are now advertising on AppLoving Meta more easily because the interest is uh bar is kind of low. So this is my kind of prediction/ assumption there.
Well, nice. Well, well, related to that then, like if everyone's just generating thousands of different concepts, like you all these different companies, more other companies such as yourself that are doing it specifically for games, if everyone's generating these these thousands and thousands of ads and concepts all the time, does that end up just kind of homogenizing or making everything look the same, especially since AI is good at kind of average but not great at like pushing past that? Uh

because you mentioned a little bit of that earlier, being kind of problematic with being innovative uh and everything. I imagine especially if you're trying to train it off of what's trending or whatever that everyone's doing that at the same time and everything kind of starts to look the same like is that been a problem for you guys are you how are you kind of working around that
you know it's a it's a funny question because I think when we started set when we did the discovery process I think I assume maybe four years ago so we spoke with guy before AI you know before diffusion models before LLMs and I don't remember if it's roia or other company just you know came and told us h look so we to see the same ads with everyone like with set let's take it four or five years from now. So it was funny and what we entered then which is still relevant is think about it as the world of options of concepts. Think about it statistically. It's like a huge galaxy with tons of different parameters and areas to explore that

you I just don't believe that you will see the same stuff like when you have a concept that is working everyone is trying to utilize that but you see that not everyone is making it successful. So you will take match tree and match three you will do the save the king and for you it's working for them it's not working and you need to tweak a lot of stuff and under save the king you have billions of different options h so I would say that the market will probably be more flooded with a lot of new ideas I don't see a situation where the the bank of ideas ends because the market is evolving everything is moving it's not uh it's not static. It's always you have a new thing because you think out of the box. And I believe that if LLM today and content generation for games or high quality ads for apps was commoditized completely and LLM were still not innovative as is. So I I I would have this

I would believe that we will see this kind of issue in the market because everyone will do thousands of ads and they will be the same because the LM comes up with the idea and etc and etc and certainly will get into this never ending loop. H but uh fortunately uh we're still not there.
Yeah. I mean, I I just imagine maybe it's just very very fast cycles now, right? Where it's like because you can create the playables much faster, you have the ability to fast follow very successful ones, but then that probably doesn't have the same impact when you're fast following it and then you maybe drop it because it doesn't doesn't have the same impact you were hoping for. But it's funny think you back to like, you know, if you watch uh like YouTubes of old stuff from, you know, previous decades for nostalgia or whatever, it has all the commercials in it. You see back then commercials were fairly homogenized in trends very because it takes a long time to make them. People are kind of looking at other people's things. There's very limited uh broadcast channels and so you kind of end up with that. Whereas now we're in an environment where maybe you can try you know thousands of things very quickly and see what works. Uh but that

seems like you may have trouble then with the data part of it, right? Like if you're fast following, is it is it because the create like if the creative doesn't work, is it because the creative didn't work or is it because that trend's over? Like how are you kind of looking at like measurements for success in such a hyper rapid environment with lots of contextual clues that may differ especially like oh I'm having to be targeting the same thing but a different country that might have a different demographic like how are you kind of managing and understanding the metrics that you're getting back.
Cool. So attribution for creatives it's a unsolved issue like to know exactly you know what is happening there and you're right on point. It's a I would lie if I say that it's so obvious to see whether something is good or bad or the causality of that I would say we we try with our partners to look at the more high level stuff like share of spend and general was and see the campaigns and again even on that there are too many uncontrolled parameters that can affect that. So I to be honest it's still a big

issue of the attribution of creatives and even more to understand the why behind each thing like even if you have a new winner you can't really know why. You can assume a lot of different thesises and yet then you can exploit each thesis until it's proven right or wrong. And the only way to prove it right or wrong is if you if it's generated a new winner and then even based on that you can have different type of assumptions. But yeah, it's like a very very very big issues that in my opinion just going to be bigger and bigger because it will be easier to create. You will have more again competitors in the market in terms of apps and games. the market will be flooded with more ads. Meaning the fatigue will be higher because the number of users that are watching the ads is not changing as fast as AI. So it will be harder for you to to find winner. It will be faster that you will have fatigue. So you will need new winners and it will be even harder to you to exploit physicises because your

the only way to have a successful mechanism here to find winners is by system like technical agent uh agentic is my addition but tech oriented machine a techdriven sorry machine with very very strong algorithmic data uh feedback loop and optimization models very strong generation capabilities and understanding capabilities And uh yeah, it's you can do it by humans. Just can
just got to find that that blend there, right? Where humans are steering it, but then the AI is doing that the hard detailed work of combing through everything just cuz yeah, as you said, it starts to become too much data for humans to process. You just have to kind of steer the boat at that point. Um, one of the one of the things that that does come out I think as a result besides just all the flood of data is uh the sort of ease of making ads for gameplay that doesn't actually match the game. Partly because like you know uh previously like making actual fake gameplay would take a lot of effort uh

or you know like you you would usually use actual gameplay a lot of the time because it's easier to record and represent that. But now it's trivial to make really fake easy, you know, copies of other gameplay, things like that. Whether that be the playables or video ads. Do you think that's exacerbating some of the problem of uh creating a lot of fake I mean, do you do you guys even engage in that yourself? I mean, you don't have to say if it's if it's something you don't get into, but uh is that something that you're you're finding people are wanting to do with AI a lot? Is it helping or is it kind of misdirecting users? Like how do you see that whole
thing? because it's kind of a weird series of things that we have in this this advertising industry that I don't think typically you see outside of mobile games.
Yes. So, it's a the first of all, it's it's a must. Uh fortunately, unfortunately, it's a must to have different types of game plays that are unfortunately not necessarily represent your own real gameplay in order to attract the users that you really need. So, this is the funny part here. you're

displaying something that you don't really have or you can have as a miniame as a you know your main game play and then you try to find the users that say hey this is what I want to see ah this is what I got but it still matches my needs that the relaxation needs the competition like the user motivation core psychological attraction for for the ad itself h it's a must today we see it with again I don't have any any studio that we currently walk in besides the IP that we're not doing misleading stuff and everyone is trying to to take it to the edge h with AI it's it's easier today than in the past to create misleading ads but AI is not really helping there a lot if you're going to the complex misleading ads because h you need new assets you need new mechanics you need a lot of usually 3D stuff and again this is what we said uh like 10 or 15 minutes ago. AI is still not good with that. And then

before that, you need the innovative idea and AI is shitty on that as well. So when you have the idea and you have the assets to generate the stuff to code them to fasten the process, it is easier, but it's still the it's still a bit hard with AI compared to like usual gameplay when you have everything you need to do it in a good way. But yeah, it's a must. We are of course very like I'm not proud to say that you know we are doing that but we are doing that because at the end you know the system the agents understand the target audience and based on that you generate different type of mechanics so it's a must today again I just the trend that I see with all of our partners like 100% of them besides sorry already the IP and there are not a lot of that is doubling down on misle misleading as much as possible you even see it now because a lot of the Chinese studios or the more eastern let's call it ones I

don't have let's call it a shame in doing misleading stuff some of the time they're taking it to a more ethically problematic level that this stuff we don't do as a company but the misleading stuff is something that everyone is taking to the edge and now we see the western studios are trying to do more and then you see the IP games are thinking about more like how can we approve to do something misleading with the IP. So everything is kind of gathered around the misleading part of the ads and I just see triple quadruple down on that. It's like getting even insane and inser eventually have to hit a point of just oversaturation that it goes back the other direction as a lot of trends do, right? So, I mean it's funny because like you know as we trended into playables, you're actually having to make more of that an actual game, right? You with the misleading stuff you you maybe were before, oh, let's kind of fake the concept or maybe we have this somewhere in the game, maybe we don't.

But now you actually like if you're making playables of it, you're like closer and closer to saying, well, maybe we should just actually make that uh if it works, right? Because we already have like something somewhat playable as at least a prototype. So, it's interesting to see it like go that direction. as you said, doubling tripling down uh is uh I mean, I guess a lot more feasible. And if if it's trending westward, I can only imagine that that is going to get worse for quite a while. I mean, assuming you see that as a worst thing, I would generally see it as misleading and probably not great. Uh but you it's a market reality right now that we kind of have to deal with. uh on the other side of the market reality when it comes to uh the the game companies. Um I'm kind of curious what you think how this is impacting the creative teams like inside companies or even outside companies that are like consulted or worked with. Uh like what happens to those creative teams? Um like are UA artists or other creatives in the the UA teams replaced, repositioned? Are they kind of managing AI projects? Like where how is the impact that you've seen so far on this process? So I do see

the smaller studios or the new ones are really enhancing AI in a beautiful way. Uh so I would say the artists are kind of repositioned. A motion designer in a 20 people studio is not the same as a motion designer as a 200 people studio. So I would start maybe with that and play developers is kind of the same on the more you know complex top animations and uh maybe 3D artist I see them also mostly 3D artists are not from what I saw not getting replaced uh but them are like way way way way faster and better using they're using a lot of AI animation is kind of the same and even animation is like less good like way less good from what I saw. So in this like compared to 3D generation which is considerably also hard I would say that the general thing is that you see a lot of I speak a lot with the high level you

know the sea levels management a lot of them want to decrease to reduce the number of people you know to be more efficient no one has something against the creative team but in reality you don't see that you see some some that are doing but with some of them that you know you have operations of hundreds of people in creative team hundreds of hundreds I didn't see a huge deduction there like in the last three years. I now hear studio because you need to generate more now because you have more competition of games. You just see people are hiring more and more. It's kind of what I think about code generation today and what Yonyi my co-founder and I discussing a lot. You're saying okay now you're way more efficient. Let's be more efficient as a company. Why do we have we have a lot we have 30 AI engineers? because you want to do more instead of uh making everything less efficient because the competition is harder. So if you will be more efficient, you might lose the game you're playing. So I think it's on steroids in game. So I would say the creative team probably

are doing pretty tough repositioning. You you you have a mental state that is a bit more frightened, let's say, with some of the students. some some really not uh but unfortunately I don't see anything you know either here or either here it's still it's still kind of moving I I I feel like the market is still trying to understand really the impact the ROI of AI and how you really use that and where are the limitation and you have you know the people that are I don't know super uh tough and say yeah with one I will do it with one people and I don't need anyone else and AI is the best But then you see this one people is doing one concept a month or two concept a month and then you say what the are we doing here? H so it's still not there. You have a lot of rumors. You have a lot of uh let's call it the thought of it but nothing practical that you say creative team is

now 60 70 80% h smaller and you're generating at least the same number of ads that you used to. I mean, it makes sense your your point about if you're suddenly able to do a whole lot more, so is your competition. So, you kind of have to do a whole lot more. It becomes kind of an arms race situation where it's like, yeah, that's that's great that you could be more efficient, but you kind of have to now since everyone else is uh and also UA is also at a tougher spot because of just all the situations that happen with privacy and and other factors in UA. it is even more important that you do a good job rather than just a job, you know, scaled higher uh higher quantity and things like that. So, it does make sense that like the quality would would be the focus then uh to make sure that that quantity is paying off and that like you're spending money to do these, right? Either way. And so, you're either making money off of it or you're just wasting a lot of time. So, it does make sense to like at least have a good at least have good supervision there in

terms of making sure people are are making sure this stuff's working however they can with whatever the metrics you get.
Um, and until like you know recently you've had more like data driven creative like video and playables and stuff generation agents but from my understanding you're you're sort of shifting also into like management agents side of things as well. How is that going to work in terms of like not just generating content but also doing more of the management of the the campaigns or like what does it look like on the agentic side? Yeah. So I would say you have like three steps that are still missing in our value proposition. I would say like in our let's call it product capabilities. It's not value proposition because the real value proposition is let's find winners faster and cheaper. So the time pan winner the cost per winner is probably probably in the long term the best way to uh to to measure a set h to complete the winner search loop how we call it internally we understand that the pace of the data that we get and the

signals must be faster. So we are doing this progress towards the deployment of the creative which you don't need us basically to do that because you can build it with the the own network MCPS but you need to do it in an intelligent way it's not just automation you need agentic workflow that really understanding the campaigns the best performance performed creative at the moment where to do the attachment of the video and the playable what happens in the different campaigns where do you see fatigue and it's not the time to replace and stuff like that We're going to handle the deployment part of the creative and the management of the campaigns that will include I believe everything besides maybe at the beginning you know budgeting and stuff like that that we will make it also supervised and also in way more controlled by our partners and with their confirmation to probably every step of the way. uh but we must must must must must have it as product capabilities as a company because first

of all it's it's enhanced the value proposition also we're getting in part of the workflow so we are not measured only by creative generation or creative ideas and it will enable us at the end to build the algorithm that we already started to build that from the data that is now way faster we will be able to build this algorithmic layer that really generate the best ideas and best creative that at the end you you reach this KPI of cost per winner and time per winner way way way faster. So it's a must for us. We we are doubling down on that of course internally and uh it's taking a not short amount of time because you need to do it in an intelligent way again not only an automation but we will get there. We also we we have the platform until now we are working with our customers only via Slack. Now we have and with kind of agentic conversation and with very very strong and hightouch customer success team but now we're doing the transition towards platform that you really speak with agents you

feed back the levels the agents speak with you and then fix them you're able to generate your own stuff and the next step which is going to be by the end of the year in analistic way is to deploy the creatives but also manage the campaigns from our platform. Will it do you think they'll eventually get to the point where essentially you are able to kind of give it a budget and say just run a whole like campaign that you do and you know try optimize it on the fly as you see fit but based off the metrics and things like that like fully hands-off in the future or is it more like there'll always be a certain amount of needing humans in the loop to manage those things to make sure that it's not just going off the rails. I imagine like obviously you can constrain budgets and things like that but do you think you'll you get to that point even within the next year or so?
Short answer is yes but you will always have some kinds of controllability and supervision but I believe so. I kind of know so but I don't really know. So I I strongly let's call it believe so and uh it will have to be with supervision because again I still don't know

how the big like you can do it with rules and more deterministic flows but again LLM's at the end you put an out input the output unless you're doing a lot of stuff it can be different and when I'm thinking about let's say set is the best of the best we're managing millions of campaigns at the same time and agent can make a mistake and you want to remove that option of mistake from the room and the only way to do it is via supervision of the customer that at the end it's his his bank his or her bank account. So yeah,
that makes sense. And uh I mean well given assuming you're you're correct and that will that will likely transpire in in the near future here. What do you think just UI or UA sorry in general looks like in the next 3 years? Uh or what is your vision of it even um that you're driving towards and hoping that like AI can help with or people help with however you think UA looks especially sort of in this difficult

environment right now as that escalates. I would believe I believe the role of user acquisition will change dramat like dramatically. I do believe that in the three years the creative team will be the number of people will be reduced. I believe the UA even managers number will be reduced and the position itself will be repositioned as the UA strategy and how to work with agentic system in order to find the best results in the campaigns. So I believe that everyone will become this strategist with a lot of a AI let's call it powder in their like fire powder in their hands. H I I expect it to be way more competitive than now. I expect the volume of creatives will be like I don't know maybe three, four, five times more per studio in terms of new concept and high quality not like AI slop. It will be I don't know it will be it feels to me that it will be tougher in this space and people will will kind of reposition

their mindset about how I can kind of control AI system and utilize them in the best way to impact my strategy. I'm thinking a lot about brand marketing or product marketing people with this kind of capabilities of understanding the target audience very deeply compared to UI managers and motion designer that not necessarily understand them like product marketing managers. I believe those type of roles will be more significant and will take a kind of authority and responsibility under campaigns management as well because campaign management will be more like of an execution mode in my opinion less than a thinking mode besides a high level strategy. So the high level strategy and how you direct and guide the system will be the main uh like the main the biggest lever and the talent that will lead that should be very like on top of the target audience on top of the product capabilities and features on top of how the trends in the market

looks like. So I believe product marketing kind of persona will be more impactful in actually the campaign management or brand people. Do you think overall that um like human creative directors will still have a place in in three to five years?
Bigger bigger even in my opinion bigger because they will have more responsibility. It's not necessarily that the team will be uh 50% bigger but they in my opinion will have more responsibility and authority because they will be in charge of the strategy of the execution layer. The execution layer in my opinion will change. But the strategy and how you guide your systematic workflows that will be a real tech system will have to have more input of thought. So you need to do more research. You need to have more context in your mind. You need to uh work with more teams internally in the studio to understand more. You need to meet and speak with your the users of the games more. So the direction roles including art, creative

will be in my opinion bigger and bigger in terms of responsibility and authority.
Well then I mean I think all that's been a lot of the things you've said have been good news for for humans that are that are worried as you said some some worry more than others uh about what AI means for their jobs and it's more like well probably just means more skills to learn. It reminds me when everyone had to learn how to type. Uh, you know, it was one of those things where it's like typists aren't replacing you, but you better learn how to use the tools available if you want to keep up, right? Same kind of thing. Uh, in that sense.
Uh, well, so we just want to finish off with three fun little questions here. See, uh, just, you know, off the cuff what you, uh, what you want to say on these ones, but uh, the first one, I think a little more opinionated, which is, uh, a a game whose whose ads you think are better than the actual game. Look, I am a sucker of like White Out Survival, Kingshot, Lord Rush, Doomsday, Lords Mobile, those type of game ads. For me, it's like the best of the best. I'm truly addicted. So, like Forex genre

is something that I used to play Clash of Clans usually in the past. I do significantly like the ads, the playables more than the game. The game is great, but the ads are like freaking amazing. the game kind of drags down in comparison.
Yeah. Yeah. [laughter]
So, we just get to the good part.
That's fun. Uh, what LLM models have worked the best for you or trending towards working the best for you? Like just give me an idea like how you see the LLM models in your work specifically.
So, this is actually a super interesting part that is becoming I will do it unfortunately maybe to the listeners a bigger longer answer because it's really interesting part. LLM is actually coming more and more as a commodity less that I've expected in the past internally like Yonyi my co-founder is in charge of the technology is the CTO the costs are so high so high and the

differences between the models is not that big and in different areas is different so it's not let's say Fab is not the best at everything Gemini is not best at everything. So they might be very good at something but not very good at other things and it's kind of become a real commodity that look we're heavy users of cloud like all of our people internally are using clot code but today the the other models including open source including Chinese models are pretty pretty uh advanced in the race let's call it like that and we're using them as well like you see a lot of codecs lately like openai uh we utilize a lot of Gemini I so we're kind of we don't care. We kind of LLM agnostic. We don't care what we use long it keeps being uh delivering us the best results. And I believe now you see the shift of the big research labs that are f focusing on cheaper and cheaper and cheaper LLMs instead of better and

better and better. Do do you find like the the CTO or other people at the company are having to constantly re-evaluate and retest as new models come out or as pricing changes? So you even see like pricing will change for or token usage allowance will change it for like brief periods of time as a competitive landscape keeps shifting keep it's will keep it will keep I I see it like it's being in everyone focus in an un not unrealistic way because we're spending a lot of money on this but you're kind of thinking yourself wait why am I real with sonet or now if it's so expensive so you all of the time on Twitter understanding the new models that are coming out. You you you the real way to understand if it's good or not, you must see it on Twitter and people that are tweeting about it and the evolves are not necessarily representing the reality of today.
So, uh so yeah, it's will keep changing.
I I imagine we'll get to at some point where instead of depending on benchmarks

or social media, you'll just have an agent itself that tests agents and then like against like sort of expected feedback and sees how well they execute. So I can just in real time tell you like this this new agent's better and cheaper all that.
The agent your agent that is based on entropic will say hey probably entropic is the best.
Right. Exactly. Better use an open source one just just for less bias. Right. Um of all the stuff you worked on as well as other stuff you've seen outside of what you worked on. What's been your favorite UA trend? Whether that be like a specific like style of UA or just like some some ways people are work thinking about from a strategic perspective. I again I think it's like a obvious trend but I personally I really love ads that has a cinematic story as an intro like this is a person look it's been trendy let's say since uh the AI improved like you saw UGC is still like gaining a lot of spend in the campaign but if I compare it to UGC misleading save the king and those stuff I

personally really love cinematics uh intros because it's bringing me back the world of advertising and target audience thinking and stuff like that and less gameplay gameplay game playing misleading misleading or UGC that is so obvious because you speak about the game the the advantages of the game but the cinematic is kind of getting you into a story bit of a fantasy and the reason that I really love games from the first place creativity and stuff like that
I think that's a great mindset that to have when you think about this stuff is is why that that makes sense to you and works for you but yeah good point about the the psychology stuff rather RA than just about the execution, it's about where is it coming from and where is it trying to go uh with the audience. So cool. Well, I wanted to thank you for for stopping by today. A lot of uh great stuff happening. Sounds like a lot of work in progress that is trending in the right direction, but we'll take time to get there. Not just from you, but also from all the AI companies trying to build all that stuff. Maybe world models helping as well. So, looking forward to to seeing where a lot of this goes. So, definitely make sure to uh for those of

you out there want to check it out. Uh it's set with two T's. Uh, so don't just type SCT if there's a second T on there. Uh, just just to make sure those are listening know how to find you. Uh, but definitely recommend checking it out. A lot of cool stuff happening out there. So, uh, thanks again for for coming on.
Thank you very much. Thank you for
and [snorts] thanks to our listeners out there and we'll catch you guys on the next episode.
If you enjoyed today's episode, whether on YouTube or your favorite podcast app, make sure to like, subscribe, comment, or give a fivestar review. And if you want to reach out or provide feedback, shoot us a note [music] at podcastnavic.co or find us on Twitter and LinkedIn. Plus, if you want to learn more about what Navic has to offer, make sure to check out our website, www.navic.co. There you can sign up for the number one games industry newsletter, Navic Digest, or contact us to learn about our wide-ranging consulting and advisory services. Again, that is www.navic.co. Thanks for listening and we'll catch you

in the next episode.