The Truth About AI Transformation (It's Not What You Think)

Every company in the world right now is claiming to be AI first, but there's a difference between transforming into something and being born as it. Today I'm having two guests who represent exactly those two realities. Leas Campamper is Appsier's AI experience architect and he has been inside the 1300 person company, a midsize unicorn as they like to call it company rewriting how it works from the ground up. And Mor is the CTO of Vibes, a company that never had to transform because AI was the foundation from day one. And I want to find out what the difference actually costs, what it gets right, and where each model breaks down. This is a podcast not about theory, but about the real story. So without further ado, gentlemen, welcome to the podcast. This episode is sponsored by Maloco. More than 2 billion daily active users,

over 2.8 million independent apps globally. That's the reach Maloco ads unlock for performance marketers who don't want to stop at the usual gardens. It's all powered by KA Loco's compound AI system. Continuously learning from billions of events [music] and getting smarter with every campaign. Hop over the world garden. Check out maloco.com. That's m o lo.com. [music] Unless you've been living under a rock, you know this is web store rumbringer. 70% of top apps are using a web store. And the best ones, they're doing it with Appcharge. Do you want a $20,000 in-app purchase? Yeah, [music] Appcharge does that. Do you want to pretend that subscriptions are still a viable monetization strategy for games? Yeah, Appch charge even lets you do that. They get [music] what high average revenue per paying customer means. And they're already doing over half a billion dollars in transactions a year. So, are

you [music] going to be the first to do a $20,000 AP? Don't accept microtransactions. Embrace macrotransactions. Learn more at appcharge.com.
In sickness and in health, in soft launch and in live ops, through every meta shift and every platform policy change. That's the kind of loyalty vibes built between your game and your players. AI powered targeting, [music] a real meta layer, rorowass optimization that actually compounds. That's what makes Vibes the fastest growing loyalty platform in mobile games. Till churn do us a part. Vibes vybs.co [music] Leas. Let's start with you. So your role is very unique. This role must not have existed probably even a year ago or at least two. You're an AI architect. What

is the day-to-day job of an AI experience architect?
Actually, it it was born a year ago and it was born soon after we decided to build our own MCP. MCP is an interface between in this specific case apps for data or apps for configuration and the customer's AI. So, we started playing with that with that interface. People used to say, "Oh, MCP, it's a fancy API." It is not. It's a completely different creature. So, I stepped into my boss's office and I said, "Listen, the world is changing. The way companies are building and treating their interfaces is going to change. The whole AI experience will be critical in AI adoption of our customers. What I want to do is to take the role of an AI experience architect and my job is to

make sure is the experience of our customers with our AI interfaces is pristine pristine I mean efficient easy to understand easy to implement easy to integrate into the AI journey in fact I want AI experience to be the foundation of our customers AI journey and this is why it's very unique. It's working very well. Our MCP is very efficient and we're building it even in a more efficient way and we're adding tools as we said you know companies are work are building APIs and every time you build an API now it's in cement right you cannot change it you cannot do anything and a year later you got to ask yourself why did I build this API like this oh I cannot change it it's already in the cement so we want to have our AI experience and interfaces the best we can like the best in the best of breed right from the get-go. Okay. When we build it and that's my job as an AI

experience architect to facilitate this exactly. So you're in charge of how Apps AI talks to your customers AI. Not AI to AI. I'm in charge of our customers AI is talking to apps fryer. It might be talking to our AI. It might be talking to our data. It might be configuring stuff in apps fryer. But that's my job. I need that my customers AI to understand us efficiently. And what is a background of an AI experience architect? The background is developer experience. I think the background is empathy. I started from developer experience and that's explaining to customers. You know, I'll start from this. I'm going to ask you how much time do you think apps the developers of our customers would like to spend integrating apps SDK or apps software? H day
this is the exact number zero
zero time they would like to immediately

understand what to do not like you know the SDK not to interfere with the code not to interfere with the with the efficiency not to interfere with stability they just want to put take our code integrate it and move on and that was my job for for the first few years in opsar and when AI came I said to myself just a minute this is exactly what what's going to happen but not just with developers with everybody all of apps customers I want to say hey I want to connect to apps operations data but I want to do it efficiently immediately I want to do it reliably I don't want it to elucinate I want to enjoy the world of AI with apps and this is the foundation so you need to first have empathy second have understanding of the of the customer have understanding of the technology and orchestrate everything to be to be exactly perfect. So our customer AI

transformation will rely on apps flyer AI experience and making sure that the experience is the best that you can give them in AI and I can I can say it works pretty well so far. I wanted to jump in to more like Vibes never had to go through any of this transformation process because you built with AI first from day one. So what does it feel like on the outside hearing the other story of of transforming a midsize as you like to put it mid-size unicorn?
Yeah, sounds uh very difficult. So for us it was a bit of a trend problem because we started at the age of AI. AI was already pretty common when we started building our codebased organization processes etc. So I'm not thinking about adopting AI tools. I'm actively building the processes of our organization. So we

build the processes of the organization from scratch based on AI which is a bit of different approach. So a year ago customer success in companies let's say they they have seen a drop in performance for customer and they wanted to analyze it and uh so what they had to do is to go to the analytics team they got a data analyst maybe one or two or days later when the customer is already pretty upset and uh maybe he's about to churn today They have access through AI to directly to the data and they can get the answers in 5 minutes rather than five days. Take that force multiplier and multiply that by all departments in the new organization AI organization

about customer success marketing user support CRM economy product etc. and think about how the entire organization is now changing because you can make decisions in a distributed manner. You don't need the analytics team as much as before to make real datadriven decisions. It's like another way to think about the same problem.
So you're talking about building process from scratch based on AI. Is it easier to build processes, new processes with AI tooling rather than transforming existing one? Because when you're transforming the process that already exists, you're analyzing how people work and then coming in with a solution or those people are coming in themselves with different solutions on how to make it more effective. But now you're essentially don't have that benchmark or that reference case. You're just coming

up with a process al together using AI. Is that more or less difficult? Our problem in that sense is much easier because in an existing organization you usually engineer the process a bit around the existing departments and people but it's it's very hard to do a reorg that is aggressive enough to fit the new state of things. Uh so you're of limited in how much you can change processes. Um when you build an organization from scratch, you're pretty much free to do whatever you want. So you can decide 90% of the decisions regarding customers would be taken completely in customer success without analytics involvement. And 90 plus% of marketing decisions would be done by UA managers without analytics when you can make so much more

decisions so much quicker. The velocity is much much much higher and so we never seen these kind of organizations before that move so fast with so few people which is pretty incredible in my opinion.
Let's talk about another thing that worked really well with uh apps flyier. So we had your CPO Barack on the podcast probably a few quarters ago and it was quite widely shared and widely listened as he talked about the transformation of the company from a company of 1300 people B2B SAS if you will and transforming into a real AI native company and I've talked to your CEO he's been AI bullish for a very very long time
so Orin and then
you were the one inside executing the strategy so can to give a glimpse from the inside how apps transformed into AI first. Yes. So first I'm a part of a team that is doing the AI leadership architecture and one of the best thing

they did like from the get-go is define this team of leaders that will push the company forward. What started it is that Orin our CEO Orin Canel is he stood in front of everybody in all ends and he just said we are all in on AI and he meant it like you can see in his eyes that everybody we need to be all in on AI. We need to you need to make the best your extra effort in learning integrating and failing. He said fail. Go try stuff and fail. I'm not going even going to tell you what to do. He actually opened the channel which is our slack channel and it's called all in on AI and this is where we discuss AI and this is where we say hey I have a crazy idea try this I have another idea try that this is the place to actually experiment now the thing with AI is the company identified also Barack of course

they identified immediately AI is moving very fast very fast things that were like let's take I like to take N810, you know, remember NA10.io? Like if you open YouTube, everything was N810, N810. So some companies say is, okay, we're all in on NA10. 6 months later, less people are like keen about NA10. I have no idea which platform I'm going to use in one year. Okay. And the company understood that a year ago. They said, "We understand that there's no way we can know which platform we're going to use in one year, but we have the best minds in Appsfire. So, we're going to use the mind of the herd." As you say, everybody use whatever you think is suited for your job. They thought like in a few months once we understand and we stabilize where we are and eventually they we estimated that a AI will stabilize and I

think we're now we're in the stabilization period we're going to gather about the we're going to gather the herd we're going to ask everybody okay everybody what's what's working for you and now we consolidate so out of five solutions that people were trying we're going to go with two solutions of coding out of three solutions of how to build an MCP. We're going to go with one. People were trying everything and now they know what works. And now we are in the in the consolidation period of okay, we understand what's going to work. We're going to build it. It takes time. It took by the way a lot of guts to say it. I'm not going to tell you how to work because people like a year ago they said okay let's think how a company is going to be in five years everybody use this platform I think that they stand out now the mistake because they consolidated too soon now we're trying to consolidate

I wanted to just add one of the elements that made it very interesting this transformation process is I believe you had one quarter where people didn't have any goals those goals were taken away and the only goal people were left with was going through the AI builder course so that everybody had proficiency in selection of the tools and automating and improving their own processes. So it wasn't just a small group that was AI activists looking at different products. the whole organization got leveled up and of course there's the one the early adopters as you said in that channel that are looking and trying different things but the rest of the company was also leveled up and was strong enough to adopt different tools but also you have internal consultants and maybe you can consider yourself as an internal consultant in somewhat but you had internal consultant with AI that helped people to set up their own process using various different type of tools that they wanted to use so there's internal help for them to to transform their

process. So I just wanted to mention that the uh the the change management transformation investment into your own workforce has probably helped a lot when exploring different avenues and and usages of AI. Another aspect is that I think it's going to be a very important aspect in the future for AI. the fact that now you have a company of let's say technical people they're the developers and in the developers you can divide them to AI believers and I AI non-believers you know I mean people said oh yeah I can definitely give my code to AI and it will suggest my next PR and someone will say oh this will never work it will only ruin my code it will only bring me like 30 commits with 1,000 bucks I will not trust it okay So one challenge is to make the AI non-believers believe and the AI believers not to go too fast, you know, not to introduce like 1,000 bugs in a single commit. But

that was in the in the techsavvy department. Now you have another department, right, of people that are not technical, right? How can you do that? How can you bring AI to people who are not dealing with the bleeding edge of technology all the time? We actually built a guild of AI builders. It was actually a guild of people that are building tools for other people. So if you're not techsavvy, but you have a good idea, you're able to go to the AI builder guild and they will help you do it. They're not going to build it for you. They're going to help you build it. Because the whole point of AI is what I called a no code haven, right? Even if you don't know how to code, you can build a great tool. You just need to know how to do it because right now there are great platforms. We mentioned NAN, make.com, you can do it with Cloud Cod, Zapier. I can go on and on. That was the the thing like a year ago and up

until now. I think now it's even better. So if you want to go with AI but you're not techsavvy at all, we are going to give you the platform to do it in the R&D department. Everybody are way into AI like deep into AI okay in other departments I think most of the people are already using AI not as you know I'm chatting and so on and actually building and now the main challenge is how you connect them okay I'm in the PR&D the product and R&D department okay you have tax heavy people the developers you have people that are lack tech he savvy but they need to integrate and they have great ideas which are the product people. How do I how do you integrate them into a single AI transformation journey? This is by the way the challenge that we're facing in other companies are facing right now. Okay. In one company someone told me we no longer say developer, we

no longer say product manager or designer and so on. Everybody are AI builders. So we put people in the same team and now we tell them you no longer have specific roles. You have you have other backgrounds. You have other specialtities. You have other strengths but this is no longer your role. Your role is to sit in the team and make the team build an AI machine that will get us to the company goal. In apps fire, we're still in in the phase of giving the tool to each role and then making the tool integrate. When you talk about this super fast decision making, not having existing process, coming up with new processes and and the velocity being just on a whole different level, what are the new bottlenecks that that creates?
There's a risk uh with uh distributing decision- making. Not everyone has the same instinct about data as the analytic

team. Data doesn't crash. In software, you can you can feel the app not working correctly. You can see the metrics, see the crashes. With data, you just get a number. This number might look correct on its surface, but it's extremely hard to know that it's completely wrong. You need a proper intuition. But since we move very quickly, you might get occasionally suboptimal decisions.
Can you give an example of of like that moving so fast and and ending up in a sub-optimal decision?
Yeah. So, uh often uh customer success are they are basically there to optimize and help customers get the best out of our system. But since the system itself is moving so extremely fast, it's very hard for them catch up with best practices. The

information spread like spreading across the organizations like the different department in the organization becomes the real bottleneck.
Understood. Okay, that that makes a lot of sense. It's so fast that people are doing things without coordination and that leads to issues between departments.
Yeah. So we need to like spend a lot of time on proper coordination using AI to do that. H for example by building skills that explain the AI what happens or give context to the AI about what is happening in the organization. H so that information spreads in AI velocity rather than humanto human meetings. So the the real slowdown becomes the information sharing between the departments rather than the decision making because previously the decision-m would have happened with all the department sitting together or coordinating the decision making but now

with AI they can move so fast and make so many independent decisions that they end up doing those and at at velocity as they make those decisions they start dispersing from each other. So it it almost creates more need for middle management.
Exactly. And uh this is also pretty much solvable with AI but by by building the correct skills that gives the AI the context.
Okay. So leas I wanted to ask you the uh the first points that Mar was making is that because they were AI first AI native essentially AI tools existed as they started their company that's easier for the company to operate effectively and build new process rather than changing the existing organization and I understand there's different type of organization to my understanding outfirely has been quite successful in transforming as an organization to be AI first, if you will. First, I envy Mo because he started from the from the

beginning. He mentioned a term that we're struggling with and everybody will struggle I think for years now and it's context and it's what what people call the AI brain. So, in order to have a good decision, AI needs to have all the data. Now when you start building your company as AI ready from day one then you're preparing all your data all the knowledge of the company customers history of the customer journey uh history of your products legacy products and everything you started as an AI ready data for AI you experiment as you go you test it as you go and just like M said when go to market teams need an Sir, it's ready because it's an AI ready data, right? Mo,
it depends on how proficient you are with the data, that specific data, and how how much intuition you gather on that specific domain.

This determines how much push back you can give cloud or the AI the tool that you use
uh when it hallucinates, which is probably the main issue with that. Right.
Right. So the quality of decision making is a bit different. So you need to be very aware which decisions can be taken in which part of the organization. So let's take what Mor said and superimpose it on a company that exists for 15 years and people were recruited when when Mah when vibes recruit people they know they're going to say listen you are going to be employed by a company which is AI first part of your recruitment uh process is to make sure that you are AI first people this is not what happened in Absire naturally in the last 15 years, not even in the last three years or four years, maybe in the last one year. Okay, we want people to be AI

first. So, how can you make a company which has legacy product, legacy data, legacy knowledge base, legacy people, if I might say, how can you make these people AI first? I think we did an amazing transformation. It started from or it continued to barak when they say you you're allowed to fail but you need to make sure that you go forward and in some cases were owning at the beginning of that process for example AI brain so even when we tell when we tell people you can go ahead and deep dive into AI and they do and they want to do it eventually they say okay you give us the platform to use AI Okay, we want to use AI. Where is the AI brain? Where is all the data that we need to give AI? Okay, and we're building all kind of internal tools. But again, even these tools are just the beginning of the process. And Mishk, I

think many companies that listen right now face people are nodding their head. They say, "Yeah, we have the same problem. Where is the AI brain?" Because we have all kind of third party tools that they tell you we're going to build the AI brain for you. You're going to take 15 years of data and build us the AI brain like out of thin air. I'm willing to see it to tell you that I've I I watch it in my own eyes. No, I didn't. I think this is a huge challenge right now for all companies.
What's the vibes AI brain like?
Yeah. So I started uh talking a bit before about the the context we're building for AI. So it's pretty much what the AI brain means. So first of all, you want to connect everyone to all the data sources because if you don't have access to the data, what are you going to analyze? Uh but second of all and more important or at least as important is to give the AI a sense of

what you're doing like how the business thinks which metrics matter for which department how you calculate metrics like often in a large data warehouse I have multiple ways to calculate the same metric like revenue what is revenue for marketing it might be different than for customers So building this entire like huge context baking in all the business knowledge and intuition.
I was wondering about customers. You know our customers they trust us for years now. Absor is the foundation of their decision making and we tell them yeah we're we're fast on AI. Sometimes they have a lot of questions. What is AI? Can we trust it? Can we trust you? Now we have very good answers, but our customers are in their AI transformation

phase. Sometimes ahead of us, sometimes behind us. I'm wondering when you are a vibes, when you have new age customers and so on. Is it easier? You know, from the questions and so on.
I'm not sure we have new age customers, you know, our customers are game developers. Some of them are pretty large companies or you you can say and some of them are tiny new startups some of them are more proficient some of them are not at all and so once the customer success at least to my experience once they have access to data and once they gain enough expertise on working with data What I saw working amazingly is the sitting with our customers analyzing the customer's data and and we can do it because we don't need analytics for that anymore. Customer success scales better

than analytics. If every person in customer success can actually work with the customers on their own data, you can leverage the customer's intuition about his own data. have much more intuition than us. When the customer success is doing correct data analysis time time again, the customer sees it's like it's working. So you have enough evidence to gain more and more trust customers. It's a matter of like data expertise spread to the people that are in touch with our customers h to gain this trust.
I couldn't agree more. I couldn't agree more. I can tell you that in Absra we we definitely we we understood again I think from day one that customers that are doing AI transformation they come to our customer success people with questions they look up to them and we

understood it and we're I am working personally with many customer success and we have internal master classes and our external master classes to allow our customer success to be the people that guide or help our customers in AI transformation. I think now when companies are looking for data vendors um MMPs you know any vendor the first question they ask themselves will this vendor help me in my AI transformation will it take me further further or will it take me back do I AI trust him we used to say do we data trust him do I AI trust him I think this is very important this may be part of AI experience or AI trust in general how how do measure that. Do I AI trust them? Like Mar said, they bring my customer success people. My customer success people go into the customer. Come into the customer and the customer present to them this is my AI

transformation. Be a part of it. If you're not part of the a customer's AI transformation, it doesn't AI trust you.
Yeah. So I agree and I would add that uh as someone on the other side that uses vendors like Appsflare and others, you can feel when a vendor is holding you back when you try to implement new AI systems. For example, Leas me mentioned their MCP. So imagine you have an agent that is extremely efficient working like 10 times 100 times the speed of a mural and it connects to everything to your data warehouse to applier to whatever and then you have one vendor that you need to automate that doesn't have MCP and you're basically stuck because at this stage of the process you need to offload that to a slower human moving kind forward and talking about a

little bit how winning looks like in your organization. If we start with vibes, what have been some of the things that worked really well, some specific moments where you thought, okay, this is actually different than it was before. Was it the decentralized decision-m or something else? Yeah, it's something we're attitude to is basically our what we call distributed analytics platform makes it possible for everyone in the organization to make datadriven decisions. And the moment I realized like the more it clicked was when the customer success team lead came to show me a new dashboard that she has built. And for context, she's not technical and she tended to show me like a new dashboard she's built to help the customer understand what his reals are. So we had a situation where a certain customer they looked at the metrics a bit

differently and uh she wanted to help them look at it h the way we do. So basically she built like a whole dashboard around it and it worked amazingly well and this blew my because she didn't involve anyone from the analytics team. She previously she didn't she wasn't like super technical but she did develop this ability by herself. We didn't push anything. We just gave her the tools and it was like magic. And this is when it clicked. You just need like in an AI native organization, you need to hire the right people that are maybe technical enough or maybe open-minded enough to adopt this new technology and you just need to give them the the right tools. You don't need to push anything. You give them the tools, you connect them to the data and magic happens. This is pretty

incredible. So in practice what we did to do that is as I said we connected to the data warehouse and the rest of the data sources. We spent countless hours organizing the infrastructure to mostly our data warehouse which is a thing that I think no organization is actually doing because it's a complete mess in most of of the organizations especially like older ones. H but it's extremely important for AI to to do less mistakes to be accurate and we built a really large set of skills like the AI brain as Leas called it h to support the h the AI analytics use case said a word that is very critical and that's tools so ft number one I can offer the

listeners, I think the easiest thing you can do right now is to go over the organization, gather the tools that everybody built and put it in one shared place for everybody to use. It's a technical challenge. It's not that hard. It's very easy to do. There are um some third party vendors right now that will definitely help you. And you'll be able to measure tool usage. You'll be able to measure the token usage of the tools and especially you you'll be able to share the knowledge of tools among everybody in the organization. So if I've built a good tool, I'm able to share it. I'll get the recognition. People would look up to me, people get an idea, they build their own tools. So tip number one, gather all the tools and share it for everybody. But there's a huge challenge that we faced on top of tools and AI brains is the flows is the

work procedures is how to get from idea to product in an AI augmented way. I wouldn't say auto I don't want to say automatic way because I don't think AI will get the idea to to production and you just think about it. Someone needs to be in the loop like a human in the loop. This is the challenge that we're facing right now along with other companies. How to do that efficiently? How to get to that holy grail five 5x and 10x? We're definitely working on that. We managed to to have some success in this area. I heard from other companies that mentioned 5x, 10x. Mishka, I can tell you honestly, I haven't seen anyone hitting that holy grail of a 10x flow like a seamless. Yeah, I have a a product manager with an idea. One day later, it was it was there in production. I don't think we're there yet. Sounds like a VC pitch. Yes, it's a

bit dependent on how your your organization is structured, right? Because it doesn't matter how you augment your own work. If you're essentially blocked by for example product decision or a product design or an analysis that needs to be done or whatever like the main bottleneck is the organization rather than the technology. How do you do in a small company? When you think about flows,
we basic lead the entire organization around AI. You default to AI. When you have a new challenge, you first of all say, okay, how can I solve it with AI and the existing people that I I already employ. But first of all, how can we solve it with AI? And when in which part of the flow is it required to have implement usually in uh our organization answer to that is when you need someone to take

ownership on the result. So if it's an analysis it depends on how revenue critical it is. If it's a small decision it can be distributed completely. If like a it's a huge revenue critical the company will crash if we make the wrong decision then we put a lot of human resources on to analyzing that specific thing. So it's really a matter of defaulting to AI because we're very flexible with processes and then like putting humans in that loop were necessary. It's very different I guess on your end of the spectrum when uh using people is the default right and people want to do their work. I think what what we're telling people is that AI is not a goal. AI is a tool. The goal is the customers. We're customer obsessed. So ask yourself how

exactly what you said. How can AI help me? And facilitate them by what do you need? Do you need tools? Do you need education? Do you need tokens? What do you need to make this happen? And once we understand what what they need to make it happen, again, we have an AI leadership. We think together with them and we build them the tools to facilitate them. AI is not a goal. This is what we tell everybody all the time. Okay. And I mentioned token usage and I think this is the this is the biggest challenge right now that that you know that it it's underlines everything AI brain costs money inefficiency cost money a very good idea in AI might cost money I think again this is this is a huge challenge people are thinking right now and tokens are not getting cheaper
no I can tell you
they're getting much more expensive Yes.

Yes. They're getting expensive. The policy of the AI platforms is getting more more rigid. You know, they're saying, "Okay, no more seats. You're going to be token usage. We're going to measure things. Your AI agents might suddenly go from, I don't know, they will go 10x token usage. You will not go 10x in efficiency, but they will go 10x in token usage. Stay ahead of the competition with Sensor Tower, the trusted source for mobile marketing intelligence. Whether you're tracking trends, benchmarking performance, or scouting the next big hit, Sensor Tower delivers the data that drives strategy. Dominate the charts with Sensor Tower. In today's hyperco competitive digital landscape, understanding your app's journey from discovery [music] to download isn't just insightful, it's the difference between scaling [music] and stagnating. Enter Appslier, the undisputed leader in mobile attribution and marketing

analytics. They've mastered the science of making every ad dollar work harder by revealing exactly where your players come from and how they interact with your app. Their platform transform mountains of complex data into clear, actionable intelligence that separates winning campaigns from budget draining experiments. And I know that many of the top game companies leverage Appflyier and are a key partner in their success. Appslier, where your app's potential meets performance.
Saturi, it's the Japanese Zen Buddhist term for sudden enlightenment. That flash where everything clicks. We're told that Heroic Labs team had theirs [music] when they realized that most studios run live ops like a horoscope. vague, kind of generic, same advice for everyone. So they built Saturi, real time experimentation, player level personalization, events that actually respond to player behavior, sayanara to

your old live ops stand, and Konichiwa to Sattorii from heroic labs. There's a drive almost in every company to automate everything with AI, everything. And sometimes that doesn't even make any sense. We're talking about simple processes where they would have, you know, simple agents working on let's say customer support or whatnot. And it might be costing them few tens of thousands a year and they make new investments like 350,000 to fully go in agentic. It's like nobody's even calculating the paybacks and nobody's even thinking about what happens when the cost of AI gets less subsidized than it is already right now. Like of course it could be more effective so it actually becomes cheaper but there's an opportunity that it becomes more expensive in the future and you've further automated and implemented AI and process that actually were working quite well and cost effectively already like how do do you see this in happening in

the market because I' I've seen some of these cases. I think right now the challenge, the real challenge is is the token usage. Everybody has a great idea. Everybody develop a great AI brain. And now I can scan all my repositories and all my code and maybe upload all of them to my LLM in a in a magic way. And the LLM just like every simple question now cost me so much money. And the real question now, the real challenge is to understand is it worth it? Up until like three years ago, people would ask themselves, is my developer worth it? Like how would you know is your developer worth it? Now you have another problem. Is my developer worth it? And is my token usage worth it? So we didn't solve the problem. We just made it worse. Right, Mo?
Yeah. I guess you could use the AI to maybe measure the effectiveness of

developments, but it's like very theoretical like in practice, it's extremely hard to to measure R&D effectiveness because at the end of the day, what you should be care about is the impact on the business. And in most businesses, R&D is pretty far from the actual bottom line. in B2C, which is probably the more like common audience for this podcast, it's actually a bit different because many developers and software engineers and R&D in general in B2C are much closer to the business bottom line than in B2B. So, it's a bit bit easier to understand if it's worth it to build like this specific feature. But in uh in most companies especially B2B companies you said is extremely extremely hard.
Talking about something that is extremely hard and something that has failed. I wanted to like we talked about

successful cases but what are some of the things where you gave it a go but it didn't work out. Something you built something you tried but you had to walk away eventually. As any B2C company, we're doing tons of AB tests. Like a good software engineer, h this problem seems extremely easy at first because what is an AB test infrastructure? You just need like a feature gate. You already measure the data like the metrics that you want to compare, split the audience 50/50 and then measure the data using cloud, right? Not at all. Uh apparently everything related to data is so extremely hard still even with AI uh because with data you don't really have any ground truth to compare and get you just don't know uh that your system is broken and you got like results you got numbers but these numbers might be completely

incorrect. So when you try to build such a such a system that looks so easy on first glance you get all the bugs in the data pipeline h that you figure like that you find found them much much after you already use the system. So AI can't fully save you there. What can save you when you build such a system is good data intuition which we're building actively very actively and apps flyier probably already have that in the organization because like they build that intuition over 15 years by the way even open AAI like the great open AI bought static the leading AB test experimentation platform on the market they bought it for over 1 billion is it's one of their largest acquisitions because it's so hard to build such a

system. So what we ended up doing is uh we took part of the infrastructure and we offloaded it to a third party platform and the rest we're working very very hard on it to make it robust and uh bug freak. A data product that you can trust is basically not doing its very basic functionality, right?
Yeah.
Might as well don't use it.
I would add efficient data.
Yeah.
How about Leas? You have a whole Slack for trying and failing.
Wow. You know, I'm going to take you a year back to my my first I think AI experience fail when we gave our customers tools. So, we built our our MCP. MCP has tools and we said hey why not let's give our customers all the tools all the operations they can do with Opsfire that was the first version or first Intella version of the MCP and

we started playing with it we started testing it and guess what we understood that we might give the customers the gun to shoot themselves in the leg because if you give tools and the tools are not safe enough customers are be able to completely deletate completed delete application. A simple question, a simple task by the the agent and a little hallucination by the agent and it will delete your application in Absra. It might do that. So that was the first fail. We said okay we're going the wrong way and currently Absur MCP has only readonly tools because we we want this to be safe and only now we understand and we research carefully how to allow our customers to have operations in our MCP to actually do actions because we don't want them to fail and this was our first failure

understanding what is the power of AI I mean it can give you the power to do a lot of things but one wrong step and it will kill your not delete your data but it will kill your application and so on since you've been working in AI you've been redoing your processes and and how the company works the the uh the natural part is people being afraid of their jobs being eliminated or jobs changing significantly so can you talk in in however you want the positively or or less positively about someone at your company or job title that looks most different today because of AI and and what does their day look like now versus 2 years ago.
I have two uh roles. Uh one is operations. operations would be considerably smaller in the future

because a lot of things that needed human operator became like basically a commodity now that AI can reason a bit better than NA10 or make could and another role would be the data analyst uh which won't disappear it would just change So until two years ago, one year ago, most of their jobs was about pulling data, cleaning it, analyzing it, building dashboards, and answering edoc questions over Slack. And the ticket was infinite. The entire organization was blocked by their huge uh backlog of of tasks. And today they're all completely changed because most of analytics can be done in the

different departments. Data analysts now are in charge of the data platform. So the role instead of doing ADOC analysis for different parts of the organization is to make sure the organization can make decisions safely and they do that by building the AI brain h that leas talked about basically a huge set of skills that encodes their expertise and also monitoring what organization is doing with AI. So ideally in two years I would imagine we we would have a a working product that closes the feedback loop on analytics in the organization. Meaning the analyst could see what people are asking the AI where the AI got it right and wrong and could optimize the AI brain so the organization can move faster and faster. So basically the role is completely

changing and uh the hiring instead of hiring uh people that have a bit of technical skills and mostly business skills we now need to hire people that are half engineers because they are actually building a full-fledged data platform.
I can relate to what Mor said. I want to give three more example. One example actually uh very quick just really came to the top of my head. Marketing video editors I think they they they nailed it. I mean they can things like productions that were pretty much imaginable like they they could not imagine it. They can actually do it in a few prompts. And now I know the people that do it in Abstrier. I don't know if you noticed their latest campaigns, they're pretty much amazing and they can do amazing things in like huge Hollywood productions are not like a prompt away.

So definitely they're changing and they can do amazing things right now. So kudos to them. I'm not sure if M can relate to that but in Absra it's a huge deal that you can have a huge
completely relate.
Yeah.
Yeah. Because we have also a creative person and with one creative and a bit of freelancers we can produce huge number of creatives which are like really good production. Mhm.
So we're you're pretty much bottlenecked by the your ability to come up with new concepts that make sense to people that
exactly
triggers them and excites them rather than the operational part of editing videos and uh maybe like meeting uh actors or whatever you need to do when you c when you do. So two more roles I would like to discuss that really changed designers UX designers people used to say now you

don't need UX designers you don't need the dashboards and so on I think you still need dashboards maybe to lesser extent maybe they need to change but UIX designers still give their touch but the tools are different now even in Figma they use it a little bit differently they share it differently and you can feel it because I can designer, right? I can go to code and tell design me like, you know, in like in whatever dashboard and so on. It will never be as good as a designer because they have a skill, they have the touch. They changed because again they're doing it with with AI. So the point is not to go ahead and design and scribble everything but the point is to give that touch that talent to the AI and and deliver and another thing that changes I think it's going to significantly change is the KB writer is the knowledge base

writer think it changes because first it is still needed it's still needed to change that knowledge base but the target of knowledge knowledge base documentation and so on is no longer humans. it's LLMs and you still need someone to say yes this documentation is right this documentation is efficient for LLM because when you ask a question right now the LLM will go ahead and it will read absur documentation and so on that documentation was originally written for humans and it can have conflicts and it have contradiction and so on but humans will never know it because they read a specific section or a specific chapter. LLM will read the whole thing and then like I know I went to uh one of our vendors knowledge base and my AI started hallucinating. I

insisted why are you hallucinating? Because there are conflicts in the knowledge base. So knowledgebased writers are needed. They're changing fast and they're changing because the goal is changing. LLM is now reading documentation, not necessarily humans.
Well, thank you on that and we'll take it home because Appfly and Vibes are two companies that have pretty much the same destination in an AI transformation though completely different journeys to get there. And what stuck with me is that neither of the model whether you start as somebody who's transforming your operations with AI or start AI native neither of the model is clean. So transformation as we heard from Leah is very expensive. It's slow and it can be painful. Uh and then again starting from zero means building without a net without references and encountering new sets of bottlenecks. So the real question is not what approach is better. It's more about is your company evolving

at all as you're going through this. So if this conversation made you think differently about how your team works, share it with somebody who needs to hear and connect with Leah and Mo on LinkedIn. And for more conversation like this, the constructor of fun wherever and whenever you get your podcast. So see everybody next week, thank you so much for everything that you shared.
Thank you very much.
You did it. You made it to the end of the episode. As a fan of the show, it would help us out if you'd subscribe and leave us a review on the podcast service of your choice. Those stars and likes, while a small action on your part, mean a lot to us and help to grow the show. More importantly, are you a member of the Deconstructor of Fund Slack group? If you have 5 years or more of games industry experience, go to deconstructorofund.com/slack and apply [music] to join. Join the games industry best professional community filled with peers always

willing to lend a hand or subscribe to our newsletter to get all the latest insights [music] from the deconstructor of fun content creators. Thanks for listening.