Assetario’s Matej Novak On Using Machine Learning To Predict Player Needs And Make A Profit

[Music] you're the CEO of this company coming up hard and fast high level view about what it is what it does thank you very much for having me Peggy really appreciate it um I have started the business with a co-founder of mine out of MIT a couple of years back because we really want to build models that understand people and we want to enable Publishers with the absolutely utmost based understanding of their customers that anybody could give them so I'm very passionate about building machine learning models that you know tell you as much about your customers as possible we're able to understand really well you know what kind of items are these are your customers looking for do they need a skin are they looking for a discount are they going to be loving your app or are

they better served in another one of your apps and uh we're just trying to you know the tagline is we're trying to build the best machine learning as a service in the in the world and I'm very excited about it we've got 30 people out of that maybe 25 or technical or machine learning guys and slowly scaling up the sales team now in New York very cool machine learning as a service that's a very cool term indeed because we've got the timing for it right we saw what chat gbt can do for consumers are you going to be that for B2B yes so Chad GPT is incredible in the sense that those guys have been able to communicate to the world how incredibly powerful machine learning is in the B2B b2c standpoint right they have really shown you that for some targeted use cases AI is simply the cheapest best and the most powerful way to solve the solution and so what acetario is doing is we want

to build the best targeted machine learning models but for B2B monetization use cases and so then when you have a particular problem that we're solving you're just best served by the power of the models that we're building so that's just it you're able to work with companies in the B2B space we're here at Pocket Gamer where it's all about games marketers they want to understand people tell me a little bit more about the nuts and bolts how you're integrated how you're able to take in that data and what you're able to tell them based on that data and by the way I love the mobile gaming Market that's where we started out and the reason we started out there is because games just have an incredible number or amount of uh understanding of their customers right so if you compare it to a standard e-commerce shop let's say Amazon they don't really know their customers very well because you go to Amazon you search for the product that you wanted to buy anyways and then you buy it whereas in mobile games let's say a really good

game you spend two three hours on it per day like Diablo Immortal is a game that I play a lot brawl Stars the game I play a lot and they really have enough understanding of what I want and if they just show it to me I'll be so much more likely to buy it what we're doing is we're able to connect to analytics data that any game is is collecting already we're able to go very deep on a user level so we're able to say this user they have seen a skin and they haven't bought it so you know let's try something else or this user all of their friends are buying diamonds you know or their Guild is now on a diamond buying spree what this user deserves a discount offer on diamonds when they're gonna see it on their screen they're gonna be delighted they're gonna buy it and they're gonna love your you love your game so it really is just very good on your understanding you know I mean we started down that journey of well you can see the guild is buying diamonds they should get a discount that I'm not always sure that app marketers fully understand what is locked in their data

as as the saying goes you know data is the new oil uh and it's true with the caveat that's you know it's just like oil it's completely useless unless you dig it out of the ground and do something with it so you need to play around with it we basically are able to support currently three powerful use cases one of them is a standard understanding of what does the user want to buy you know what is the right size and amount and content of things that they're buying so that's a live Ops use case then we're able to Leverage The around 350 million monthly active users that we're predicting for to build a really good understanding of a global spending pattern which we're leveraging for a second product which is uh Global pricing uh and then for a third product which is a very strong user acquisition use case is we're able to forecast really well the future spending of every individual user we initially started as a light bulbs use case but it turns out that it's an incredibly powerful information

for user acquisition Specialists and so for example the way they're able to leverage that product is by you know using our understanding of their future spends to say which campaign is going to drive money for me and which one is money losing and so just on that use case where me we want to make sure that no user acquisition manager ever loses any money ever again because after one day we're able to tell them losing money or making money and so nobody has to wait for 30 more days and that's just a very practical example of machine learning you know in the industry pricing sounds very interesting because we are in an economy where you're telling me you can tell me pricing and I'm thinking that is a very unpredictable changing variable give me an idea the patterns you're seeing I mean if you could give a couple of ideas to games marketers saying well you know what this pricing range is really hot that one forget it very interesting right because if you look at the best businesses out there in uh in-app

purchases let's say it's Netflix right or Disney Plus Disney plus and Netflix have higher prices in the US they have cheaper prices in India and it's only fair right it's obvious that you should at least follow the things that Disney plus has let's say you have double the price of India let's say your two or three countries are half the price of your tier one countries and then you can go much deeper you know people are looking at let's have cheaper prices in Android let's have more expensive prices in iOS you can go deeper than that you can start with a couple rules to say let me just hold GDP per capita of a country and let me scale prices by that and at minimum that's going to drive a lot of value for you and there's obviously a lot more sophisticated ways to go about it by taking in real-time GDP data by looking at location for example even in America right the difference between New York and Alabama is Monumental in in economic conditions and

that's just a very easy example of how much more you can personalize do you integrate with the apps Flyers do you integrate with lean Plum clever chap other companies so you're able to say to a marketer exactly what I'm integrated there so I'm seeing the data and getting a more holistic view what is that allowing you to tell an app marketer an advice you're right so there's a lot of data sources that if we combine we're gonna have the most powerful predictions so obviously let's say what's important for marketers we're able to collect data your first party data from let's say your amplitude uh analytics or Firebase analytics um then we have a lot of Integrations with mmps where we're able to pull your cost data your cpis and your contextual data like these are the campaigns that you're targeting so now we're all of a sudden combining campaign information we're combining user level data which uh is generally something that lives in two different systems already right which is

a non-trivial thing to combine and then there's a lot of metadata that we're personally working on very hard to make sure that you know when something changes in particular Indian States let's say of you know Gujarat or something like that we are able to react to it and you know reflect the economy conditions for example so there's the anonymized metadata that we're able to see in the industry there and then there is all the data stores that you possibly have in your business end-to-end integration so that we can combine them all into you know a full prediction if you look at a normal marketer like oftentimes it's hard to even combine your cpis and ltvs so in a dashboard let alone make machine learning predictions on top of it so you're into Data big chai mate I mean you you have your company you come from MIT this is in your DNA tell me some of the surprises even for you in the data that you're looking at where you're like man I didn't think a marketer could even

know this it surprise me I was honestly most surprised by how much value this is generating so I remember I mean we're just starting out we're we're students we had our first you know client and uh I mean we just started to personalize in-app purchases saying Hey would a user like to buy a hundred coins or 50 coins you know do they want to be show be shown a bigger offer or a smaller offer like in the most simple use case you can think of and you can easily get 40 50 revenue uplifts on those targeted offers by literally just changing the sizing of offers if you surface the right offers I mean you can get incredible increases what that means is that you can have a fantastic game really really good game really good product right it feels great it plays great it has incredible retention and if you just aren't surfacing the right offers I mean you're losing out on half your revenue and like maybe three times the profits it's incredible and the other thing that I

was surprised with is uh you know gaming is a margin business right let's say you know if you can get a dollar in LTV then you're spending 70 cents in cpis you know and then you're just trying to get a little bit of that you know margin as as your as your so any LTV improvements which are really Revenue per user improvements they're still getting them for the same price so like a when we have clients for example in China big games right more for example Century games is a long-standing client of ours they have hundreds of millions of Revenue and when we improve their games LTV by 30 the games Revenue goes up by 2x like it's really exponential there it's I was just so surprised how exponential everything is in in gaming so it's like AI is a service but there's a level of like customization at scale isn't there it's very important to make sure that we can customize as much as possible

because uh monetization and user acquisition this is in the realm of core business right so in order for our stakeholders to be satisfied with having a system automated system handle a core part of their business like discounting like you know surfacing those offers they really need to have as much configuration in there as possible so that the customer has a feeling of control right so they you know there's no stress of hey what if your AI goes haywire and then we have two days of downlift so it's it's it's a lot about finding ways to you know automate but in a way where humans can still apply their expert knowledge through as much configuration as we can possibly support it's uh it's very fun yeah so tell me how you give control back to the app marketer as well for example a couple ways let's take an example of uh of you know an app marketer in the user acquisition use case that are using our Revenue

forecasting engine right the way we're giving control is we're able to have front end where the customer is able to click through what are for example the most important Revenue forecasts that they're interested in you know are they interested in you know after one day of the user being in the app give me a two-month revenue forecast or maybe are they interested in only 14 day revenue forecast which are going to be more accurate but still the most relevant for this customer another way we're able to for example have it be customized is by changing the way that things are predicted you know some customers for example are interested in having pure Revenue data right Revenue numbers but some customers are only interested in saying is this user going to be a payer or non-payer because if for example the only decision they're making on is payer versus non-payer then they can get better performance of models uh by being more specific and then on anything that we do where we're actually changing the monetization in the game we're able to scale an A B test up and down every

single client is running an A B test with us when they start with us they only deployed on 20 of the user base and they can dynamically change it when they are full with us we have customers that we have been with for two years and we're still only handling 80 of their user base because we just need to make sure every single month that we're providing the value that we're providing right and so those are for example some levers that are important to pull and you know surface also good times for you mate when I think about the rising importance of the subscription model they're beating it pass to your door so Futures are very cool or we basically were trying to Port our models into subscription apps as well to make sure that we significantly increase our Market size right and so you know we're already personalizing um battle passes for games so we thought let's go into dating let's go into fitness and that's actually very fun a lot of gaming Publishers they're very

Tech heavy they have a lot of people internally they know how to code uh whereas in subscription apps you know it's it's more important to be able to deliver external value and I love the markets yeah it's not the Balkan gamer crowd uh but we love them as well I think we're pioneering uh performance pricing so on all the products where we are increasing Revenue we always show you this is so much extra money we generated in this month and so you're just gonna pay us a small percentage of that and that's the that just shows the kind of relationships that we're going to build our customers you know I really just want to provide value build good product products build good predictions you know do a good business that really makes sense for our customers and makes them rich foreign [Music]