User Acquisition AI, Past, Present, & Future - PHIL SHPILBER of GameChangersSF

foreign [Music] [Applause] we're gonna get moving along here with the global Trends track so next up is a talk on user acquisition AI past present and future with Phil Spielberg from gamechangers SF hello uh so good not to be on Zoom thank you guys for coming out I'm really energized by uh meeting with people in person so give yourselves a big hand of Applause for uh for coming out really well organized show so thank you for having me my name is Phil

Spielberg I work for a company called Game Changer SF I started the company about 10 years ago to do user acquisition and marketing uh kind of the old-fashioned way what we now call the old-fashioned way with humans doing user acquisition and I had a really interesting experience about eight years of eight months ago we were acquired by a company called Pixis AI it's a leading marketing AI company funded by SoftBank and what the experience I've had for the last eight months is really unique I've watched uh AI kind of do work that we've done you know human work that we've done for the last 10 years and we got to compare results and really think about you know what should humans be doing today with the kind of technology that we have available and what should we be letting AI do so what I want to talk to you today is about is about what I'm seeing what I'm seeing right now where I think things are going what's exciting what's scary you know what's what's happening out there um how do I advance

I should have probably asked that before I started it's not I could just have like hand signals or something oh there it is all right yeah yeah thank you all right so no conversation about AI can really be complete without talking about Chess and the reason is Engineers nerdy Engineers like me have been obsessed with getting machines to beat humans for like 50 years and I remember this actually in 1997 Gary Kasparov lost to deep blue right it was the first AI that beat a human and I know like if you're not a chess player that might not sound impressive like you could just put in a bunch of stuff into a computer and it'll beat a human turns out there's actually like 10 to the 40th power positions of Chess available you can't program that that's like a number that starts with one and goes with like it has 40 zeros it's a number so big I think it's bigger than like the number

of atoms in the known universe so you can't like you just can't make a giant database so the way that this was programmed was we programmed every kind of known opening every known high-level game that existed at the time and then we let it think like make move combinations a couple the you know 10 moves ahead something like this and we were able to beat a you know a human you know Gary kaspara for that time was probably the best chess player that existed and that was like a big moment we're like okay we you know we can get you this kind of machine to beat it beat a human but the way this played is it really mimicked human play it was like you put in rules and you taught it how to play like a human and that got us kind of to an interesting place but then to you know fast forward like 20 years Alpha zero which is based on a Deep Mind by Google this thing uh knew nothing about Chess right and what we did was we taught it the rules of Chess and then we had it play itself for about couple hours and after that it could be just

destroy anything out there we're talking about humans machines toasters you know whatever um just to give you an idea of like if you know anything about ELO system it's a it's kind of an interesting system that lets you compare things like if I play a player here and that player plays another player and this kind of happens at a large scale you could sort everybody uh kind of top to bottom and so like novices people just start playing they are like at the bottom amateurs like these are people that get pretty good like I'm at the top of that who keep it as a casual hobby they get up to 2 000 experts kind of between Grand Masters these are like there's like the Thousand best players in the world are up to 2 900 Magnus Carlson the current World Champ chess champion that's his rating to 2 uh 864 right this is probably now the greatest chess player better than kasparo Alpha zero is like you know it's just like there's no

point playing this thing if somebody asks you to play that AI you should not it's like it's like racing a Ferrari on foot like you would not want to do this thing it's like you could put a sane bolt out there and it wouldn't matter he wouldn't you know he would be a blip in the rear view mirror so this is all kind of relevant because um you know this is this is actually very much where AI is we're like getting into this deep mind kind of area and so again the reason I put this up is I want to tell you that this is what we do and so we've done over 400 game launches uh we've been doing this for 10 years so we kind of understand how humans work really well we've got a good process and when we uh started to kind of implement AI in our user acquisition efforts we realized a lot of the stuff that we were doing was just allocating budgets based on row as for example that's a very mathematically solvable problem uh you you don't really want to have humans do this thing especially if you're starting to compete with AI because the

human brain is made to look at like three four data points at any given time and say like yeah that tiger is going to try to eat me or not right this is the way our minds evolve what it's not good at is taking this huge amounts of data and trying to make sense of it in any way so what we've tended to do was really simplify things you know look at audiences in a very simple way look at bit and but you know bid Management in a very simple way and so the early AI that we implemented it wasn't really even AI it was very rules-based we would put in like oh if the CPI gets Above This turn off the campaign I want to allocate you know this kind of budget to this territory right this is the kind of stuff we use to automate and it was very much like early chess it was like here are the rules uh try to behave like a human right and so you got okay results doing something like that but where we are today is the AI doesn't really need to know much except what what you're trying to achieve and then it takes in a bunch of data and it starts to allocate these budgets in a way that we never

could right we could never consider the amount of data points that AI could do and so what I want you to what I want you to talk to you about today is kind of the three takeaways I want you to have is if you're doing UA today you've got to be messing around with AI you've got to be starting to experiment with it because your competitors absolutely are and if you're doing this thing just like human you're like racing the Ferrari eventually so you know whoever like I hope you talk to us but whoever you know whatever you're doing you need to be working some AI into your user acquisition um the other thing I want you to think about is okay in that world where we're increasingly relying on technology to do work what should humans at AI be doing like does this mean that they're going to be no more jobs in user acquisition well no I think there's really great things we should be doing and we should be dividing the labor it's not like we don't you know ride bikes anymore now that we have cars right there's still like things that should be done by humans and then I want to talk to you a

little bit of like after seeing eight months of this where I think the really interesting applications of AI and marketing are going forward so the number one thing is you should absolutely be implementing some kind of AI in your user acquisition the first thing is like it used to be you had to install sdks and do all this kind of stuff you know today like our technology is completely codeless it's just a browser plug-in so implementation you don't have to bother your uh your development team so there's really no excuse not to be uh getting on this AI I'll show you some of the data but we're seeing a 20 to 80 percent you um row as Improvement or whatever kpi row as tends to be the the kpi we're looking at improvement over like the best human teams and it's and I'll show you why that's happening and then again to to make sense of these large data sets humans really simplify things computers you know they don't need to do this kind of thing they could look at ginormous data sets and actually make some sense of them so just um you know this is like the

average game result we've got uh over a month over a month like looking at uh the way we do our a b tests is we have a human team do uh do UA we look at the row as then we kind of take the same parameters and take the campaign and let the uh the AI manage it and we're seeing on average a 58 row as Improvement and that's just doing a bid and budget management allocation along channels kind of giving it a lot of degrees of freedom going to different GEOS different targets basically letting AI pick whatever it wants with some parameters right don't go over this kind of budget there's some still some guard rails but they're not like hard and fast rules so the more flexibility and freedom you give AI the better results it'll come up with and if you look at some of the stuff it's doing it's not like what humans would do you look at some of the audiences you're like I wouldn't have done that um but the results speak for themselves so I love this one because when I started in the industry the the thinking about

the very old thinking about targeting is gamers are 18 to 45 year old males right it sounds kind of hilarious now but that's the way we thought about it the kind of newer thinking was okay other people play games we're going to do look-alike audiences uh right like Facebook look-alike audiences is kind of a targeting Innovation but it was still very biased because it looked it kind of unless you had huge organic traffic it looked it mimicked what you already acquired it just sort of doubled down on your strategy and did this kind of thing now it's completely behavioral like you could see what the audiences are you could do small tests like thousands maybe hundreds of thousands of tests until you see kind of the individual level or the whatever aggregate right we're allowed to look at with uh with the way the Privacy is working these days and you could pick out those audiences and you don't have to talk in these terms anymore about like you know are they RPG players or whatever they're just you know people that are going to play your game so we needed simplifications right humans need simplifications AI doesn't need

simplifications so we shouldn't be doing targeting probably going on so humans oversimplify data sets but humans are also much better at strategy like your AI is not going to come up with your marketing strategy it's not going to try emerging channels it's going to do a very good job on stuff that that is mathematically solvable so when I try to Buck it off like what should humans be doing and what should AI be doing don't have humans doing math it doesn't make any sense like we're not gonna you know we're not doing math on launching Rockets to space anymore because why why would you do something like that it's the same thing here if it's mathematically solvable offloaded have ai do it have people thinking about what should the creative strategies be at high levels right we'll talk about AI in Creative generation we're getting there but you still need to prompt these things you still need to tell it what to look at you still need to build repositories for it to look at you still need to look at emerging channels that you can't automate and figure out how to

tackle those so humans are in a way I think we're being freed up to do much more interesting work then sit there and figure out what we should be bidding on an audience in you know UK on iOS like why would you want to do work like that it's you know it's not interesting and it's not very valuable this is why you don't want to race a Ferrari right like in a month the AI can look at 1.5 million data points on average it can take 2 000 actions it would take a human if a human could do this kind of work if you could retain an employee that would do this all day it would take them 400 hours to do that so again this is not something that is feasible but it is when you use AI to do it so like I said the you know strategy is still much better done by humans uh new channels like what should your Tick Tock strategy be what kind of you know what kind of influencers should you be engaging uh AI is not going to be doing any of that anytime soon I didn't used to have time to think about this kind of

stuff and now I do and I'm thankful for it and then thinking about you know creative at a high level what should we be communicating yeah we can generate um creative like we can you know we have text to to images and all that but we still need to come up with what we want to make and that's what I think people should be doing with their time so that's kind of where we are that's a quick summary of what I'm seeing right now um I think it's really interesting to talk about where things are going so we have three uh modules right now that we're using in in AI the one I talked about mostly is governance AI that's mostly like figuring out bid and budget uh kind of stuff this is what I think is most advanced right now and it's doing a pretty good job and that's where we're getting like the 60 row as Improvement we're just starting to get into targeting AI That's the stuff about like finding the gamer grandmas like we're still pretty early on that the creative AI like it's between the last time I gave this talk like the whole lensa AI

thing exploded so people understand a little better what's going on here I'm going to tell you we're nowhere near having this thing at any kind of reasonable point for gaming maybe in like some uh verticals it's it's feasible but we need video and it needs to be based on what your game looks like that's going to take some time but it's already starting to get there so I think like that's going to be one of the the kind of next phases of this what I think would be extremely helpful is is testing so testing ads right now we make a bunch of ads and we put them in the wild and we have users we have you know customers basically like vote with their clicks and their downloads on what the best creative is well why do we need to do that why do we need to spend all this money doing that can't we create AI that will actually test our ads and mimic humans uh I think that would be incredibly helpful it'd be much faster to make ads that way and be a lot cheaper so I'm pretty excited I think this is actually one of the earliest things we're starting to mess with this kind of stuff and it's great because you could you can do the AI test and then

you could do a human test and you could see if it's predicting and you could train those models so I think that's going to be one of the the first things all right so content creation uh yeah like uh stable diffusion make a very scary version of fill right like I don't know why these things don't smile but that's uh that's coming to our games when you need to make like if you need a like an ancient Beast fill like this is yeah I'm available to be licensed for games I think um so content creation is going to get pretty cool right you're not going to need to uh like if you want if you have a game model and you want to put a different outfit on it you don't need artists sitting in some studio somewhere cheap and doing this this you should be doing this with AI um it's actually getting really interesting it adds what I think is going to start happening what I'd like to see is we'll just plug AI into the game and we'll see which are the most compelling moments in the game would it like the emotional turning points where's the retention the highest we can take that content and start making ads with it we don't like that would be

really useful data and we already have it we already know what the interesting parts of our games are but the marketers and the kind of developers can't you know like for whatever reason we don't have those conversations as much as we can as we should and now we don't need to have those conversations right I think that's that content creation is coming I don't know if you guys have seen this this is opening all kinds of interesting questions but um there was an art show I think it was in Colorado and uh this is like stable diffusion AI um I think mid-journey won the the contest and it opened up all this kind of kind of shitstorm about like what is art is you know what's art really right like and these are interesting conversations we're gonna have but it sort of doesn't matter like you're either gonna drive this bus or you're going to be under it so whatever you know whatever is going to happen I think you should be driving the bus on content creation and kind of figuring out the the ethical and legal questions as you go so just as far as content creation every image almost every image in this

presentation was actually generated by AI so I don't like have to go to my artist actually we have like a whole team that creates presentation art like I started wondering what if I don't even talk to those guys what if I make my own art I almost did it um remember there was a chess pyramid in there I tried to make a chess pyramid it gave me this kind of stuff I really couldn't use it so my art team made that but this is like the point right like 90 of what you need for stuff like this can already be done by AI so how much easier our job is going to be how much more time can I spend thinking about what I want to talk to you about versus making these images that's what I think is exciting so all right the back to the three takeaways that I want you to take um be doing something with AI now right because you'll be you'll be running a foot race against Ferraris uh human work is going to get a lot more interesting uh yeah like there might be some you know Less jobs but they're going to be more interesting jobs um and finally I think we're only at the beginning right I think we're going to be inventing this future for the next couple of years so it's a really

interesting time um I should mention Eric my uh business partner wanted me to mention that if you want to come tell me that I'm crazy and then this is just insane stuff we're at Booth g19 so come come tell me what you think about this and what your experiences are so far thank you yeah I'll be happy to take yeah it's got to be some questions about like the ethics of AI and like I'm yeah um thanks very much for that um so far how have you had pushback from team members who have found their roles becoming obsolete and if you haven't how do you plan to manage that pushback and keep people motivated no that's that's a really that's like the core of the question right um what's interesting is my team has grown while we've been implementing uh the AI models like and it's just the nature of their work has changed so I

mean I think if you look at what's happening in the tech world I think AI is not the driving Factor behind job losses I think there are other factors at play but it is a legitimate question I think I think there will be fewer kind of traditional AI drop like uh UA jobs sorry the aiua sounds so close I think um I think it's almost like we're not going to stop this progress it's just a question what I tell my team is your job security is your skill set that's all it is it's not this job here if you could keep advancing your skill set if you become the best at using the tools that we have you're always going to be employable so that's all I could say I mean we've been dealing with this since the Industrial Revolution

[Music] [Applause]