Keynote: AI Trends in Performance Marketing - Phil Shpilberg from GamechangersSF

[Music] thank you [Music] I'd like to welcome to the stage please to talk about AI Trends in Performance Marketing from game changers SF uh Phil Spielberg now do we ah there we are Phil Phil thank you very much come on down and it's a round of applause for Phil from game changer [Applause] set up all right not a lot of switch over time so let's go foreign so I'm talking about AI today as it uh relates to user acquisition but I'll talk a little bit more broadly as well talking about AI right now is really interesting because there's so much going on I feel like every time I do one of these I'm sort of Reinventing it

because in the last month since I've given the talk last so much changes so in in essence like I'm trying stuff out on you right now so bear with me as I talk about this so um I've been in games for about 20 years I've worked on 400 different games mostly on the marketing side also a little bit in production when I was at uh 2K Games I come from the traditional kind of console or mobile a console and PC space before moving into Mobile I started Game Changer SF 10 years ago Game Changer SF is a advertising agency for mobile games we've specialized only in games since uh since we started we started to develop a lot more technology in the last few years a lot of our customers started to take user acquisition in-house so we ended up having getting more into the tools business and providing tools to our clients for stuff that we've developed for our own use and so the evolution has been more towards in-house UA and so naturally we were moving more and more towards tools and about a year

ago we started working with a company called Pixis who had some really interesting AI technology like they raised about 100 million dollars from SoftBank on the strength of their technology and it was a really interesting combination because the guys that founded it are in India and Bangalore they were all Gamers right they all kind of dropped out of college because they were playing so many games and they sort of had to start this company to like make a living and they've done pretty well with it but they were dying to get into gaming but their technology at the time was really focused around like online retail banking this kind of stuff so they acquired Game Changer SF to get expertise and access to gaming and for the last year we've been uh we've been implementing AI into gaming and so what I'm here to talk to you about is the the kind of stuff that we've seen in last year from doing a bunch of games and converting from sort of like the old school you know human managed user acquisition to incorporating a lot more AI into it and kind of talking about our

successes and failures like we've seen some things work great some things work not as great and so I'm going to try to sum up what I've seen and what if you see this a month from now it's probably going to change because we're seeing stuff all the time so that's kind of the background I feel like no discussion of AI is complete if you don't start at chess because I feel like this is where like engineering nerds try to figure out if we can get computers to think more like people and there's nothing better than pitting it against like a chess champion right so it really mimics how things have developed in AI back in 97 I remember this I was like graduating college uh deep blue was competing with Kasparov who was the chess champion at the time and they played a few times and he kind of won easily but this was like a turning point in 97 the machine beat him and it sort of like never turned back but the way it happened was really interesting the the way that uh deep blue was programmed they you know IBM got a bunch of Chess Masters they had

him advise and they basically loaded it with a bunch of information opening database closed data you know end games a bunch of Grand Master games and it basically thought like a human it like that's the way you train a grand master you like memorize is all these classic games and then you you know you look at a situation go has this happened before if not you start calculating and so this AI sort of calculated its way to it and kind of brute force it could beat a chess champion that was then today it's completely different the top computer programs what they do is they don't know anything about Chess except the rules right they just learn the rules and then they play themselves for about like 10 hours and then they get so strong through doing that playing billions of games and combinations that they Crush everything so this is a really interesting kind of evolution because this is kind of what happened in UA right initially the AI was just it really wasn't right it was rules based like if the CPI does this shut it down like kind of guard rails right and then

it then you built in rules that sort of thought like humans well today it doesn't work like that it's the next generation of AI you basically tell it your goals and you give it a bunch of Freedom right like you still set some guard rails you don't want to spend like 10 million dollars before it's optimized all this kind of stuff but the guard rails are getting like smaller and smaller and the more decision making power that the AI has the more it comes up with sort of interesting Solutions like if you watch this AI play chess like if you want to go on YouTube and watch Alpha zero play chess it'll baffle you if you know anything about Chess at first you're like it's not doing anything that a human would do and yet it's beating humans and kind of breaking all this chess theory that we came up with for 300 years well it turns out that we simplify things to a great extent as humans right we are like these are some rules and things like that when AI is not bound by that it actually comes up with really interesting Solutions so that's kind of the background right and so what what's happened for us is we've

done about 400 plus game launches over the last 10 years and so before the acquisition by Pixis we were a very traditional kind of marketing Shop with a bunch of technology that we developed to kind of like create ads we have multivid which is a really interesting multi-variate uh ad creation and testing platform but we were largely doing things the traditional way we did you know creative Services marketing strategy user acquisition data science all that stuff about 12 years ago we got access to this really interesting AI technology from Pixis and we started deploying it so the first you know really interesting thing uh that I want to tell you before we get into this is we realize that gaming is pretty Advanced when it comes to user acquisition these guys were getting ridiculous results in uh in like finance and online shopping they were like oh we've improved Roi Roa has 60 70 tell your clients that's what's going to happen turned out that the reason that was happening is a lot of those companies were running their stuff

pretty badly right they were throwing in that's Facebook throw some interns on it right and or they would just let stuff run and never make any changes until it just became terrible well that turned out that's not the way things work in gaming people are actually pretty good at this and they really focus on it so we do see row as Improvement but it's nowhere if you saw this presentation a year ago you would see like 50 60 that was outside of gaming in gaming 10 15 20 is a pretty good result so with that in mind um what I want you to take away from today like if you don't you know if you don't walk away with anything else these are the three things I really want you to know uh that I've taken away from AI in the last year like you really should be experimenting with some sort of AI right now in your user acquisition I'm not saying turn it all over I'm not saying even maybe give it most of the dollars but you really need to start like playing with this and using these tools because like the smartest Market out the marketers out there are doing it and you're so you're compete like we're all competing for the

same Impressions and all this stuff so definitely start messing around with it and start understanding like what it looks like the other thing that I'm finding is our team has actually grown significantly since we started implementing AI but what we do right now is much different than what we did a year ago AI like yes it can get better results but it's sort of the day-to-day like do I spend money here or there which channels do I choose it does it much better than people do it because it's kind of Ruthless calculating Arbitrage right and you don't really want people doing that what you want your people doing is thinking kind of like getting back to the basics of marketing like what should our strategy be what do we want to tell people about our game what should the creative look like what influencers do we want to work with what user generated content do we want to encourage what new emerging channels are there what are the Trends they have more time to go to conferences and learn stuff like that's actually uh what a I think a lot of the benefit of AI is that that it frees you up from the really mundane parts of user acquisition and the other thing is we are really

like just taking baby steps right now I'm gonna give you where I think some of this stuff is going from what I've seen and it's like it's pretty exciting it's gonna be pretty interesting for the next couple of years so first of all um you know like this is what I told you it really is kind of a no-brainer these days to try some kind of AI like you don't have to install sdks anymore it's pretty codeless at this point so if you you know if you wanted to work with Pixis I imagine one of our competitors you could just sort of plug this in right it runs off attribution it's uh you know in our case it's a plug-in to the browser that just creates a new column in your in your ad groups uh puts stuff in an AI group you tell it what the goal of that group is right and it'll try to achieve it pretty straightforward um we generally see a row as Improvement you know it's not always dramatic as I said but generally it'll find a row as Improvement the biggest thing I'm finding is it saves a lot of time and resources it kind of takes away the

mundane work like you know you realize sometimes when you lead UA teams that like sometimes they don't look at stuff in a long time when they should look at stuff AI doesn't take breaks it doesn't like need weekends it gets sort of you know look at stuff all the time so just kind of what does AI look like today like I'll tell you what Pixis AI stack looks like because that's the one I'm most familiar with we currently have three modules that run on four channels performance AI is like the bid and budget management it'll decide like which campaigns which GEOS which platforms which networks should get the spend and what you should be bidding to the extent that that's the way the network works right what should you be bidding and it's pretty efficient we get a pretty good boost uh with this the targeting AI is a really interesting one uh in channels like obviously not Google but other channels where you have a lot of targeting options it'll figure out like behavioral data and it'll create new new audiences right this is one of

the more challenging things to do to keep coming up with new audiences and finally creative AI this is the one that's like the most potential in the future right with generative AI but generative AI in games isn't really far along yet because first of all there's a whole bunch of legal issues that have to get sorted out right you can't like if you create something on the you know mid Journey can you put it in an ad murky right but the bigger problem is it doesn't really look like you're a game and so that's sort of a problem right now and then uh finally like the generative AI is making big strides in uh in static you know in like Graphics but video we're still pretty far off and that's and then it has to look like your game again so we're we're I think like a year away from really having a generative AI that can make real like impressive ads but we can generate copy like it's great for that it's kind of like a Chad GPT type copy generator where we could load in everything that you've done and all the things that work and then we can come up with new uh new

copies so it's starting to do that it's starting to rotate creative like it has a library it knows when creative is starting to get kind of stale and it rotates in new stuff and it'll turn it off if it's not doing well like all that stuff is uh you can now do with AI but when we think of creative AI I think we're thinking of General of AI right so finding new audiences is pretty amazing like we've one of our early clients we started running uh targeting Ai and they kept telling us well that's not our audience we're like it kind of is your audience but like this is the whole point it finds non-obvious audiences like the kind of overlap with other audiences we get less targeting options now but it's really important to constantly be creating new audiences and targeting them well this AI can create sort of market like micro audiences test and then move budget around to the best performing audiences that's a pretty uh pretty like good application of AI today and then um like I get a lot of questions about what should people be

doing like are we going to lose all these uh UA jobs that you know it is pretty clear that AI will do certain things better like do calculations uh it lacks bias on audiences things like this it's not you know even though it can make creative it's not creative right it won't uh think about things that it's never seen before that's where people are really good it won't tell you what the strategy for your game should be so marketers it's sort of it's funny like a lot of old things are new again like uh marketing mix analysis is kind of coming back around and uh people who haven't been in marketing for a long time it feels really new well you know that thing's like 50 years old so and it's a great tool but it just wasn't being used when we had this kind of straightforward Performance Marketing stuff that we were doing and we had all these great targeting options so that's like the stuff that our UA people should be looking at what are these classic tools or new tools that we should be implementing instead of kind of calculating where your budget should be spent so I think this is uh where human work is really interesting right so

better strategy try new channels try new creative on those channels work with creators um and you know provide better strategy that's where I think the future for humans is so what does um what does AI look like when you're running it so this is like a month in the life of AI for a typical game um so 10 to 20 bro as Improvement is what um we can expect when we plug in AI it takes maybe 60 to 90 days to get to that point uh but it you know it's pretty consistent and it'll kind of stay like it'll stabilize better than what most people can do on those channels that I showed you which is I don't think I talked about it but it's for us it's Facebook Google Apple search and Tick Tock and those are just like API channels where you can make real-time decision decisions based on data coming in um so you know AI can analyze an incredible number of data points that people just would not be able to do I think like in this particular game it saved over 100 hours of human work so

that's you know again like what can you do with 100 hours that you didn't have before this is like the interesting part um interestingly it I looked at this it recommended 20 000 actions um the way the AI works is once you kind of you know you go through like a training period and then you look at its uh recommendations and you could say yes no after you kind of hone it in after everything over 80 confidence interval when it's like I'm 80 confident this is going to make a this change is going to improve whatever your goal is row as then it'll make the decision so it'll suggest 20 000 actions it'll take like a smaller percentage of them few hundred actions but that's the way it works it kind of and you could see the the other interesting thing is there's a lot of black box AI out there like Facebook and Google won't show you what's happening when they make these decisions when you have this this is meant to be on your side right so it's when it's making decisions you could actually see what it's thinking and why and that's one of the things I like best about having kind

of AI on The Advertiser side is when you make when you see like a recommendation you can see what's behind it and you you know in some cases you could override it you could tweak the system but you see its Brains it's not just doing stuff so okay what today and when I make these slides it's always a little dangerous because it's possible that in the last few weeks like you can do something new but broadly what I'm seeing today what can AI do in user acquisition it could automate bid and budget and and budget allocation fairly well among the channels where it works it does a good job finding new audiences it does a great job generating copy and it does a decent job generating images like if you're looking at uh the stuff I have in this presentation you'll get an idea right today what AI can't really do is automate all channels right so there are still a bunch of channels that like you that you don't have an API into them and so the best you could do is like maybe you know download data put it in the AI

and have it give you recommendations but that's still not an automated kind of AI thing and that's going to take a little while as I mentioned they can't really generate video ads for games if somebody can tell me about it but you know I've seen early experiments and I think we're still a year or two or more away from being able to like plug in and make a generative video ads for games I think that's coming but I I don't think it's there yet the other thing is self iterating uncreative so we have a system called multivid where we do this but it's not automated right so what I envisioned one day and I'll I'll give you like what I think is coming next but I think there's a lot to be done with uh with AI that basically looks at performance of ads understands why an ad is working and then makes changes on the Fly Ai and improves performance in that way so um that kind of brings me nicely into what I think is coming because I think this is like this is the really interesting part of what what do we think is going to happen in the next

year or two so one thing we waste an incredible amount of time and money on is testing creative so we make a bunch of ads and you'll hear things like one out of 20 ads work one out of whatever not all ads work right and we have to spend money testing these ads to see which ones will work well we're going to create you know we're on the verge of seeing basically AI personas that start to mimic the way people behave and we'll be able to create these Ai personas and feed ad feed ads in and then um figure out which ones are going to work and the way this training happens is you just sort of do it side by side right you keep running ads in like the channels you are spending money and then feeding the the results into AI I think eventually when we get really good at like segmenting things um it'll it'll be able to tell us which of our ads are going to perform best so we're going to stop wasting a bunch of money testing these ads okay this is the one I'm really excited about um if you've ever worked with a development team like this is back from my console

days and you try to get assets uh like video while they're making the ad you need to make a trailer for E3 used to be they're working on this game and they're like do you want us to finish this game or do you need like this trailer uh it's difficult right getting stuff out of games and then at the same time there's a lot of data inside games about what actually players engaging in what are those moments like those big moments that you could share in ads well I I think we're going to be able to plug in AI into game engines and feed like data about what's happening and then extract like the most interesting pieces of that game and be able to use it in advertising without having to bother the developers who are rightfully working really hard on making that game great um I think that's going to be really interesting when we could start like figuring out exactly what in the game works and feeding that into the marketing I think that'll be one of the more interesting developments in the next few years um on the really fun side of things deep fakes I think think are gonna become bigger and bigger but not in the way

that like people are warning us about right I think um there will actually be really interesting endorsements and things like that which will be able to happen without with the approval of like personalities but without their necessarily their participation um it's really interesting like I've been talking to some friends uh that are in the VC community and they're experimenting with some of some of this like producing content like there's some people that are very famous and they have very busy schedules and they want to do a podcast they would love to do a podcast but sitting down and recording a podcast takes a lot of time and you know some mornings you're not into it like you're not feeling it you know deep fake AI feed you know 10 000 hours of a person speaking it could crank out that podcast and it'll sound great they'll be excited every time and that's going to have a pretty big implication in marketing right it'll make our life I think easier in some ways harder in some other ways but it'll be it'll be an interesting World we're living in whether the fake stuff it's getting really good it's getting extremely good

um so the the final one I think is the um it is sort of mundane but it's interesting like our multivit technology today every ad we create we create a bunch of variations because what happens is like traditionally be testing you usually have completely different ad Concepts you'd have like you know one kind of ad that had one message and then another had another message but we started to notice uh when we were writing these ads is actually a lot of the small creative decisions that didn't have to do with messaging but simply what you put first or like which character from the game you used or which game piece of gameplay you used had tremendous uh effect on the way the the ad performed because we live in this world where people are scrolling through their feed and they decide in the first two or three seconds if they're gonna actually watch this thing so what we put there became incredibly important and it was kind of separate from the message it was more about like visual novelty like what would get people to stop scrolling so we started to create every ad with a bunch of variations to make sure that

our Not only was our message incorrect but the creative choices we made to get people's attention were correct well that's like pretty time consuming and expensive and what I was talking about with real-time kind of being able to change these ads is I think we'll be able to like load in a bunch of assets and an ad will start running and you'll start getting feedback and the AI could actually make adjustments to that ad as it runs until it starts performing better or it'll just like turn it off like it's not it's not going to work but that's going to be pretty interesting and we are like experimenting with this but the automation of it's going to be you know it's when we get the generative AI that could actually make the videos this is going to be a really interesting part of that so I don't know if you figured this out but every single image on this was created by AI like no artists touched this is probably the nicest presentation I've ever had and like like this is not what my artists are spending time doing they're making ads so that's pretty cool I think you know um so anyway the the three takeaways uh

that I want you to walk away with again is AI is like in the very early form right now but it's really interesting so definitely start experimenting with it um humans do some really interesting stuff that we should get back to doing like figuring out classic marketing and how it applies here and let the AI do the heavy math stuff and uh yeah this is just the pre-party I think it's going to get pretty interesting in the next few years I'm pretty excited all right thank you guys [Applause] [Music] [Applause] [Music]