AI growth hacking

[Music] okay hi guys I'm Justin chap Wes key with improved at AI and I'm on a quest to okay this the next slide I'm on a quest to discover the perfect player journey I believe inside each game there is a version of that game that leads to maximum player enjoyment maximum player engagement basically for each game there is its own perfect player journey and I think we can discover that using artificial intelligence but first I'm going to rewind 20 years IBM built a supercomputer named deep-blue designed specifically to take on chess grandmaster garry kasparov in a series of chess matches of course the grandmaster lost and humans pretty much haven't stood a chance ever since at the game of chess now it isn't fair to call deep blue artificial intelligence it essentially brute force the game of chess looking at millions of moves that

would defeat the grandmaster and so even after this defeat humans pretty much assumed that we would reign supreme at games involving higher-level reasoning intuition or even things like gut feel well that was until last year google's deepmind group developed a piece of software called alphago to play the game of go pretty much every go master that has been put against it has defeated and at this point even only a year end we're pretty much assured that humans will never beat computers at the game of Go now this actually is machine learning this is using artificial intelligence and learned to play go by playing many many many games against itself it exhibits higher order reasoning pattern matching and during game two at least so little last year another master commented I've never seen a human play this move so beautiful so beautiful now so now we have computers showing maybe some sort of creativity now the criticism we could leverage towards alphago just like deep blue that it was designed to play a specific game unlike

humans that can't generalize and play lots of different games well it turns out that that same group at Google using software very similar to alphago when trained on a series of Atari games was able to achieve better than human performance on a majority of the Atari games that it that it was trained on so if we map this out going forward almost any game that we can train a computer on its going to be better than humans at pretty much determinator Tsar definitely going to win now what does this all have to do with the business of game design well I would argue that the business of creating games is itself the ultimate meta game we can think of a game as a sequence of experiences that the game is presenting to the player if the game makes the right sequence of moves presents a fantastic sequence of experiences it will be rewarded by that player with higher engagement 5-star reviews more revenue you know and

referrals to their friends if the game plays a poor sequence of moves if it presents a poor sequence of experiences to the player there'll be a higher turn you'll get those dreaded three-star reviews and you know it won't be a success financially but so how do we make this a little bit more quantitative in a way that we could start applying artificial intelligence to this well I think there's a fantastic framework from dave McClure called the pirate metrics framework who here is familiar with pirate metrics anybody heard of this oh cool okay pirate metrics is an acronym Stan it's a 8 rrrr standing for an acquisition activation retention revenue referral you're probably heard of those things so again ARR R or R say it with me R yeah there we go all right this gives us a way I think to start to quantify what the perfect player journey might look like it's actually fairly comprehensive in that even with these five simple metrics if we look at the players ability the

players enjoyment of the game can be measured by retention can be remote measured by the referrals or reviews and of course our financial success is being measured by the revenue now none of us want to be slaves to some sort of AI Overlord masters that are just telling us how to design our games so I think there's another way we can think about this we can essentially deploy artificial intelligence as a sort of cybernetic enhancement and artificial intelligence prosthetic as you will that we can outsource our brains and allow it to take care of tactical details of game design and well we focus on the stuff that super fun story character look and feel etc a good analogy for this there's a tres chess trainer app that I'm playing the white character and there's an artificial intelligence in there that makes few suggestions with these green lines now this is in real time and I'm essentially outsourcing my chess cognitive capacity to this AI to make moves fairly quickly

and play a fairly strong game of chess now I don't have to do what the computer tells me I can take creative risk and try new moves and the artificial intelligence will act as safety rails to kind of bring me back on track so I think this is how we can deploy AI in our games to take care of tactical optimization for us while we focus on the higher level details so to step through the pirate metrics framework I'm going to look at each one and show how we can deploy artificial intelligence tactically today so acquisition acquisition is all about finding the best players for the lowest cost who here is in marketing or user acquisition anybody anybody anybody okay so the user acquisition people already have a prosthetic when you take out ads on any of your search ad networks it is learning which creative is performing best showing that creative more often some of the more sophisticated ones can even tailor that creative for specific

audiences now if we want to get a little bit more sophisticated we can understand that Facebook look-alike audiences and Googler Google similar audiences are a machine learning algorithm you train them on a target audience and it matches other players that might match that audience so one way that we can use this as sort of a hack is to create look-alike audiences segmented by customer lifetime value so we can take our existing players in our game segment buy high medium and low pay pairs or even fast payers so the people that generate the most revenue in the first 24 hours create look-alike audiences based off those segments now what this allows us to do is be relatively sure that we can spend more on our customer acquisition on our higher paying segments than our lower paying segments the nice thing about this is we don't have to get into really complicated lifetime value modeling which is actually quite difficult once the the player engages with the creative

and goes to the App Store now is the next step in their player journey so even though they haven't played the game yet they're still experiencing the brand of that game there is an opportunity to use machine learning here and optimizing app store presence there's two companies store maven and split metrics can both do optimizing your app store images etc and again they use machine learning algorithms to do this so again we don't have to be machine learning experts we can outsource that part of our brain to these these companies next step in the pirate metrics framework activation activation is all about delivering that aha moment to more players so getting them into the game realizing how the game works the first thing I would say here is again we're looking for the perfect player journey the perfect sequence of moves so almost nothing in the first hour first day first week of that player journey should be truly random all right it might look random but we can use machine learning to start to find a sequence of experiences that's going to really get those players engaged a good

example of this would be in like a match-three game where there's a bunch of different levels the person is engaging with we can use machine learning to figure out which order of levels leads people to maximum retention through that activation phase so basically using AI to script that that player journey retention this is all about keeping more players coming back this is really the core of your game and artificial intelligence is going to be less important than your core story your core dating mechanics etc but this core part of your game is also going to be where you're taking the creative risk and trying to find those exceptional experiences that really keep players coming back you're really looking for that Unicorn content that people love to engage with and I think this is where that analogy of that chess trainer comes in you can take a lot of creative risk and then use artificial intelligence as sort of safety rails to get you back on track so if you create a stinker of a

level a I can guys that get rid of that all right and I think it really frees up the creative process so we can have higher creative velocity and take more risk if we get tactical tactical on retention many games have daily retention bonuses you have all these different items all these different numbers there is no reason that human beings should spend a moment thinking about this kind of stuff let the computers figure these kinds of things out so we can focus on the core creative work going into revenue from my perspective revenue is all about not leaving money on the table we might trick ourselves into thinking that we can just keep growing per player revenue but the truth is there's going to be a maximum willingness to pay for each player based on how much time they're engaging with the game how much their value they're getting out of it what their demographics are so their goal is related it just have a well monetizing game and understand that there will be some ceiling to that so things like basic price optimisation you can even get into dynamic pricing if you

don't like dynamic pricing you can get into dynamic discounts dynamic incentives for your incentivized ads dynamic ad timing anything that is involved in monetization there's probably a way to personalize that for the audience that's coming in in Angry Birds too there's a great example of this you have this bonus pop-up and so there's different items that are being offered for different quantities there's a timer on how long this offer is available there's a price again human beings should not be wasting their precious minds trying to figure these things out let the machines figure out which offers are performing best last our referral get more people to refer more people this one unfortunately is actually pretty difficult to quantify if I recommend a game to a friend over beer it's going to be very difficult to track those metrics but there's two things we can start to look at to start to quantify this there was a company branch metrics that has a free deep linking

solution that is pretty good to be able to track refer to installs and then Mixpanel has a really neat blog post called getting serious about measuring virality and its how to set up analytics funnels both players to kind of match them together so I can just sort of give us a start for how we might close that loop but we're not quite there yet I'd say I'm going to add a final art that I think is very important is review earn those five star reviews what's what's interesting about this I think there's actually a way to quantify whether or not people love your game and that is simply to ask them do you love the game you can use this kind of pop up obviously you're trying to drive reviews to the App Store but you can also ask them on a second date do you love the game yes or no using that information you can feed that back into these machine learning systems and they can tell was that player journey fantastic up until that moment where you ask them

for that rating all right and be able to learn which place move now which versions of the game were performing best so I again please join me on my quest to discover the perfect player journey using artificial intelligence and again I think using the pirate metrics framework can allow you to think systematically about how to apply these tools how to basically develop artificial intelligence as a prosthetic as a cybernetic enhancement allow you to spend less time thinking about the tactical details of your game and more about the important higher level design thank you thank you bring us back in time so you could do a question I love to but you you refer to deep my IV I know damage recent low a low it was working on a game with them before they got bought and in some ways that's the biggest tragedy of estate mind not what

they're going to take over the universe with the Skynet the fact that their talent will have got in that team for game development is not going to make games but you know I think it's fantastic do you have any questions there's one of course can I do you expect me hi Michael phobic video we have a signature to do with deepawali ai ai I mean the Titan X is the viable machine for deep learning so given that the hard works becoming cheaper more ubiquitous and these are becoming more demanding and required what's the next step you decide you know what I do want to do AI for help design my monetization retention are all right what's the pirate's favorite letter R yeah think so but the first love is the sea yeah what's in it what are the first steps you go at your buyer Titan X which is like I said uh but easily acquired people in the machine ya know what well so the the nice thing is your hardware is easy to acquire but the experts or not so yeah so the the limitation here

is just the number of people that can do this kind of stuff and that's actually why I say look at this as a tactical prosthetic four different pieces of the puzzle you're going to have more and more companies that can do different components of this and unless you're a huge organization that can pay huge salaries for data scientists that can do this stuff you might have to wait a couple of months until the solutions are available may I do that I'm happy to help people with that but the the limitation is going to be talent and just but I think very very quickly more and more companies are putting these solutions again split metrics in store maybe anybody could use that to optimize one specific component of that yeah [Music]