Delighting your players with live ops

[Music] yes I represent the Amazon Appstore AWS Amazon game check unjust Amazon and I've got the best job in the company I get to come and meet game developers worldwide hear about what's not working from you guys take that back to see out of the where buildings full of really smart people working these problems and if we come up with something that may be a solution I get this slide back out here and talk to you guys about it that's pretty cool and today I'm just kind of want to give you a brief intro with the track with live ops and kind of providing your customers live ops the first thing I want to do when we kick off the LiveOps discussion its kind of

talking about what why box isn't so why pops is not you know brand new game mechanics it's not creating downloadable continent that's content not necessarily operation it's not bug fixes it's not feature deployment they're not future development but for every one of these live ops isn't there's somebody's going to argue well yes in some cases it actually is or the line ops team might actually deploy that for example in feature development you can bet the best feature development teams are going to get a lot of input from their live ops team to find out what they should really be developing as far as effective features for the next release so perhaps it's best just to say what live Ops is why box is making your existing game code the game code that is currently in production with your customers the very

best that it can be so keep that in mind for live ops as we as we move forward now what does a live ops operation typically look like what are some of the things that lime Ops announced assault well I've seen live Ops teams asked to increase revenue to increase retention and engagement to decrease churn all of these things so it would be understandable if you thought that somehow live ops was going to be this magic bullet it's not LiveOps is not take a terrible game and all of a sudden make it profitable with no churn if the game isn't fun LiveOps is not going to make a whole bunch of people play it it's gonna make whatever game you have out there better but it's not going to turn a bad game into a good game now what levers do people have to actually execute on those tasks I've seen live ops teams create events and I'll actually show you some examples of

that a little bit later they affect game balance they can do player management they could work on in-app purchasing they can work on a be testing and most of these are self-explanatory but I want to drill down on two of them the first one is game balance game balance is a great lover for your live ops team if you can give your live ops team the tools they need to use what do I mean by that just telling your live ops team the game balance is broken you need to fix it doesn't help them I mean what are they going to do now if you've given your live ops team the ability to change the strength of armor to change the power of a weapon or the speed of a player now your live ops team has something to work with so if you want your live ops team to be really able to execute on the tasks that you're giving them give them variables that they can work with give them the levers that they can move to actually impact

what you want them to get done so how does that work with in-app purchasing I mean we've all submitted games to app stores right you've got to submit your IEP catalog you get a SKU number you submit the price all that stuff is locked in stone in an app store so how much can you really tweak from a live ops perspective what I want you guys to consider doing is giving the AI ap catalog presentation variable so that the live ops team can control what the paywall looks like you may not be able to change a blue vorpal sword from $0.99 to four dollars but if you have a $0.99 blue vorpal sword a $2.00 red vorpal sword a $3.00 Greenville sword then your live ops team can decide which vorpal sword to put on the paywall for different cohorts at different times giving them the power to go ahead and tune monetization approaches with the I AP catalog a lot of developers don't

think about that because they don't think about the levers they want their live ops team to have after the game goes live I'd like that to stop today right now and I'd like you to think about the levers you're gonna give your live ops team in operation all right how do you actually really excite your customers and really delight them with live ops it's in the execution and good live ops execution starts with data a lot of data data not just about collections of customers not like daily active users I mean you know DARP who is nice but dark who doesn't tell you how to fix itself you can scream to your live off steam I have too much churn what are they going to do about that now if you tell them that 80% of the players are churning out while they're grinding for soft currency that's useful if you

tell them that 80% of your players are churning out after they finish one level before they start another that's useful so what I want you to do is I want you to gather metrics on player behavior not aggregate player revenue or things like that yes that's what you want to improve that's one of the KPIs you'll use to decide if you're successful but I need you to get to the layer beneath that I need you to get to the player behavior level so that you can give your live ops team data to work with so they know what to change it sounds daunting don't worry we have presenters who are coming on later in this track to tell you how there are already existing tool sets that will give you a lot of this kind of operational ability out of a pre created package so stick around in this room yet you'll get that data so after you've figured out how to get that data executing live ops well is is how you're gonna finish up with that and creating delightful moments so yes you can change the game balance

but what do you do for example in the case of players churning out while they're grinding for soft currency how do you make that experience better well obviously it's a well if it weren't a bit of a grind they wouldn't call it grinding for soft currency would they right so how do you make that less of a bad experience how do you make that less unda lightful if you're live ops team actually has some access into the contents of loot boxes one of the things you can do is hey we've noticed that you've actually opened 50 loot boxes without taking any damage congratulations you've just earned a something special something special could be something free from your in-app catalog it could be a couple power-ups zero cost of goods sold for you guys or it could actually be an actual real physical good something tangible and I mean I work for Amazon we sell all kinds of real physical tangible things we'd be

happy to work with you on how to sell widgets or gift cards or whatever but it wouldn't it be delightful if while you're grinding for soft currency all of a sudden it acknowledges that you're grinding and you're grinding well and it gives you something as a result that's kind of cool right certainly unexpected and that might very well encourage them to keep grinding and not churn out so let me give you a couple of examples about how apps that I've worked with have gone ahead and gotten some of their goals accomplished with live ops the first is a game called ostrich among us it's a cute little thing where you have to pattern match the dance that ostriches are doing I'm glad the photographer's not catching me do and there's a photographer again they do they do this cute little dance that you guys need to match and if you do it well you get high points it's a casual game but they were noticing that they were having trouble getting users to try it and find out how addictive dancing dancing like an ostrich really is so

they decided that they were going to have a very special one-off event for new players we were at a game conference in Moscow and they created a tournament just for the people who were there at that conference okay if I created a special tournament just for the people in this room at this presentation well you'd feel you were part of a specific group a certain crowd if I gave you a code to enter into a game and only you guys would be playing for a specific prize well if the prize were kind of cool and it could be an app currency or a gift card or whatever chances are you do it and they did the ostrich amongest team got hundreds of plays almost I think 800 or 900 plays of this game during the day of that tournament it was really impressive to see how many new people came on board and still played the game so that's a

really good example of how the ostrich amongest team three people in New York City use live ops to actually improve their user acquisition next example a game called ghost pop ghost pop is a little bit larger team but it's a fun game where you run around to the flashlight you've got to swing the flashlight to hit a bunch of ghosts that are coming at you ultimately from 360 degrees and when you hit the ghost with your flashlight they pop hence the name of the game and they needed to increase user engagement because they weren't seeing people playing long enough and have enough sessions per week so what did they do they figured out that some players are really good at getting a ton of distance down the down the track other players are really good at killing every single ghost and not just outrunning them or dodging them some players were really good at ramping up a huge score even if they didn't get the farthest or kill the

most ghosts so what do they do they said well you know what if you happen to be a runner we're going to create a contest just for you if you happen to be a really good ghost killer hey we've got swarms that'll help you and for example here's an example of the screen that they put up massive hard rush if you kill a ton of ghosts and you know that's what you're good at here's a year's event that you might actually feel really good about entering because you know you can kick but in this particular part of the game so you'll go ahead and you'll enter that one how many waves can you survive how far can you go and the nice thing is these are programmable recurring events so the live ops team doesn't have to do a huge reset they just set this in the tool that they were using and it just runs everyday when I pulled the data for this there were over six thousand people who were actually enrolled in and playing in these events at the time of their one-day events and there was still like

five hours left for people to play now I'm not saying that those six thousand people wouldn't be playing the game if it weren't for this particular live ops operation but it would be a lot less likely we think and their data certainly suggests that's true so this is another example of how they're delighting their players by offering them an opportunity to do what they love about the game great tool great way for live ops to run events a little more detailed case study is from a hidden object game called seekers notes by my Tona my toda is a bigger company they're not kind of small like the other two they've got over 300 people they have the top ranked hidden object game with over 70 million downloads that's kind of big for a hidden object game but it takes a lot of resources to actually make that game as a result they really wanted to find a

way to engage existing players more get players who were going through the tutorial to engage more reduce churn and actually reengage players that had already turned so what they decided to do was create again a series of special events that the live ops team could go ahead and create and tune and then offer rewards that were tuned to the player cohort that they were really interested in capturing or recovering so here's kind of here kind of the steps on how they did that and the first thing they did was identify the cohorts now it's easier way to identify hey people who have turned out that's a great cohort only problem is it doesn't tell you how to get in touch with them does it it doesn't tell you how to reach them or how to contact them so they needed to do a little bit of homework when they were developing their cohorts and figure out okay we're gonna have to go and message to our

Facebook group hoping that some of our churned players will still get the Facebook announcements and actually we know what our demographic is in that churn group that we want to reach so they actually bought some paid advertisement that they knew was going to be pretty successful at reaching players in the cohort that it turned out so they figured out how to get in touch with their cohorts second they decided on monthly events and tuned rewards to what they thought the business value of that particular cohort was and then they went ahead and they rewarded the players who actually completed engagement to the satisfaction of my Tona so only when players reach the level of engagement that might onnum wanted to see did they actually incur any capital of expense of fulfilling any of the rewards and what were the results of this as a result of the live ops team managing some of these special events they are increased with new players in

the tutorial average revenue per user by two hundred and ninety percent 2.9 times the revenue from new players just by getting them to engage in a special event at a level that was appropriate for them with a small I think they were offering a $5 Amazon gift card well worth the profit I mean the the it was certainly worth the effort they put in of their daily active users 41% more of them were actually payers as a result of that event pretty cool churned players they got a lot of church players back in the game by offering them an opportunity to participate in a rewarded event and that increased their average revenue per user in that cohort about 220 percent active players they broke active players into two groups active players who had never been in a tournament before or had never

participated in a special event before they increased our poo by 63 percent and they had 27% more payers and their daily active user list as a result of that then they had players who had been in monthly tournaments before and had done the monthly tournaments with seekers notes before when they added a prize or award to that tournament they increased average revenue per user 33% and had 24% more paying customers in their in their dau so pretty good pretty good notes on this all up by having these rewarded tournaments sponsored by the live ops team again none of this took any dev team effort to do at all all of this was just a LiveOps team creating these special events identifying cohorts and figuring out they wanted to offer prizes they got retention up across the board which is a huge win for them and they ended up improving the average revenue

per daily active user by just a little bit over 200 percent for the 30 days following the event when you take a look at how much money they spent sending out five and taller five and ten dollar gift cards the worst cohort was a hundred and sixty percent return on investment woohoo that's not bad and that's all coming not from your dev team but from giving your live ops team good data to start with and good to find cohorts and good to find goals and rolls so what do I want you to do next next I want you to go out and get this book by Dan and Chip Heath it's called the power of moment and there's really important questions like what kinds of events or special things are going to occur that are gonna make your users think wow that was a great game or that was a really super thing that happened I want more of that this book is going to answer those

questions and it's going to teach you how to how to design more of those moments into your game and because you're going to create a lot of variables around this you'll be able to tune that with your live ops team really good next thing I want you to do is talk to me I'd be happy to walk you through any of the Amazon technology I showed you in the course of the case studies are the examples if you actually don't really want to come up and talk to me but just want us to send you stuff go to bitly amazon engage and we'll share with you all the different stuff that we're working on to help you engage with your users and help your users engage with your games guys thank you very much for your time I appreciate the attention [Applause] [Music] you