Engage, Retain, & Delight Players with Live Ops on AWS | PGC Digital #1

welcome everybody thank you for joining us my name is Tabitha graves I'm the community enablement manager for AWS team tech and I have here with me the amazing Chris Finch yeah maybe not amazing exactly but I'm definitely a Solutions Architect as he mentioned an evangelist for game tech at AWS I really do want to mention that I appreciate everyone the audience giving us your time today we're here to talk about what live ops is and how you can use it to engage your players keep them around they can enjoy the game that you've made for them which is a topic that is as a game designer for myself big big important one and I hope it's one that's of interest to everyone here as well so I will stop talking with no further ado Tabitha chicken now LiveOps can significantly impact the player experience in the game that you are creating it also takes very careful planning and coordination on the backend to create that impactful experience for your players so we're going to begin we're not gonna talk about port night

well we're gonna talk about them for a second so the black hole event last year was the largest game largest live event in games more than seven million people watched across all social platforms on Twitch alone there was 1.7 million concurrent viewers and to stay relevant right now live ops is absolutely critical and having that part of your game development cycle is important it's not just important to the players but who expect fresh concen and live events but also streamers now who need new content to feel their ecosystem as well yeah that's very true and well it is important to acknowledge the success with fortnight's when you talk about live ops because they need credit live ops with some of their big successes it's just as important to acknowledge II don't need to be as big as epic or have as many players as for tonight to get results for incorporating live ops in your game butterscotch shenanigans for example they recognize the games industry is a crowded and competitive place which is something we all know because that's where we work and these guys leverage LiveOps practices to gain a competitive advantage in that space

they actually is back in automation by website api's and serverless compute specifically AWS lambda to help them locate and build daily challenges for their players so it's content that's synthesized from both our official content and user-generated concepts and this happens regularly schedule daily schedule so the players always know one to look for something new and that's important because when they always know one to look for something new they come back to look for it so that's how they keep their engagement now we're gonna look at a live out to flywheel that we created so you see in the center it starts with planar engagement it's at the center of everything and you'll see that the examples that we just discussed were basically combinations of experiences routed by player engagement in those games so basically all of those the player engagement in the middle is revolved by the user experience so at the highest level each game is fueled by a cycle of design and development decisions so you see design is at the

top developments at the bottom you know as players are playing the game that is launched whether it's in you know full production alpha beta the players then engage with that content and then you start collecting data so player data games ADA and then you're able to analyze that and then make adjustments and pivot to that information continuing the cycle of design and development now player engagement is at the center of everything and live ops is simply the bridge that connects the players or in the developers to the players to let them know that they are listening right and I mean so ultimately you're looking at how do I know what my players are doing and how do I know what makes them happy right and as a developer the important takeaway here is that for this to work you really need to have live ops built into your game design from game from day one right you can't incorporate player feedback into your design iteration if you don't have it and the best way to understand if people are engaging with their game use to analyze the data they generate while they're playing it and some of the most important considerations you're going to be making for these game for these players is going to be the first

couple of cohorts people play your games you know that feedbacks can be super important seeker not collecting that data if you're on and analyzing that data you're not getting it into your design from day one you're missing out on a lot yeah that's perfectly that's correct and there's also there's no community consensus of what defines live ops so what we did is and what we're going to discuss today is basically the pillars that we found to be most consistently called out by the game dev community so let's jump into the first one with community building now community building is a little bit different and we saw in the example above with utilizing epic games with fortnight it was something like a live event but community building in general is is fostering that community through you know social - borz specific team objectives basically getting those players to come together and explore the game with their friends so in the case of the live events that we discussed

something like that you need in order to handle something like that you need without over-provisioning you need something that would be able to scale up scale down and really meet the demands and handle that traffic of your players so something you know like on-premises servers is a good example of one way that you can handle something like that with AWS Amazon ec2 with auto scaling you're actually able to set it up and it'll scale up and down and you in it'll scale as people are connecting to your server so it really it is able to meet those demands yeah and the important thing there as well is that it's not you know it's about saving you money by not having Hardware lying around I think you're not going to use but the same time it's also making sure your player experience is great because if you don't have a card where you you can't always necessarily guess that you're gonna be the next big hit and you know I hope everybody here makes next pretty bigoted right but it allows you to not over provision which we saw a lot you know back in the day using on-premises

servers you know it's hard to tell how many resources you actually need where this is pay as you go and pay for what you actually use yep cool so why is the economy monitoring but one of the things that's important about your games community is actually its in-game economy for games especially that allowing game trader auctions a lot of players love this kind of interaction is the source of enjoyment and other players learn to depend on them what they know their names are not to provide the market for rare or bought goods within their game and understanding how the drops in your game affect that can become is really critical to maintaining and healthy in-game community as well as managing your financial expectations like from I ap when those drops can be both farmed and bought from an in-game store right so understanding that economy web can be super super tough though because it's all really interrelated and gnashed and it's kind of you know really convoluted you got a farm boss x2 get Phoenix Peaks as drops and then by miracle water from the stores you can create sort of hacking so you can farm boss Z to get a

libero closet all these relationships get hard to track as they start to cascade together and then it gets even tougher as they start to release new content and try to take make older content Marc stuff accessible to new players because you're adding more layers to that web of interconnectivity and you know it's it gets tough and in fact with aji being bought you can actually find yourself in a situation where your your your economy is is creating a loop where people can farm something and make way more money than they should be making even to the 20 words it's messing up your I ap income so one creative way of solving this problem we've seen has been to let the designer model economy almost live really in a graph database now for those of you who aren't familiar think of agrestic graph database as a basically a big 3d model right so every vertex is like a chunk of data and then every line between them every edge describes a relationship between that connected data then inside every vertex you can have all sorts of attributes and metadata and stuff like that and the edges can be pretty obstructive that's what a graph database is why that's useful is this really really fast and efficient at

showing that kind of connect to data it's showing the relationships between pieces of data so you can actually use a graph database to store all those items their relationships to each other and then make it query like show me all the connections to the sort of hacking and you'd see the beaks and the water and boss X connected to the beaks and boss a connected to what the sword needs to do to case so you'd see all these things like in one shot as opposed to adding to play test over and over to understand how they affect each other and then as you tweak the the purse to the drop rates when you get new content and you can actually see the effect of that all across your ear economy so it takes what is often an organic process to kind of check out what's going on your economy makes it much more systemic but may either your data mean right let's eat we can see the numbers across the entire economy now as far as the solution is concerned you can always build your own graph database server on ec2 instance you know there's there plenty of open-source solutions out there that you can use for that if you're not interested in handling the structure yourself you can always use like Amazon DEP tune which manages the infrastructure for you while you can the

third part of community building is social campaigns so with social campaigns like social dashboards as you are creating new content you know launching your game or maybe it's a different you know campaign you want to see and get a handle of how the community is feeling about it so being able to have that type of sentiment management you you're able to predict and actually see like an overarching view of what everyone is saying online through forums or through their other communities so one way of pulling out this sentiment analysis and creating an actual like social dashboard is going to be through natural language processing so that's you know one one of the open AI or open services that you can use to find insights and relationships we use we have a service called Amazon comprehend and that's a natural language processing service that you don't have

to build your own infrastructure for it is pay-as-you-go it's one of our managed services and you're able to see the different results from the campaign's that you are working on and really get behind and see the actual community's sentiment yeah real quick as well one of the nice things about a tool like natural language processing is it doesn't just show you sentiment behind what people are saying but it also highlights key words things are being repeated over and over again so it you can have it you know be yourself a daily report button page through and see what your community is talking about not just how they see and as well how they feel about it so that's that's actually a really good conference a great tool for that so um in addition to all of the cool community stuff you also need to know what your players are doing inside your game super important because that's how you get your raw data that's how you get your unvarnished data right people don't always necessarily know what they like when they're talking about it but they know when they're doing it so understanding how your players are interacting inside your game is super important so one of things you want to do to do that is to to encourage them to

play the game in ways that others like them have enjoyed it right so we're going to talk briefly about recommendation engines as a way to ticket people to do things they enjoy now recommendations often get a kind of a categorization as oh that's only for AP and that's certainly true if you are creating profiles of people and how they purchase stuff in your in-app purchase store you can certainly say okay people who played the game like you also purchased this thing so that that's certainly a way to go about it sure but you don't don't get pulled into thinking that you're stuck with IEP recommendations only when you're dealing with a a recommendation engine you can use a recommendation engine to suggest anything to a player that they or their character can possess or consume so get out of the mindset of just thinking about the shop right you can also do it to say see you've got a game that's got quests and it looks like do you enjoy a stealth approach well what you can do is you can check out okay other players who also played the game like you with stealthy approach stuck this quest line

out to the very end because they enjoyed it maybe you should give it a go so it doesn't have to be a IEP recommendation you can be a quest recommendation engine even though you know critter and we'll Bremen floating you're saying hey you know this guy who's got a quest I think you might like it so your player goes over there helps direct their behavior towards stuff they're going to enjoy by the same token you could point players towards items that enhance their performance you know there's a quest line or a an activity that can wind up with something that will help they play you can point them to that if you've got a PvP R and they get to this point in the quest line in the abandoned ship what you can do is maybe create a secondary objectives say hey you know I know you're trying to do this kind of thing go to this line and get to this point and then you know kind of move on from there but this will actually get you where you want to go so you can again you're starting drive player behavior towards things they want to do and stop wasting their time doing things that aren't they're not going to feel rewarded doing as always you can do this one of two ways you can run your recommendation engine on your own hardware or you can use Amazon personalized which is a fully managed it's basically an API which handles

hardware configuration and manage and maintenance and provide provides you with a single API to query against kind of all those interests for another strategy for measuring and responding to player behavior is going to be looking at threats so looking at what does normal not look like or what does not normal sorry and you're able to detect anomalies and respond to those different threats that might impact your game you know something like that is either you know fraud detection maybe player management you know player abuse something like that you're able to do this actually through a art through a IML then able to detect those anomalies and so something like identifying specific player behavior or maybe it's like a general trend so you don't actually know what you're looking for but you're just looking for a blanket of something that is wrong or something that is abnormal but you're able to do

this with for AI ml frameworks one that we have is a sage maker and that's a fully managed service again you know and it provides the ability to build trained deploy and deploy all those machine models very quickly just a real quick side note on that sage maker is definitely one of our more advanced tools and again pattern matching is really about about patterns right I sorry um machine learning is about that about detecting patterns responding patterns so when we're talking about threat detection you're talking about looking at as Tabitha mentioned anomalies and kind of groupings and being able to report on that and seeing those anomalies or groupings before you start getting reports from players what you're trying to do here you're trying to solve the problem before it becomes a problem and that's what I can help you with and then what is a live procedural engagement so one of the things you're going to be doing if you can if you're doing this live ops thing and you're collecting lots of data on your players that you're going to be getting a lot of data from them right so you need a some

kind of a pipeline to be able to accept that data and dump it someplace you can do analysis on it that's really the basic piece of of live ops is collecting a data and analyzing that data you'll you'll need a pipeline for that you can use something like Kafka you can use something like Amazon Kinesis whether you want to manage it yourself whether you want to Amazon to manage entirely up to you but both all of these solutions are really about being a central port where you can get lots of data add it and it'll go someplace where you can work on it now what there's two ways to deal with the data once you've got it one of them is to kind of look it over in regular basis right now spreadsheets do you know charts and stuff like that and say okay see a lot of people are not interacting with my new content I wonder why they kind of going in seeing who is and what point they're leaving the data tells you that right one of the things you can do though is since you have that data passing through you have the opportunity with a tool like Kandi's analytics to look at the data as its passing through the pipeline at like points and windows in time right so what I can do in near real-time is find either anomalies like

we talked about before with with machine learning you know anomaly isn't groupings are really easy to detect so if I detect an anomaly and I'm getting geographical data where my player is in its own I can say okay this player is out in the middle of nowhere that's interesting what's he doing I can kind of look and see what he's doing I can throw an alert to the system and if the system then has code to tell it how to house you engage a player who's gone off on his own is doing his own thing maybe a dynamic encounter maybe an encounter within you see that you've wanted to have but only two very specific small group of people you can also do groups of people I've detected an ad hoc group of people over here by this by this the city you can launch an encounter at them so you can actually do not just um rant of design iteration like we talked about early in the process but you can actually react to players in how they're playing the game right as they're playing it and engage them and you know give them more options to play as making in a way to make the world more more alive as they're playing it so that's that's where your veteran live procedural engagements again the tools you're going to look at either something

like a Kafka or real-time analytics pipeline or Kinesis which allows you manage that stuff you turn after an hour last pillar is going to be new content and features and all of this this last pillar is actually a culmination of the two pillars that came before so through community building analysis of the sentiment and everything that's going on to managing and measuring player behavior looking at the effects of everything the new content and features is is the effect of analysis and everything of those so for the first one with promotions you're looking at something like seasonal events so if your game has you know in eSports keep it capability to it to where you have different seasons or maybe you have in your game you know different holidays you want to be able to see and promote and differentiate those type of events

from you know other times during the year something like an event calendar for players that you can create with different campaigns but with promotion that's all about you know push notifications so sending notifications of updates or patches maybe that are happening maybe real-time leaderboards you wanted to make sure that they get those notifications so with a push notification service they're able to get that real-time or whenever they log on to the client something that one of the services that we have this game is on pin point and what that is is it's a it sends out push notifications but it also has analytics on the backend it has a fully customizable dashboard and you can actually have player segmentation with with that dashboard so what that means is if you have if you wanted to notify if your game has different classes you know major druid warrior you can have predictive segmentation of notifying maybe one class if something changed if

it's a card game and and something changed on that card game maybe some cards aren't allowed anymore you're actually able to send only to those people that utilize those classes so you can actually have different user groups and and segment your players that way or if you wanted you can send a broad message to everybody cool so this is probably not going to be much of a surprise to anybody but you need to develop fast or at least fast you know people are talking about multiple releases in the day sometimes even and getting those releases out fast and without air is super critical and new engagement the second you start doing releases that aren't where you're missing your deadlines or where you're allowing bugs to filter in that are part of the build process you start losing the trust of your players and you start losing your engagement and your community now it all stinks so getting those releases up fast is critical so a lot like when we talked originally about flywheel and building live ops into your

game from day one building continuous in degree and continuous delivery mechanisms into your process from day one really save you a lot of headaches now for those of you that are aware see ICD continuous integration continuous employment is about automating the process right it's about automating the build process the test process and the point at the deployment process so that you're not just pushing the button to get the build and then having to do all these manual things to get it out there to people you're pushing the button and it's going to all the places where people get automatically so there's a bunch of different steps to write pipeline automation helps to reduce error for your process since you're not manual copy files will be things multiple destinations and so forth so be sure your building see ICD in from day one it's going to be as simple as just having a tool like Jenkins throwing the build processes and depositing artifacts in the right targets build it from day one you're in a much better place and that is really super critical to be able to handle the requirements for people that are playing modern games you know they expect to see fast bug fixes they expect to see responsive developers and

and you guys want happy players so you want to do that too so taking the time to build see ICD into your build process early on is really worth the investment again doing it early allows you to build the community to because you're showing your early adopters that hey you know I'm responsive I can work this I'm you know I'm gonna be able to give you what you want in a very timely fashion this is all super important stuff so build it in from day one in addition to the Jenkins and chef and all the other usual CIS CD tool sets you know if you consider something like a TOS code pipeline which is an edge CD service that allows you to automate those pipelines which lets you deliver working updates to your players quickly and efficiently it also well helps automate the testing process so those errors that that can fall through as a result of build process will get caught and will actually go out like you closures and just like fast development is important so our fast updates so you know patches downloadable content anything that's new and exciting you want super fast and delivering after a year of launch you're

able to do that with a content delivery network so something that so one of the services that we have is Amazon CloudFront it globally distributes that content with low latency high high transfer speeds we have things called edge locations around world that are specifically designed to be at where your players are playing so to deliver those type of experiences that they're expecting to have all of the updates on the development that you've been working so hard on so going back you know we talked about the flywheel in the beginning and we always like ending with the flywheel again we understand that your world revolves around your players and creating those unique experiences for your players in your game and hopefully you learned a few ways of creating those experiences for both your player community and also for your team making your game yep thanks everybody really appreciate it those email addresses are there for real they are definitely to be used feel free

to reach out to us we love talking to game developers I've been an indie since show me the codes on Dirk basic if any of you know what that was I'm sorry but I was there for that so I I'm I love the community I love participating so and I love talking about this endlessly so feel free to reach out anytime you're our guy that was fantastic I mean I've got a few questions there's a few questions coming in I've got one at you we put continuous development into actual units right the early on but I do know that that is something that does take a real mind shift in terms of advancing what would you say the biggest kind of pitfalls to try and avoid when you make that change you know from kind of like batch release to continuous rate for game developers especially I mean we we just want to get stuff out there right we we are one of the few groups of developers that is universally and truly

passionate about the stuff we do right we make cool games and we really want them to play those cool games so a lot of the time we resist process because it seems to get in the way it gets between the fun part and the fun part right you've got this fun part over here which is building the game get the fun part over here which is people developing the game and if I can just give them the game I can don't worry about all that crap and I'll spend this time upfront building the CI CD pipeline right it's super - super tempting to do that and I mean if if you know even if you expect to only ever have five people play your games there's a chance that a million people might play your game you know you never know when the next scene is coming from and it could be you you know it's it's not amazing odds there's a lot of people that you know that don't do it's that that stick around doing a couple of games a year for a small group of smaller players and love it and that's fine too nothing wrong with that approach but if you do wind up with the players you want to be able to support them because ultimately again you want people to enjoy your game so the big

pitfall if it's in your head right it's in the get past the okay I just want to do it I understand you just want to do it just do it this way and you can actually do it faster push the down button right well we gained some me personally I guess if I don't want the dev team to touch it I want LiveOps team to do it and I mean this same pro it's just I push the push the pipeline back a bit yeah I mean that's more about release to user pure sure you're getting stuff into the cloud they're ready to go and then managing the consumer experience independently yeah absolutely you massive and that saves you a whole bunch of pain when things go wrong as well so let me just go into questions so will Amazon leverage its DSP to create a robust ad network for gamers or gaming similar to what unity already has here my former Unity ads going what yes I'll have to ask that question that's the job of a cup their names you know that's a

really good question I don't have any I don't have any information on that currently um I mean if you look at how what we provide tends to be solutions oriented right you know if you look at everything we've ever provided from the Amazon Web Services portfolio you're looking we're looking at things where we provide technology that we've used ourselves right so you know we've provided easy to because we need it we've hired s3 because we needed global you know storage that was accessible you know across the world we need provided all these AI services that are on the API level because we recognize that not everyone's a data scientist but people still need the insights available through machine learning so while I know it's a crappy answer to that question I don't really know the answer to that question but I would assume that if we do it'll be based on something that we've done and has been successful enough that we think our customers can use it and say so I think yeah I think in G adds to the customer

and a lot of it is runoff off your servers anyway so but that's just it I've been I've been f two years so don't quote me on that I met you so that's one is can you share a dashboard of how Agora what a graph database looks like sure yeah actually um if you look at the neo4j is one of the the big to graph databases that are out there I've got a page actually can I can I link a page somewhere can I send a share screen to this so you can pull the page up onto a web browser and they should be in a share and then that should hopefully just work yeah your screen you can also share a window in where's the share screen thing on the second course if any has a big green button is always something you miss but that's the UX problem with all right so here's the page if you look at neo4j

it's actually it's a neo4j developer at graph visualization they actually have yeah they have tools that let you do that so this is kind of what you're looking at what you're looking at a graph database right this is their visualization tool yeah and so ultimately that's what it's it's you can get these things that are much more complex net and aren't so obviously radial aligned and pleasant to look at lot of things are messy as heck right because you're dealing with a lot of things that are connected a lot of things that are also it's more like a big messy thing of goo actually when you look at it but this is a good example of a nice simple one that's easy to understand and follow and if you are interested in this topic I love graph databases I've been using them for lots of stuff for a lot of time now it's a great way to deal with relationships between players you need players to classes to guilds to each other to friends lists so you know all that kind of stuff to understand how people grew up and interact with each other you can get some great insights using graph databases without machine learning let alone when you can actually add machine play but that's what you're

looking at when you're looking at a graph database and again if you want to talk about it like to hit me up on my email there should be more having a chat so I've got two questions left and we've got two minutes left so let's see how we go do you have any examples of any how clients use game tech in the past oh gosh so many yeah I mean I'm kind of flat foodies later I have a lot of our case studies in the back and I had like I know we did for honor used game lifts for instance to to handle matchmaking and multiplayer game lift is one of our larger multiplayer services we have folks using our back-end services I'm struggling right now because they don't have our references off the top of my head tell me that you got anyone offline I mean butterscotch shenanigans we talked we talked about butterscotch

shenanigans I know on our blogs the AWS game tech blog specifically we have we invite customers to write about their experiences and we post them on our blog so if you're looking for something like that that would be a great resource to go and view high-level all sorts of different games from you know a couple person studios to to you know Triple A studios there's so many use cases around around everybody yeah and so I'm looking at you know if I'm looking at our Capcom uses us your box uses us Capcom did it with containers and no sequel databases there's a bunch of stuff on the psycho you know I it's aws.amazon.com slash game tech we have all sorts of customer stories you can look at there I'm a huge fan of containerization as that I guess the whole idea of a just where's my mind because I'm old enough to remember some

old I'm back in 98 we did what we think was the first ever rack system we literally soldered laptop boards to run quake servers mrs. Beatty Brita calm we had some ridiculous stuff back in the day to run I think a wire play which was a dial-up modem based on my gaming platform so yes I've been there I do have containerization compared to running massive sudden minis or laptop based rack systems it's like such a huge journey oh we're in such a cool world for multiplayer right now you're absolutely right containers are you know that that you're running one or two when there's a couple people online you're running you know a thousand of them when you get your spikes if you have your events you've got it a brand new hit it's and it takes care of itself it's so nice exactly makes life so much easier