Superstar Session: The Live Ops Readiness Checklist PG Connects Digital #2

Kristen Cox is going to be talking to us you know about their their what they do what they're doing and what the experience is and how we can be successful in live ops so hopefully Kristen will join us shortly ha here we go hey Kristen how we doing oh you're unmuted oh that's ok great hello how you doing Oscar no too bad not too bad glad to have you here and again thanks again for the the support you guys give us in sponsorship and now I'm a little bit earlier but I hope it just means that you get more time for questions later yeah so I'm gonna pass over to you absolutely and I'm gonna start screen sharing right yeah there it is yeah now

this has been this is really fun you know this has been great watching watching the show so far you guys own a fantastic jobs through transitioning to this digital format but today I am here to talk to you guys about LiveOps readiness so this is going to be I think pretty breezy hopefully we will have some time for questions at the end I'm just gonna talk about ten things like the big 10 things to think about when you're getting ready to do live ops to start though I'm gonna say a little bit about me so I'm Kristen Cox I'm the director of live operations for the Xbox game studios Publishing Group I've been with Xbox for about 18 months now but I've been making games for 17 years pretty much exclusively live games I've done a lot of mmo's and a lot of like live connected games sort of in in the larger triple a double a space right now though

a lot of my job is working with teams who are going to do live ops maybe for the first time or they're just really expanding their use of live ops in in a way that they never have before so I'm working with a lot of teams who are getting ready to really do live ops and a lot of those teams I have are not new to games but they are new to live office which I think it's a pretty normal place for a lot of teams to be some teams do come up in the lab ops sort of right away it's the first thing they do but a lot of teams are making games and then start adding in live ops so as I've been working with these teams it's helped me sort of encapsulate across the board what it is to do laughs they have a lot of questions like what do I really need to do what do I really need to know and so I put together just a quick sort of listicle like it's you know 10 the 10 things you really need to have nailed down as you get into live ops so I'm just gonna run through those and hopefully it'll also help you if you're

in a similar situation or even if you've been doing layoffs for a long time sometimes it's good to just get a nice refresher so these are the 10 things that I think if your team can really ask these questions of yourselves and have good answers you should be very well situated to have success in live ops so we'll start with number one and number one is retention which shouldn't be too shocking retention should really be at the heart of any lie box game what you really want to ask yourself is how are you going to keep players in the game long-term so this is a question that's mostly about game design and it really should be asked fairly early in game hopefully if you're making a live offs game retention is at the heart of your game loop the heart of what the game itself actually is um but even if you know going in from the beginning I'm gonna make a live ops game it's worth stopping and just asking this relatively simple question what exactly is going to keep players in the game long term because for a live off game to

really thrive you are going to have to retake the long term it's much more difficult to keep a live box game going if your game has sort of a very distinctive beginning middle and end and it takes you know on average you know eight to ten hours in order to complete the experience obviously for a live ops game you want something that players can continually engage in that can mean a lot of things the answer this question could be anything from you know progression and growth mechanics which is the most traditional answer it could be novelty you could say we're going to have new experiences over and over and over again we're going to keep adding them and that's the things going to keep people in long term the important thing is that you haven't answered this question what is your retention strategy how are you gonna keep players playing so that's the first one retention followed pretty closely by number to monetization if you're making a live ops game you really do have to ask yourself how are we going to make money because very few live ops games use the the most simple sort of

monetization strategy and games which is make a game completely finish it package it up and then have the player pay one flat fee for it before they play the game which is just sort of by to play classic retail style monetization strategy that's actually pretty rare for a live ops game so you're gonna have to think a little bit more deeply and ask the question how exactly are we going to make money you should have a clear answer to this you know I know that sometimes it's nice to think like as a creative well we're gonna make a great game and then somehow the money will come but that's really the way it really works so it's good to sit down and really have this conversation with yourself it's also incredibly even more important if you're gonna have a live game because I usually you are going to need to have revenue flowing in in order to continue to operate your game since live games rely on being continually updated it's unlikely that you're going to finish the game and then have less

need for support and staff and all the things that it takes to keep the game going so you really are gonna have to think how is money gonna flow in how confident are we in the money flowing in what rate do we think it's going to come in what is our strategy for monetization after that we go into social to me this is an incredibly important question to ask how will your players interact with each other and I think that even if you're making a game that's strictly single-player even if you're making a game that's a super simple match three single-player game you still want to ask the question how are your players going to interact with each other even if you're making the decision that that interaction is going to be primarily completely outside of the game on third party platforms and even if you're not going to take a strong hand in it and that's because LiveOps games really thrive on players abilities to invest in your game and so much of that investment comes down to the ability for players to talk to other players about their

experience in the game and to form those kind of social bonds I highly recommend that any team making a live ops game of any kind ask this question even if you think you know that the answer is oh we're not doing anything social consider if maybe you should be doing something at least a little bit social you should be accounting for how players are gonna interact with each other because you need to you need to have a plan for that like even if it's very basic even if it's just we're gonna start a beneficial discord channel doing that ahead of time and really knowing what you're going to be getting into and having an understanding and a mapping of how players are going to talk to each other and how they're going to interact with each other is key now a lot of games will have very detailed social interactions inside of the game and so if you are making a game that's heavily reliant on social this is also a good time to stop and ask what exactly are the mechanics were going to be using to do that do we have actual social features inside of the game how are we putting them together

what's the UX for that very frequently I see developers make games that are inherently very very social but not prioritized creating social features and those like get to the bottom of the list or they're not even considered or you know they don't even realize they might need to go and build something to facilitate that social interaction and make sure that it happens cleanly and then that can cause you a lot of trouble down the road if you're not getting the kind of attached and you're not getting the kind of retention you expect from a very social game because you haven't really built the systems to facilitate that social interaction so super key how will your players interact with each other next we get into something a little bit more technical networking and back in text so how is your game gonna connect be connected and how are you gonna scale online today this is basically something that every team really has to contend with it's it's very rare nowadays for any game even once they're very light on live ops to have absolutely zero back-end or zero

networking to contend with so make sure even if you're a very very small team that you're asking this question how exactly are we going to be connecting how are we going to scale for very small teams you might be like well I just assumed that the platform is going to handle that right like I'm gonna be on I'm gonna be on Apple I'm gonna be on Android that's gonna take care of most most of what I'm doing but make sure you really investigate that and look to see if it's really gonna take care of all of your needs you know if you don't have an engineer and networking an engineer on your team consider sort of like reaching out to the community and getting advice to try to figure out what exactly you should be looking at even if it's not a major component of your lab ops game and a lot of that is because the games you know this has been going on for 30 years now live games have had trouble with scale you know I always tell teams make sure you plan for success because one of the worst situations is to have your game

fail because too many people showed up to play it that's that's a situation you know nobody really wants to be in you want to be able to capitalize on success you really want to be thinking what are we gonna do if a lot of people show up and that's both just in raw power which to be fair these days is actually much easier right we have access to lots of very powerful cloud networks that allow us you know games to scale organically dynamically you don't have to worry about provisioning servers anymore but you need to think about that side of it you also need to think about the way your game actually works so is it going to be a good experience if suddenly there are a million ten million people playing your game is that going to overwhelm the way you're doing matchmaking is that going to overwhelm the way you're handling lobbies or the way that you're handling the ability for players to connect to each other is that going to change the balance of your game is it going to make it harder for people to join up into groups is your guild system gonna work all of these things are things you should be asking yourself how is the game in a scale what's gonna

happen when I'm connected and all these people are playing next once you have that connection what is your live content strategy so how are you gonna add content to the game pretty much every LiveOps game relies on some form of content strategy whether that's balanced you know in a competitive PvP game and maybe you know new modes or Maps or if it's an RPG and you're adding in tons of quest content and and land and classes and all these things pretty much all kinds of live ops games must have some form of content strategy so you need to ask yourself how are you going to add content to the game I'm what cadence are you going to add it what kind of content I really recommend the team's actually sit down and write this out well ahead of launching and really ask themselves what is their strategy going to be why are they choosing that strategy how do they think it's going to work what kind of results do they think they're gonna get if they do this kind

of content up-to-date frequency if they do this kind of content being updated and really lay that down even if when you launch the game you find out a lot of your assumptions we're incorrect it's so much easier to figure that out if you go in really knowing exactly what those assumptions are and what you expect to get and then be able to compare what you're actually getting to your expectations as opposed to just going live trying some things out and then trying to sort through the data later so highly recommend really ask yourself what are you going to be doing for your live content strategy married along with that is the next one number six which is content management tools so if you're going to be putting all this content into your game you probably need to figure out how you're going to manage it this is something that the games industry is really struggled with because traditional game development that sort of done as you know fire-and-forget or you know boxed product style games is really built around the idea of building tools to get

to the finish line you know it's the idea we need to build these tools so we can finish the game so we can you know completely get down to you know work through our bug queue have a final gold master and then put it out the door even though we've been doing live ops for a very very long time we still oftentimes don't have fantastic tools and if you're using off-the-shelf solutions then especially for your engine you may not realize that if you want to build live content management tools you're probably going to either have to do it yourself or you're going to have to go out and get a second set middleware to do that content management and when I say content management I mean everything from the pipeline for putting content out into the world to the way you're going to visualize that content like for instance if you do have a competitive PvP game and you plan to be making valid changes all the time do you have an easy way for your designers to actually see all the numbers in the game and make quick changes to it and then test those

changes in a way that isn't incredibly time-consuming do you have a system for making changes to the game without pushing new client versions you know do you have a way to manage your backend configurations on the server side all of those things are tools that oftentimes get T prioritized honestly by teams because it's really hard to build the game to begin with so we're really focused on that and not so focused on how are we gonna manage all this content when it's out in the world so definitely take time to think about management tools content management tools 7 this is a subset sort of o6o the build test of deploy life pipeline because in a lot of ways build test deploy is one of those content management tools but how are you going to release changes and content to the players build test deploy or build deploy is really the backbone of making iterative updates into your game and it's one of the things that I think teams should spend a lot of time

thinking about even when they're early on in development the faster you can get a build pipeline that is iterative and flexible and start using it even before you launch the better off you're gonna be when you go live because when you're live the backbone of your game like really the sink-or-swim for a live ops game is going to be how quickly and how easily you can iterate on your game so you really need to have a fantastic plan for how are you going to make changes to the game now I think we still tend to call these build pipelines even though nowadays this doesn't necessarily mean that you are relying on a strategy that requires you to push lots of builds or lots of versions of the game to the player your build test deployed I may be almost entirely on the server side and may be almost entirely supported by bits and bytes in the cloud as opposed to actually pushing them onto your players device even so you need to

make sure that pipeline is working really well and that it has the ability to do things like rollback in case you put something out that you really really need to take back because it's causing a huge problem in your game or and that you have the ability to do staging that you have the ability to do a lot of testing on all of your changes that are getting being pushed out before they hit your players and you have the ability to do multi-threaded development so that if you are in fact doing lots and lots of updates you have can have your team working on multiple versions of the game at once but still be able to safely deploy just one set of changes without touching the other one so those are all really important things to think about and as I said highly recommend that your team actually gets a system completely in place and you actually start working this well before you actually launch the game next analytics and data so you know

if you you get your game it's got great retention and social features you have a strategy for putting content into it you have a strategy for releasing all this content you have a back-end all these things then you actually need to understand what is happening in that game so that you can make intelligent decisions about making those changes because you can have a fantastic content strategy and a good set of tools and a goods build pipeline but if you don't know what you're doing to the game you're going to be lost so you need to ask how are you going to track and analyze player behavior what is the data you're going to be collecting and what are you going to do with it what system are you going to use to cut that data so that's tools right and storage you know data flow all of those things need to be accounted for but this is also about policy I'm a big believer in the philosophy that you should only collect what you're going to do something with I don't believe in collecting data that

you don't have a use for you should be very deliberate in my opinion on what you collect have an actual plan what data are you collecting and what are you gonna do with it how are you gonna look at it how are you gonna evaluate it I strongly recommend for my teams that they put together a written strategy for data before we launch that has a set of goals and targets for everything that we're going to be tracking and if we have no goal for that data then we have to really ask ourselves why are we tracking this it's true that you could get into a situation in the future where you wish you had that you have a new need and you wish that you had old data but that's actually more rare than you might imagine and a lot of times you can't address that by getting the data moving forward to me that is the lesser of the evils of collecting lots and lots of data and drowning in it having a hard time managing it and not knowing what to do with it I would rather be in the situation of saying oh I just realized I

have a new bit of data I want to start collecting than be in a situation where I am NOT able to make efficient use of my data because I don't really know how to manage it or regular because there's so much of it and I don't even really know what I'm looking for or what it means so this is also about process ask your team whatever how are we going to collect data on what frequency are we going to analyze the data what are we going to do to analyze the data how long are we going to store our data and then how often are we going to act on our data even if this ends up changing quite a lot after you go live and it probably will because you're gonna get to know your game a lot better and you're gonna get to know what doesn't doesn't work it's really worthwhile to have done this work before you launch so that you go in confidently and you're not sort of in a post-launch state of panic trying to figure out what are we doing what are these numbers mean so really sit down and think about what are you gonna do to track player behavior and why are you gonna do it and then what are the

actions you're gonna take based on that data and you're gonna be doing a lot better and you'll start to move a lot more quickly after you get out so many games these days do a soft launch and spend some time sort of in limited markets and the key to sort of getting through that stage really quickly and efficiently is having a really strong understanding you know what you're going to do with the data how you're going to collect it and then how you're gonna optimize to get to the next step and that moves us to a specific kind of data monitoring which is live monitoring so how are you gonna monitor your games performance this is a subset of data and analytics but it's a really specific one so every live game is going to need some type of monitoring even if it's all the way over on the end of the spectrum where you're monitoring is if a player writes a bad review on you know the platform I will know there's a problem all the way to we have an entire team of people who do nothing but have a 24 hour on call schedule to make sure that all of our services are up and running think about

where you want to follow that continuum and I would say really think about it I think it's it's sort of something a lot of live teams go iyo obviously I want 24/7 you know NOC support and I want on call trees and I want everybody looking at all the time but I would say especially for smaller teams really examine whether or not that's going to be necessary for you and also what you can actually maintain for your game it is true that every live game is going to have some amount of you know monitoring that needs to happen and there's going to be some amount of services that at some point will fall over someone whether it's on a local level of a single player or an entire outage will happen so you do need to have a plan for what you're gonna do when that happens but it doesn't necessarily mean that you need an army of people to be monitoring your game so really think about what is gonna work for your game at your scale finally we get to number 10 which is community so to me the backbone of live live ops games is community really having a live ops game to me is all

about growing and and shaping a community because that's what really creates stickiness when you are launching your game I strongly urge teams to think deliberately how are you going to grow and shape your community this is a space where developers I think have a lot more influence and they usually think they do if you have a deliberate plan for how you want your community to grow what kind of community you want to have you can take deliberate steps to set norms put policies in place case figure out what kind of monitoring you want to do all those things before the game goes live and then you can have a lot more control about shaping your community I'm not saying that you can it can be perfect or you can prevent all kinds of you know negative or toxic behavior but you can have a greater hand in it than you might realize um and it also gives you a leg up to start using organic growth we spend a lot of time in live ops thinking about acquisition and when we think about acquisition we usually think about paid acquisition but please don't completely ignore organic growth it can be incredibly powerful and

it will deliver you some of your most devoted fans so spend some time before you launch thinking about your community where it's gonna live how it's going to be grown and how it's going to be shaped so those are my 10 things if you know this can hopefully be helpful to you if you can really sit down with your team and go over these even if you think yes of course we have answers for most of these I think it's a worthwhile exercise to really look at all these because if you can have fantastic answers to these questions and good plans you really are gonna be in a great state for live Bobst i mean if you're doing all of these things at a sophisticated level you're basically a live ops master at that point because these are the things that are going to be coming up over and over and over again for you so that's the checklist really quickly I'm gonna if you'll indulge me for a tiny little plug I do I have a podcast that I co-host called the art of live ops so if you like listening to developers talk about live ops this is a podcast that I host a

chamber games Kurtzman we interview in every episode a different live ops developer and talked them about their experience in live ops and their experience making live ops games the first season is completely out there's like 18 episodes we're gonna have new episodes starting the land in July so check it out the art of laughs okay that sounds fantastic I'm gonna have to go and subscribe that immediately anyway that was fantastic you know nice simple ten points I could not even question any one of them yeah you know the priority of them I mean I can spend hours debating with you about how to structure a data as well but that's anyway um we have a question I mean we've probably a loads of questions that will come out let's start off with the one again is this from anonymous attendee the very popular question make it do you think indie developers are too worried about user acquisition and

retention from the data and cost point of view and are not taking enough advantage of fandom and community that can push them forward for free um I mean yes you know it's a good question I do hear a lot especially from small teams now that they're kind of being overwhelmed with this idea of games as a service right that there's this this idea that it's this big thing it's expensive that you need to have a data scientist and they need to be in there doing these and those things can be fantastic and you'll hear great stories from teams like teams a king and super cell that have done amazing things with systems like that so that's fantastic but if you're a small team I think it's still true that grassroots is one of the best investments that you can make and when you see teams that do serve rise up above the noise because that is really what we're fighting against at this point there's so much content that you

want to get noticed and really your chances as an indie of getting noticed by trying to yell louder than everyone else is pretty low so you want to take that kind of grassroots approach and sort of try to come in with your players right build that fanbase and get them to evangelize you yeah if I know what some wrote a book about James's service oh wait I did but it's like yeah absolutely that's the key point isn't it and it's like why shout across the room when you go up to somebody and spending a lot of our time shouting and spending lots of money I am not trying to say that the the publishing side is a bad thing at all for me it's amazing and it's really good and it's really specialist skills but you know focusing on only you a and marketing is not going to get you the retention yeah I think it's I agree like it's very unlikely the truth is also that when we look at the most successful LiveOps games you

still see that they're all very focused on communities they're very focused on people having passionate investment in the game and that generally doesn't come from people like seeing an ad that might be the beginning but also what could be the beginning is literally talking to you the developer on a disc or channel no I'm cynic I'm not shouting you know the thing I think is hard for a lot designers in particular to do is to realize that they're giving up their baby it's not that game anymore I mean really is it I mean that is hard right I'd some watching who's a fantastic LiveOps developer who works for Microsoft Asia made an analogy that I love he said that traditionally in the West when we were coupling games are coming up game developers and designers wanted to be movie directors but in LiveOps you want to be a nightclub manager and that's a really different mentality identities actually which I

thought that one that's a great line I know it's so good it sums up doesn't it yeah and it's also you know part of that is being a knight I mean it's it's it's being of the retailer it's making sure that you've got the the staff the entertainment you've got the the VIP section there's so many elements that metaphor that just match yeah and your job is really to make sure to create a space and an opportunity for people to have a fantastic time I love it because it really is true that sometimes it can feel in LiveOps like well what are we doing we're just building tools for other four players to make their own fun like Who am I as a designer and it's like well I can never have an experience in my home that's the same as going to an incredibly you know well produced nightclub that that's not something that I'm capable of you're giving me that possibility yeah no I totally agree we have another question here so until not long ago developers thought only about

releasing the game and not about the longevity of the grain all right totally through this and LiveOps seems to be a new thing that I don't agree with I've been doing it for 22 years so you know I'm a bit biased that should be thought of from the beginning for teams that have only just realized their need for live ops what is the most important thing to start with we already know your top ten but if you wouldn't start with kind of one way of thinking what would you be so I'm gonna go very meta on this and I would say that the first thing you want to do is it's at the team culture level really I would say the first job that you have when you're moving a lot of Ops is to get your team to embrace iteration embrace the idea of never done embrace the idea of never perfect embrace the idea of getting it out there even when you're scared and iterating on it it's hard but you got to get the team in that mentality um in a lot of ways it ends up

being really break out of that waterfall world yeah because the one thing I think for me it's actually about thinking about pipelines and what I mean by that is is like you've got to create a sense I'm gonna I'm gonna go fully Columbo on you it's the Columbo TV show yes and it's stupid thing but homie speaks for we told me mention Columbo it's the best TV show in the world because you know what to expect you know who did it so you know what to expect and yet you still watch it right you're filling in the gaps and try to predict what's coming before he says but just at the end just one more thing great if we can set up but every week feels like a Columbo moment where they're looking forward to what it is that's gonna come next and they want to predict it before we say just one more thing then I think we've absolutely nailed live ops mm-hmm

I love that idea because it really is that's a dialogue with your players right it's not about just creating something interesting it's like you said it's about playing with the expectation of inviting them in to say I want you to guess I want you to be involved in the process of anticipating what it is I'm going to do sometimes I want you to be surprised sometimes I want you to be satisfied because you did guess it correctly and that's a good thing absolutely yeah totally agree I think that's it for us at the moment so that was amazing thank you so much for that I would like to find out the name of that person who mate came up the night stalkers I am going to steal that absolutely my next webinar I'm sure so Kristen that was fantastic thank you so much yeah thank you for having me thank you guys absolute pleasure you