Panel: When is the Right Time to Start Thinking, Testing & Delivering Live Ops? PGCD#2

we have a panel uh i'm going to invite steven lee from deca games to come up and introduce his guests he has an amazing array of people um but if i won't try and uh jumpstart things i'll let him come on board the the plot is to talk about when is the right time to start thinking and testing and delivering live ops wow you got a few things to cover there stephen you're on mute still absolutely yeah i mean this is a topic that definitely can be uh we can talk about all day only we had the whole day to talk about it oh wait i'll leave it to you sir all right thanks oscar great so now everyone is coming in we're all here is that everyone yes even

patrick success just waiting for video um here we are there we go okay awesome great well let's get started um hello everyone thank you so much for tuning in to this panel um it is a great pleasure to be moderating such an esteemed and experienced high-powered group of people if we were to kind of pick people in the industry uh to talk about live ops i i would find a hard time identifying many more people than when we who what we already have assembled right here so uh you're in for a big treat so um what i'd like to do before we get started in case um some some in the audience are not familiar um let's just go around and introduce ourselves so please tell us who you are uh where you are what you do and how about uh to also tell us the last game that really hooked you or maybe the last game that you spent money on um i will actually start myself to give

you guys a second to think about it so um i'm stephen lee uh based here in berlin germany for deca games we like to refer to ourselves as a liveops publisher so we help developers and teams to take over operations of their existing live games and in some cases we acquire those games and that gives developers more flexibility to move and focus on on newer projects who would like to go next feel free to chime in [Music] shall i start i'm i'm mark mark robinson from founder of delta dna nine years ago i think it is in counting um nine months ago we joined the unity family so i'm gm of delta dna within unity and it's been a great experience for us all based in sunny edinburgh one of their ironys about lockdown i was telling my colleagues um on the panel is that the weather has been blisteringly hot in scotland which is totally unusual um because of lockdown my pals and i

haven't been on the golf course so we've been playing golf clash so we've been unlocking the golf clash clans and getting competitive with other people around the world which has been a really fun experience so that's me i'll jump in so hi i'm james gortzman uh i'm dialing in today from seattle uh i've been in game industry for about 20 years my last studio experience doing live ops was over in china where i was actually running a studio doing free-to-play versions of pop-cap games including plants versus zombies instead of the early days of of smartphone mobile uh but then i started a company called playfab which was doing sort of back end services pretty broadly for gaming and we were acquired by microsoft about two years two a few years ago so i've most recently been the general manager of play fab at microsoft uh and uh and i'm actually just now starting to look actually even a little bit more broadly into sort of the broader cloud services for gaming at microsoft including azure and other things and what that might look like so you know stay tuned that'll be kind of

fun um in terms of gaming actually uh i have three kids one is a seven year old and she's been playing a lot of roblox recently so i've actually been doing a lot of roadblocks you know exploration with her you know for those of you who aren't familiar roblox is becoming this game platform with a whole bunch of interesting live ops happening inside of the games inside of roblox and so all the techniques we know and love for mobile games and pc games are being adapted now inside the roblox platform it's kind of fascinating to see these early often very young developers starting to adapt the same techniques that we we follow uh professionally so that's fun well i will join in next um so hey i'm nick uh i'm also based in berlin i've been working in live ops and product for about 11 years now and i'm currently a freelance consultant focused exactly on those areas so i work with a lot of different companies to help improve their kpis particularly retention monetization and uh in terms of what i'm playing recently

i still can't quite prise myself away from marvel strike force um yeah too much progression everywhere i keep upgrading stuff it's uh it's an ongoing struggle um i i can go next hi oh sorry all right um i'm samia i'm darling from bangalore india uh i've been a producer for live ops mobile titles for about um four and a half years now so uh i feel like on this panel of legendaries i'm like a magikarp um so i've worked at ea mobile uh and gsm games and jumpstart uh but actually i started off in the industry a lot longer and um i started as a community manager for like a hardcore pc mmorpg uh back in the days when social media didn't exist uh and when you know live ops consisted of us going into the game with our own accounts with our own characters and you know manually running events of spawning

boss monsters in the game or holding tournaments and um you know tallying up the scores ourselves and updating the forums remember forums um so yeah it's been quite an evolution since then um and as for what game i think right now i'm not playing a mobile game recently i have to take a lot of subnautica below zero but still in early access but uh it's been really zen sort of cruising around the bottom of the ocean and you know exploring all these ecosystems so yeah that's me all right well uh i'm patrick mcgrath from sunny berlin i i'm actually surprised there's so many people from berlin here uh i'm currently product lead of june's journey top hidden object game in the world very proud yeah i have a probably 10 ish years background in live ops got my first start at nc soft when mmos were going to more live op structure and they were doing

the free to play and uh worked at studios and companies like z2 pop cap natural motion and and now luga so uh you know on some of the bigger titles that they've had and lots of experience and yeah excited to be here great thanks everyone um so uh with that let's get started with kind of the meat and bones of of the the content so uh let's start with kind of the title title of this panel so when is the right time to start thinking testing and delivering live ops um does anyone have something that jumps out at you in terms of timing certainly the thinking portion the answer is yesterday should have thought about ashish thought about it last week and implemented it yesterday yeah well i'll also add it's also never too late so you know absolutely the earlier is better but never too late you know we've seen a lot of games that

are you know maybe they think they're doing live ops but they're really just doing very occasional updates or something and they're on a slow glide path to kind of irrelevance and then they start to accelerate events and start to do more engagement they start to pick up the pace and you can see those curves turn right around so it's never too late although it's certainly better to start earlier it's fascinating how you know we've all sort of reoriented our thinking from the sort of the launch and launching crash that you know especially when delta dna started we were trying to save a lot of flawed games now these games last for years and that's really notable that you have to really adopt long-term thinking and long-term strategies for games because genuinely the engagement with players can can last for such a long period of lifetime value can be ignite over accumulation of experiences over a long time i think that's the fascinating thing that we're trying to reorientate our thinking around right now

yeah i think as james said actually um a lot of it is about like kind of just getting in front of it being early because like honestly once you start running live ops running a live game suddenly it's like you're in a game of space team or love is in a dangerous space time if you've played those like you're constantly spinning plates in five different directions and you're you're usually struggling for tools and resources and to make sure that you are getting ahead of your planning so the sooner you can like think about building tools or automating some of those systems like you put yourself at a massive advantage yeah i would add that it just adds to the probability of success especially this day and age um it's hard to succeed against uh you know this cpi ltv uh battle that we're all facing and if you don't have proper live ops in early uh and i mean this is a portion of it like when do you start testing it well

ideally in soft launch which means that you had to think about it earlier than that um so yeah i i really think fundamentally to give your game the best chance uh but to james's point for sure there it's never too late that makes a really good i think an interesting point as well about automation because you really need to free up thinking time to plan ahead so you can't keep repeating the same things that you've already figured out whether that's campaigns or events or promotions or whatever you've got to have a real really strong sort of automation element to live up so that you can just get on with the new thing the new stuff the new thinking and that that's critical for unlocking the value i think but it's automation with like with the ability to change a bunch of variables at the same time to keep things fresh right so i mean one of the biggest challenges i found in live ops is people have come to me in the past and

said okay we want you to start running live ops on our game i'm like okay where's where's the economy depth where like what can i sell what can i actually use to incentivize players to take part because sometimes you can have an event that's just fun but a lot of that time you need to build that up with like progression and rewards and other things that players actually want so like actually having the economy ready behind it is also really really crucial one thing i want to add one thing i'd love to add is is something we've seen which is to really start to understand how to roll live ops out of a game is a huge culture shift for studios that are not used to it so it's a completely different mindset and so we you know my colleague kristen who's actually just speaking she said it takes some time experience five or six years for a studio that's traditional to sort of package goods or you know kind of old school to really adapt the new mindset and so given that this is a very long journey not in terms of the game technology but your own mindset you know just getting something out there and then starting to

learn and because it will change your mindset you know if you're designing a game with the lines up sort of mentality from the beginning you will design it very differently than if you if you don't have that and so you know yes it's good to start earlier but you know a lot of it also has to do with just that that changing culture shift and and you can't rush that it really is not something you can switch and expect every designer on your entire team or every producer and team overnight to sort of get it it's a very different shift yeah that's sorry one of the things i would add is uh the evolution of what live ops even means has assisted the the cultural shifts um i mean i remember back in the day there was still uh conversations of premium versus free to play that was eight years ago seven years ago at some places uh so yeah you definitely go from a studio that's having these types of debates to even start uh you know respecting free to play and certainly some of the larger

titles uh the supercell put out have helped the industry kind of realize the potential of free to play uh and what live ops does in that business model and so it is that cultural shift but the evolution of the market itself has really assisted in oh this this is a viable and you know players actually like it more or at least i should say and live ops is quite a confused term i think i mean it's a bit of a catch-all but i think it you know at its essence what i think of it is it boils down to attaching individual players to the right content you know that's a complex formula you've got different playing styles you've got different sort of levels of competency different different archetypes in your player base if you're thinking ahead like we all encourage people to do then you should have lots of content to deploy we've moved far past the sort of the concept the basic concepts of time promotions

new halloween offers holiday offers and we're seeing some real sophistication now where campaigns are very precisely targeted to players based on their playing style and that's where that's where the industries go in as we see it that you know the sophistication is really ramping up because you know the thinking is there the culture is there for the studios that have adopted this approach and and the tool sets are there as well to hopefully deliver the personalized experiences so uh we've successfully answered the title of the panel so thank you everyone for coming um just uh just kidding um so let's let's kind of dive in a little bit more and help developers um kind of through the development process so you know you've all pointed out correctly that the earlier you think about liveops is better so let's talk about pre-launch um what are you guys seeing or doing with your partners or within your companies to prepare for effective live ops feel free to throw in tactics examples

um what has worked really well for for you guys from your experience i guess i approach things from a more like economy driven angle um and definitely one of the things that i try to do before like launching live ops on any title would be like like i mentioned earlier ensuring that you have enough kind of depth of economy so that you have enough items that you can distribute but also making sure to as great a degree as possible that those currencies and items feel balanced against one another um i feel like this is a step which is often quite overlooked and a lot of the time when people design an economy in the first place they kind of put placeholder numbers and then often forget to come back and change them um and only when play testing realized oh well we should have updated that but actually again it's one of those things if you can get it done before launch and actually have your currencies balanced then your liveops are inevitably going to be more successful because you're actually going to be

offering value for money instead of overpriced like overpriced content patrick you also mentioned something about testing before right i'd love for you to elaborate on that you can yeah absolutely um so even before going into soft launch just evaluating your feature set making sure that the motivational loops are there um understanding what you want your players to be motivated by and this like very tightly uh goes with the economy um you know making sure that the reward space is there and also the value proposition is is well understood right because you can have a some currency or some reward that economically is valuable but the players don't like it and so these are the types of things that you want to go into soft launch testing as well as uh basic frameworks uh so if you have a sales framework you have an events framework uh you should certainly have crm so like

messaging within the game uh and going into soft launch you want to have you know variations and different balances uh different types of of these these events of course i'm not saying hey you should have three event types but there's easy ways to have a variation like reskin some some very low cost uh uh options that you can have just to see how players are engaging see how players are you know monetizing around these things how they're progressing and and yeah this really will uh set you up for success if you can come into soft launch with this planned out and uh you know you you have this hypothesis driven attitude towards this feature set we see quite a switch of gears um going from soft launch to full launch i think you know we we have seen um our clients and we see

experience where the numbers can change once you've done the full rollout depends on the geo that you're doing and you're doing you know you're targeting yourself lunch at so there's a lot of as patrick was saying there's a lot of opportunity pre-launched to do game experimentation level balancing economy balancing all of those fundamentals and getting getting them right because that's actually where the lifetime value comes from fundamentally but as we move through to proper launch then you're actually needing to very quickly validate some of your assumptions around lifetime value you're looking at your ad placement strategy you know a lot of the the change in dynamic is seeing ads as a retention mechanic now whereas before we probably all agree that ads were a churn problem actually getting the data to show the appetite for ads for different player types versus the consumption of ads that's an important mechanic to get

right but it needs to be tested out in the real live world because we do see behaviors changing from soft launch to to launch reasonably dramatically and you need to be able to reorientate things quite quickly to take advantage of the full launch data that's coming through it sounds like testing is is a huge huge component that um you know you guys are you know pretty integral for your process are there key um numbers that you're looking at where you whereby you see if you see this happening in a soft launch that gives you the confidence to to move forward what are some some markers or flags that developers should keep keep an eye out when they're prior to launch i think this is something that from genre to genre dramatically change like you know you look at some of these games and they only care about up to day seven retention and so really

it's just okay what is day one retention and the engagement like how many minutes uh did they play in the game where it would i mean we really want to understand uh long-term retention do we actually have something here that has a long-term business uh case um so we're certainly looking at retention uh then probably uh engagement within okay what are the players doing uh where are the pain points in the economy and monetization of course like we all run businesses we need to get get paid um [Music] so i would say probably in that order in soft launch at least uh from my standpoint in being a casual game that has uh you know we're looking at one to two year ltvs to justify our cpi's um but again like genre designer this this can change dramatically

but not just casual games i think that's a good point even though i would say you know in some ways the more hardcore the game is both the higher the potential ltv can be but also the less important is to focus on that upfront i mean the more important it is you get the engagement that the engagement and retention dialed in first because you know you know you look at the game like riot games i spent a lot of time talking with my friends over riot and monetization is such a distant thought for them you know with league of legends especially it was 100 engagement for for a very long time and that really is even now probably the most important kpi for them because it's all about the engagement trusting it with the community a very active community of players monetization will follow excellent um how about in terms of um you know once the game has already launched um you know you guys are trying to understand how the game is doing and keep a pulse what are some important factors that you guys keep in mind when you're thinking when you're

considering live ops for for either your games or the games of your your clients or partners well the good news is that the actual metrics can continue to improve you know this is a journey and you can continue to refine refine the game you don't none of us sit back at launch anymore and just wait for the money to come that doesn't happen what you have to do is actually double your efforts and refine and maybe test and you automate um but really understand the player experiences and how they're changing through time you're thinking ahead in terms of content and matching the content to those players because we're really seeing the impact of a lot of accumulative events here you know it's not one one magic bullet will solve your retention it's a lot of accumulative events and that just takes time and effort and constant attention but if you do take that philosophy and it's back to james's point about culture if that philosophy is in the

business and how the games you're built and managed then these kpis continue to improve and um you can be on top of that curve i'll also begin i know marketing you know delta dna and i think analytics and understanding your players is so you know so important because the part of that culture shift we were talking about earlier it's not just the idea that i'm in for a marathon now i'm going to be continually operating this game kind of day in day out forever it's it's more about the the role of the player in your design process it's you know i think a lot of games early on you know just as we were waiting in the green room coming in my colleague christian was talking about um it's like running a nightclub as opposed to being a movie director and i think that that idea of chief designer is nightclub impresario is so critical because you really are creating an environment for players to have a great time and be really you know meet other players and have a really you know engaging experience and so when you take that mindset the designer

it's almost like the player is sitting at the design table with you alongside you you know you're on the same side of the table you no longer cross the table saying here's this experience i'm going to build and give to you the player you really are you know learning what the players are doing the behavior and oftentimes your own you know understanding of your own game as designer shifts over time as you start to see you know using live events as opportunities to experiment with new gameplay mechanics opportunities to test out new new uh you know ideas make for a more fun experience and as you try these things being able to see what's working what's not working and listening to your community and seeing what the players are reacting to and then being able to kind of you know tweak over time toward that it is is part of the really fun thing about these live games now is we have this very emergent sort of paths that games can follow where over time it can shift and that's i think the most exciting part about this this space yeah i mean it's really about adopting um a mindset of experimentation you know i think not being afraid to take risks but also

uh designing with the goal of um tweaking or hitting a certain kpi and that can be quite difficult you know as you mentioned a mindset shift uh entirely so it's um it's quite interesting in that sense i think it's helpful to think of it as a different design challenge as opposed to what we're used to instead of saying you know as as you all mentioned this is what i think is going to be amazing um you know saying this is what i think our players are going to really like um for which they will reward us with x y and z you know engagement and monetization and retention so um yeah that sort of mindset is pretty critical nick i know you mentioned you had some thoughts about about player community as well yes i i feel like community is like they're really useful resource definitely uh to keep one side and obviously i'm stephen i'm familiar with like how deca has

rebuilt certain games based around their communities um and turned them into very successful products again and um i think they can be a really brilliant resource i would say if you're looking to them to provide guidance on what you should do in terms of liveops strategy that's maybe a little optimistic they can certainly help validate theories that you might have but throwing out like random surveys to what kind of event should we run next usually just ends in disaster um but i mean coming back to a point that mark mentioned earlier as well about how you need to really try and like separate and target users in different ways and how different users consume content i mean you have people particularly in let's say like the mid-core space you have people coming in who will spend a ton and consume content extremely fast or ones that will spend a ton and will play once a week and you have people who never spend and grind and people who never spend and just be a casual player and all of these require very different

approaches really so coming up with a kind of an all-encompassing live up strategy for these groups is not really an option a lot of the time unless you can effectively segment your audience and deliver interesting rewards to the right players at the right time it's interesting on the subject of community i think it so depends on the kind of game that you have and i think it sort of stands a spectrum you know you have uh games where community where player-to-player engagement i should say is it's very strong like your pvp games and your battle royales um and you know discussing strategy uh and you know forming gears and stuff is really really important on the one hand uh and on the other you have games for which that might not really be true by i you know what comes to mind is social casino for example players really into slots that's not really the most social of activities you know you might want to talk about it when you won the jackpot uh but you're not really going to talk about it when you lost a lot of coins for example so i

think it really spans the spectrum um and in terms of investing in your community i think it makes a lot of sense to do that um for the heavily you know player-to-player style games where every single player matters you know even if someone is coming in for five minutes a day playing a couple of matches and leaving that is still um you know adding to the engagement of other players as well uh the players who are your high value players in terms of monetization and engagement so it makes sense to sort of invest in community management for the broader community uh whereas for games that are on the other side of the spectrum they're more single-player experiences i think it's it's gives you the best roi i i suppose i could say if you focus on uh you know your high value players the you know not just monetarily but in terms of you know social influencers if you have any or engagement um and it's it's really really important you know when we say community is is key to your game i mean that is

true your players are of course you know the the gods almighty you know who are the source of all your um you know they keep the lights on and they provide you with food on your table um but it's important to sort of uh balance what they are telling you uh with what you're seeing in the game in terms of metrics and how they're actually engaging and what i found uh you know over the years is that players may tell you exactly what they want or what they what they think they want they might spell it out in you know uh no on certain terms but um instead of just following exactly that it's very important to realize that um they are expressing a need you know and we as designers and game makers can you know try to figure out what need they are expressing and give them experiences that fulfill that rather than exactly what they are telling us you know in those exact ones um okay i'd love to hear from uh each of you about uh for if you have an example of a

game that you've worked on where um that you'd really love to share in which liveops experience was really really positive very effective or perhaps on the flip side a horror story a mistake that was made a lesson learned from live ops it can be from your own games it could be shouting out a game in the industry that you think is doing a really good job or a really bad job of live ops um let's go around the horn and uh let's start with james i'm on the spot um you know i've been actually very involved recently with minecraft within microsoft you know and i don't know if minecraft is necessarily a title we think of as being especially focused on liveops but you know it's one of the longest kind of enduring communities in the in the game industry at this point you know almost i think 10 almost 10 years now not more and um you know it continues to grow you know we're like like a lot of games with effective live ops uh it's a bigger you know bigger audience today than it was you know you know you know 10 years ago so um and

and key to that really has been one of the things we've added that has been a big focus is the marketplace which is an opportunity for players to create content and essentially sell it to other players and that marketplace has all sorts of challenges with it because now you're actually you know besides the kind of physical complexities of how you actually arrange for player to player payments um there's been a lot of learning around how do you promote that what sort of events do you run around it what sales do you run um and so just managing the economy of sort of a marketplace like that has been a whole new level of learning you know beyond just what you typically do in gaming um in terms of actual kind of you know training i love trainwreck stories myself uh i don't i don't unfortunately have a great one i can share um you know most of the because i'm on the on the playfab side you know one of my roles is providing a lot of platform technology that other parts of of of microsoft game studios and others of course use and so you know my trainer is typically involved in servers not working or you know key services going down at a critical moment um and i think the the only thing i'll

add there is just you know in a world where these things are getting so complex you know the the number of dependencies in a given chain of technology keeps growing so the hardest part now really is just keeping all the pieces humming and having enough capacity in the system to handle those spikes because the thing about live ops is you know we still see you know games that suddenly go from zero to three million dau and you know a few hours and uh and that you know no matter how good you are and how much capacity you have that's going to challenge any system and so those are usually the systems i get called in for super thank you for that anyone else want to chime in with a real world example sure i have a so uh like somia actually i used to work on kind of hardcore korean mmorpgs uh the wild west days of free to play um and we we basically had to break the game uh using the the tools that we

had um so that we could offer like significant incentives to players who would spend and we used to run events where if a player spent a certain amount they would unlock an additional reward a tiered spender and basically there was one tier which was tied to um what we called like a gm service that meant the game master would actually perform some kind of operation usually to like insert a gemstone on your sword that would make it more powerful or that kind of thing but we hadn't considered that actually amongst our gm services was also a wedding ceremony which basically meant that a game master had to go into the game and perform a wedding ceremony like typing out like do you take blah blah and you know throwing fireworks around and that kind of thing and we suddenly realized that actually a lot of people just quite wanted to get married in game so a ton of time was spent on just people having to go into game to constantly do wedding ceremonies instead of actually

like driving revenue as we were hoping so there you go a slightly ridiculous example amazing nice nice engagement uptick yeah very literally um i i think uh one game well june's journey i'm particularly proud of just from the standpoint we were talking earlier the the cultural shifts that are needed uh to to move into a mindset that really desires live ops and this is something that uh when i came to wooga it was just starting to develop and the speed and momentum that we built up over time you know we had one sale that we could run in june's journey for like the first almost nine months of the of the product like uh and it was not something that was very flexible you had to keep all the currencies the same and these types of

things so now we have like event you know multiple events different frameworks multiple sales systems these types of things that were developed uh internally um so really proud of the the way that the the team took that on and it was very fun to develop these systems uh ourselves especially uh with the depth of segmentation that we have and just really understanding the player motivations player journey and and just trying to essentially delight the players because this is how you're going to get the best of all kpis um so this has been a really a great experience uh of course in in the past that had a few but this is uh certainly uh a proud one a recency bias as well one one example of trends that we've been seeing is um your shift you know live ops is all about sort of in-game messaging and crm um

obviously but it's also got another personalization component which is dynamically changing the game for the player under the surface and i think that you know that is a really interesting area um for the application of player management right now whether it's changing the level difficulty responding to the sort of the competency of the player or adjusting the reward on a rewarded ad you know i think that there's a big opportunity there to deploy um you know some really nice best practices and player management to actually fit the game around the player and that's that's something that we've seen um quite a lot of our clients and the industry in general start to adopt um you know it's in it's sort of first flushes i would say but it's definitely something that could be uh could be a focus um over the next next year too mark that was an excellent transition um in the few minutes that we have left

i'd love to take a a few minutes to talk about what else you guys see in terms of where liveops is headed right we've come such a long way in the industry um where do you where do you guys see um the art of liveops headed in the future other trends or other things which you'd like to see happen ways that people are are attending to their nightclub if we're going to keep that analogy going well i remember you know there was a session at gdc a couple years ago about machine learning in in live ops and games and i think we had like one person who had one example and it was still seen as this kind of cutting-edge thing and i think virtually we're going to talk to now is doing some trying to do something with with machine learning whether it's turn turn prediction you know some simple example you know to to mark's example this is giving us now you know very advanced personalization where on the fly you're changing difficulty curves to you know optimize engagement or something like this and i think we're just going to see that

increasing uh you know it's something we certainly are looking at and um i think that's the role of ai machine learning and automating optimizing live ops i think is gonna be one of those major trends for the next couple years totally agree uh to to just go almost to the feature level i think there's a couple trends that have certainly caught fire um probably in the last year not to say that this is something new you know akin to the the battle pass it's been around for quite some time uh that caught fire but really uh gamifying the purchase funnel uh having that have features and events around just purchasing things i've seen this quite heavily in in a lot of different genres on mobile and just like things as simple as purchase events and making that fun whether that is iep

or just hard currency um and then also rng more and more genres are getting into using rng in a positive you know very fun way i think there's a lot of uh psychological triggers that bring enjoyment to to player bases uh of course depending on how you use it like definitely taking out the component that can be predatory but uh you know as long as you make it fun for the players this is something that more and more genres are starting to take in do you want to define rng for anyone who doesn't know that acronym oh uh random number generation so think of oh boy i don't i want to stay away from saying the the bad word of loot boxes but uh and anytime you know you're playing uh wow or destiny and you know you kill a baddie and they drop something

rng in the background saying oh what do i drop here and so even when you're chasing prizes and opening up chests and doing things uh in the core loop uh this this i think is something that is becoming more and more uh widely used even in genres that naturally uh it doesn't seem to fit but i think there's there's some good synergies that are now just to the sheer creativity of of people uh starting to to show up and feel much better i think there's another couple of topics here as well that liveops is sort of starting to influence one is some of the techniques that have been learned in liveops are feeding their way back into game experimentation you and level design and other topics like that we can certainly apply some of the disciplines of liveops back into the development um community um the other the other direction i see this going is that more and more um which was the sonia's point was

um every player counts we're not we're not thinking of spenders and non-spenders anymore we're thinking of blended economies and we can see that the interaction between in-app purchase and ads is a really powerful one you'll add provoking a purchase following a purchase with ads or not as the case may be really understanding the player experience for those blended economies because we've seen we've seen you know the component of the overall economy that as generates is it is really nigh very important um and we see that that sort of blending of the economy does add significantly to engagement as well you know something like a 6x was the last study that we did in terms of engagement if you had the right blend of economy so there's there's more complexity coming i think is what i'm saying and um you know we have the toolkits and we have the thinking and the cultures to to take advantage of these now yeah i

would just add to patrick's point briefly um that actually there are a lot of genres which are now trying out techniques that wouldn't usually be accustomed in that genre so you've got this kind of birth of like a hybrid casual mix where casual games are taking techniques from mid-core games and trying those out and trying to drive progression in different directions but through live operations a lot of the time um and just a quick word regarding loot boxes as well even though it seems to be like a taboo subject i think it will continue to be a controversial thing in the coming years um as governments take more of a stance on it and i think actually moving to more retention driven like monetization mechanisms like a battle pass like is actually not a bad thing for the industry as a whole i'll throw one more trend in there which is something that we're seeing at microsoft which is really interesting as enterprise live ops this idea of bringing live ops techniques from games into non-games and i i i don't usually

gamification because i hate that word but we're seeing because i sit within a big company we're seeing lots of other divisions kind of perking up and seeing what we're doing games and trying to apply the things like hospitality so i've talked to for example hotel chains or airlines who are looking at how do they take their traditional loyalty programs that are also very boring and sort of try to introduce notions from events and live ops into into things like loaded programs which i find fascinating and obvious to see if there's more of that kind of spillover happening as as every business in the world is essentially transformed into a software driven interactive application excellent so uh it looks like we are just about up on time if there are no questions i'll leave it to you guys to if you want to say one final word otherwise oscar will we'll take it away does anyone want to jump in with a final statement or or perhaps a word of advice for um developers who are tuning in right now who are just now really thinking about live ops in a holistic way in a serious way

for the first time how much time can we have we have one minute sorry go ahead oh i was just gonna say you know a lot of people have experience with these like very huge and cumbersome systems that have you know data science all this stuff behind it but you can definitely go a very very long way with just basic configs in an excel sheet or in a google sheet um and then get what we'd call like you know very mcgyvery with it you know put the bubble gum in the paper clip and try and do this thing so there's there's a lot of ways that you can just be creative and uh you don't need these state-of-the-art systems but if you can afford them they are nice yeah for sure i was gonna say um you know if you're just tuning in and no matter where you are uh as a company or as a studio you know whether you're just coming up with a new game idea or you're in the middle

of development or you have a game that's been running for a you know for a number of years and you're thinking of adding or enhancing liveops i think the basic uh you know advice is the same which is that your core game needs to be engaging on its own uh which is you know on a session recession or day or day-to-day or a weekly or a monthly basis if that's not in place fix that uh then you can think about building your content cadence out and your feature release plan um and yeah i mean i'm not going to say anything that that's new and breakthrough you know at this point but um i cannot stress enough the importance of you know tools and fixing your tech debt and um if your game is at a stage when you know it was built a really long time ago before i walked into the picture for example and the architecture is just not there um it's imperative that you invest the time and resources to fix that uh because it really is an investment in your game's future you know for the next couple of years five years maybe even ten years

um so yeah this is a drum that i will eat until you know it breaks so yeah i know we're out of time so i'm not gonna say anymore great on that note um thank you everyone uh for tuning in thanks to all the panelists for an amazingly enlightening session um as i mentioned at the top we i'm sure we could be talking about this all day um if anyone listening right now is really is interested please feel free to reach out to any of us you can find us through all the pockygamer literature and through the meeting systems um and with that we'll i'll hand it over to oscar thanks david that was great really awesome stuff and and you guys have you know knocked out i think what's interesting for me is i i think we've got the stage where people have seen that we've reached a threshold point in uh the acquisition side of the business you know there's there's lots of effort and work and you know analysis and tools being put into recruiting new players but i think what we're now seeing is that some people are really i'm starting to understand that

it needs expertise to actually understand the difference between making a game and running again and you know i think you know seeing people like you know nick and yourself and you know what you guys are decorating and obviously what we're trying to do i think there are starting to be an increasing number of people are trying to bring that expertise to the teams that can't afford necessarily to hire the data analysts and the operations team and so on and so forth you know unfortunately not everyone's got the kind of resources that patrick has at work you know that kind of awesome scale of game that they've you guys have got um and and you know like so realizing that there are these tools out there the way that unity is acquired marx team and also the chili guys and the way that you've got play fab and the microsoft guys there's this i think crystallization going on right now in live ops and given i've been doing this for 22 years it's about bloody time so anyway um i think you guys did an amazing job um you know i think one last thing i was going to just ask you stephen is like um do you think that people really

understand how profoundly impactful it is to change your mindset from releasing the game to live i'd love to see what your view on that subject is uh i think that certainly some people here right now are thinking about things in the correct way but um as a lot of the other panelists uh alluded to it requires sort of a huge fundamental shift for for the industry as a whole and and it's good to see that that with all of our successes here um the industry is moving in in the right direction absolutely great well thank you very much and i particularly like the way kathy uh sorry uh uh kristen uh a previous speaker talked about them yesterday yeah sorry james yeah my brain not working there two people to read names for i was talking about it as a nightclub uh the idea of uh games that are nightclubs they're not um you know they're not rockstar performances in uh wembley stadium or wherever else uh and i think that as soon as we make that mind shift the better

thank you one and all that was amazing thank you guys thanks everybody