Content and Events Planning for Live Ops Panel | Zynga, Murka, Storm8, Funzio

the focus of the discussion for this panel is going to be how to plan a content and event schedule for liveops and with us today to speak on this topic we do have brock david ceo from merca drew 11 lead pm on words with friends at zynga and jeff witt who is a co-founder at funsio in storm 8 and currently operating his own games design consultancy so in the interest of time because we are running late we will have links to everybody's bios um it's also in the event agenda but wanted to go ahead and dive right in so guys when we think about liveops planning and sort of planning methodology i wanted to first talk about so what are the primary forms of content when you think about liveops and then what kind of planning methodology you guys use for each of those categories of content and maybe we can

start with you drew sure so i'll disclaim that i've not really worked on sort of the the match 3 style games where you're going to have sort of more of the the content cadence that you're necessarily selling forward into and i also haven't worked on the heavier iap driven games where sales are going to be a huge part of what you do right like if you if you had a slots pm uh in my seat then they would tell you a lot about how important sales are how much sort of your your store is a living part of your game and how you need to be optimizing that i can tell you about sort of the the importance of different kinds of content of content novelty and of time horizons right so when you have a game where you're trying to get people to engage right words with friends is an ads driven business primarily you want to make sure that you have sort of a different layer of stuff depending on the the

window that that you're giving people so this is something that our creative director sort of leans on a lot is that you have your hourly events right stuff that you can come into and experience on a hour-to-hour basis that makes the game feel alive you have your daily stuff your weekly stuff your monthly stuff and so you want to make sure that that's sort of well-balanced and then you plan that by looking at your calendar months out and saying okay do we have the right amount of content and the different the right types of content to engage people so part of it's going to be you know just your your regular content treadmill of features right you're going to update all of your stuff that refreshes on a weekly or a monthly basis then you also want to make sure that you have your your limited time events that are going to drive whatever key metrics you want to be inflecting you can do that in a deeply planned basis on a more ad hoc basis right if you're an economy pm that says hey you know we have a really high

clear wallet problem right now you might say oh well we need to run this kind of event and we can get this into market in a week or two weeks or three weeks and so we can get that out and sort of solve this problem that we have that that we have with our business okay brock so i i agree totally with what drew just said i will just add to it that when we are planning our live ops in our activity first of all you have the anchor of product release and this is something that most of the time you cannot you cannot change so this is the first anchor that we are putting in the monthly calendar after it really cut it to live ops level which is kind of between one to one and a half week let's let's call it like a mid time so first layer you you have the long long term retention and let's talk about like a collection collectively something like two months three months then we have this uh one month's

planning which divided to three or four iterations and uh after it you have in in daily basis mini games boosters me offers and so on and you are balancing in that way that if the liveops is a little bit more challenging this time or a little bit more difficult you have in a day-to-day things that are giving you more more more easy content and more keep keep the users on look so really to cut it into like big blocks small block and smaller blocks which give them all all the time the entertainment yeah i really liked how drew said you know talking about live ops in terms of solving problems or improving things and i think that when it when it comes down to it you want to use live ops to either solve uh problems in your core game whether it's a lack of incentives or players not doing what you want them to do or to optimize what is going what is working already so to juice point like if your players have too much hard currency or too many resources

you'll want to hit them with events that will sync some of that if you if you find that your players are monetizing very well on research but you're running out of research you'll want to introduce more research trees or gear stuff like that and this is where analytics just becomes so important knowing where your players are at in terms of balances content consumed and you know the rates that they're consuming content will help you plan out okay i need new research i'll probably need new research trees every you know couple months or we're running out maybe we should raise the level on max research this month the cadence for new content again based on how much uh how fast your players particularly your spenders are consuming it um all that you'll want to look at your analytics and see what is the best sort of way to sort of best frequency to release that and then the more like uh gameplay-oriented

events are really there to in my mind to fix problems if balances are too high you'll want to push them to gameplay that will help them burn you know those balances ultimately uh so once you set aside sort of fixing problems in your game then you take what you know then you look at liveops as how can you make what's working work way better afk arena i think has gone into this pretty good stride where they recognize that their sort of hero fusion system is super deep and does monetize very well and so you see them with everything they're putting out it's essentially just refocusing on stat checks stat checks stat checks to make players seem like they're not powerful enough because they recognize they're monetizing very well just on euro development so for them everything that they do all the new features that they put out a lot of them just have stat check components which will just reinforce players to continue purchasing cells with lots of

gotchas to you know try and increase the max level through their hero fusion system so when you're thinking about like what how to sort of frame your live ops for your game i guess to summarize what problems do you want to fix what liveops co stuff would help fix that whether it's sales new content or events and then take look at what's working for your game right now and what uh liveops stuff will help optimize that and those are sort of my two-pronged approach to looking at uh liveops got it i thought we could kind of shift the discussion to the content and liveops teams and so could you guys speak to the kinds of teams deployed against different kinds of games like in terms of the size of teams the functional roles and i think that when we look in the industry we do see different kinds of teams different sizes of teams like a supercell team could be very small and then for some other games there's literally like 40 50 people on a live

ops team so when you guys think about team structure role size how do you guys think about that how should someone that's working on a game plan for the kind of team that they'll need to be successful with live ops and again maybe starting with you drew so i'm gonna have the least to say just because the ops pros are are the other two people on this panel so i'll let them talk about sort of how big or small to keep your teams i can i can speak to what words with friends does so we have you know a live content operations team they own the the live ops calendar of the game they have the ability to go in and change that and they're responsible for understanding you know what what people are doing at any given point monitoring that and adjusting as needed the feature teams that are developing these sort of systems that people are engaging with right words with friends is sort of a a very systems driven game there's a bunch of different game modes that you can play and when those come around and for how long

is a pillar of how we run our live operations those teams run their features and sort of get them to a point where they can be continuously operated where they're more or less done optimizing that feature and so they optimize it in liveops right they release it and they say okay how can we add to this how can we how can we improve it at some point you sort of have to say okay we're mostly done investing in this they build up the playbook they hand it off to the lco team the lco team runs that feature more or less in perpetuity and so that that moment of handoff needs to be accompanied by a requisite number of tools that allow for the the ongoing operation of of that feature right you you can't just keep piling up work on a team that's going to own your entire liveops ecosystem without also giving them the the ability to scale themselves right and so if i had to sort of lean in on on one part of this it's that you can't just sort of build stuff ship

it and then walk away for for the next thing that you build you have to make sure that if your team isn't going to grow which is going to introduce inefficiencies as i'm sure barack is going to tell you about there's there's an element of making it more efficient to work on it on a sort of per unit basis and so i would go with that brock yeah so i think that the the the amount of people that you talked about the size of team is a little bit the further discussion but in case of what what to do and the how we're doing it so we divided for a long term teams that which is like similar to the features team which develop and then we have a search of a trans transition period in the transition period you have the dynamics team in the operations team that really managing the day-to-day in inside the games and controlling everything but when we are launching a feature there is like a transition period where you need to test it to a b test it

sometimes you you need to roll back or to change things if it's like feature thing or if it's bugs so we in the first time it's really like like a pinpoint and so it goes and back until it i think if we are talking about heavy feature it's something about basically after 102 iteration which is like totally goes to the operation in the dynamics team and we are really data-driven and motivated when we are launching a feature so in the dynamics team we have a lot of analytics that expertise by the way inside the feature so we have like analytics that expertise in the live ops that are running day to day the balances and so on but also you have the the ba team that is responsible to understand the reaction of the users and the reaction many times uh related to how the feature was developed what is the content and not only to the balance and things like that so it might be related to the to the future team and i think that the the character of

this of this team might be like a little bit 24 7 team they need to leave the game that comparing to the feature team is more like long term okay we release we are less feeling it but the the dynamics team the operation liveops team need to work all the time like 24 7 to understand to read the the mind of the users and then they really the way of working is is completely different and if we're talking about medium and big games i think that these teams need to be like at least like 10 people the divided between operators uh monetization managers and the analytics something like that yeah yeah when it comes to at least design and product i find it helpful to kind of separate between feature development and live operations just because having one team two both they tend to focus more on one than the other to the detriment of the game but you do need communication between

the two teams because otherwise design for feature development will design a feature meant to be used some way and liveops will use it completely different i would say for designers you know specking out features um one section i like to have is how do you intend this feature to be used for live ops pretty much any significant gameplay feature game feature is going to have a live ops component whether it's for events or um sort of how's it going to you know inflate or over time what is the sort of content release cadence for stuff within this feature and so at the very least if you're a designer and you're designing a feature have a section there that sort of gives your vision for how you intend this feature to be used in the liveops capacity so at the very least you're going to have the live ops team could could look at that kind of see what your vision is and if there's a conflict between the liveops team and and the the feature design team that very least you could catch it early sort of figure stuff out so that the

liveops team doesn't run with a feature or with content in a way that it wasn't designed to work like that okay and so maybe the next question i want to ask because i think there are probably a lot of folks in our audience who are actually currently live operating games but if you guys could talk about maybe some of the biggest lessons you learned or the the single biggest lessons that you've learned to help some of the other folks in the audience avoid the big mistakes that'd be great to hear so again maybe starting with you drew in terms of like some big lessons you've learned operating words with friends lessons learned so a couple come to mind so one is understanding what you can do and in what time frame is deeply important right so i talk about product management being sort of a very interdisciplinary discipline and so part of that is being able to know what tools you have on what time

frames sort of being your own producer in that way and so a good example of a fire tool that we had to go through where we got to relearn this lesson was actually during sort of the initial wave of code right everything sort of shut down from i don't know march to may or even june and a bunch of game companies saw a pretty substantial influx of players and a bunch of people returned people obviously had a lot more time on their hands and so we we had to figure out okay what do we do with all these people right we have all these people coming back to the game they have expectations about how well it's going to keep them entertained what can we bring to them that improves their lives sort of gives them the experience that we want to give them and we have to be able to flight this inside two weeks right and so being able to know what you can do sort of at the top of the hat having events that you know hopefully aren't too art driven that you can turn on in a

pretty short time frame and say okay here's our normal calendar and here's our calendar if we just like turned it up to 11 right we have to really run live ops as hard as possible what does that look like what are all the things we need to do when do we need to start doing it and when can we sort of have it live by and so understanding your game and understanding your capabilities and building for those scenarios right investing not just in your next event next month but in your capabilities to live operate the game and scale it up on demand are are deeply important and any any live option should be able to sort of assess that within their game and know whether or not they need to invest in that for the long run all right so first of all i might agree with the point of understanding your capabilities and really the the scope of the team and what you can do it's very very important because liveops is very sensitive and when you're doing a mistake it's affecting

dramatically and very fast on your users so this is a first point that i completely agree the second is that sometimes less is more and when you have a hard time in your game and you feel that you are running out of dow or you have a small dope in revenue you are starting to use so called the big guns and when you are putting okay you are starting something and sometimes levels taking time to affect yes it's like a chain but you're saying okay it's not enough and then you're pushing something else and then you put something else and then the the balance is becoming a little bit more problematic and the user are becoming a little bit confused because they don't know what to chase and you need also focus on your users so i think that sometimes to understand it okay the relax okay let's put something according to the plan let's put exactly what we think maybe to add something else more but not to to do an x out in anything so really less is more and we learn it a

bit in the last period and the last point which is my third point is that economy is uh super important if in live ups especially when you're releasing a new live ops and understanding the difficulty and the segmentation of each user to not not create an inflation of coins or whatever and to create the right balance between the effort of the user and the the gift or the benefit that he's getting you have like the the point that it's need to like you need to catch this point which is very very important especially when you are releasing something new and you want to have the user you have a good experience good job yeah so um i think you know with live ops it's important to always try and be optimizing on something even if for events that work well or for sales or new content that performs well there's always improvements that you can make so whether it's you know

in an event with a leaderboard sort of tightening the leaderboard tiers to try and create more competition at different parts of the leaderboard or using like a goal system to plug in sort of like additional incentives and parts of the event where players are maybe not engaged as much there's always something you should be able to do and i will note on barack's point like um inflation definitely is something to be careful about because we've seen it ruin a lot of games but it's also kind of it can be an important way to keep uh your players especially your spenders engaged and spending slight inflation over time will keep your you know your spenders trying to always optimize your stuff but it has to be slight enough where it won't deprecate their spend too quickly um so for especially for rpgs and strategy games i do think inflation can be an important tool just very mildly used but yeah always look at

all the tools in your bag to try and optimize what's currently working and then be inventive like drew was saying like um sometimes like if you're hitting sort of a lull and spend or engagement or whatever try and use your existing framework to put out like an event and then just be experimental and see if it works like and um sometimes it'll work sometimes it won't but it's too easy for i've seen a lot of game teams become complacent and like their live ops strategy because they've seen it work in the past and they you know they do the same things all the time and then that over time is just going to get less and less effective so just always try to be optimizing uh however your content or live uh strategy is so i just have one last question we'll open it up to the audience so last question is around just generally speaking when we talk about content optimization i was hoping you guys could talk about

specific tools or frameworks or ways of analyzing your initiatives to let you know if you were successful or not and i know you know i think zinc is famous for like you know um kind of like having a beat strategy as well as expected outcomes in terms of prioritizing features but starting with you drew if you could just kind of generally speak about some of that stuff for you guys that would be fantastic sure yeah so zenka has you know like you said joe the the sort of bigger beat investments and also the the sort of smaller quicker hit investments i think when it comes to live operations our optimization framework is heavily driven by experimentation and personalization right one of the things that when you sort of get up to a certain scale that you get to do is give different experiences to different people right a lot of people especially at smaller studios worry about the the nature of oh well if

i give you a fundamentally different experience right different numbers to different people you know everyone has access to the internet people have forums people talk are they going to get upset at me or do the benefits outweigh do the cost outweigh the benefits of giving people fundamentally different sets of tuning zynga games have for years learned that the the core purpose of entertainment products is to entertain people right if you're not giving people a challenge if you're not giving them something interesting to engage with then they're going to leave your game and find it somewhere else and so that your your core responsibility as a game maker is to entertain your audience part of live operations and part of the the tuning of that is to give people a good experience the way that you can measure that is sort of many different ways right we we sort of think about our games as business owners right we say does it make money do we have a lot of people playing it and i think that that misses one of the purposes of metrics which is to understand sort

of the the game as it is experienced by your players right people like your game and so they come back retention is an outcome it's not sort of the the sole driving purpose it's a way of describing that people are actually having a good experience similarly when you look at your live ops you say okay what is what is the thing that we want to do with this right we want people to i don't know play the system more but why do we want to make them play the system why is that a primary good oh we want people to play the game more because we think that you know increasing this amount of novelty for this amount of time is going to get them to retain better or we want them to play the system because we think that they'll really enjoy it and so you you start from the point of saying okay here's what we're actually trying to do and then you say well what if that if we were succeeding at that right in a perfect world where we're definitely crushing it what is the metric that actually tells us the story of we are we are successful so i'll give you you know a clean example we have a feature called

called solo challenging words with friends right and it's on a weekly cadence and you know it's it's a ladder you you play against an escalating sort of difficulty set of bots and it resets every monday and so one of the things that we look at is very simple it's just feature retention so if you played it last week uh what percentage of you played this week and you know we look at many many other things right it's it's a revenue driver for the game it's something that interacts with our core loop right some people play the core loop less because of that we care about that um you know people play that a lot it drives ad views and so we sort of track all of that but what i want to encourage everyone to do is think about the context in which your live operations uh sit in your game and what you're trying to do as a game maker for your players and then really think about okay what if we were succeeding where would we find

it and if we were failing how would that manifest and then track those metrics it's not a one-size-fits-all solution yeah right yes so to the point of making a challenge i completely agree i know that this is like continuous debate about the whether to do more hard challenges to let's say more engaged user but i believe that again this is entertainment and i completely integrate if it will be too easy it will not be fun at all because it will not challenge them uh so so this is a a first point and second when we are developing a new feature which will become a live ops we are trying to give a certain kpis about first of all to categorize it if it will be a retention uh live ops if it will be a revenue if it will be for new users so for uh for more engaged users so we are really putting the kpis in numbers and in percentage how each feature are going to uh to to increase our our initial

kpis and then we are put when we are putting it on live we are trying to all the time to change it and from iteration to iteration that we are learning with our segmentation with our maybe a many a b testing in each group so we are really segmented it and that is testing group by group the the challenge level and also the the balance like what they need to to spend in order to to achieve it and the length of it the finishers and everything and it's becoming all the time and i think that it's becoming perfect after something like four four even five iterations when you are really learning and all the time even like you can have a live ups for a few months and each time you will change something small and you will see it's becoming better so it's like all the time it's continuous working on it and i really say that to define first of all your kpis

categorize it to define the kpis and then to adjust it all the time according to those kpi to achieve your goals okay take us home jeff um yeah you know i i definitely agree with what both drew and barack said like i will say um you know one good thing about always iterating on your live ops events is it keeps players from having an expectation of what certain events are going to ask of them player expectation is one of the things that you should try and generally avoid setting because then you're kind of locked into it so if if you if you run an event and and it's the same you know five or six times and players suddenly learn that this event is always like that and when you try to change it it may disrupt the engagement of players for that event because they're used to having it be a certain way so having always trying to optimize something tweak stuff i think not only does it keep it fresh for players but also lets them not

expect to see certain things every time they engage with the event which uh gives you more leeway to optimize them um and i will say i think there's a nice balance between trying to you know monetize off of events and other live ops and um you know being too hard on the players i think it might be important for if if they find that a certain sort of liveops strategy is successful but ultimately taxing on the players sitting down and just really trying to think is there a way that you could achieve similar results without having it be too much for the player um i do believe that there are designs of either events or sales or other stuff that you could probably get similar results but just it's better for the players quality of life in the game and i think when you when you feel like you're burning your players out maybe just see if there's a way to fix that without

hurting your bottom line okay all right so that is basically it in the first generation of mobile gamers came off consoles from titles like super mario warcraft and league of legends now as we enter the era of mobile 2.0 with three and a half billion devices global gaming has mainstream easy to play wherever you are and something for everyone the categories have proliferated the best gaming companies in the world need deeper insights a framework to capture the market opportunity and now introducing game iq a revolutionary analytics product that delivers a taxonomy of genres sub-genres and comprehensive feature analysis matched with app annie's legendary market data

you have the insights to level up your performance designed exclusively for the rock stars of mobile from heinrich we have a discussion we often have when operating our game is when we are doing too much and overwhelm the focus players for example too many events in the calendar competitions shadow what is your take on evaluating what an optimum is in regards to that i'm happy to jump in on this one so i think it comes down to measurement right you you're to build on what jeff said before you do want to optimize on something but you also want to make sure that you're not substantially cannibalizing whatever it is that your game is trying to do right so if you're if what you see is yes you your new event drives a bunch of spend but it's substantially cannibalistic of all of your sort of core game spend

to the point where it's not actually driving a bunch of net news spend then you may be over investing in whatever that new stuff is um because it's it's primarily replacing rather than incrementing whatever it is that you're trying to do and so uh focusing on measuring that is is a super important part of your live ops okay next question what are the best ways to assess the effect effectiveness of in-game ui and ux and make recommendations on optimizing them i haven't seen anything from a ui ux perspective i don't know if you guys have seen anything like that so our approach here is generally a lot of taxonomy around sort of what people interact with and i mean when i uh at a previous job we had amplitude and one of the best things about amplitude uh hopefully no one's no one's paying me for this this is a genuine love for amplitude

uh they have something called sankey charts right so they basically say like everyone who fires this event goes on to like do this other thing in this distribution and the idea is if you are you know clicking around and you interact with this particular game element do you go on to do this other thing more or less right and so i guess from a purely qualitative quantitative sorry perspective you can say oh people who click on our store are more likely to spend it's like okay that's not very interesting but you say oh people who you know complete this this action are more likely to spend okay that's a little bit more interesting how do we streamline our game to get them to that magic moment a little bit faster and does that correlation hold up even after we optimize for that user flow right and i i guess you know you could just measure kind of like you know do some of that hot spot analysis and just conversions and stuff like that on different ui elements but yeah i personally

am not very familiar with any other methodologies there the next question from the q a are what do you think are the pillars of an effective survey feedback collection system uh i guess so from my perspective i would say one i wouldn't use a survey feedback collection system i think you've got to match it to the audience in like for example like some of the surveys that are run off of offer walls and that asks these deep complicated questions for someone who's just trying to get 10 gems real fast is probably not the right approach i think that um surveys and feedback collection systems that are more action-oriented right which is why a lot of like the hyper-casual guys try to put as much in the creative as possible and so they're really their feedback is really around actions are you clicking on you know the ad or not are you tapping through on a fake app store to try and

install a game or not and so those types of kind of surveys or feedback collection systems that are about actual actions seem to be the most accurate but um any other thoughts on feedback and collection systems or surveys paul i mean at riot you guys you guys do a lot don't you in terms of we do we have a central insights team and and there's a variety of proprietary tools that we use and we use qualtrics and some other stuff that we integrate um on league which is you know built on ancient ancient code from billion years ago we've had to implement a lot of proprietary stuff but we use something called the player feedback tool that targets different cohorts of players after they do certain things and we can kind of hit them with surveys whenever we want it's a little janky but it works we've been able to to implement stuff like that and then we have a good um analyst team who can help parse that information to help the pm's uh make adjustments but it took a lot of investment to get there but it is helpful if you have those

people on staff to do that right uh next question do you have any frameworks for looking at the health of your in-game economy and evaluating what can be optimized from our pricing content quantity and content type perspective jeff do you have thoughts on this i know you do a lot of work on game economy i mean really it comes down to like looking at okay what are player balances um are they consuming what you're putting out how fast are they consuming it and you know compare that with how frequently you're putting out new content um basically it's just i mean new content and player consumption are you know there's a push and pull there and you know if if something's not working then you have to kind of look into the the data and see why players aren't responding to what you're putting out and um in some cases it may be that you know if if it's an rpg maybe

the stats on the new content aren't good enough and maybe you need to inflate a little more um or if if that's not the case it could be that there's not enough challenges in the game and you need to you know add more pv stages or make your events more difficult or something like that um but all that is something that you know your your analytics should be telling you as to how players are behaving and you just make sure whatever your what plan you're coming up with that it's really targeted to fix the problems you're seeing in your analytics don't just do something because you see other games doing it and it seems like the right thing to do uh each game has their own problems and and your your live ops should be balanced to address those problems okay and then from sashank vendrangi do you see the live ops tastes of a casual player evolving like are people more open to relatively mid-core systems because players are now

fairly used to match 3 and other mechanics any thoughts guys i don't really look at casual players very much but my my belief is that there probably has been a little bit of evolution in terms of it's kind of like how a lot of the games that were considered hardcore a few years ago are now considered mid core so i would assume that that's kind of happening on the casual side as well but that's only a guess for me any other thoughts on that uh yeah i tend to agree i think there's a couple instances of casual players doing things that are i mean i would at least consider a little hardcore behavior like whether it's you know for a farm game logging in every 15 five minutes or 15 minutes to deal with crops or with idle games dealing with like prestiging which is definitely not a casual sort of feature um so you know i think

how do you how do you sort of bridge uh casual players to a more mid-core title um that that's a good question i think you know it'll take a good design for that but i i definitely believe that casual players are capable of more complex gameplay if introduced to them correctly uh next question from jvmhong javian what's up what are your top three games that have done live ops really well i know brock's gonna say scatter slots right i said there are many good and successful it's very very complicated the question and i think that you can see a lot of similarity between many successful games in the industry basically they are all so-called taking inspiration between one to each other some of them are more innovative some of them more going by the stream but i think that by looking on by the

way new new games that are being launched here you can see in successful games that are start the ranking that you knew you saw you see you were ranking very fast becoming a top ranking i think you we look at them and you can learn a lot i don't have something that's specific to the point right i mean i think that i'd i'd have to say you know fortnight they're kind of going crazy right but they're they're kind of unique i think i think a ballpool's done a good job especially with the implementation of the battle pass i would just say just be careful because the danger would be applying battle pass to a consumables based economy don't take that success and try to replicate it in a different game with a different kind of economy a lot of playwrx games have done really well from a live ops perspective as well so those are some games jeff what are some of the games that maybe on the rpg side um so you know i have mixed feelings

about afk renix on the one hand they've recognized where their bread admirer is which is hero optimization and everything they they put out is is just about you know again hitting players with those stat checks part of me feels like i mean it's working for them but at the same time i think they are not pushing harder on more social and engagement oriented um events which i mean i think they get a fortune i think there's opportunities there to really have fun gameplay and and then and make you know a little more money um on the flip side you know legendary is an interesting case and i would never want to be in legendary shoes because it sounds like a nightmare to to run their live ops cadence but it works like they utilize liveops very effectively to try and optimize their revenue it is

a little too burdensome on players and it seems like a nightmare to um to run um but you know if you're looking at a game who's trying to really like take their you know initial success and you know uh take it further uh legendary you know at least was was doing really well with that you know two or three years ago i have not stayed on top of what they're doing um versus you know uh raid uh i feel like is not doing a very good job with live ops they seem to feel that more engagement is better and not they're they're focusing more on engagement than trying to actually boost their monetization which i think long term is going to be problematic if you're trying to sort of grow your revenues and i think they've set themselves up so they could probably implement some unique events that i think could monetize pretty well but they are not for whatever reason um so yeah there's a comment from the audience that

township's events are pretty cool by the way from sashing fundrangi you know there's also a little game called league of legends that i think has been doing a pretty good job in live ops paul from a riot perspective are there specific games that you guys look at like those guys are doing things right or in general i'd say there are some games that we definitely kind of admire i think we've learned a lot from just being fans of blizzard games and how they've done the live ops and services especially wow you know i think we learned a lot from how they do things um but i'd say i'd say to be honest um we've taken some good learnings from some games in korea you know a lot of the nexon games in particular of what felt right to offer players what type of events to offer players and then in time just evolving league as we got more refined with our hires and with our methodologies internal to know like hey what what moves a needle in certain ways or not like you know it took us a while to get to the point of really understanding

you know the different competitive cohorts and versus a normal player cohort versus a casual player core and as we kind of built those capabilities we're able to tune accordingly but um but yeah i don't know and we have a another audience comment pokemon go was terrible at first but has really kept the dedicated players engaged i have to say like i don't play pokemon go but i i was surprised that you know through covid which for a location-based game now the games actually continue i mean i don't know what they've done but they've done something right so that is pretty impressive and maybe what we could do is just take one last question and we'll wrap it up with this if you had one book to recommend that influenced your career work the most what would it be uh jeff what do you think um i really liked daniel kahneman's uh thinking fast and slow behavioral economics um it really is what sort of defines

free-to-play in my opinion and so um that it kind of taught me to not try to try to sort of get away from conventional thinking sometimes as sometimes it's just plain wrong so i would definitely recommend that brock so there are many many books uh but one of the great books that i read is uh um why first leadership like start with why and this is a a very very good book uh but i think that it's it's it's how you are how you are managing and what you are doing in order to to lead your people and to lead you your business to success so i think this is a great book all right and paul i know you put something in the comments but uh can you talk about it yeah i'll talk about it so i'd say if i i have a stack rank it's it's uh plain to win um what strategy really means and it's a book um written by some people at um procter

gamble and it goes through defining product strategy is outcomes what are the outcomes you're trying to do and what do you and what it goes into is what is what you're not doing is just as important as what you are doing and that can help you cris crisp up where you put priority and put your resources and that was really seminal for me in my development i've got two answers myself so the book that has impacted me the most is this book called principles by radelio and the book that's impacting me the most right now i'm actually listening to it on audiobook but it's it's called you can't hurt me by david goggins he's a former navy seal and for a guy who has not had like a really who hasn't had like a lot of business experience or like even like deep education i mean the guy's like a genius so anyway i would recommend that as well but all right guys i think that's it thank you uh speakers for staying late and answering questions um i definitely learned a lot and i want to thank you guys

so much for your time thank you everyone for joining the conference and definitely want to do more okay folks glad you could catch this presentation so good to have you join us this was actually part of a larger digital conference around best practices in live ops so click the link to basically get the agenda for all of the other talks that were part of this conference and links so that you can watch the other talks as part of that conference also if you haven't checked it out yet check out appiany's game iq product game iq is a great way to get market data on the mobile gaming market and app annie was the sponsor of this conference and finally if you want to be notified of future talks and virtual future virtual conference events please make sure to click subscribe to the game makers youtube channel and click the bell notification so you can be notified of future events coming up we've got a lot of cool stuff coming so stay tuned and great to have you with us catch you

later