Keynote: From Live Ops to LIVE OPS PG Connects Digital #2

so we're going to kick off with a company that uh uh we're all familiar with uh we've got uh uh zoe williams here from king who's gonna talk to us about going from live ops to live ops and i'm really looking forward to seeing what she has to say about that so i think um hopefully uh zoe will be joining us on the screen shortly yep hey zory how you doing um so i'm really looking forward to this zoe i'm going to hand over to you uh to introduce yourself over to you great thank you so morning everyone let me just share my screen for you so hi again i'm i'm zoe um i'm in business performance at king so my

home is within the farm hero saga game studio but i also do some work for candy crush friends and ads but as today is about live ops i'll focus on my experience within farm where i spearhead the live ops team alongside a colleague from crm production and game design but before we go there um just a super quick view on king as a company so we've developed more than 200 games which are enjoyed by players worldwide we actually had 273 million monthly active users in q1 um so king's been part of activision blizzard since 2016 but as a company we were founded in 2003 and we have our major game studios in stockholm london barcelona mammo and berlin so i mentioned that we have a 200 games our players play but um there's four super recognizable global franchises so just quickly on those we have the candy crush saga

um so that's where candy friend sits candy soda just to name a few uh we have pet rescue saga and we have bubble witch saga and of course um we have farm hero saga which is where i sit and that's in the london studio um candy crush friends is also in the london studio but going back to my talk title so from lowercase live ops 2 upper case liveops um that's kind of what we're going to delve into today so we'll look at what a live op is across the industry or we'll look at what i mean by the lowercase liveop and the drive that we've taken to go from lowercase to to uppercase so best place to start i figured was probably at the beginning so what is a liveop what does the industry the games industry view as a liveop so i've read various articles i've spoken to other people in the industry i've even googled it um but there is doesn't seem to be a set like a standard three or four items that define a live op i think it's a different per game people view it differently

so you just take direction direction um from the generally agreed opinions so instead of defining a live op industry wide i've just picked my favorite descriptions so this is actually um an article for pocket gamer and it was that live ops is about introducing features as chocolates in a box so each are a small nude light that keeps you coming back for more and i really love that one i think that encapsulates really what we're trying to achieve with our live ops um they keep the game fresh um increasing the life of the old content by just giving a little bit new um but also from another point on a more tech specific site it's about making changes to the game that don't interrupt the player experience so we can put our live ops in without the players having to stop and re-download etc so it's all about being agile and bringing bringing new content forward so in terms of farm hero saga specifically our live ops are all time limited they all use new content in addition to our standard gameplay and they use over-the-air content

updates so again that's making the changes without interrupting the gameplay now for those of you who aren't familiar with farm as well as saying you should go give it a go i thought i'd just give you a quick view of what the main game play looks like so it's a it's a match three or more puzzle game so you can see at the top here that um this player has 12 moves left so they have 12 moves left to collect all the targets up here so in this case the player needs to collect five more apples to complete the level within 12 moves you collect targets by matching three or more so if you look down here we see we've collected players collected five waters and we call them proxies so five watercolor seeds so that'll be five going up to his target on top now this gets more complicated as you go through the game we introduce things that we call blockers that i won't go into now but they just make so in terms gamepl what our players see when they play live ops they still have this this core gameplay so we have our site

our saga which is the main progression the players are going from level one to level 3600 wherever we're at at the moment we're continually releasing new levels for the players um and there they um exist on a live system so you have five lives and if you lose them all you have to wait a certain amount of time to recharge what we've also done though on the side is built an energy system so when players are playing our live ops which i'll give an example of in a moment um they use little balls of energy so they have five that they can use there and this just means that by playing a live up the player isn't losing an opportunity to continue to progress in their saga um we found our players particularly enjoy this um the behaviors we see are that players tend to burn out their lives on the saga progression and then move over to live up and of course also some players just enjoy the live op and other players are stuck on the saga so live ops and nice escape so getting into more of my title um the lower case live up i've grabbed a few examples for you

so if you see here this is how we display a live opt to the player so this is what they see um in this case there's nine levels and they need to play all nine levels they get rewards as every three levels um to get to the big reward at the end to essentially win the live up so here they click on the flowers to access each level so we call this one flour um we have another one it's got chickens and eggs we call it chicken so if you see here i've called these reoccurring live ops these players are showing these throughout the year multiple times i'll go into them in a bit more detail later on but for now those are our lowercase account reoccurring now in a small step towards the upper case so i guess we could call this a capital l libor perhaps um we have our unique liveops um so there are two types of these some of them are directly linked to to farm itself so we had our sixth birthday so we built a live art just for the sixth birthday the players would just see it once and it's heavily themed so in this case here's the frosting that

are the level stations that's how players play the levels um so another example here was a level 3000 players again that's linking to farm so in terms of our other unique live ops that we do these are more linked to events that players are experiencing in their everyday lives um so for example here winter but we're going up to the holiday season so we we gave them the live ops theme towards that another example could be new year's eve so we may give them one that's more about celebrations and fireworks for example so that that's kind of our lower case live up view so they're all visually different they are giving players a new way to play through content but there isn't any new mechanics so how do we transform from these lowercase to the uppercase the the best way to describe this is with a chart which obviously is about right for me given that i'm in business performance um so our previous schedule we um we aim

to run live ops every week of the year so bearing this in mind on our previous schedule our previous liveop calendar we were looking at 60 unique live ops so that was really taking advantage of things happening in the world that players can relate to which makes the game more relatable but also taking advantage of things within farm so that players felt close to our game forty percent are that reoccurring light bulb so that's like the chicken and the flower now that seems pretty high but there's a couple of reasons for this um for one this is where we do all of our well a lot of our testing it's a great place to ensure that we're still hitting the mark of what the players want so we can ensure that players are still being driven by the reward system we're using that the players still enjoy the sort of um blockers that we're using within the live arts this is a great chance just to learn and understand and also it's not possible always to have brand new unique content coming out all the time so this gives us the breathers and the gaps between where we have our new content drawn um drawn up and then moving to the current schedule

so that's on the right here and you see that this event this uppercase liveop um is has now been added so we've taken mainly from the unique live box to create this event live up um so the reason for this is we want to keep our reoccurring live ops that be that that testing um ability that we have in the filling the gap the players still have content and we don't have to design new new items for it that's still really important so 30 is it's still the reoccurring light bulb although we are working on brand new um skins so that's the visual you see um so we're working on brand new skins to go into our rotating schedule of reoccurring live ops just so it is a little bit more very for the players um but the biggest question is around the 20 unique now unique is a step towards the uppercase live up as i mentioned so it made sense to me to take from the unique and put into the in the uppercase liveops section the reason we still have 20 unique again comes down to what are we able to do with production

what support do we really have so and also it works well for events like for example halloween so i mentioned bubble witch saga that's one of the other king franchises and it has a witch so that's actually more suited to halloween but it's also an event that our players live through so it makes sense in that case to give them a unique live op so it's going to look like halloween it's great but it's not our area it doesn't link to the farm specifically so rather than invest that dev time in an event around it instead we provide them with the unique content so that's kind of where the unique ones are coming up in our current schedule so in terms of making this move um what what changed was there a change in our thought process was that was there a change in our priorities for us to kind of move from lowercase to this uppercase liveop well it was actually um a structure in the studio a structured change in the studio which enabled us so we now have an events track so the studio is listened into three tracks three views and one of them is events

and the goal of our track is to keep the content exciting for players by keep providing them with a full and a fresh live ops calendar always meeting player needs and introducing new game player mechanic where possible so why is this suddenly possible why why does making a track make a difference well it's meant that we've got a dedicated development pod so this means that we have clear visibility on how much time developers have enables us enables us to plan more efficiently so we could look and say right well we have a few options one small ones medium one's large and kind of in a tetris way fit them into the development calendar um which means that we can get a lot more out previously we did have the development support but as their um priorities weren't directly linked to to the events to the liveops track it was harder to slot in what we required in addition to that we also have an event central team now so again events uppercase live um central team now and that that team actually has time to dedicate to the live ops um

which again just ensures that we're really focusing on the right things and really really driving forward so on that um this events team and we have a member from crm um and the big benefit here that this point is the around the marketing and player insights so they have great relationships with those teams and they can ensure that when we're doing our uppercase liveops they're in the best position to ensure that we're getting the marketing support um that we're really working with with the guys in the community to really reach out to the players and another great thing about the crm side is they have access to tools which means we can put things in the game without using developers so this is great for small things for proof of concepts so for example um previously we ran we just we tried out um challenging players to win play 20 levels in a row and within 24 hours and if they manage that they got a high value reward just we wanted to understand whether that sort of thing drove our players and we were able to do that with crm tools and then we could take those learnings there and decide if it's

something we want to build into the game more permanently in the future um we have a business performance representative so that's myself um that we're all about testing learning and optimizing so as well as looking at the new and making data driven decisions on where to go next for liveops we also look at what we currently have is it still meeting player needs our players still interested in this can we make it better so it's just continually surveying the whole product calendar from lowercase to uppercase we have our game ux designer and so the focus here is on the player journey and the player wants as well as obviously designing content they're there to ensure that when i have my data and have my alignments from data that we're still thinking in the eyes of the player we're still putting ourselves in the player's shoes and then finally production um so this is all about the process they're the umbrella function they kind of they bring all of us together including the dev team and they're there to they're there to help ensure that everything moves run

smoothly everything runs together as it should and we have this really well i guess uppercase live or machine um so as a team it's our responsibility to we run ideations and and create briefs for new things to go into the game and we base it all on data of course um we then kick off um the we kick off spec with new specs with design to start that process and then we put it into the year live op calendar so we're running the calendar as well and then when when it comes to the events going live we're the ones doing the setup and we're doing the monitoring so we're really responsible and accountable for the whole process so at this point i'm pretty sure i know the question on everyone's minds can we have an example well sure um so this example here is called emilia's adventure we ran it earlier in the year so emilia is one of the main characters on farm and she's a little pig up top and um

for this uppercase live up we had the new art and the new level set as we could would for our unique live arts but the new mechanic in it was this new narrative layer so this was focused on adjusting how the players interact with live ops so previously they play maybe because they'd like to get to the end so in this case it's 18 levels so they display to get to the end rewards throughout but we wanted to understand if our players would engage with and enjoy going on a journey with our characters as they played through the live up so in this case when you went into the live ops screen um for amelia's adventure and there was a story panel that came up and it was emilia's plane's broken and needed fixing help her fix it so as you play through the live op you're getting more bits of the story telling you how you're working for amelia helping her fix her plane and eventually get it up into the sky so this was kind of the uppercase element but brought it forward this was the new mechanic we chose to focus on for this uppercase libor um and it was it was a full event which we're trying really hard to do with our uppercase liveop

events so we had full marketing support as well as featuring on apple and google we involved player support in the community so we um in the forums we had we built up some hype around emilia we had conversations going on in there uh we even ran a q a session with our head of farming community so it was the the interview was pre-done and shared with the players and that was all done taken very well by the players and again all of it is linked to this kind of strong emilia and her adventure so i realized i've given a lot of information so just to recap on kind of the live ops within a farm and we are reoccurring liveops so those are the ones that are around businesses our business as usual so they feel they fill the gaps but they're also a great opportunity for us to test and to learn and understand what's driving our players we have our unique live ops which players only see once and they tend to be linked to either events going on in farm as a game or

events going on in the world so players can kind of connect to them a little bit more so sixth birthday was an example but another example was i gave the winter but also we just ran a spring event that could just be down to the seasons and then finally we have these uppercase liveops these events so here's mutants adventure and here's just an example of those kind of story panels that players are seeing as they played through so the next steps i mean where does one go after full capitalization you know once you've achieved this uppercase liveop well the truth is i don't think you ever truly achieve it you're continually learning you're continually adapting to your players and continually building new bigger and fresher things so i think there's two key paths to follow the first is around data um so we and i mean this both quantitatively and qualitatively so in terms of the quantitative approach again as i've mentioned we're running tests we we're always trying to understand

what the players are enjoying what they're not enjoying what does work what doesn't so some questions we may run tests to answer a one could be what's the value to the player of providing a new level set of levels versus the new visuals so that helps us understand what what players are engaging and interacting with most when they play do they do they mind if they're playing the same if they had to play the same level twice but the overall the visual of it is different um or would they prefer the levels were all super different but the visuals are similar so that just tells us where to focus our efforts um another question is so i mentioned that we have the saga progression uh it's good to know do our players want content that takes them away from the saga or should we be looking to kind of take our live ops in a different direction a direction that allows them to interact more with their saga progression um another another one do players prefer community play so do they want to play

together as teams or do they prefer competition and again this is a super important question because it really helps drivers and again deciding where to go next where to take our calendar next um and again another one here into reward types how can we better drive players or rewards or again what's the best way to surface to the player on that's the quantitative side in terms of qualitative we now have something called voice of the player so this is run by our player insights and it's it's every other week um and it's a chance for us to hear what our players are saying so it's information taken from surveys that we run after our events um from chat in the forum from complaints to player support we have an idea section in the community where players can put ideas for changes to the game and then they can vote on ideas that they like so here we could see things that players want us to get rid of perhaps or things that they think would make their gameplay experience better so all of that is put together

in the voice the player sessions and then given to us the game team so that we can always take these things into account as well so it's a really important combination so that's data and the other is tech so tech's essentially keeping our game alive the tech team are always looking at the game as it is as a blueprint to understand how to to be better how to build better so what are our current bug fares what what's currently negatively impacting the player experience what could we add to make the game smoother so these are questions that they're asking every single day and working on every single day and it's equally as important if not more so than the data because as i say it's the tech that keeps the game going so on that live ops success uppercase live success how are we defining success well the first success is player engagement so we have metrics from our tests so um i may run a test and look at the game rounds so our players playing more rounds as a result of the content we're giving them or i may look at daily active users so

our players returning to the game more as a result but then also we have the surveys we have chat in the community as i mentioned voice of the player so that's a really good way to measure if we're being successful we want the players happy and success too it's yes it's money it's the in-app purchases are our players being driven to spend in-game now we find that if you have success one then success two will usually follow so i mean on that ultimately for the lowercase liveops the uppercase liveops events special content special features and any other thing that we'll hear live ops referred to as today um at the heart of everything is the player so they're individuals who continue to return to our game because they like or love the content that we're providing so as long as we're ensuring that the players are at the center of what drives us we can't go too far wrong so thank you very much for your time this morning um this is actually hunter on the side here he's another one of my main characters any questions cool we have loads of questions and basically you just made

the whole of the track redundant because that was so good you covered out so many areas um okay so we have uh loads and loads of questions so i'm going to start off with psy and so is saying what tools do you use for uppercase liveops in terms of a b testing analytics and things like that um so we actually have an in-house tool that's been built for um for our tests so usually we're usually always running an a b test so and it's using the in-house tools so when we set up um an uppercase liveop or an event we tend to have a small segment of players who aren't seeing it and the segment of players will see maybe so for example with emilia's adventure the uppercase liveop we had a small segment of players who didn't see it with the narrative and so then we were able to run the tasks using this in-house tool yeah i mean i think that in households great um because they let you choose the things you want to do obviously not every team has the chance to to build that because

um i mean i've worked for people like unity in the past and it's like it's always amazing how many people try to reinvent the wheel but obviously as king it's a lot easier for you guys you've got your own engine for goodness sake um so what's the best way to balance data player insights uh marketing and business i mean i think there's a particular side you had which covered this but i'll let you kind of uh try to answer that question so what's the best way to balance all those things um so this is going to be a really biased response because i'm business performance so i'm all about quantitative but as it's the quantitative data that we're using when we're running our a b tests on the events we tend to start there and then use the other data so the qualitative sides the marketing thoughts to validate what we're finding so if my tests say that players want us to i don't know delete the saga map um we need the qualitative side to kind of confirm that so it's more validation and i think for me it's about using um qualitative for framing what questions to ask and the

quantitative to find the the kind of the significance of those questions and it's obviously easier when you've got lots of players to be able to sort of work out what statistical significance is in terms of those decisions and i think it's a bit harder if you're if you're starting a new game because you don't have a scale of audience yet um but i think it's still the case of where you've got to use data uh what what data you can get your hands on definitely we're very lucky with the with the number of players that we have playing because it does mean that we're able to do those tests where only a small segment of players don't get to see the brand new content yeah so we're not we're not impacting too many but we're still getting significance yeah and it's a conversation we had yesterday um on the game maker insights side of that balance between using data effectively but tying you realizing that data is going to give us our history it's not going to tell us what will happen and getting that balance between like using that to drive our decision making but not letting it override our vision for the game which is kind of what you were saying

um so there's a question here from um uh janus i'm gonna probably butcher the name there uh can live op in capitals become a reoccurring liveop with time that's actually a really good question so i think it depends on the nature of the i'm really regretting this title now on the nature of this uppercase live it really works it works um yeah so as the goal essentially is with each uppercase liveop we should be introducing a new mechanic it depends on the individual situation so for for a narrative one i guess it could become reoccurring depends how you look at it um because we may now do a new narrative with amelia so that could be a media story part two so i think we'd still call it uppercase but it would be recurring yeah i think it's an interesting one because i mean i look at it as um i try to work out cadence and schedules for reoccurring events as my primary driver but then i've got to think about when i've got content releases or feature releases how i use those

and then you think about the kind of real world and maybe even in-game narrative moments that you want to celebrate um in fact one thing that kind of informed my thinking on it was the subway surfer guys i got i did about two hours of consultancy so nothing relevant um but it was interesting how they uh added the idea of traveling to cities and that idea of going to rome and then a year later then they went back to rome it it unlocked a way of what's that a reoccurring lie but always an uppercase live up because arguably it's an uppercase liveop because each incident of the repeat is unique in its own flavor and is being treated as its own values that makes sense it does it sounds like i'm going to have to make a whole new criteria for my next talk occurring oh but i i mean i'm massively into the rear carrying thing because i think it's about making the live operations process efficient yeah um you know and i you know and but you you can't make it tedious

as is that i think where you're coming from and i think that's a really valuable thing to talk about uh so king are obviously a massive company with considerable resources this is coming from anonymous um this anonymous guy is really kind of asking all the questions um what would be your key advice to smaller indie companies i what is live ops basic what is liveops basic i think that goes back to that kind of the definition that i pulled out at the start so and at the end actually so it's it's balancing keeping your players at the heart of everything and there's providing new fresh content so it doesn't have to be big it doesn't have to be awesome or amazing so at no point when i was saying uppercase live ops so they say it had to be changing the world it's just about providing something something new and fresh so i guess my advice would be just to talk to your players even if you've got less of them maybe that they'll engage more maybe they'll interact a little bit more and the more authentic you are i think that's that's i mean we talk about data being important and it absolutely is but i think authenticity goes hand in

hand with it i think if you use your data without authentic connection with the players it's not going to get the flavor is it it's not going to get that experience exactly that's why our voice of the players are important i think because it really does you know we have an ideas section where players put them their ideas for the game like i love that we have that and that we take that into account and i love the fact that you used um becky's um article as the first thing because becky used to work for us when she wrote the article so she was there a graduate who managed to help move on to being a producer um anyway so um so water thanks you for the talk um and uh asks um who gives the final go no go for an event or is that a joint effort within the team or does it need to go to a product owner for example um so when we the team so the the four of us crm production design myself um we put together a brief and then that is presented to the head of studio with kind of a that will cover what we want what the hypothesis is for

the event why we think it will work and the expected um the way it will run the expected i've lost my words [Laughter] impact expected impact absolutely well actually i have a slightly different view on it but that's because we do live ops as a service and our our viewers that we've scheduled it and once it's in the schedule it's approved by the product owner and it's and obviously because we're working with third parties it's therefore has to have been approved by the client in the process as well so whoever the developer is but for us it's the admin you know the community manager admin whatever the role is depends on how we're working with the team it's their decision on the day because they're the ones that are going to press the button live so if there's something that's going wrong they need to be able to roll it back so our view is that we empower the people who are going to run the actual event on the day to make the final call but they're backed up by all that

management support now that makes sense as we're the ones who are turning it on on the day so if it goes wrong we're the ones turning off and problem solving it so we're kind of everywhere but um when we're running when we're going forward to get this sign off it's more for brand new things that are going to take a significant chunk of development time so if we're using the reoccurring or the uniques there's there's less of that kind of solid process and there's more of a review of the overall calendar for the year yeah but i think it's like any feature um sort of scheduling process i remember when i was at playstation home i was responsible for as home architect i'm responsible for all the platform development and i had to get all of the regions in sony to agree with anything if anyone knows anything about sony you'll know how much of a nightmare that is particularly with the time zones but that's why everything had to be done early and then released over time when it became available you couldn't actually schedule anything until it was actually done um but yeah anyway that's my my take on it um so we've got probably got time for one more question i've got about eight

let six left so i'm trying to work out um which one uh i suppose i'm gonna do it because it's it's topical what has been the impact of kobe 19 on live ops for you guys gosh trying to get me in trouble with this one aren't we thanks um so during covert players have been inside more so we found that they are playing our games more and engaging more um so as a result of that because we're kind of aware that we are getting more players in the game because of kobit and we've run unlimited lives campaigns twice so players can just come and play for as long as they like so we're seeing different player player behaviors than we are usually because we're giving with running these campaigns where we're just giving players extra things just to help them along their way during covered i mean that's a cagey answer

it is an interesting space because right now we've seen um you know obviously a very difficult situation for a lot of people and what do people do in difficult situations they seek play because it's a great distraction it's good for self-esteem it's good for self-confidence it's good for release and so i think it's understandable we've seen enough uptick in people playing games particularly mobile games um and you know what happens next who knows yes as the restrictions change i still think games will continue to come out a bit very positively and i think particularly big games which are easy to find like yourselves um i think will always be one of the sort of bigger better benefits beneficiaries beneficiaries uh for this particular kind of like model um so yes i i think you know it's an interesting problem and um it's gonna have an impact on on ad use it's gonna have an impact on an in-app purchase but what we don't know is when the recession hits how it will then further effect but that's not a game specific issue

that's an industry issue and our job as game people i think is to make that transition that process as easy as possible and just to fulfill the entertainment needs of our audience yeah definitely and we can say we continue to run tests so we should see when that step change happens and then the idea would be to continue to read our data hear what our players are saying just adjust and adapt as we need to exactly and i think that's it you know the only thing we can we can do is to be as responsible as we can to our audience and the data they show and on that note i have to say thank you that was a fantastic talk that really was brilliant and i'm so glad that you uh you did that i know you were a little nervous earlier but it did not come across at all so thanks very much for that