Panel: LiveOps and Support Strategies to Shape the Future of Your Games.

Hey game folk, good to see you here and thanks for coming to check us out. With me today is Stijn Tops, head of live operations and monetization at Supercell, Sebastian Brandt, director of player services huge games, and Vlad Vladislav Abramov, player from So player support lead at Cyber. They're going to give a little bit more about what they do when I ask the first question. So in

interest of time, we'll keep it going. But really what we wanted to talk to you about today is to look at, you know, live ops and everyone's definition of that is very different. But just this is the live ops track, and how the support solutions and the live ops itself are merging and coming together, and what you should be looking at in terms of what's coming or how to improve that whole process within the game. So I'm going to jump straight in and ask some questions, but a little bit about more about me. Four and a half years now at Keywords, most wonderful four years. Fit, fat, and healthy, because that's what happens in gaming. You sit in front of the computer for way too long, and somehow the calories find you. But the the role really is about how do we bring better solutions to the players. Everything is about the player happiness loop, and that continuously keeping the players engaged and keeping them happy. So Keywords, we've been investing a lot of time and money into technology, AI, the human intelligence,

which is critical these days, and bringing that all together in the live ops player support player engagement space. So, if you want to pick brains, learn some stuff, here for the next 2 days, just come see me. Uh coffee, you can give me a coffee, get as much knowledge as you want. But, I'm kicking off um Stanev for a very short description of what it is you do. Um Well, how do you, you know, when you give a very quick definition of what you believe the live ops means to Supercell, and share briefly more about your live ops strategy and live ops team and how they're structured. Sure, sounds good. Um yes, I mean, very quickly, I mean, live ops at different companies it means a bunch of different things. Think at a high level at Supercell, we think of ourselves like a continuous supply of new content and events um to excite and glad our players. Um with a clear end goal, which is generally either engagement, retention, or monetization, or preferably a combination of all three. Uh but essentially, like we're trying to, you know, make our games as exciting and as lively as possible, uh but with a very very clear KPI in mind. So, we are trying to improve a metric that will essentially lead to an improvement in in the business or product uh KPI. Um live strategy at Supercell, I would

say that varies strongly strongly by game team. Um we are sort of like looking at a few um kind of like core pillars, um engagement being one of them. So, we do try to make our games as exciting as possible. We try there to be new content, new events. Um you know, we try to improve metrics like um retention early on. Uh we want to make sure that there's lots of uh you know, play time, but essentially all the KPIs are engagement related. So, it's sort of like core pillar number one for us. Um number two is like monetization. Um you know, there needs to be content that players want to buy, that they're excited about, that they want to convert on. Um finding the balance between those two, I'd say that's probably the biggest differentiator between our titles. So, we have games like Brawl Stars, um which at the moment, you know, we're very very focused on engagement. We feel like that's actually our best chance of improving the the overall game health um and improve the business metrics. So, you You once you figure out for a game like what are the levers you can pull, what are the things that actually work for your game, um that's what we're trying to double down on. Um there's on the on the other side games like Clash of Clans, um where we're much more focused on improving our

monetization metrics, very stable user base, very stable engagement metrics. Retention's been strong for the last 11 years. Um and so that's the title where we say, "Okay, actually monetization here is actually the key lever that we can pull." Um and hence we double down on that. Um I would say that as Supercell and and as a as a company, uh we've been growing a lot. Um by Supercell standards, there was 400 of us last year. There's 540, I believe, um as of today. 140 people that we've grown over the last 12 months. Um we're going to add a bunch more people. And a lot of that is actually, you know, aiding to our efforts within live ops and monetization um as a function. So, we want to deliver more value to our players. We want to double down on engagement, double down on monetization, like we just make sure we deliver a better experience um that is worth players' time and also their investment. Um and that's essentially my role at Supercell as well today. It's like I'm building out the function. I spent 3 years working on Clash of Clans and on Brawl Stars. But my role today is basically hiring new talent, developing new talent, and making sure we have all the capabilities in place so that we can we can grow as a business, grow each game. Uh we have pretty ambitious targets. We have

Brawl Stars in the room, we have Hay Day in the room, we have central functions in the room. We're all ambitious to grow, and we think that we can um just by delivering, you know, better live ops value to our players. Um and yeah, just doing a better job and giving them more um to engage with and more to monetize on um as well. Right, thanks for that. And just just note that the point that's being made here is that a game will spend probably more time live than it took to develop in the modern era. So, it's really, really critical that you get the engagement balance and and that whole support and live ops right. Uh coming to you, Sebastian. Um can you share with us your live ops strategy, a little bit about yourself, and how do your support teams assist and add value to that? Also thinking a little bit from a technology human balance. Um well, uh I would say it's not much different. The focus on monetization and engagement when it comes to players. Um, but um what is important is that my teams, which is player support, VIP team, and player inside team, fully support this um

uh this all these actions. So, basically for us um it's crucial to be knowledgeable and to um to transfer this knowledge also to our also outsourcing partners um to make sure that um we can support the players uh in in every time they request they request help from our side. Um when it comes to player services, uh we are involved from the beginning when it comes to uh new features, new monetization um events. Uh we are we are checking if the if the banners are uh are up to date, if they are kind of like logically structured. Uh then afterwards, after the launch, we are kind of like testers. Uh so so we make sure that everything works on production. Uh if there is anything that QAs didn't didn't spot. Um and afterwards, we collect feedback, obviously, and we provide it to the product to the monetization teams. Uh we talk to our players one-on-ones.

We do, you know, video calls. Uh if if we talk about bigger features, um we we do surveys. And um together with business analytics, we provide uh we provide um opinions and and we try to kind of like give suggestions and to to improve or to change. So, basically, uh we need to make sure that um that we are all we all communicate well. Um we are aware of what's going on in monetization calendar. Uh we are close we are work very closely with the product department. Uh our basically player player service agents and then the account managers and obviously player assistants need to be knowledgeable and uh I need to I need to say that many times uh player player service agents they they know more about the game than actually their the developers, right? So so this is basically the goal, you know, communication, knowledgeable and being knowledgeable and um

and trying to improve the um the product and uh for for the sake of customers, for for for user experience. I think that's it. And we'll we'll we'll drill into some of those things a little later in the conversation. I know that um that's been pretty high level, but coming to you, Vlad, a very quick description of what it is you do. And and the first question for you is like, so how do your support contribute to live ops and strategy Cyborg being different to what you've just heard now? Yes, thank you, Peter. Uh I'm Vlad, working for 2 years at Cyborg, quite a newbie in the industry. And uh again in the question um the live op strategy at Cyborg um is pretty uh simple, but at the same challenging, because we keen to provide uh our players with the highly engaged content and they've been asking for. And it's uh it's not an easy task to to do that because you need to know your huge audience, like uh 150 uh million monthly players. Uh and especially when you have three weeks

update can cadence, it means that your team updates the game with everything new, with a new destination, with a new character, with a new boards, with everything new. And I mean, obviously, you sometimes have bugs, that's naturally. So uh that's why your support team uh they they need to uh have um the the flexible tools um in their possession, so they will be able to cater their workflow uh to update their knowledge base FAQ, uh to create bots, automations to deflect tickets. Uh and uh we at Cybo also constantly listening our players audience on social media forums, Reddit, Discord. And one of the examples how players support team actually had an impact on the live ops strategy that we noticed when our players they've been looking for one popular

character in the knowledge base. They just type the name of that character dozen of times. I don't know why they just wanted that character. We collected the feedback. We delivered that to the live ops team and they released a successful promo. So that's a kind of of a collaboration we have. So let's just stick with that because obviously we've given a high-level overview of apps. Let's drill in a little bit more to sort of how this all plays out, what technology, what tools, and how people go about stuff. Vlad, you recently changed the support tool platform. Can you talk about the reasons for doing that? And like what do you see in terms of the gains of that in helping this whole live ops collaboration with support underwriting underwritten by support? Yes, sure. Six months ago we actually switched to Helpshift. Before we had another help desk tool and we actually had a lot of reasons Peter

what why we wanted to switch to another tool. I have three main ones to share. The first one I would say we wanted to get rid of emails. That's something that Rovio Pascal guy also told about before. That's the same pain point that we had because when you force your players to leave your game and open the external mail provider to type the reply, uh to type this reply. That's uh that's just decreasing your retention. And uh the second reason uh I would say that we actually wanted to integrate more uh automations, bots, and AI into our player support uh journey uh because we just yeah, we just wanted to keep up with the times and uh uh deflect more tickets, uh save money, save time. And the third one, naturally, we just wanted to increase our customer

satisfaction, that we actually did. But yeah, that's the three main reasons. Yeah. And uh it's been uh quite a journey so far, but happy you're meeting all those objectives. Yeah. Yeah, of course. So, Stan, coming back to you, let's drill a little bit more deeply into that. What sort of is the data that is key to you in helping make these decisions? Cuz these guys want the secret sauce. Uh obviously, guys like Supercell Supercell has been highly successful. What's that data that really drives these live op support decisions? I'll give the high-level framework. I think for us, there's sort of like four categories that we're looking at data-wise. I think one is obvious, right? So, I mean, we have we have our data analysts that are embedded within our game teams. We're looking at sort of like the overall health of the business, health of the game, product metrics, right? Everything from DAU to retention to, you know, re-engaged player retention to monetization. So, like all sort of like the core metrics that are key to to inform what we do. Um every sort of new event, new piece of content, uh new feature that goes into the game, there's hypotheses on which metrics they will improve. And so, kind of like on a core level, we're following up on those assumptions, making sure those are hit and trying to improve them

over time. I think the previous talk uh there was mention of right you you never launch V1 kind of in a perfect shape. That's very much the case for us. Like there's initial assumption, we try to hit it, then we iterate and try to improve the metrics as far as we can up until the point where well, we think it's a good investment of our time and and move on to the next thing. I think that's one. Two is player uh research. We've been doubling down on that quite a bit over the last couple of months in particular. Um so, like supplementing our kind of like heavy quant data that's coming in anyway, we also like player opinions on um how they find our content. So, let's say doing a bi-weekly or monthly survey within the game, um how do you find this content? Uh why did you buy it? Why didn't you buy it? Um you know, what could be better? It's kind of like a mix of just very basic comparable quant where it's like a rating from 1 to 5, supplemented with a bit of open-ended qualitative questions, but without, you know, risking a player churning because it's just too much or too demanding. Um the third group is, you know, for us think as as a as a company, um the community team has been a huge value add to Supercell, and so that's, you know, through social channels, that's through our creator program, that's through our Reddit, that's through X, that's through Facebook, that's just comments that our players are sharing, and then two-way

communication between our our community teams and then our player base, uh which has been pretty vital in terms of supplementing um what we know on the quant side, which is like qualitative info on, you know, why is that? I think it's quite easy to look at numbers and make your own opinion. I for sure jump to conclusions. Uh they're sometimes wrong, they're sometimes incomplete. Um and I think a lot of like, you know, player sense check can also like help in and understand like, okay, why did this number move um the way it actually moved. Uh and you know, I'm sure that everyone here that works on monetization will know that, you know, there's plenty of tests you do where you're 99% sure that it's going to work exactly that way, and then that 1% is exactly what happens. So, that's just part of it, and that's, you know, that that's part of the game. And the the last group, and this is more on more on the player support side. Um so, we have a uh an embedded um game champion on on every single game team, which is essentially our way of getting all our um you know, player support interactions kind of embedded into into our game team as well. Um you know, for the for the 2 years that I worked on Clash of Clans, this person literally sat on my left-hand side, so it was kind of continuously aware of what I was doing in terms of live operating the game, new events, monetization. Um so, he knew what I was

doing, and he knew what he was doing. Um you know, he was in touch with, you know, hundreds or thousands of player support agents, um where we have players that are coming in with with problems, with bugs, with feedback. Um and that's through HelpShift within the game. So, we have, you know, players that want to reach out to us that want to talk to us, they can engage with us anytime. Uh and for us the the game champion is sort of like our way into the team to make sure that whatever feedback they have, whether it's a bug, whether it's a problem, whether it's a suggestion. You know, when when there's general themes and then kind of overall um kind of like broad strokes of opinions that come true, those get embedded into our process. So, they come into our road map, into our live ops plan, into our monetization. Um if a number doesn't, you know, if the number isn't delivering as I expected it would, my first port of call is the community managers to see, "Hey, what what is social saying?" And then this game champion to ask, "Hey, look at tickets. What's coming true? Like what are the the general concerns or problems that you see and that we can use?" So, yeah, those are the four and I'd say player support is a critical one in terms of tying it to a player support function. them close. So, good friends, not enemies. Right. Um Sebastian, coming

over to you in terms of taking that technology level a little further and looking at how technology can help this whole support live ops interface. Um you've been a bit of a pioneer in the language AI, the machine translation piece. So, three-part question for you is like what was it before versus now? Uh what was the thought process of why you would want to use something like machine translation in the live ops environment? And then how did you test to see that it actually was giving you the success you wanted? Um so, I always uh I always thought that um we cannot only focus on, you know, English-speaking audience, you know, like we have we have other markets that needs to be taken care of and um and we were always looking for solutions uh how we can how we can improve um our our our um our lan- language language uh language skills or language uh

um tactics when it comes to player support and and VIP management, etc. And um uh I tried many things. Uh we we've done um you know, uh machine translations like, you know, Google Translate and stuff like that. Uh we were using Keyword Studio services for outsourcing agents for specific languages. Um uh we were also like we were also um going for companies that were offering I was in the past before AI um machine translation with manual support of this of of this machine translations. So, all of these options they have some um we had some issues with that, you know, like either it was costly, uh it was not time efficient, etc. etc. So, um nowadays it was now it's much easier and with this tool that actually helps if this uh has

offered us to test we were really lucky to to to to check check it out and um we found out that it helps us to reduce the time of response uh to players. It helps us to increase CSAT especially for languages like Asian languages. They were the most tricky ones uh uh and and we went up from as far as I remember we did an AB test and for the non-translated tickets and the translated tickets we saw an improvement of as far as I remember 0.5 when it comes to CSAT. So, it was a really good really good result. And also it's a really good tool to optimize the cost. Uh and by that I mean you decide which languages you want to translate, which tickets you want to translate, which segments of users you want to target with with dedicated translations.

So, we're looking forward uh to to using this tool in 2024. Um and as As test went went went really good. So, um uh we decided to to take it on board. So, so uh What it bring, what what will it will bring? Um We'll definitely reduce the cost of of, you know, of agent speaking um native foreign language and Exactly. And and we hope to see improvement on of CSAT and time to response Cool. across all the markets. So, if we had taken that um technology approach on your on your approach with the recent shift to the system, um now that it is end-to-end with the the whole platform tying into the agents, have you seen some notable outcomes in terms of before-after results with the key things that are driving uh in a live up space,

the support interface with KPIs, FAQ automation, CSAT? What are you seeing in terms of that trend with utilizing a more modern or AI-driven approach? Uh yes, Peter. We noticed significant results, to be honest. Uh first first one that I can mention that's that we decreased percentage of emails as a incoming workload by uh 10 times. So, it's like around 10% of our workload now uh it's emails. And having having done that, we also noticed that we improved our customer satisfaction just only with this one move. Uh and it it it just yeah, it's significantly increased. That we were really happy about that. Um One on the on another hand, we increased the average full resolution time per issue because when you remove from your

agents' workload those simple easy questions or tasks um that uh covered automatically by bots and automations, then um they have time uh to solve uh more complex ones. And it takes that time for them to to to chat with our with our players. And uh I also noticed that players they like uh chat uh more than emails. So, uh sometimes they're just uh spamming us with different questions. So, uh that's also pretty interesting. And uh the last thing is that we actually uh leverage a good automation level from from the start. It means that right now uh we are covering um more than 12% of our workload. It's just the beginning that where we start with. And it means that we save tons of money, tons of time of our agents. Um so, yeah, really happy with that.

Stan, coming back to you in terms of when you look at this whole live ops environment switching to the support, how do you define what outcomes are you looking for that you can give some guidance to in terms of what are the key measures in in, you know, future proofing the game? Well, it's a lot of question. Um I would say like, you know, starting with engagement. I think that that for us is sort of like the bread and butter of what Supercell's been been good at for the last uh well, 10 plus years. Um you know, the we're always thinking about, okay, which KPI do we want to improve when we bring a new feature to the game, when we we're bringing a new event, a new offer type. It always starts with a hypothesis, starts with an with an assumption and that we're trying to hit. Um if it's engagement, um it's quite often retention focused. I think that's for us, you know, when in games with, you know, anywhere between 10 and and 40 million DAU, that for us is, you know, the the easiest way often to move the needle. If the game retains a slightly bit better, um or we, you know, we we churn a few less players, our percentage points at least, that adds up over time, uh week over week, month over month. Um that's sort of like how we do major

shifts in performance. Um I think that's that's a a very clear one where um issues with you know be it your player support be it your community those are often the first things to flag any issues or concerns there. So those are things we're we're trying to improve constantly. And I think on the other side like I think on the monetization side I think that's for us that is where we have a ton of of space to improve. You know our games historically are very good at engaging very good at retaining players. I would argue they're under operated and they're under monetized. That's something that we're we're we're working on every single day with with our growing teams and I think for us trying to find that balance that's going to be the biggest challenge right? You bring in players under certain assumptions. There's a an unspoken player contract. This is the game they signed up for. This is the experience they want to be part of. And we're trying to sort of like rebalance okay how do we bring in new ways to monetize players that are that are exciting to players that is for content that they're happy about and that is you know basic metrics it's improving ARPDAU it's improving first time conversion it's improving repeat conversion. Very simple measurable metrics

that sort of like on a high business level make a huge impact for us as a company and I think that's it's an exciting journey. I think for every single game that leads to different strategies. For some games like the engagement lever is much easier to pull. Like I said for other games monetization is is the way we're we're going down. And I think within live ops you know you can look at participation rate you can look at completion rate you can look at like you know monetization per event but at the end of the day if it doesn't feed into like these higher level product or business metrics it doesn't really matter. Like it doesn't matter if you're moving around players from A to B it's overall picture that we're trying to improve and so that's for us the yeah the core challenge but it's also super exciting and lots of opportunities so expect more from us there. Similarly to you Sebastian having seen I guess the explosion of mobile and then now a free to play model sort of going across other channels and everyone trying to kickstart should we call it web three or the metaverse what are you sort of seeing some of the challenges lying ahead in terms of how you Is it

going to be more of the same in how you do the player support and the live ops, or do you see because there may be a shift in channel, there would need to be an a higher adoption of technology? Um when it comes to player support, um I definitely think that if we if we move to to um web browsers, uh this the the the player services player service needs need to adapt. Um we have been with Helpshift for more than 4 years now. And uh we are trying to take um all the new ideas and the the functionalities that you are offering. For example, the live chat is really interesting for us, and that that we want to try out definitely. Uh it's it's it's kind of like when we mentioned the email conversations and going to live chat, it's a com- it's a like a completely uh it's completely uh changing the

the approach. Uh we want to be fast, we want to be uh efficient, and we want to help players as soon as possible, and not only on mobile, but also on web. Uh so, uh we will take advantage of of of of these new features that you are offering. Um one of the things is obviously everyone's spoken about feedback. Um so, just very quickly going from uh left to right, uh how often do you look at doing surveys, um or is it like after every ticket, do you really bug the players? How are you collecting the feedback, and how is the system helping for that? Uh the feedback that we collect it's actually from uh first of all, that's social media. That's where our player experience team are helping us. Um that's also the tickets uh that we have that our uh player support a team is collecting for us. And uh I usually like reading uh Reddit uh forums. I go there every day and uh just

look through the latest post to understand what the airplane are doing. And Discord is a huge ser- server that we have. Uh so they're quite beneficial. Just Yeah, I do it it it you need to have the time to go and check by yourself even to understand uh what's the audience are doing over there. Because you you you need to be there on that front, too. From your side, Sebastian, any tips on that? So uh nowadays, when we have AI, uh we are taking advantage of that. Um for example, for HelpShift, we have integrated the solutions that uh will give us the sentiment, will give us real-time feedback on the tickets. We don't need to go manually anymore through all the tickets and we will have if there are any issues, it will be spot they will be spotted immediately and and reported to um to um

to the teams responsible for fixing. Um we do a lot of we we send out many surveys. Uh now it's basically um it's a it's a part of of the of the product life cycle. Um uh it's it's always a survey. We we always ask uh questions uh to players about how they feel about the feature, what they would like to change, etc. etc. Um we monitor social media, obviously. We we do app reviews like everyone does, I think. And based on that, we we create like um a bigger picture uh and we present it to the product. We I I think we are the one to push for changes if if they are necessary, if they need it. Lab up side to support coming from your perspective. How are you feeling that that feedback is taken in standard and and is it something that you also drive in your team or you expect more from the support team to drive to you? Yeah, I mean I think it's like it's actually like Sebastian said, like it is

part of the product life cycle. I don't think it can be something that lives entirely on its own in like a in like a in like a separate room. Like I said, for us it's embedded. Embedded literally means sitting, you know, as as part of the team and then giving feedback where needed. I think it flows in two ways. Like when there are I think often when it's bugs or when it's problems, it comes, you know, from player support into the game team just to make us aware of the problem. Also how big it is. I think that's actually a huge piece of help shift and and these tickets is not only do we get the feedback on what's wrong, we can also see, okay, where is this player in the game? How many players do we think are affected by this? Is it Is it, you know, five people? Which is probably okay. Is it a, you know, 5,000 people? It gets a bit bigger. Is it a million people? Okay, now we're, you know, all flags are all all alarms are blaring. So, it really varies. Um, I think when it comes to product feedback, it's probably more the other way. So, it's us asking player support um, if they got, you know, input put on specific topics. Um, so they kind of sift through these different answers or or different input they received. Like, okay, did anyone mention this? Is it a problem? Is it an opportunity? Um, so I think it's a two-way street as it should be. Um, I think the the one thing I want to highlight here is that for us, you know, and also in the comments of, for example, on on social,

um, you know, we have people where it's essentially their full-time job to make sure that they're aware of how the community is feeling and how the how the, you know, both from a community side and from a player support side. For us, I think the most important piece is like, how do we make that value add, you know, in addition to the the the the quantitative data that we receive. I think on their own, both of them could be quite dangerous, right? You We have a very vocal community, which is wonderful. It's a It's a great position to be in, but it's often a vocal minority from which you hear, especially through social channels. I think for through player support, it's a bit more balanced, but you have to care a lot about the game to come voluntarily give your feedback and wait for a wait for a human agent to to respond to you. Um, but that together with our quantitative data, I think that's for us where where the real power lies. Like we see the numbers, we get the qualitative feedback. Um, I think the more proactive player support can be, the better. Um, But I you know, having them as part of our team it was like this when I joined Supercell. I couldn't imagine working in a different way where they're not next to you cuz otherwise, you know, you're missing this whole angle or perspective that is pretty valuable in how you work. Um I kind of covered us with all those questions and there's a

little bit of time left and so maybe there's one or two questions because it's always great if we can get if anyone wants to. Otherwise, I think we've stayed very loyal and on time and and hopefully given you that insights into the piece between support and how the live ops piece is now all merging into that one end-to-end platform. So, anyone got a question out there? I I do I do have a question that wasn't in this and that is regulatory framework. The regulatory framework is now starting to shift. Lots of pressure coming in around privacy. The digital act is coming in. Anyone want to grab a very quick one-line or two-line thoughts on how will that impact and how we create live ops and support environments? I can go very quickly since everyone everyone seems so eager. I mean I think from our side like we're we're we're we're closely following everything that's going on. There's a lot of moving pieces I think this year when it comes to privacy, when it comes to web shops and all that fun stuff fun stuff. I think from our side like you know, the the main thing we're we're focused on is is is is player quality and I think on

the on the on the privacy side that's around being transparent to our players as we we we've always been and just making sure that we continue to to deliver excellent value and then the same thing happens on on the store side. There's a lot of stuff that's changing. At the end of the day like all that matters is that our players get the best experience possible. And that's something we're we're aiming to deliver. So, it's pretty pretty vague but you know, keep your keep your eyes open. Cannot say a lot of things because I've been busy with implementing Helpshift. So, yeah. I didn't dig into the topic yet. So, I think I need to to do that with legal, but yeah. Finishing up with you, Sebastian. Any thoughts on that? Yeah, so we all know that retention is the key nowadays, so every account closed, deleted is is a pain for us, so but like you mentioned, uh we need to make sure that that we stay aligned with the regulatory and we make the the process as as easy

as possible for the player, you know, basically. And also we embedded this process in HelpShift, thanks to thanks to the tool that we are using. Uh it makes us really easy to report on that and also we were we may we are able to um to collect the reasons why people actually are leaving the game, you know, like and then it helps us to improve as well, you know. Well, that sets us at the end and guys, thank you for sitting with us and listening through and hey, Oscar, back to you, sir. Thank you very much. Thank you. I mean