How to Maximise Engagement (& Fun!) Through Live Ops Strategies

so now we are going into the final we're going to close the morning with a double-handed sort of fireside session uh featuring um the customer engagement specialists [Music] and the super hot for fireside a blockchain based uh sandbox game uh folks uh as as as run by my my dear friend sebastian who's having a he's all over the bloody news at the moment can't you cut i can't open a california press release that your name in it so um congratulations on uh on on uh everything with everything with the sandbox the moment it's going uh oh jackie's back are you wearing this jackie no i don't think so that was nice um anyway uh thanks guys thanks being patient we were in about ten minutes behind but you have your full uh time for your your session now so um is it is it mom chill is that right this is mom sure yeah mom sure cool all

right it's a cool name i'm good we're gonna have to check the etymology of that site i've never heard that one before um but are you guys fine to um to launch him yeah i think that's uh that's good i'm going to share my screen real quick and uh i just wanted to um start by um thanking seb for uh joining us today so the topic for today is really on how to uh maximize player engagement through personalized live ops offers so i feel like it's a topical one especially in the context of previous conversations we just heard about kind of the battle between monetization and engagement for players and uh and how do you strike the right balance and we have some interesting data to share we have some anecdotes uh that we can share as well and um this is uh mom show i'm co-founder and chief product officer at leanplum and i also have seb i've known seb for what seven years now we met i believe in paris uh the first time over seven years ago and it's been a great journey being a partner together so seb i'd love to have you introduce

yourselves as well absolutely well today i'm putting my pixel hearts which is my previous mobile game company i co-founded in 2011 and it's true that we spend a decade into creating mobile games that are focused on two things one is turn players into makers building their own stories through city builder games with ips and on the other side turning players creator as well letting them create their own games with sandbox and other franchises and as part of this journey where pixel this game studio we founded 2011 was acquired in 2018 by animoca brands we have been really putting a lot of effort and more focus into liveops into a data-driven approach that's our leveraged input platforms we were i believe one of the very first gaming clients to use limp lump solution to engage our players through values in-app marketing actions and i think that the topic of today

would be to to review some of the best practices we had and everything that lean plant platform has enabled us to do so and a quick word based on the ct builder we launched as you can see here we we launched game with garfield with snoopy with goosebumps loud house wonder park a dance family and we're about to launch a new city builder that makes more than seven titles focus into this specific category always leveraging that expertise that we grow year after year thank you seb um and a little bit of intro on lean plum um so i started the company uh back in 2012. i had spent many years as an engineer working at google and i built some of the core a b testing infrastructure there as well as working on a growth team specifically for youtube and i took a lot of those learnings and applied them at lean plum and and seb is right all of our early customers were gaming customers uh our first product to market was an a b testing platform and so to this date a b testing is kind

of at the root of everything we do a very data driven company but we've really uh expanded to what we call full player management now and full player management in my mind is really the opportunity and the ability for a game studio to connect all the various touch points together that personalize the player journey and to me those are first the crm and the life cycle marketing that has a significant impact on retention that we've seen over and over again the live ops and monetization capabilities the ability to show offers especially the personalization aspect of that as well as the in-game experience ultimately the game design the actual if you have ads the ad loads um even the store itself all those aspects of the game experience are vital and uh really important to be personalized and really really linplum is that one-stop shop to connect all these three together into a cohesive solution for that and as you see you know obviously we're very proud to work with pixel for seven years plus now but we also work

with a lot of other companies uh small and large uh including electronic arts and big fish as well as morka mobility ware so really the spectrum both on the casual as well as on mid core hardcore gaming as well and um and one question that i get asked is why does it matter to personalize the player journey and connect all three of these aspects together and in my mind it's always about providing a great free-to-play experience uh because you know when you when you think about let's say you're starting to play a game and you come in on day one you play a couple of sessions you're already showing the game developer the kinds of things you like the kind of content you're really excited about and when you get that d1 uh push notification it better reference the kinds of things you already showed the developer you uh enjoy the most like the kind of characters you chased or the kind of upgrades it would be interesting um then you come back back on day two the liveops offer comes in and it shouldn't be just a generic sale that everybody sees or it's the same sale for everybody it should really be a

personalized sale for the characters that you are pursuing already or interested in maybe it's a personalized tournament or event for you over the next few days and then as you progress in the game day three day four day five then you really start to personalize the gameplay itself maybe you're starting to uh tweak the level difficulty the ad load all those aspects depending on who the player is and how you react to them and that really makes a difference to be able to connect those aspects together into a personalized player journey and it also allows you to really create a win-win scenario where the player is having a great free-to-play experience and at the same time the game developer the studio is able to monetize and engage in a very uh efficient way so i think that it shouldn't really be one at the expense of the other it can be both as long as you really strike that balance of of relevance and personalization and really for today for the conversation we're looking to dive in a lot deeper into that great um so a little bit on those three

areas that we touched on obviously life cycle marketing liveops and remote configurations so when it comes to life cycle marketing and i'm actually going to ask seb to chime in a little bit on this uh in a moment uh for each one of these but you know specifically lifecycle marketing typically owned by marketing teams although product teams also the game teams typically um would come in as well sometimes um but this is kind of your standard calendar of events uh sales i'm sorry events that are non-sales related anything that's engagement being able to optimize the player's life cycle i would say probably 85 of this is focused on re-engagement um and the main channels here are in-app messaging push email one thing that may strike you as odd is that in-app messaging is a really important channel for re-engagement um that is actually our own uh finding um we've what we've noticed working with several gaming studios is that players don't just stop playing all of a sudden they kind of slow down and it's really important to be able to detect that early

phase of churn in order to re-engage them through the right offers to the right gifts with in-app messages so it's obviously push and email really important to be able to bring people back but that's usually uh sometimes too late so being able to capture that early sign of churn is actually really important being able to detect it to begin with and being able to respond so this is life cycle marketing we all know that crm live ops capabilities are typically split in two one is being able to run a live ops calendar of events and sales and the second part to it is being able to automate the player journey through what we sometimes call life cycle ops these are things like defeat offers streak offers spending activity offers um any kind of offer gift that is being attached to the player life cycle and and what we find is that those are really great ways to uh boost the effectiveness of those promotions for the events and sales again channels primary channels in-app messaging but push email also really important channels

especially for vip players when it comes to pushing email specifically and then the remote configuration is is really interesting and this is uh this was actually one of the original products we built as i mentioned with the a b testing solution and the idea here is that you have the ability to control any aspect of the game and we recognize at least a lot of gaming studios have built in-house back-ends and they already have a game config available what we really strive to do is to enable our system to sit on top and really act as a gatekeeper that points to the right configuration for different players right so the personalization aspect the a b testing the segmentation aspect that really makes a big difference and it could be used for as little as just controlling promotions in your store to a lot more advanced use cases like being able to optimize the gameplay level difficulty being able to tweak other aspects of the game like ad monetization rewards cooldown periods for ads

and a fun little one that we've seen customers do is actually control other third-party sdks remotely with lean plumb being able to turn on and off maybe a game tune sdk or any other sdk uh when need be uh so uh seb would you mind just sharing a little bit about your experience and kind of how you guys kind of see those three areas yes absolutely mom chill well as you alluded to just several easy implementation of of those three pillars that can be leveraged you just mentioned some what we did at pixel i think we experimented with quite a lot of things that are possible here whether it are ib a b testing campaigns whether they are remote configuration and special packs the one i'll think triggered like the highest uh return or conversion behind and more importantly we are really regarding the life cycle marketing and the re-engagement first of all so it's a common practice when you run ct builder games to run live ops events

so basically we have some games such as newbie downtile that's been live for six years for which we make i give you 60 events so far so almost one amount very regular basis and something we've noticed is like it's not because you run an event you create an update you change the icon the packaging description the market the first featured screenshot that it's enough for players to find out that there's an event going on it takes even more effort and that effort goes through by leveraging those communication channels such as in-app messages and push notifications to let them know that today is the day that the event is starting so come if you want to engage with it and get a chance to win the rewards for that you get after the end of the duration of that event that's usually 15 days to three weeks at max so definitely push notification to

notify users that your event is start changed in-app messages for those who unfortunately might not have turned on their push notification is still useful if they come back and launch the game with sin as well the use of in-app messaging as a great way to engage the audience both within the game but also beyond so typically we are trying to leverage social media and community management as a way to engage our players to continue the journey outside of the game so when even when they are not playing the game the fact that we use ips like peanut or adam's family again we know they are a fan of those brands we know that they want to see more content of those brands so we keep posting regularly on social media anecdotes content teaser screenshots etc and sometimes we organize contests on social media and we want to make sure that our players are being informed about those contests so we leverage the in-app messaging to take them outside to follow and participate into

social media campaigns that well boost the social media campaign and then brings more player back into the game so it creates a rather beneficial acquisition strategy as well for us the last thing that we've leveraged i believe was the remote configuration we tested it back in 2016 if i'm correct as part of a previous version mobile version of sandbox called sandbox evolution where we were essentially changing the order of the display of certain campaigns and certain prices for virtual items through a limp configuration based on specific behaviors of the user so defining if they were first-time purchaser or they never made a purchase until now but they kept watching only video ads we would trigger those different options and convert them we did use the same practice with

targeting user segments on snoopy downtail game as well and there what we found out is like there were a lot a certain number of users who were participating event after event so they were coming back each month to complete those events but they never triggered an in-app purchase yet so we felt like they are highly engaged non-monetizers and what could we do to to create finally that trigger where we creating custom packages of offers in a purchase offer really targeted at them where we well only them receive them specifically with that with a special discount or an increased quantity of virtual currency of characters and content in the pack just for them so that we brought that psychological barrier for maybe some friction to the first app purchase and then well we had highly engaged retain users paying users and that was highly efficient

yes um yeah thank you thank you seven we're actually going to touch on this in the next slide um here which is specifically uh about a question that we set out to answer given our large install base and the fact that we have access to all these you know 1800 campaigns we analyze over a period of time about 670 million messages across in-app messaging push and email and the question we asked was what is this actual relationship between personalization and monetization uh because obviously personalizing the player journey the the free-to-play experience for the player is better but does this actually lead to better monetization and and what we found in the process is that yes you can increase your revenue per message by personalizing your messaging but there is actually four distinct layers about how you go about personalization and the four that we found are kind of the baseline starting with demographic segmentation this is the well-known one where you upload a user attribute or list of attributes for the age gender country ltv you name

it and all those attributes sometimes for some of our customers there's uh hundreds if not thousands of them especially if they have uh more kind of mid core game um available with different upgrades and and all these uh other characters and collectibles you can have you want to store all that information on the player on the player profile and target based on that this is a really good starting point um and we do support the ability to store you know up to 2000 of those on the user profile and believe us we we have customers who actually go that far but what we found after is that when you apply behavioral segmentation especially recency and frequency with aggregations or without this actually can double the amount of revenue per message that you get relative to demographic segmentation and so really a kind of a good rule of thumb here is if you have a player lifetime value ltv instead of just looking at ltv if you look at recent spend this is

actually a lot more impactful and it does make sense intuitively maybe somebody has spent a thousand dollars in their lifetime but they haven't spent anything in the last three months so chances of them spending right at this moment is probably more predictable based on their recent behavior versus lifetime behavior and specifically when it comes to recency and frequency being able to look at total game play the kinds of games they like the kinds of characters they chase aggregates such as sum totals for the spend or minimum or maximum and so on and so forth this is a really powerful tool again we're still talking about segmentation at this point um but it actually does give you a boost up and when we work with our gaming clients we actually measure for every single one of the messages that goes out how many of them what percentage of your total messages are targeted based on behavioral segmentation uh because we do see that boost in uh roi and revenue per message and then the next layer up comes the

behavior trigger delivery um so obviously all of live ops and as we mentioned lifecycle marketing as well being able to affect the player journey in the moment in real time as you're leveling up level 20 as you're buying a a specific bundle being able to offer a follow-on bundle with a streak offer or something like that being able to react in real time actually gives you an extra boost in revenue per message um yes so real time is the best predictor of behavior so recent current session is better than the last 90 days worth of recent behavior which is better than lifetime behavior so the further you go into real time the more uh relevant more personalized and more uh i guess opportunity for monetization you have and then the last piece there is obviously the messaging content and personalization so being able to personalize the actual message that goes out so it doesn't feel like it's the same message for everybody maybe you're referring to the player total number of coins or other things like that maybe their name if you have it or username

so being able to climb up this ladder of personalization we know is hard our job is to make it easier through focusing on ease of use and out of the box tools especially when it comes to developer effort we know that a lot of these things such as behavioral segmentation advanced segmentation requires sql even if you're using a third party tool and so we want to make that out of the box as much as possible because we know this is a game of operational efficiency so the more your teams yourselves can focus on being able to personalize the player journey um without a significant amount of effort involved in you know in getting developers to code things up for you or or other perhaps other blockers being able to really focus on that enables you to actually achieve that ladder personalization where every message that goes out feels personal and relevant for your players and as we show here on the right it also means more revenue per message um seb do you have anything to comment i

know you already mentioned behavioral segmentation is something that you guys found through your own testing actually works really well yes it worked well and i was looking at some questions as well in the q a and answering them and it's true like what has been really helpful is for us is likely to integrate into the company culture and at least if not in the company could do at least in the game design team culture this uh life cycle marketing the idea that we want to drive more engagement and more revenue from by lz liveops and the update of the game rather than at the launch and that is something that they need to integrate as part of each update like it's not an update is always features but it's sometimes future it's mostly content and it's often like marketing so thinking like how do you define like which offer like which segment why why is it going to be fun why would someone would want to convert potentially and if there is a monetization trigger

behind and so uh what do you design and don't design 10 000 or things like our team scale doesn't allow that so we tried to focus on just a few offers and make sure that those were visible enough and well targeted so that well that will trigger those conversions behind great uh thank you seb and uh one um question we often get is what is the actual distribution of of the uh these campaigns so how far um do people usually go and uh we recognize it's actually a long way for us to really get to that promised land of every message feeling very personalized and the player experience being you know really the player journey being personalized we saw that about less than ten percent of people actually or campaigns actually are real time our behavior triggered uh so there is a long way for us to go uh we recognize that and we're constantly innovating on our upfront to be able to move people up this ladder and really study what really is the blocker for

for game developers to be able to get there and another kind of slice of the data if you again look at that ladder of personalization the bottom layer is about being able to send scheduled communication um through in-app messaging or push or other things like that and what we found is that triggered communication being able to trigger off of behavior is performing better in fact for both bush and email we saw that revenue per message which is the metric we measured using an attribution window of 24 hours we see about 5x increase one thing that would perhaps surprise you here is that email is actually our best performing channel but it's also a little skewed because it's a lower volume channel typically used for vip players so we do see um correspondingly higher revenue per message as a result of that however both email and push about five times higher revenue per message when you trigger the message versus just sending it a scheduled uh message um so that's uh that was

really insightful as well and um and then last uh before we close out um i know said we worked on a case study together this is a different aspect of personalization where we actually take into account timing but would you like to just mention um just a quick result that we were able to study together on this yes indeed like we were in that case study uh back in the day on the first sandbox game and back then through the a b testing of when was the best time to reach out to user through push notification we were able and also what kind of content both on the title and the text description we were able to increase the percentage of open rates and have fused the engagement in the app and monetization through the offers because we always combine a special promo with a push notification and in-app messaging to notify the user like it's not uh just one thing that works it's always like the combination of all so that it's a journey for user inside the application

once they get and they want to open they go through the whole funeral behind and it's been easy to set up with implement that full customer journey so that we could us as game designer work uh test try it out ourselves and then launch it to the slogans or the segmented objects yeah here you can see a 16 increase on the open rate and a 17 increase on the revenue rate which uh by momshims i think tells me it's a great matrix we've been happy with yes uh definitely seven and again going full circle back as as i started with uh we started as an a b testing platform so everything you do in lean plumb is is based on that foundation of data and testing and so we really pride ourselves in that and we want to help teams be more data-driven more test driven as well so we always encourage all of our game developers to a b test everything in terms of algorithms in terms of new strategies

there as well so i think that's uh that's all uh from us so seb um and i are ready for some q a um so i'm just gonna stop sharing here so we can take a look hello hello that was very good that was really good um uh thank you for for both for being along and sharing that i'm going to pick up a couple of questions from the audience uh um okay there was a kind of conversation um uh early in the previous panel uh about the kind of friction between um kind of creativity and i guess and um and but business right or or uh you know processes and business so the question here is how can developers avoid being too money thirsty and create a balance between you know i guess what the audience wants in terms of finding what the company needs in terms of money is that obviously everyone's more money right that's that's the thing but

is you know is there is there a tension there i mean sebastian you kind of created a lot of different um sort of games and worlds do you you know what what what is it drives you is it are you is it a mixture between the kind of creative idea and the under the kind of business or or do you kind of come up with the ideas first and then apply you know structure around right all right i'll try to to keep the answer rather short and i don't want to derive too much into the other world and metaruses but the idea is i think the right approach is the way to think like user first and community driven so what's in for my user in this update before even i think to take their money for monetization purpose like why would those users spend time in my app my game rather than into another one do i give them enough fun enough value for their time that is like they're going to engage into a gameplay that's rightly balanced etc and if i create a monetization offer

like again is it what i'm selling to them of any value in a sense like you could sell for a wearable but wearable you just unless it's does it give extra capacity to your character extra skills does it help them to increase maybe the progression speed etc or you can sell a character into our city builder like this character he needs to come with more animation more stories more gameplay time behind something in again that you feel like there's a return of value for the users that's been our approach and uh one of the main reasons that our games are able to monetize five six years after the launch is because we don't try to monetize at every single event absolutely we just tried to keep people engaged over a long time and at some point you know we'll get it right we'll get the right offer for them and they will convert into a paying user while they are engaged and engaged again that's the most important

specifically in such a competitive market as mobile gaming with new titles coming all along and the attention span getting shorter and shorter from users yeah yeah definitely i won't take you down the old sir they'll sir sandbox is a whole different world different conversation maybe later in the week um i guess both of you i'm going to try and give you two more questions then we'll probably wrap because we've it's been a long warning it's been very insightful one what would your advice be for developers that have limited resources for marketing liveops i mean in terms of is there like a balance of how much effort you should put in how much budget you should put into the live ops versus the kind of initial build and the other the various other projects you need running i mean live ops is the game to a degree as you go forward but yeah yeah i i would say with uh with liveops specifically you tend to to see probably higher impact more immediate impact and being able to monetize as well as impact the gameplay um so i would definitely double down there one approach that we've taken is this notion of prototyping so being

able to really create entire events and experiences with only interconnected in-app messages that are super lightweight maybe just testing an idea or testing an experience you don't want to invest too much in building an entire built-in tournament experience or things like that you just want to maybe shape a few different connected messages together this is something that we've seen work really well so being able to uh get off the ground quickly um that's a really important aspect especially now with ios 14.5 i think we're starting to see probably more and more uh people turning to liveops uh in the process as well so i do think that more people will be experimenting with uh running more sales more events so that's something that i think will be my advice cool another question is would you say that the distribution between marketing and live upside and the remote configuration is equally shared as one way or more important than the other i mean to me it feels a little bit connected but because you sort of separated them out

and you're in your uh in your structure before is remote con i i would say it's uh it is connected because that's what we see our customers really seek out um you can't really just be sending crm messages and they're completely disconnecting from your monetization strategy with the events and sales you're running or from the game play uh you can't really provide that end in a player experience that's the fully personalized experience and um so we maybe have the selection bios that we search and seek for those uh developer studios that are actually interested in personalized the full player journey so most of our customers tend to use this for all three of them they don't always start with all three of them but they end up uh somewhere there fair enough and finally for me why why lean plumb i like it it's a distinctive name but what's uh what's the thinking yeah so lean came from the lean startup at the time it was a really hot block that came out and we wanted to really have this instill this notion of of the lean methodology

of testing everything being data-driven uh plum you know i often when i'm on a panel i joke that you know we started in silicon valley so we there's in silicon valley used to be an orchard so there was a lot of fruit companies that came out like apple and maybe you know plum lean plum one day is going to be a big company but may or may not have to do with uh the domain name available but i'm not gonna say that well maybe if i can if you look at back in the days like remember like the app store used to take a long time to validate your apps before they could go live and really it could be one week two weeks now it's very short so so back then it was even more important to have that capacity to test rather rapidly deploy new configuration av tests without submitting a new app each time and there was a pain point if i remember correctly that clean problem was solving for for us and their clients it's a good thumbs up there from you can't be a bit of uh be a client thumbs up i'm gonna have to end it there guys um thank you so much for your time this

morning um uh we're gonna bring our track to an end but have a lovely a lovely day wherever you are in the world i didn't even ask you where are you guys where are you both sebastian you're in ferris yeah i'm visiting in bulgaria but i'm usually in san francisco are you in san francisco i am usually there but i'm in uh bulgaria right now just because i think i'm getting up earlier sorry about that okay guys well enjoy bulgaria enjoy paris um and we will see you all again soon thank you so much awesome more fantastic thank you