Liveops Lessons from Temple Run & Bejeweled Blitz ft. Eve Chang

hello everyone and welcome today I'm joined by Eve Chang who has worked on some of the biggest mobile games ever launched games like Temple Run Temple Run 2 games like the jeweled Blitz and so what we are going to talk about today is first we're going to talk about uh Eve's approach to live Ops especially working on some of these games that have been very successful secondly how do you make design and product decisions for live games third the biggest Lessons Learned working on some of these games and then finally just ease take on the current environment and some of the big challenges and difficulties that we are currently facing right here in facing the end of 2023 and going into 2024 today's episode is sponsored by data. to access estimates for rankings downloads Revenue usage or engagement for millions of apps on the App Store and Google Play sign up for data. but to kick it off Eve thank you thank you for your time welcome to the

podcast and I thought we' just kick it off with maybe a little bit more context in terms of your background just so that the audience can understand some of the experience that you've had sure thank you Joseph so my name is Eve and I've been in the video game industry for about 15 years I started on PC free to play and when mobile games start picking up I realized that was the future so I jumped down to that and I was fortunate enough to be working with some of the biggest titles in the mobile G industry including like Joseph menance B Blitz and Temple Run and Temple Run to and now I'm working on my own projects based on my experience and the expertise I learned over the years great and Eve I thought we would start with basically when it comes to live Ops and having worked on some of these games like temp run and V blitz could you kind of describe your overall approach if you were to think about like a framework for how you believe believe mobile game

studios should be thinking about and should kind of structure the way they do their live Ops could you kind of give us a high level perspective in terms of how you approached it sure I always felt like we always have to answer three questions one is the mark what is the market void what is being offered on the market that players are not getting that there's if there is a void does does my game can my game offer that that's the market void question I will ask ask then I will ask what do play what do players want what do they want that that doesn't exist or exists but not in in a capacity that can be to its fullest and third of course comes back to getting itself what do we get what how how what is our objective for the players to experience is it live Ops is it is it is a new content what what what what drives the players coming back to the game play again and again so I I try to answer those three questions by first looking

at the market looking at what's available and looking at the Players communities and to see what are their voicing and what they talking about what are they excited about and what are they asking for right and Ju Just to go a little bit deeper on that and just from a practical perspective so how does that kind of manifest in a team discussion or get operationalized so what I'm what I mean by that is so you ask these questions the market void what players want things like that and do you kind of first brainstorm a bunch of those things and come up with a bunch of initiatives and then where do you get some of those ideas or how do you determine yes we are feeling a market void or no we are not feeling a market void if you can go into a little bit more detail sure for I I'll give an example for p Blitz it was a very old game uh that everybody loved but it's obviously aging with time so I looked on what was going on in the market and I saw that most of the games that

our Match 3 that develop after Blitz all had some sort of character to interface with the players no understanding the demographic were mostly female and mostly wanting to have some sort of connection to the gang itself I saw that that's what the Market's driven and that's where uh we were short on U not the markets are providing plenty of that but we we ourself as a game does not provide that and and so we discuss how can we how can we Bridge the void and looking at our product to see which which systems that could benefit from having some sort of character interactions to help the players to have a better experience I see so so it sounds like you kind of start with in this particular case you looked at Market competitors you then formed a hypothesis that you know similar to these other games that were building a personal connection with the player that you you believe that you could do something similar and so then

you what you designed that feature you implement it you rolled it out and then how did you how did you decide so a couple of questions one how did you decide to do that first or second or what the PRI the prioritization question and then secondly how did you try to validate either before you launch this feature and after you launched this feature whether it was the correct thing to do or not well first of all we looked at existing performance of existing uh metas in game and we realize the ones that are more competitive are actually underperforming so the hypothesis was formed that because the demographic would they probably are less are more solo Jour Journey uh players than competitor uh players so therefore we looked at a whole bunch of system that could help people who are mostly solo Journey players to have a better experience and how we can utilize that system to help players experience more of the game of the different metas that that they may may may not explore on

their own so we look at all the systems and then my bar was go back explain to your grandmother if she can't understand how this works it's not for them so we came up with finally came up with a system that was easy to understand that actually helped the navigate to different parts of the The Meta system of P Blitz and then we decided to implement that and that took a long time to nail down because seems simple but to things that look simple are usually very comp complex to make it look simple but uh we streamlined it in a way that any player can come in and they would not they would not need to navigate through all different points of the game they can just go to the quest system and they will it will direct them to all the systems they want with a character assistance her name is Gemma and she will help you uh do whatever Quest that she's asking to do right and one of the things that you mentioned is that you were thinking about this feature in the

context of a what you were calling single Journey player or something like that solo Jour solo Solo solo Journey player and so I'm I'm wondering do you have the concept of basically player personas and then are you are you considering these different features from the perspective of different player personas or did that not factor into how you roll this feature out it definitely had a factor into it obviously with the limited data with privacy nowadays we have little ins ites into player behaviors just Bas we can just gauge from the player session length and what they engaging most and what performs well to take a gander at what could the possibly the personas are and so definitely take that takes a huge factor and obviously that's a little bit of gamble because we have seen some metas that are not performing to Target but we don't know if the hypothesis of that they are more solo Jour Journey players has been proven before so this is something that has never been proven

before so we had to do a test to see how well the players respond to a more specifically Focus to this kind particular personas and it did perform really well they want outperform the hypothesis by 20% we were floored we were like what and um over time after three months is still outperformed hypothesis by 10 to 12% so that indic that data collection that data verification tells us player indeed are enjoying the solo Journey more and they're spending more time and they're interested in spending money once they're having the fund that they're looking for and so overall it had a really positive lift on revenue on session length on even a uh slowing of the decay of the ma as well okay and so like the second topic that I had mentioned in in the intro that we would talk about is how to make design and product decisions for live games and so to the and so just kind of going back even to this example of this personalized Avatar type of feature when

it comes to like let's let's say there were multiple ideas and and you and your team says oh we should do a Gemma or someone else says no no no the big problem is like balancing the levels and somebody else comes up with another idea and so when you have these different ideas of how to improve the game and these different to your point you had an ideation process then you had specific hypotheses right that you know we need to create more personalized experience by having a relatable Avatar to blah blah blah blah blah versus these other ideas how did you think through whether that was the which which scenario or which thing you guys should work on and whether that was right or wrong did you just try it did you try to validate it before you put it into product or did you validate it you know what we're just going to try it see what happens and then if it doesn't work we'll roll it back what was that approach the approach was to look at existing features and try to do a forecast of all different feature ideas what the possible impact

are for the kpis and obviously these are best guesses and at the end of the day the product owner has to make a call on which one that makes makes most sense and in this case because we saw the meta that will come more competitive w't performing as well and this one even though it was never done before we my hypothesis was that it would help players navigate through more systems and therefore would extend their session length time and we did forecast based on that so right so so there's kind of two two things at play that I want to kind of disaggregate for the audience so the first thing that you mentioned is you were looking at impact so something like I assume you guys use like an ice rice or zingga style expected outcomes model to say okay if we design this feature here's a potential impact it could have you then you kind of stacked rank stuff and decided what so that prioritization seems like it's was pretty standard for in terms of the approach that you guys took what about like for this Gemma

feature in particular let's say you think you're like yes this is this is a feature that will have this impact on retention or monetization or whatever but let's say there are other people in the team that are like no it's going to have a negative impact it's not going to be good how did you resolve the differences because ultimately game development is Art science data blah blah blah combined so how did you guys resolve towards a perspective on what that impact is potential impact is and if there were differences in opinion like you know you've got two people on your team who said hey this GMA feature is amazing and two people are like oh my God it's the worst feature ever how did you guys res we had that but great thing about B Blitz because it was so so well known so popular it had a huge fan base and we had a Super Fan Club where we actually have selective about 80 to 100 people who signed NDA who can preview features that we might be working on so we can

get some qualitative data and so when when my team is up and AR saying this will never work and we don't believe in it I detest this idea so we like to take it to the qualitative fan base that we have and see what they think and and the result was complete last SL would be like 8 to2 like the percentage wise of people people voting and so obviously that's still risk because it's qualitative but I helped persuade the team because they see that the players are aligned with the process that we were working on so it was worth trying obviously if it did not perform we will either look at the data to tweak it or like you said go back but got it okay so so even before you launched it there there was like a practice of getting player feedback product validation through speaking to an existing player base and those players that are playing your your game a lot those are your that's your hardcore audience so they should know or they should have a perspective on whether something is good or bad so that that makes a lot of sense

yeah so it's not a vacuum for sure but there is some guest work as you know with with with game designs yeah yeah it's not yeah that that's the thing with with our craft right Eve it's like it's it's not it's not a perfect science for sure okay and so then maybe now rolling into the third topic that we talked about Eve you've had really great experience and so whether at EA or imaji or other places when it comes to maybe some of the biggest lessons learned in terms of live Ops and some of the mistakes I think it'd be awesome for audience to hear about some of your experience some of the mistakes you've made so that we could potentially you know get to understand what those mistakes are so we could avoid them in the future my biggest lesson learned is to listen to the players and not just qualitatively that's important but knowing who your players are is so important for example for Temple Run 2 during covid we completely pivot the content to the psychology of the players

out there they are feeling anxious they're feeling isolated and tempor run itself was a player running away from demon monkey in a desolated area and to us how do we how do we cater the game to towards the player to have a better experience while releasing some of that anxiety so we made a very Serene beautiful map called Enchanted Palace that looks like kaj Mahal and U something BR inia somewhere that's comb that that's mystic and then we had two one's a Healer one's a herbologist as the characters so we help bridge that that sense of anxiety with the game the game mechanics that they were actually collecting herbs they were actually helping people and so that was very successful people really respond well to it they even said that this is the map they will come back to play just to look at the environment and so that gave him something a piece of mind that that usually tempor run traditionally did not have so I think some learnings from U

mistakes that I would say is that whenever we look at data and try to make decision solely on data that usually bites us um unexpectedly for example we realized that people were were turning off in in the foui uh early on between 5,000 under 5,000 meters so the solution obviously to us is to make it easier so that people don't die as often so that they can get past the 5,000 meters turns out the turn rate was still the same and even though was easier and that actually impact the revenue because they were watching L as to to do save me so was like okay let's go back why is it why did it not work it's easier why did they still turn out well it was easier but it was more boring so what we end up doing was very very funny it was Art change we just put signs to tell people ahead of time is it there's going to be a turn all we

did is put signs up and all a sudden people had a better experience because they feel like they're in charge they can prepare for the turns and in so the Improvement was was a lot more obvious and the the the turn rate was improved for for about I think had to left lift about uh 7% so that tells us not to only look at data driven stuff but also look at look to players what their experiences are and try to try to make decisions based on their experien versus a data driven decision for me that's a that's a takeaway yeah I think that's a really strong um lesson to convey and I can also just tell you from my experience I have seen so many tests that have been poorly implemented right and it's like oh yeah this survey told us this but then the survey was off the back of one of these like incentiv I get 30 gems to fill out the survey where people would just go tap tap tap tap tap tap tap tap close give me my 30 gems right and so you got

the whole notion of garbage in garbage out and so there's I I think there's definitely an art to like how you implement specific tests and then also to your point and I think the Temple Run case that you mentioned is one of how do you interpret the problem statement right so then ideating and coming up with okay here are the various potential reasons you know one of the things we do is we we build a logic tree to try to break down what the potential things are but then being more thoughtful on that it could be because of the challenge but maybe there are other reasons at play that could be impacting the the loss in retention or the the 5,000 meters as you had mentioned yeah and that sign that that that sign thing works so well we're implementing every map throughout okay if that's successful that was but that was completely not there was no data that was showed that that would be effective so I think you just have to have to see it from players perspective to try a few things and try to see if what works and be willing to uh change

and pivot and ask the right question if the question wasn't answered correctly maybe we didn't ask the right question right any other lessons that you'd like to convey to the audience Eve I mean you got some good ones by the way so I mean no worries if you don't have any I think one thing I do say that because we are very driven in the mobile Industries to get people to play longer I think it this always comes back to players what they want for example for tempo run we did earthday content that was met with a lot obstacle from the team because they thought dirting up a map people would be upset about it and instead of picking up coins we're picking up trash turns out players loved it they want they resonate with the fact that they care about the Earth they want to help clean up the their environment and they they care about social cause it actually United them in a way that we did not foresee and the best part was that Revenue showed that it worked but

the Apple WAP actually came back told us was tin Cook's favorite Temple Run two content so oh wow great so it's so I think that's those are the things that I think we keep on listening to play looking to what they want I think that's the biggest lesson I have because we can make a game we like but ultimately the game is for the players and I just we should like it but the players should love it yeah and I I think the the point that you're talking about is a really complicated issue in my opinion right because we I I I can't tell you the number of times in my career where there are the people like there are people who are what I would call like the Steve Jobs or Elon Musk they know what the audience wants like Steve Jobs when when you know he's like yes players want or like like our audience wants something like the iPhone or the iPod or the this or the that but and there's so many people who think they are Steve Jobs for game design right and they lack the humility of saying well actually it's

not what I think it's what the players really think and so to your point I I really hope that people are a little bit more humble in terms of that aspect okay so Eve any final Lessons Learned or should we move on to the market well I think one thing that I noticed more and more because gaming is such a big lucrative market and everybody's trying to get piece of a pie more and more I seen people about the product like a product and which it is a product but It ultimately it's a game I think we have to remember the fund no matter how much money you want to make if the fund isn't there doesn't matter what the forecast is doesn't matter what the data says if the players are being treated like a wallet it's not it's a very hard battle to win but if they're having fun they'll they'll they are it's entertainment for them they will happily spend money when

they're enjoying the experience and I so I think to me the user perspective is so important that I always say I'm a player first kind of producer Because unless we provide the fun we provide the experience for the players that they enjoy they not coming back and that's what life Ops is all about it's about bringing them back again and again again yeah I I think what you're saying really reminds me of the well this is this concept I first heard from scopely but the the the concept of two ends of the Spectrum in terms of monetization on the player side is like the willingness to pay which is that to your point players enjoy the game so much they're looking for reasons to pay in the game very similar to Riot games right in the games that you have so much fun it's like oh yeah this this cosmetic whatever I want to pay and on the other side the back end is the payment optimization like how can you drive sales how can you drive monetization techniques to just maximize the amount that players pay and

sounds like from your perspective you really are more around the philosophy of fun and willingness to pay and to drive that aspect of monetization rather than just really trying to squeeze as much money from players as possible yes but I also believe good design should have monetization already naturally built so that they naturally want to spend money so we don't have to squeeze because when they get here they naturally are looking for something that's the that's the best of the marriages for me yeah so it should be a light squeeze it should be like yeah yeah exactly yeah I got got it is that if you are willing to spend $5 doll but they give you a deal so good you end up spending 15 and you have a good time you're gonna come back and do it again but if you squeeze them so hard that you get the $ five dollar out of them then they may never spend another time again that's a bad experience right and I think this last point also then connects and ties back to this final topic that I wanted to cover with you which is really

around the market and the current market environment and just so you know I've been speaking to a lot of people in the industry and I would say in particular I would say even like the Chinese leaders and people who I have been speaking to are so negative on the mobile gaming market for the next few years like they literally to go into hibernation they've seen dramatic impact of performance of live games that they that they have have launched from from China and so I wanted to get your take on the current market environment and also maybe one leading question is and when we talk about this last lesson that you learned I also believe that in the current market it will it's kind of Shifting towards you should be more focused on willingness to pay rather than payment optimization just because of where the economy is you know we've got record credit card debt like people in my opinion need to feel better about what they're spending money on sure and so handing it over to you

Eve in terms of your current take on the current market difficulties and things like that and maybe you could kind of describe not only how you would characterize the current market difficulties but then what do you think developers need to do to thrive and survive this this kind of New Market environment that we're heading into I think there are multiple reason economy is definitely a big part of it but the players are smart they can see when we're trying to squeeze a d out of them and when we do it when we do it well they tolerate it when we don't do it well they don't tolerate it and more and more so like I just mentioned previously I think a lot of people are very data driven very product monetization optimization driven that it leaves the players not a desirable experience so I I actually I truly believe to understand what players want out of a game and if you give them what they want then that that will help us sustain through the difficult times

because as we know you know coffee is great for doing depression doing doing uh Heights it's because people want coffee whether they're having a hard time economically whether they they're making money coffee and tea always do well throughout the times games can be like that I think just we have to give them what the players want and give the sense whatever sense of satisfaction the sense of achievement the relaxation or the sense of being social with their friends whatever it is that your game provides if you can give them that Ultimate Experience I think that's the key to survive and to thrive in this market current state okay well Eve that's all the questions that I have it was kind of like a a quick discussion here but I feel like a lot of the lessons that you have mentioned and some of the learnings that you've shared are super valuable and hopefully the audience can learn from that do you have any last minute message

for the audience or ways in which they can contact you sure I am actually currently very busy trying to put up on my website it's neuros sisters Studios and it's about neurology it's about psychology of the players and it's obviously about females in the game Space but it's not exclusive to it's not exclusive to the female audience it's just more female focused so n sisters studios.com okay and I will put a link to your website in the show notes but Eve thank you for your time and sharing your lessons I I think it's very valuable and for our audience we will catch you next time thank you so much Joseph bye bye thank you [Music] bye

[Music]