Legendary: GoH 5 Lessons from 12 Months of LiveOps Growth

[Music] alright let's get started hey everybody my name is Ethan Levy and I'm here to present legendary game of Heroes 5 lessons from 12 month 12 months of LiveOps growth and actually between the time I put this presentation together and now we've had an additional four months being live so it's actually the top 5 lessons from 16 months a little bit about me I've spent 14 years working game development worked on over 50 ship titles across console PC mobile mostly in free to play space mostly some interaction of design and monetization for the past 2 years I've had the pleasure of working on this game as part of Network legendary game of Heroes if you haven't played legendary I think it'd be could be quickly summed up as puzzle and Dragons westernized so today I'm gonna talk about over the past 16

months as we've been in players hands I think I personally and the team have learned a lot and we've grown a lot and I wanted to kind of summarize some of the top things I've learned over this period of time just to give you a feeling for what we're talking about this chart shows ARP table growth over the course of 2016 this is each data point is a 30 day preceding average so I didn't just want to talk about peaks and valleys but actually numbers that represent really sustained ARP Dao or player engagement metrics so this is our growth over the course of 2016 in terms of ARP down you can see in the middle a big kind of baseline resetting as we hit worldwide launch and similarly you know it's not just about how much each player spends but it's about how much time each player spends in the game and our player you know our engagement time per day for your average player has also grown looking just at that 2016 period we have

increased 652 percent in terms of our ARP Dao and 136 percent in terms of minutes per day spent and that's if you're looking at a 30 day average that's I'm not just giving you like our worst technical issues to our peak sale day that's that's a 30 day average for each one of those this growth has really enabled remarkable results on the iOS store we've been top 163 countries 49 countries on Android we've gone top ten on eight in eighteen countries on iOS and we've done this all with the team of 15 people so today I'm gonna be talking primarily about feature development and the lessons learned from future development and the reason I'm focusing on that is because that's where I spend the vast majority of my time but it's really important to note that features alone don't produce results

like these right as I said we've got about sixteen fifteen or sixteen full time people on the team a bunch of shared studio resources and just world-class people in every one of these pillars and it's taken a blend of live operations UA and business intelligence feature development player support all held up by a strong studio operations staff and all held up by a world-class studio culture and so I'm here talking about features and the impact of features on growth but it's a this game is the complex ecosystem in all mobile games like this our complex ecosystem and it's not just one thing it's the full team in the full Studios effort that I'm talking about today so as I said it's really world-class execution across all these pillars that have been powered our live ops growth so with that let's talk about the top five lessons I've taken away when reflecting on this period number one focus on inflection so if you look at this this is a list of all of our major updates over the course

the year I left out some small point release some small point releases that are just bug fixes and they range from you know you can see our soft launch in early November of 2015 up to commander skill which we're actually releasing this week and by about six or seven on Tuesday we'll be able to know how well players are receiving our new commander skill with the crusade event so I've taken the ARP Tao chart and I've broken it up each color by kind of the period of the major feature release and so this is why I use those 30 day preceding averages so that you could see on each day for the next 30 days what the average are so because I was really interested in the trends and in spotting where things change and as you can see there's a giant baseline reset when we go worldwide and that's because we're looking at per player metrics and when we went worldwide

we got an influx of featured player thanks to features from our friends at Apple and Google we were very lucky to have that and we also started scaling our user acquisition and so once we started scaling up our players there was a baseline reset so for right now I'm gonna soon zoom in on that soft launch period and that first month in January is is pulling in a lot of days where there was not a lot of people so really starting on February through worldwide launches where I feel like these numbers are really valuable to look at so I said my number one my top lesson was to focus on inflection what am I talking about so you can see in with our 1 2 release we built the support for events but we didn't yet have leaderboards on those events and so we were missing the competition angle of it and we didn't actually start running events we kind of laid the baseline with our 1 2 and you can see what that first trendline is is showing where our bow was heading on 1 2

and then as soon as we turned on leaderboards turned on competition and started running events on a weekly basis you can see the inflection in our ARP data that that caused so when I look at this 16 months worth of updates I look at four releases and I think these really had immediate measurable inflexions leaderboards and turning on weekly events our raid boss event harness which was a new type of event we released in in May of last year optimization on the Android platform and our guilds I look at those and I can say like oh you can point to really immediate inflections on all those and there were a bunch of updates either major or minor without immediate and without those big impactful jumps they still had to impact and each one had a positive impact but it wasn't that really satisfying level up so you know

why am I making the differentiation well obviously if I focus focus on feature development I feel like each time you release a new update it's kind of your turn at the baton really I'd love to hit a home run every time I'm up there and we're home runs came from was adapting and innovating on best practices from the marketplace but we hit plenty of base hits along the way and those base hits I would say we're either bundles of smaller features that didn't tied together into a cohesive whole immediately or something I've started calling bank shots so what do I mean by a bank shot I mean when you're trying to inflect a specific metric or you're trying to motivate a specific behavior but the way your feature gets there isn't a direct line it's more like oh if I hit off this wall and off this wall then this is what the player will do so to give you a really tangible example late last year we released a Guild Wars feature and it had a couple different motivations and one of the motivations

was to increase the appetite for Heroes in our game you bring five heroes with you into each round of combat and you really during the course of the week probably only need five great heroes for that event until you go to the next event when you need another five heroes and most of those heroes might be the same or now that we're running different event types the same as you did two weeks ago and so we released this guild war feature where it was the first chance that one player has to attack another in the game and we release skills and counter skills so there was interplay between one players attacking deck and one players defending deck and that was five attackers and five defenders and part of the idea was that we would increase the appetite for rare characters over time thanks to an evolving meta in the counterplay between skills and counter skills and so you can kind of see what I mean by bank shot it wasn't a very direct line so now with perfect

beautiful 20/20 hindsight I can look at that and say well you know I wish that if we had one to increase our appetite for heroes we had done it by delivering a game-mode that required more heroes if we want to make it that you need more than five six star heroes then we should make a game-mode that requires 36 star heroes at the highest levels of competition something like that that to me is a much more direct attempt at it anyways so the the lesson here as I said is before committing to an update ask yourself is this going to directly inflect our business and I think it's kind of a tough question to ask we had a what I think was a very successful moment late last year when we started going down the path for a feature that was called world boss and in the room when when we decided to pursue world boss part of its appeal was it seen like a low effort feature that we could get out fairly quickly that would have a nice inflection and so we started designing this feature we got it to what I would call about 60%

resolution we started reviewing it and it seemed like in order to really make it a feature that was up to our standards of greatness it would be much more meaningfully effortful than we initially anticipated and when we looked at the amount of effort it would take to get world boss out and the amount of effort some of the other things we thought would be more impactful out we said well let's stop developing world boss and choose one of those things and work on that instead and even though that meant throwing out a significant amount of design work and potentially even causing some inefficiencies in the development and release of new features I think it was worth it I think we made the right choice by choosing to focus on something that we believed in could be inflective relative to the amount of effort we were putting in a second lesson build for your bandwidth so all teams have bottlenecks and when I look at kind of the middle period of 2016 ours one of ours was building game data as I said there are a lot of there are a couple features that I look at and I

feel like we made immediate measurable inflections in our software and then there are a lot of things that I look at and I say well we put great tools in the toolbox and some of those were still pulling out and using today right nothing everything that we have done has been useful over a period of time and helped us increase how much fun events our four players make the software better make more revenue per player but I think there's a period where it felt like we were building a you know we were building more tools than we could use it their full potential and that was because we had a really turbocharged world-class engineering team and not enough manpower on the p.m. front for the amount of tools that we were building to use them effectively let me give you two examples one example is an early feature that was a lie mission is the design intention was it was a really low effort feature where we wanted to increase session frequency by giving a snackable timer based mission the

inspiration Point was missions that you'd be familiar with from mobile strike or game of war and at the time you know this this failing I can take ownership of this is my failing when I set it up I set it up with no loss aversion and when I look at what works well in those missions and mobile strike is if you don't come back every six hours and collect all your timers you've lost resources but the way I set visa was that there was no loss of version once you get past a certain rank there are always four ally missions available for you and so there's no part of you that says I have to come back to the game to set one of my allies on this ally mission and when that 15 minute timers up I better come back to set the next one instead they're always available the rewards are fairly consistent and not that great and you can look at it and go once your high-level player and just sort of forget about this whole section of the game and so this this feature hasn't had a retune I think since we use we use allies and other

ally missions as part of our weekly event so as I said they're valuable tool in the toolbox but at no point have we put the effort towards going back and raised raising up the level of what these do and it's not really clear to me that that effort would be paid off relative to putting that same amount of effort in another area of the game another example would be the enemy skill system that we first released I think in 1/2 or 1/3 kind of early on our design intention was to increase intention by adding progression novelty to the campaign and you know our game is like candy crush match game below with an RPG mechanic on top with a bunch of heroes with stats and skills and part of the intention here was that your enemies would drop gems on the board below that would add a little bit of skill play in and change things up and make levels more exciting we could use it to make the campaign more exciting and events more exciting etc when we first implemented enemy skills

we put it in one section of the game which was the daily dungeons and it really took a long time until those enemy skills and the skill system in general started to bubble up and play a meaningful in role outside of that initial use and again I look at that as our bottleneck on design and data creation right we had to be very tactical every day about what the PM's are doing to level up the software and so we have things where I feel like we just didn't use the tools to the full potential when when but now I mean we made those investments and in the past two weeks one of our new and very talented team members has been pumping out unique battle skills and enemy skills for our event characters that I think like finally make this part of the game shine and where I've had some of the most fun in our events recently with our characters because they're starting

to feel unique so over time that investment is paid off but when I look at that period of time what I think the lesson is is to adjust your output to match your capabilities given the mismatch we had in terms of how much could be accomplished between this world-class engineering team and not quite enough PMS I wish we had done more better right I think we could have made even more inflection than we made by investing in fewer features and bringing them out either at higher quality or bringing them out at a faster cadence and that's that's one of those great lessons you can see with a lot of clarity in in hindsight lesson number three live event live ops events are the heart of your game I joined the legendary team I think it would be kind of on the tail end of pre-production on at the beginning of production we didn't have clearly defined milestones but I think getting to version 0.9 was like

we're building the base game and then that year of growth was like we're building the events on top of the base game we're building the event structure on top fed and I I think that the key lesson I've really taken away is that live hops live up events are the core of your game when you're focused on this sorta game and there is no separate other base game and I wish we had just started there so this is this is a slide I already showed right this showing the inflection point when we just started releasing started and released our first event that was driven by leaderboards we saw an immediate inflection and you know ever since that point we've run one event every week up till you know we have an event ending tomorrow at 5:00 our weekly events pull on a core LiveOps suite as I said everything contributes to our events but kind of the core of it

are our two harnesses which are special dungeon and our raid boss harness and actually next week we're leaning into an older part of the software the activity map and combining it with the raid boss harness to make even a new event type but you've got your event harnesses the leaderboards the drive competition initially it was just solo competition now it's solo competition an additional global leaderboard for top players and the guild leaderboard all within an event we've got a pop-up system that plays a critical role in the game and we've got guilds and localization every event we time it the execution of it so that all the text is localized in the key countries that we localized in which was that was a fun moment of LiveOps growth getting things together early to get everything localized but that's kind of the core of our live op suite so what does that mean when I say we've run an event each week well we have some form of limited time content which is a special dungeon an area where you fight

special raid bosses which have unique rules relative to the base campaign game and now with these Crusader events we're experimenting with using the campaign map to be part of that limited limited time content so those levels are there those bosses are there and when the event turns off they're gone every week there's at least one new event boss I spent screenshots from our phantom of the opera' event because I think it was one of my favorite examples of giving the art team something that's not immediately obvious in a puzzle RPG and I think they really executed it an amazing level I really enjoyed these characters but so limited time content brand new boss competition on a leaderboard as I said we've grown over time from having you know even AB the gate we had segmented League based leaderboards but now we have a solo leaderboard a separate global leaderboard that if you're in one of the top positions in the game you get both solo and global leaderboard results and

guild leaderboards each leader board gives rewards and that boss character is the exclusive exciting reward that you can get only from that event we also sell at least one new hero in a new card pack with each event and we use our pop-up system our event-driven pop-up system to run in-game marketing and ingame messaging about what's new what's exciting what's valuable decks you should build cetera that's kind of the core suite a slight addendum to rule 3 is that left unchanged event performance will decay over time now what do I mean by that when we launched at the gate we just had dungeon events and as I said we were in the road to soft launch we had a lot we needed from soft launch to worldwide launch we had a lot we need to do and so for a period of time we were pretty much running the same dungeon event week after week after week that's all we had and over time we

players interest naturally leveling off until we released a our second event harness our raid boss harness which really turned around the trend there and allowed us to run two different events you know dungeon raid boss dungeon raid boss back-to-back and so to keep things fresh on top of it not only do we have like two major categories of events but as I've been saying all these tools we put in the toolbox we use to change up events to improve their performance to make it clear for the player what's important though you know in my first point I hit on immediate inflection and trying to hit a home run every time if I look in that one two and I look at pop ups pop ups at the time just a driving dine an in-game marketing on its own might not have seemed like one of the most inflective things we've done in the game but over the time as we've leaned into that and learn to use it better and really focused on our marketing messaging that has probably become one

of the most impactful features we've added to the game and it was easy to look at it and go like oh we added marketing pop-ups we can put images in front of players and text array but it's actually like over time learning to use all of those tools in the toolbox has driven this incredible growth in terms of ARP Tao or has been one of the main things driving growth in terms of ARP dowan and player engagement over time what I put up there and then this kind of hits to the point that you know I'm focusing mainly on the impact of features but these changes in our town and player engagement are not just about features it's a complex ecosystem and so you're getting a combination of feature inflections population veteran C and event optimization so new things that change the way players behave in the game the metrics naturally changing as your player population grows over time right the if if you have a successful

free-to-play game that means your players your veteran players are sticking around for a long period of time and so as the mix of new players or early game players to veteran players changes your ARP dau and daily engagement our natural gonna grow over time because your veteran players are naturally the most engaged and tend to spend the most and so part of that's just the shift between those that's natural over the growth the game and part of its intentional through feature growth and part of it is through event optimization but as I said at the beginning events with leaderboard competition and exclusive content to buy and win are really at the heart of what I think in modern free-to-play mobile game is and really uh if this was day one of a new mobile game new free-to-play mobile game this would be one of the primary design consideration siddur a shion's lesson number four was to believe in the power of guilds I think if you look at the top 150 grossing games and you look

in the mid core section it's you can see that there are a lot of games with gills where it's been really successful there are games with guilds where maybe it hasn't paid off but I think you know going into this game I had never operated a game with true guilt and releasing it has I think been one of the major inflections that the team has made during the course of this games live operations so at its heart guilds are a player can join a group of people they have a shared chat room where they can strategize they have a guild emblem now which they didn't have it launched but they have a guild emblem to symbolise who they are the leadership has some tools around setting different ranks of players in the software in their guild so you can promote some people to the council or give them the title of Dukes and duchesses to just show how they're doing in the guild so it's a chatroom the ability for the guild to compete as a unit on a guild leader board during

each event and the ability for players in the guild to earn special gifts we we made this item gifts that when you earn it you can only send it to a guild member so for instance I may earn a gift due to my placement on the solo leaderboard at the end of an event I may purchase $100 I ap pack and get ten gem gifts to spread the wealth with other players in my guild and I can't use them in myself I can only send them around around now guild did not have an incredibly long period in soft-launched it was the last thing we launched right before worldwide so the reason you're looking at inflection downward is because this graph ends at the point where it crosses over and the inflection happens so this is just the soft launch graph so let's zoom out to the worldwide launch graph and as you can see there's that as I said that natural resetting when we went from soft launch to worldwide launch and we scaled up meaningfully but when you look here on this ARB cart you can see that

inflection that the guild system was a part of for when I look at it I think introducing guilds was a major inflection point for the game so let's go back to that previous chart I was showing where I'm showing ARP tile on top and player engagement on bottom and as I said this growth isn't feature centric it's an ecosystem so part of that growth was due to our user acquisition strategy part of it was feature inflection from the guilds and subsequent features we launched part of it was event optimization and part of it is that population veteran see so what do I think that guilds major contributor to the software was people who are in social groups and are in an engaged social group are gonna stick around for longer and so guilds by creating more veterans helps improve the stacking of veteran players in our software and increase the portion of our player base that are veterans and that lifts all of our metrics on the per player basis and

has a meaningful difference in our in meaningful change in our absolute numbers so let me go back to my lesson guilds belong at the heart of modern free-to-play game so if I take those two and combine it and then and if today was day one on a new mobile free-to-play product the first question is how is this game built for guild events that's the primary design consideration in my mind number five this one's real short one complexity of featured design increases over time so I'm just going to talk about two features one we launched early on leaderboards one we're building right now relics and Scrolls and if I just in a high-level segmented League based leaderboards that drive competition and a set of relics that you can equip to a team to boost their stats are not wildly different in a vacuum in terms of effort but at the point where we release leaderboards the game was much less complicated so this is the overview page from leaderboards these are the six main

screens that leaderboards had to touch from a user facing perspective this is the overview page of our currently in development relics and Scrolls PDF and I think it's like 32 or 33 screens and as I said in a vacuum these two features aren't crazily different what's different is that the size of the game has increased and the number of points each one of these new features has to integrate with increases so as feature richness increases so does the complexity of adding new features to the core loop and I think if you've heard any of my previous talks I've hit on this a lot that building features into the core loop in a way where they market themselves as critical to players discovering them and I'm having the impact that you want on the game so my counterpart he day who I think was the who is really the architect of the game economy he came to this game with an incredible view on the needs of of growing the game economy over time and he knew and planned for that growth on the UX side I don't think we did enough

to plan for the growth and anticipate years of growth of features being added to our game and needing to add them to the core loop and each feature we just have to do more like UX refactoring to make sure it really works and that it's surfaced to the player and I think this is natural there's just like you know there was no way to not have stat inflation in our game over time and power creep over time and it's a challenge just to plan for it and make sure to provide a great user experience I don't think it's possible to grow your games feature depth over three years and not need the sort of refactoring over time but I think better planning for it at the outset would have made our lives easier and presenting great features to players over time so wrapping up real quick these are my top five lessons focus on inflection build for your bandwidth live ops and Guilds and live ops events the guilds compete in our the heart of your game build tools that allow you to combat the natural decay of events and that and plan for the fact

that your work feature design complexity will increase over time and you know really the most important thing to take away from this presentation is great results are possible over time you can do incredible things during the live ops period of your game during the soft launch and worldwide launch to change how players interact with your game but it's not just one of these pillars it's all of them it's world-class people working together across live ops you a feature development player support supported by a great studio built on a great culture this is what it takes it takes all these things to achieve greatness with the modern mobile free-to-play game so thank you very much when a little bit long I probably have time for one maybe two questions then I can take questions for anybody who has them who'd like to stick around alright well thank you all very much up we've

got one hi we're good page Thanks and what's about impact on revenue of sales of pop ups like game over on mobile strike mm-hmm so the question was what is the impact on revenue of sales and I think there are two types of sales and it's very important to think about them differently one is a sale where you're selling currency so I used to go into I used to play mobile strikes six times a day every day for months now I pop in once a month and each month that sale the amount of bonus gold is inflated right they've inflated that over time and so what that means is that they're selling currency and a lesson a key lesson I've learned is that selling currency can be a really effective way to boost your revenue in the short term but generally the more currency you sell there's a hangover on the on new currency purchases after that happens

unless you've designed the game so that there are new things to always be soaking up that currency so selling currency is really effective but you have to plan for the after-effects the other thing you know a thing I think we focus on selling a lot more is new things that are really limited time and that's worked really well for us in aspect and there you're just dealing with a different sort of inflation you're dealing with the power inflation over time another quest part of your question was about the pop ups and how important are the pop ups pop ups are key to promoting and explaining the value of new things that were putting on sale in our game Nielsen in the back row and I think within at least the last week if not the last two there have been times where he goes what's wrong with the pop ups and it's because he looks at the hourly revenue and he can tell instantly that something's wrong with how we're messaging to our players so as I said before like the the power of messaging to your players what you're selling to them and what the value of those things are to their play

experience it can't be overstated Thanks yep I'll just take one more hello this is Victor from Gameloft you said that you had kind of problem with kind of contempt fatigue with again weekly bends mm-hmm what is for you the best way to contract that so with through game play or so the metagame let's see ah so the question was early on I said we had a problem with fatigue during events and it's just kind of a natural part of the game I really think I think that the best way to combat that you know you're not going to be able to release a completely new event harness every week and it takes a long time to build each one of those and so really focusing as a live ops team on not novelty having the tools to introduce new things and releasing new client updates that make minor improvements such as as I mentioned the global leaderboards or launching this week commander skills like just introducing new things over

time and using the tools you have to make things feel as new as possible because you're going to new skills this is pure gameplay no you're commenting new leader poor system is metagame so what do you think it's both sometimes you can change this watch is the other I think it's both I mean I think it's you have to each week basically look at the tools you have and say like what's the best thing we can deliver or do we need to deliver absolutely new things so no easy answer sir no magic go for one of those no all right thank you all very much for listening and please fill out your review let me know how it can improve and hopefully I'll be back next year to tell you about another 12 months of growth