Brawl Stars': Learnings from the Removal of Loot Boxes

[Music] so welcome to our session about learnings from the removal of loot boxes and braw stars you probably seen one or two headlines out there and as we say in Finland R is currently doing pretty okay um what what these nice headlines don't show though is how much it took it to get us here and today we're going to talk about the removal of loot boxes but also our key learnings by taking brows from historical lows to historical Heights right now and spoiler alert we're going to talk a lot about engagement show of hand maybe how many in the room have played brawl Stars oh wow oh sweet sweet awesome that's quite

a few um for those who don't play Brawl braw is a fast Bas 3 versus three multiplayer battle game and Battle Royale it's made for mobile it was released into Global launch in December 2018 and generated a little bit over 2 billion USD lifetime Revenue so far the game is made by a very diverse team of people mostly in Helsinki but supported by some of our colleagues in South Korea we we um have 23 nationalities in the team and we are also supported by a bunch of people in central teams within supercell and on the outside so we're working with Freelancers and we're working with uh agencies on the outside to support our work my name is Fran Korg I'm the general manager for rosters I've been with supercell for a bit over 9 years now 9 years 3 months and 12 days or something uh I moved to Finland you have

understand that so I'm still counting days maybe um previously I worked for Blizzard entertainment in Europe and a in a role in PX awesome and my name is Frank Yan I um currently work as a game designer umbrell stars and mainly focus on systems and prior to this I was also at blizzard but in the um most recently on the Hearthstone team I was there for about six years and two years at at supercell and I got my start when I my dad told me that I'll never amount to much if I played a lot of video games so that really fueled some uh passion to prove him wrong and no you don't have to be called Frank and neither did you have to work at blizzard to get a job at supercell we're hiring so yes let's go so the talk today is separated in three sections really it's before during and after the removal of loot boxes and the first part before we go into anything else

please keep in mind that what we're presenting to you today is our reality it's our learnings they're very specific to Broad star to the history of rers and everything else so your game circumstances might be different uh we still hope that some of the learnings we're sharing with you in a moment will be applicable to what you do um but we wanted to set the expectation that you know just because it worked here for us doesn't necessarily mean that it works for you before we talk about the removal of loot boxes let's talk about how loot boxes actually looked like in raw Stars before and they were pretty much part of the game from early soft launch we went through a bunch of iterations in the mattera But ultimately ended up with this uh series of three different boxes you have the regular braw box that's the iconic thing on the left in the middle you have a big box that's worth three boxes and then the Mega Box uh the PC resistance um and that one is 10 gacha PS

um now these boxes contain both horizontal and vertical progression in process so it was your primary mean to progress in the game and it was our second most important SKU after the battle pass which is called brw pass and bra star this is how it looked you open a box had a few tabs with escalating rewards sometimes you got lucky on got a brawler and that was the whole EXP experience now let's talk a little bit about the early days of broad star initially things were great um so you see there on the left side the period from December 2018 to December 2019 and then January 2020 to June 2021 on the left is a global launch that again was December 2018 the big Revenue Spike there in the middle what a happy moment it's a termination between bro pass the first

version and The Game launch in China and as you can see the numbers came down quite hard afterwards um overall what we saw in our game is that the revenue did not always follow the growth of the Mau it was still meaningful and over these 18 month or so the game generated the first 1 billion USD however things didn't stay like that this is then the period from July 2021 until October 2022 so coming out of the summer period summer 2021 uh Europe is a crazy important market for us and everyone in Europe in summer 2021 traveled because they could travel again and we have this weird thing that in most games vacation means people play more but in Broad Stars when people go on vacation they play less because the social graph around them is missing so it hurt us so every after

summer we need to bring people back to the game somehow and guess what they didn't come back and there's some external factors but also probably a bunch of internal factors of us not hitting the mark with our updates in October 2021 we released an update to stem the tide against the decrease of our kpis and so we introduced a feature for clubs where all the members in the club could participate it was a really hardcore system with leaks and a shop a special currency Etc and we thought you know we need to enable social to get higher retention y y y um we also introduced new two new power levels to the game before that we had nine power levels for each roller now we had 11 and to make that interesting why why would you level up your character we introduced new Ultimate abilities there were passives actually called gears you could equip and they were pretty complex you had to collect them different pieces then craft them

a concept completely new to bra stars um and well initially you see that both Mau and revenue stabilized somewhat but there was also the seasonality of course in quarter 4 But ultimately this update didn't hit the mark at all with our community and they told us in no unclear terms that the fund to cost ratio for this new feature was not okay then February 2022 uh Russia in the Ukraine um for us that meant that in February essentially we pulled out of Russia immediately and Russia at that point represented 25% of our diu and 20% of our Revenue so in this downtrend already we had uh well an even even harder battle to fight then later in the year um we decided to rework gear the community

especially hardcore Community was still pretty upset with this update uh so we simplified everything no more crafting gears could just drop now it's just one level um and it worked out again somewhat da and Ma came up uh Revenue stabilized somewhat but yes it didn't really do much so after seeing a decline for quite a long period we knew we needed a change change position okay great uh around this time I actually got hired so uh I got hired during a pretty tense period not going to lie it was uh a bit stressful they kind of said hey uh we need to find uh we need someone on systems designed to come kind of help us recover the game um and when I when when I came on one of the key questions that we asked was how do we get players excited to come back to brawl stars um and the answer was quite clear

unlocking Brawlers is one of the most engaging aspects of the game particularly we have five Rarities of brawlers the highest is the legendary as you see here and these are the most exciting Brawlers to unlock because through boxes you know there's a chance you unlock one of these and it's a really exciting moment you see it in videos people record and it's really really exciting so at this point you know we had maybe like eight to 10 of these legendary Brawlers and when we did the math it was like on average it takes a free-to-play player almost a year to unlock a legendary brawler so we we actually viewed this kind of as an issue um so we believed kind of like the best solution would be to remove loot boxes from the game this is a pretty dramatic jump from what I just said but I'll go through a bunch of reasons and maybe reasons that maybe weren't that great in iight but uh so why did we believe that we should have removed loot boxes the first first was we were in a decline and

we felt we could take bigger risks with the game the second was that we uh wanted to provide more choice to players and clear goals around brawler unlocks you just saw a common problem there the third was there was a lot of scrutiny and continues to be a lot of scrutiny around uh loot boxes in the media and uh you know imagine a situation where we say we come out and we say brawl stars is removing loot boxes that is a very newsworthy moment and we we felt this could be what we needed uh the Silver Bullet so to speak and then certainly even within supercell the other teams had kind of been discussing like you know maybe we wouldn't do a loot box um and so we thought that in the worst case we'd be able to pass on some learnings from brawl to other games uh naively though and this is a huge uh huge butt we decided to remove all random rewards from the game so regardless let's go down the second

part of the presentation which is around what did we do during this process of removing loot boxes well uh we had some key goals um not surprisingly we wanted to provide clear goals and uh and expectations for when players can expect to unlock their next brawler we felt that because loot boxes as Frank mentioned and I'm not talking about myself and the third person uh we need a new monetization hook because loot boxes represented quite um you know strong monetization for the game it had to be really easy to understand and I can't overemphasize this enough and this will transcend the talk that it had to be really easy to understand people have to come in they have to get it and this is a really core pillar of ours so this is what the replacement system looked like we call it the Star Road two RS and uh what happens is you want kind of unlock braw ERS

sequentially so basically we tell you in the beginning which Brawlers you unlock and then you kind of uh collect this new currency called credits and once you earn enough credits as a freeo play you unlock that brawler um the order is predetermined in the sense that everybody gets the same path so there's no unfairness here and uh we have to however we have to account for different Rarities we have five Rarities each has a different price point that had to be respect Ed in the system and uh so as I mentioned we have a credits as the new currency and we said no R in the system right uh however this is uh one of this this leads to the first learning which we had over 60 Brawlers in the game at this time and imagine like if we had come up with a system where you got a choice between all 60 that's a very tall order um and we actually rest with the design of the Star Road for many many

weeks uh we went back and forth around the questions of you know how do we account for you unlocking a rarity because imagine if players are excited to unlock a legendary Rarity brawler but it takes them many many weeks but they can choose to unlock a legendary they could be stuck unlocking a legendary brawler and not being able to unlock Brawlers for a very very long period of time so we had to account for this in the design of the new system in arriving at the this also how many choices do we give you we had to kind of find a way to limit that and uh you know what about new Brawlers added to the game how do we account for that our ultimate decision as you saw in the prior video was we we had these really complicated design documents and ultimately what we realized was look we're just going to allow players we'll give them a subset of choices and they'll just be able to freely switch between which brawler they want to unlock until they have enough credits and then they uh get to unlock that brawler and this is kind of the details

of what that looks like so this player is first unlocking Jackie she's a super rare brawler our second highest rare uh the our second lowest Rarity and then we say Hey Now You're unlocking uh either Brock or barley you make a choice we wanted to kind of have you commit to that choice so at least you kind of saw your progress towards um that brawler and then the closer you got actually the higher chance that people would monetize to kind of top up the rest and then you kind of move on to the next and we always just limited the amount of choices for you in this new kind of system uh what this meant was that uh the Star Road accounted for brawler unlocks and because I mentioned earlier that we removed all random rewards uh this is some foreshadowing uh all of the rewards for vertical progression so powering up those Brawlers became deterministic so you knew exactly what amount you were getting when so uh some of the pros and cons well the first the I'm just going to mention one key Pro and one key con the

pro is that it's very predictable you know exactly which brawler you're going to get you have a choice so it's like really easy to understand and the con though was that it's really not that exciting and to unlock Brawlers anymore in this new system because you knew there was no there was no surprise when you were going to get them and so engagement was at risk here uh regardless I think this comes to our first learning which is and we this is something that comes up often in our design discussions is when wrestling with a design simplify and uh you could probably cross out the first part of that sentence and just the learning is just simplify because a lot of our designs I think that resulted in some of the historical highs we've experienced more recently we do think are they're really easy to understand and brawl is played by a largely young audience and when they come in it has to be really obvious what the game they're playing is and how things work so I can't overemphasize this First Learning

enough okay so we decided this was what we're going to ship with yolo and uh Frank will now talk about what happened so at the end of November uh we have the broadstar World Finals usually it's an Esports event and during that event we had um a small community talk on stage which told the community that we will remove loot boxes from the game and the irony lo and behold we introduced a new box to send it off the Omega box the box to ENT boxes and so we created this community event where the community would have to do some takedowns in the game we're talking about millions and millions of takedowns and as they progressed on this path they would unlock rewards um because we wanted that the community feels good about changing system and it was really really important for us so far that well there on board with that change they got excited we could also

reattach people and so we created a simple Community video where we talked about it which got had 17 million views um just like a little Community video so it reached a lot of people created a lot of hype and people were really really excited about this so that started late November early December in the same period we had a lot of other things going on besides the Omega box event we also have our traditional seasonal free base in the December from December 12th which is our anniversary to the 26th we had a free brawler people could unlock with a quest we had a new legendary that garnered a lot of interest and the first legendary was cheaper after the system change and last but not least seasonality quarter 4 is always higher for us in audience and spend is also always higher so the initial results were pretty promising you see pretty clearly when things happened so first the Omega box event uh early December late

November and then the update itself with a stardrop on December 12th uh and normally even on update days we have a du and Ma drop so it was pretty clear that the community got excited and a lot of people came back to the game and yeah so zoomed in really in the diu the initial results looked awesome diu was up 60% month over month which is like wow hey did we just pull this off however it was super hard to get a read on what was actually going on because there are four mentioned things we did so the next learning is really about and hindsight hindsight is great but the next learning is about reducing noise when you ship important things you really want to make sure that you can read those changes in your data in other ways and we just couldn't the socials were full of stuff uh most people are still talking about the removal of boxes so it was

impossible to read socials the data showed us well what you just what you just seen it looked pretty good and pretty promising even retention jumped up temporarily but yeah it was impossible to read for us so our suggestion to you if you ship a big changing update or something really where the key is in measuring the outcome yeah avoid avoid doing a ton of other stuff around it and uh don't pick Christmas because that's uh we were in a hurry for some reason yeah no kidding so now we're going to the period after we shipped the Star Road we had two more updates um right after and both of these updates had been planned as the original change in February we Shi masteries masteries the idea was pretty simple give players another way and give players more agency to change this on rails path the regular unlock process of

rers was all of a sudden just by playing more there was a cap there as well but uh that cap would kick in after like eight hours of straight play or something you could get additional currency to unlock Brawlers to upgrade Brawlers some extra Cosmetics for the Brawlers you you loved um yeah and then there was a second update in April which introduced a new currency bling it removed an old currency called star points uh and introduced the catalog in short what this did is that we replaced a currency that was entirely broken it was uh exponential star points was something we would give out for competing in the game in different places um but an average player would maybe get 50 star points in a week and someone who's super super invested would get 50,000 and it's incredibly hard to manage that uh from an economy perspective and so we couldn't we couldn't produce all that content for these people with with 50,000 and it

also didn't make sense um but we also couldn't lower the barrier for other people so we removed it we replace it with bling uh bling became a pure cosmetic currency we made it clear from the beginning bling can be used to use uh to buy almost all the Skins all the Cosmetic items in the game except the higher Rarities of items um so we wanted to make cosmetic items more accessible for free-to-play players as well we hoped it would generate some hype in the community and the catalog is the second feature here the catalog was our new way to to distribute skins um when brus launched we had like less than 20 skins in the game and so you had a random random four skins showing up in your store every day um for 20 skins that works when it shows up you buy it that was the only way to buy skins um but by this moment we had over 300 skins in the game so waiting for the skin you really really wanted was uh really cumbersome so we resolved it we removed that barrier

people could just go and buy skins quick show of hands these two updates how many of you think those changes were positive for our community oh wow wow okay all some hands all right good stuff good stuff yes both guess everybody is negative then okay both features were in fact embraced by our core community so if you looked on reddits or Twitter people were raving about it they were like a the developers they finally get their together and it's like sorry P um we didn't rehearse this part no we did not uh I'm getting the evil eye from our Pia um but but yeah the the reality was unfortunately not that Rosy our theu

kept declining so the Star Road hit brought a ton of people back but as you can see even by the time the Mastery update arrived we're already below the November numbers in in diu uh and masteries and bling did not really change that trajectory at least not immediately um these are the D30 retention cohorts both for bling and masteries the important learning here for us um and that's maybe also a pretty obvious thing to say but big changes just because you feel like oh we need to make a big change we need to shake up things those big changes are rarely a silver bullet it's not like we make this big change everything will get good uh so starro did not fix our problems at all and yes big changes can be part of the solution but but by themselves it's more like rolling the dice and see if it will

work or not um so for us we transition actually about mid last year to a place where we take selected big bets and go much deeper with making incremental less uh controversial improvements to the game both for us and our players yeah basically incremental improvements um and this learning of like that you know you don't need to change a lot to change everything I think that was also a key learning coming from this and for us it was also another secondary learning maybe if you will but people really don't play at least rers for the systems at all they play it because of the gaml because the gameplay is engaging and we spent possibly 80% of our de resources on systems at that point uh now we don't anymore and it shows awesome all right so as Frank mentioned

the Star Road didn't succeed in recovering the game which is a huge gut punch because that's what I was brought on to do uh at this point I was like maybe maybe I won't be here much longer uh but I think the key here was that we didn't understand what our players wanted in the sense that we were kind of in this Echo chamber because even when we were showing and we were showing our design for the Star Road and how we were going to remove Loop uh loop boxes from the game to to a bunch of other teams and some folks were really critical but because our pretense was we've already made this decision to remove loot boxes people didn't actually question that and so this Echo chamber effect was real very real and that gets to kind of the fourth learning is you you really have to find folks to challenge you particularly on big changes and particularly on what Amazon calls like one oneway door decisions because you can't really go back from this like once Star Road is in it it it's it's there it's there forever at least for now um and so uh I mean we did

get like critical feedback but it was more on the design of the Star Road and not so much I actually it was like second month in the job uh I went to show one of our more vocal teams that were like the you know original brawl Stars creators that moved on to a new project and I got grilled for half an hour on like the Des the first version which you don't see here on the first version of the Star Road so so they do give us very impromptu feedback and it sometimes comes really harsh um and now on the team culturally we're very happy to call each other out and talk about you know hey I don't think this is the right design and we have very constructive uh and sometimes very colorful uh discussions and that we don't hold it sacred like uh maybe this is part of the Finnish culture but like people are very brutally honest and and happy to voice their complaints and then that actually comes back to to really help us so all right uh we hit our lowest monthly active users ever

around q1 of 2023 and at this point like look confidence my confidence level is shattered but you know despite this team morale was actually really strong and as a team we felt quite empowered to turn braw around there wasn't a lot of this self victimizing there was just a strong focus on well look we're in this position we're now the underdog what can we do to really save the game um I mean certainly if it did come through it didn't come through the team and maybe Frank took uh took the brunt of it from other folks but not I had full hair before that um and as a team we just started to really reflect like why didn't we hit the mark uh we looked up certainly a bunch of internal data I mean internal data is really good at predicting historical trends not really good at predicting the future and then we started looking more

closely at our player sentiment our community team is very in touch with the community and uh there's a lot of very uh good dialogue and uh I want to highlight one particular from traces 777 who writes brawl stars without chests like he's referring to boxes is not fun please return them and I guess he had like another thought there after that cuz CU he kind of trails off but we got the picture we got a lot of onear Google Play reviews in that period like a lot yeah so uh whatever plans we had for the rest of the year were actually trashed this is Ash he's a trash bin brawler in our game so we thought that'd be a good reference point um are you struck by something because I was this was about March April time frame and the fact that we were willing to to just scrap our plans for what the June update was and

we had planned out the next couple of updates we ship every two months I was really uh inspired by how quickly the team uh kind of can make these decisions because in June we made the decision uh we shipped this decision which was we had to infuse rewards once again with excitement that the deterministic reward structure that we had with Star Road and uh vertical progression really was not working for us players were not entertained when they came and played the game anymore and it showed through in our numbers and um yeah so what did we do in June well we added random engagement rewards so you earn these by playing the game back into the game and this is what they look like uh instead of a box they're now a star totally different and what happened is you click and there's a chance that the reward can upgrade uh from our lowest Rarity which is rare to our highest which is a

legendary that was a Mythic one that you saw there which is our second highest Rarity and every time you tap there's a chanted upgrades and uh we do think the design of this definitely helped us and I think there was like two key things the first is if you are considering designing random rewards uh this anticipation is really important that the fact that the player uh has this chance that the reward can upgrade is actually a very very important piece and that's why we have the tapping for upgrading built in because and ours is a two-step because the first step is what will it upgrade to in terms of Rarity and the second step is what is the actual reward the other really important thing is that it's really important to have a high highs in the sense that your jackpot rewards need to feel like jackpot rewards so the so the legendary which is our highest Rarity that needs to have some really juicy rewards in there and actually when we launched it we did not have a really good legendary reward this was from the from the team

we actually had to improve upon this over a few months to kind of get it to where it is today that you uh can enjoy um just to be really clear uh we also added cosmetic items cosmetic skins into in the old box model we did not have skins Skins are actually a huge exciting point to unlock and so we added those also in and this system complements the Star Road in the sense that we did not pull the Star Road out this was just something that was added on top instead of deterministic rewards in a lot of places and uh we do think that a key aspect to design that worked was we made it really obvious it was on the main Hub it's really easy to understand uh you get three of these a day and you just have to win so there's no elaborate quests if you're playing bra hopefully you're you're winning at least above 50% and you'll get these um it's you can't miss them so what were the initial results

let's take a look at retention first again D30 D30 is that we're using all the time uh we almost don't care about D1 D7 Etc um so we're very focused on the the longer taale and that starts in our opinion with D30 so blink catalog update and the startups update which also included another piece which is an re-engagement program before that you would come back to brawl star and there was nothing so if you were gone for two years you came back there's probably 20 new Brawlers uh five new game modes a lot of new systems and we didn't reintroduce people at all and now we did that we learned from Clash of Clans actually we had a similar program and ours worked really really well so the retention worked out and as a result of that ouru started to recover uh or the decline started slowing down at least now the in

interesting piece about this about this graph specifically is that um again like it's summer period uh the lion chair of our player base is in Europe a lot of people go on vacation as I mentioned earlier not we always see a drop we always see a drop into the summer period um but the drop was curiously flatter now we didn't release much new content in that period so it's really people get just more sticky to the game and the learning here is uh probably the most obvious from our perspective but random rewards are indeed exciting for our players and and maybe that's so because our players grew up with a gcha in the game but uh the the amount of people buying into the completely on raes experience they were just much smaller group than the people who wanted an exciting gcha

experience so were we able to recover a little bit more in the future now so this is our D30 um until February and you see the stardrops update there on the left um and that was just the beginning of the recovery the first update we shipped after the stardrops was hyper charges and that update was in September um it improved the value proposition for power level 11 in the game so we replaced gears from that slot so we didn't want to introduce a new power level we moved that a little bit lower and now you had on power 11 the possibility to unlock a hyper charge and we only introduced six at the beginning and we were highly inspired by Clash Royale at that point who just had Shi card Evolutions so there's a lot of cross team learnings if something works it works and we all want to use it uh

Clash Royale might just have introduced stard drops they're not called like that they're boxes so they're completely different so what what hyperchargers do that as I said they unlock at Power 11 but they really give you hero moment every game so if you're a good player you get probably two three hero moments and if you're an average player you get probably one hero moment and in that moment so you charge it along your super and in that moment you just get faster stronger and tougher and the supers change this is an example of how that looks with Shelly so she's just on speed runs really really fast her super is much stronger than before and it has a wider range so it modifies the gameplay of um that brawler we lean into kind of their strengths the first batch of six hyperchargers was received okay uh I think we we managed to to reach

about 25% of the original Target for Revenue um so that was not great um but we quickly took the learnings from that and iterated for the next releases so hyper judges arrived here again our retention pretty much flat at this point shortly after we had a community event uh the First Community event after the Omega box event so we didn't do one for a long time yeah um shooting star drops it was an event that was triggered by the community again the community could play takedowns were counted and depending on where the community would end up they would get bonuses and we also added um hyper charges to startups at that point mhm and last but not least we also turned on a life ofs modifier to get two star drops every time you would get one before and again Community event was Super

Hyper it was pretty clear that people liked stardrops and wanted more and what more and this was like a very slight change for us like you just get double that's the event and then you we have this progress bar that we track as to like how many takedowns the community has received and you just it's just a bar that moves up and that's it that's the design then we also introduced the mega pic the mega pic uh I'm not actually show how we ended up with a pck I wanted a pinata but but anyhow so we had the mega pick um that replaced the previous social features which were Club league and quests Club League was an event going on every other week every other week you got tickets and had to play on three different days to get rewards um now the mega pick is once a month for 3 Days very clear on a weekend in the old world you had to play 24 to 36 matches per month depending on the length of the month now you play 15 a month um

previously you had a choice depending on where you were in the game regular matches or competitive matches which are rather long there so three pcks B now we have just a simplified format it's a the competitive format without bands it's just best of one and we also introduced modifiers to the game to make it a bit more light-hearted and fun um before that you got Club coins from from your club rewards and then there was a store from that store you could buy specific items uh only for Club coins we removed all that complexity now in the MEAP pic as your Club is filling the mega pic you just get more and more star drops as a reward at the end and yeah we had League ranks before and no more ranks no more hardners our goal was to get the old system engaged less than 10% of our diu and AO was 20 for the Mega p and currently we're sitting about 40% participation and this

cut we cut a ton of complexity as Frank mentioned it took them that long to explain the design this new format is literally for three days a week you come in there's a mega Pig game mode you click it you play and then you earn for the for the for the club and that's it that's the basic design the stard drup effect the community event shooting start drup effect and then the mega pick release here and once again our retention improved a little the next update we shied took our old braw pass and transition it to a new pass and the easiest to compare those two is this Slide the old pass was a bimonthly event 7 to n weeks the new one is four to five weeks every month always starts on the first Thursday of a month uh the old one was available for gems which is our premium currency 169 gems 4 IP usd99 um now the interesting piece here

is that on the free track of the pass you could earn 90 gems for free which means that freet to-play players could buy the pass every other season for free and so and yeah and the value proposition was 13x and it included an exclusive brawler that was part of that season now we changed it quite drastically it's now I only the regular pass cost $66.99 and the big pass cost $9.99 it's called Plus there's no more exclusive brawler in there which uh I had some sleep as nights over um we can give out actually more gems for people because they cannot spend them on getting the raw pass anymore and we were able to increase the value proposition for paying players because we wouldn't give that to all the freet to-play players anymore at the same time we buffed the free progression so we took what they earned before in the premium pass every other season and the regular stuff and all merged that into the new monthly season so the freet topl players

actually got a buff out of all of this so they felt pretty good with it and um and of course pays also feel pretty good with it because all of a sudden we give a lot more value so how would the community respond if the BR pass was only available as an IP well again that could have gone really really bad for us and we counted that in three ways we communicated really early that we would make a change early November the first new season would start in January to give a frame of reference we also as I already mentioned we buff the freeto play progression and we were in general really really transparent with the community who would get what in the new world so we showed a table on socials even like two months before and so people could compare content creators could create content around it and actually like you know oh yeah they're not lying to us this is actually more and Community sentiment went to a place where people all of a sudden were rather

positive with all those changes uh which was absolutely not expected the backstory here that is kind of important is our community manager uh Danny um shout out to him he came to us quite early on and he saw that we were moving to a IAP brw pass and he's very in tune with the community and he made a very strong stance almost like a die on the hill kind of stance that if we didn't execute properly and buff freedom play with this like it was going to be a non-starter and we were going to flail really really hard and we've seen other games like out cuz we do our research and like it's really hard to do this transition if you're free to play looking at this and saying now I have to pay and not every two months every month um so we just tried really hard to get in front of it and be transparent and in plain English tell you hey this is what you're getting in the new world the exact almost like the exact numbers like Jeff who's one of the other design really talented designer on the team like crunched a numbers and work with Danny on this and the community folks to

to really get this and I do think this is it ended up being uh really well received as a result of all that hard work so so shout out to to all those folks we also added 5v5 game modes that comes to speak about the gameplay side we finally decided to actually increase the team size a little so we would have more people working on gameplay that happened about mid last year and so the first results showed in December when we were able to ship some new game modes the community had to ask us to have 5v5 for a long long time and we always oh there's so many problems with this and then we just decided to do it for the community and it was really positive and so we had some more community events to send off the another currency then the raw pass update arrived then we had another event where you could play together um to earn more rewards and then maybe about star jobs yeah and uh after introducing them in

June we continued to kind of add them to other key areas of the game so we have a couple key areas we consider pivotal to the game one is trophy Road this is your main progression everybody talks about the game in terms of how many trophies they own you earn trophies for playing matches we added them there uh we added them to MEAP pig as Frank mentioned earlier we added them as live Ops and and offers that you saw the community events all of them largely center around this new star drop and just getting more star drops and we add them to the pass on the free track and lastly more recently in February update we added them as we revamped our rank mode which also had very low participation rate so we we felt we had to kind of go back and rework a lot of the Key Systems and over the last eight months the trend has been to revamp old systems that a very low participation rate make them easier to understand and add you know more random rewards in uh in the process so zoomed out again a little bit and

looking at Mau this is how this looked so you can see things consistently becoming more flat at summer even going up a little and then from there just kept creeping up um and so the increase between our low point over there which was June 2023 and February we increased our muu by 2.4x zoomed in a little looking at the average diu that actually improved by 3.9x um so it's not only that more people are playing the game it's also they're playing it more intensively and some of these changes like the braw pass being shorter always a brw pass waiting for you Etc surely have to do with that and the last but not least the revenue um it by 8. 8X from June last year to

February and March is on track of being a growth month as well so that feels pretty good for the team so yes call that we call that doing okay in Finland the fans in the room appreciate it when you see the learning here is that when you see retention based momentum really double down with everything you have you know these hype moments and driving installs that's easy there's always a tool there's UA you can go massive there you can invest a lot of money or you find other ways like content creators to work with creating temporary momentum for a game is not the difficult exercise the difficult exercise is to improve retention and when retention is there everything do you do every hype moment every U USD you spend on UA all of that you know becomes much more powerful so we were able once the stard drops and the re-engagement

calendar changed our retention to something much more meaningful that allowed us then to invest heavier in UA it allowed us to push more with the community side because all of a sudden we're driving people into the game and they're actually sticking around so at this point you might wonder okay we had boxes there's obvious nervous when you remove loot boxes as your primary monetization how does that change in terms of paying behavior and what does it look like in a world where rewards are largely deterministic because you can't monetize on stard drops uh largely speaking well this graph uh can show you that story so from January to December 2022 is the l h side that was when boxes were in this is just looking at uh the equivalent of boxes in the old world versus the New World so we don't have braw pass on here for for because it would mess with the

comparisons and in the new world starting December um you know you could basically like buy coins you can unlock a brawler uh by just spending paid paid gems You Can level them up spending paid gems and or or gems in general and items we call like the gameplay related stuff and you can see actually uh it's actually we were shocked by this that it's been relatively consistent and we did not expect to be this stable and now we're actually trending up as you can see towards the end of that so this was like definitely surprising to us that we were able to transition off of you know Loop box based monetization largely speaking so that gets to kind of the recap which is I'll just go over the key learnings and then I'll kind of summarize with a kind of a final a lesson that's on here for folks to take a easy screenshot of the first is when wrestling with a design simplify and I can't stress this point enough like a lot of the designs and features that we've done since have been a lot of

simplification while trying to maintain depth that's ultimately where we think the best design lies in terms of Elegance the second is to reduce noise particularly around big changes you saw us introduce Omega box a bunch of other December stuff probably should have moved star Star Road to February for instance um the third is that big changes are in our case not a silver bullet that actually if you look at all the incremental Improv improvements we made since June uh those were all like relatively small in terms of versus a Star Road took like six months of development each of those updates uh for better for worse we kind of develop in two months and we ship every two months so we double down on one major feature every two months and um we kind of uh we do that so it's a lot of this is like incremental improvements the fourth is to really find people who who can challenge you and on the team some of the folks are here today they challenge

us every day uh especially on design related stuff so so hats off to everybody that's here um and and everybody who's watching remote and on the team that are working on the maintenance right now that we have going on um that's really really important culturally for us the fifth is random rewards are really exciting especially for engagement purposes and uh the last is if you see retention based momentum double down and maybe like the thing to kind of wrap this up for us is ultimately the player doesn't care about any specific thing you do uh when they come and play the game they're subconsciously asking themselves what are they entertained and if the answer is no for too many days in a row you get what we had in June and I think what we've done and a lot of this is we don't know how much of which contributed to this kind of historical highs that we've hit five years into the game but certainly players come back and they're entertained and they're telling their friends uh to come back and uh we think a lot of

it is maybe viral spread through our the school networks uh so thank you for for having schools in in this world to help spread brawl stars and that concludes our talk um so there's a lot of stuff that we didn't get a chance to cover but um you know certainly there's a lot of details here and a lot of work done by a lot of the team that is not captured in this presentation but um certainly you come and ask us about any of the specifics there and certainly we're hiring hiring a lot of roles and so um if you are interested in joining our grant this adventure please uh consider that too that's it thank you for your [Applause] time I guess maybe do we have time for a few questions not sure we have 10 minutes for some

questions hello thank you so much for the talk um do you have a Target retention rate that you kind of know is going to lead to the growth that you're looking for um and just like like what's the kpis that are indicating that you're heading in the right direction versus you know um maybe not what you're looking for is it D30 pretty much it I don't think we have a Target retention rate per se it's like we we're constantly looking at how the cords do uh we we talk now a lot about new player retention as well uh depending on when your game is um returning player retention is probably more important in brers last early last year the amount of returning users coming into the game every day started outpacing ing the amount of new users for example so you know we had to really take Serious that

we had to measure returning player retention Etc there's no no specific Target number in mind um other indicators really like as long as the graphs go up that's it um but but there's a gazillion of different kpis we're looking at every day really and just turn rate's also top of mind for us and that was that's been steadily decreasing and just to follow up really quick it game changes does that uh like how do you separate those from things like system changes and mization the the way the team is structured now is that within supercell we we transition to world where each of the live game teams which you call sells they essentially Independent Business unit and within those we have subss uh run usually by a lead or a captain and um in in broaders we made the separation that we have a feature development team and we have a gameplay team uh and that's how we split today is still like the I would say the

the feeder development team is still much bigger in terms of headcount than the gameplay site but this is mostly because a lot of people are flexing between uh between rols you know we have a few people actually in the room who are working like as a for example ux UI artist who will also work on gameplay stuff when necessary or we have client programmers and server Engineers on the team who don't mind flexing in One Direction or the other so we're trying to stay as flexible as possible on that side other game teams are structured in different ways so at Super we made the decision that each of the life games can ultimately decide how they want to work and do you stagger releases between the both of them no we don't we are having goals for both things every update cycle gotta cool thank you so much thank you and there's a lot more focus on gameplay iteration now than ever before because ultimately that is really really important for us rather than just purely a systems change hey thanks a lot for the talk um I'm wondering just generally in Supercell like in current teams and also going forward what the feeling on loot boxes is is there like a like a

systemic movement away from them or do is it sort of each team is kind of making its own decision as they go there's no mandate from the company so it's completely down to the game teams but I mean in terms of how the game teams are feeling about it generally do you see kind of a movement away from it or is it kind of all over I would say it's pretty much 50/50 still so some some game teams still believe in it and some other game teams want to try something else but there's no there's again Independent Business units each game team can make their own decisions and there's no pressure from the outside to go one way or another all right cool and actually just a followup the I mean so the difference between loot box and like uh random rewards is basically one of them you can pay for and the other one you don't right well it's really like I'm I'm not sure how hard that definition is yeah um the the way we're using them right now is mostly engagement based um we have introduced them as part of chain offers as freebies um we might in the future offer

them for gems again it's like you know we we're always trying to do the best for all players and we ultimately will experiment with things to see how all players accept them um yeah I would I would expect more experiments from our side with with random rewards thanks cool Hello um so one of your bullet points was uh to try to reduce noise around big changes so you can measure the success easier um how do you do that because I know that you mentioned that summer was like kind of your slow season and if you know that Christmas time you know everyone has Christmas money and there's more spend and I'm sure there's some changes that you'd want to piggy back on Christmas um how do you like balance that tension between like oh this we want to make this easier to measure versus we want this to piggy back on Christmas I mean depends if you know you could ship the thing a little bit earlier to have still some clean data from October or November before going into the Christmas period you can of

course ship it a little bit later like I guess it depends a lot on the game right in our case January is usually on the similar level Revenue wise for example like uh December still so yeah I think I think you you could work around it but I also don't think it would be a problem if you have just this one big change because if you have historical data from previous years you know oh my game is usually going this direction so then you can compare a little bit easier maybe but for us it was just too much we were in this mindset that we always wanted to do really as much as possible and squeeze it into every update um and I I do from today's perspective I think that was a mistake and we would like to move away from that and we've started very conscious Journey now to move away from that and be very deliberate about that like oh no we are not shipping these two big things at the same time awesome thank you so much welome hello Franks um your journey has

been incredible since beta and you've had to Pivot many times um and thank you for bringing those insights to GDC uh supercell is also known for dropping projects that don't work out so what metrics kept brawl Stars alive what metric yeah Frank was there for some critical critical periods of time when the game was still being killed so you probably best answer this of of I think it's really like it sounds super fluffy okay but besides metrics of course if the metrics say this is an absolute no go from our experience then the the team will probably kill their Darlings but if things are a bit more on the edge then then there's a second component and that is the team how does a team feel with their product we're not making top down decision so it's not our leadership team saying you got to kill this game they will challenge you probably if it doesn't look great and if you don't have answers you will probably come to the conclusion yourself that you got to kill it but um but in br's case

for example it was very much on the edge you know we before we made the decision decision to go Global between May and June 2018 we were in the situation where we rolled it also out to the soft launch to Android which we hadn't done before and that changed the numbers already a lot but it was still like H you know it's like could go this way could go that way then we compared against clash and Royale data and we're like oh look pretty bad actually but there were a lot of uh people in the team back then the the data science in the team he was really conservative so he always you know assumed the worst he was like Frank we're going to hit March and to go to 13 million duu and then we're going to die and that was not super inspiring um as you can imagine maybe so it it was really about like trying to give other perspectives other angles and like hey there's possibly was a best case and you know Clash of Clans was released in

2022 does it still make sense to compare a game in 2018 which is an action game with a very different audience so it's like sorry there maybe a convoluted answer but there is a mix between the kpis need to look good we have a lot of Benchmark values there but there's also the team component if a team believes you know then you can mitigate Risk by having a smaller budget for GR Global for example marketing wise thank you for believing you I think we're out of time yeah I think we're out of time sorry we're out of time but Frank and I will be outside of the room if you have some other questions definitely thank you thanks again [Music]