Balancing League of Legends for Every Player, from Bronze to Bengi

[Music] so my name is Greg Street I head design on League of Legends players call me ghost crawler they call me worse things as well but we'll get to that I've been asked to remind you about keeping an eye on your phones and filling out the review forms at the end of the session and here we go so I want to start by explaining the title balancing league for every player from bronze to bangi this is bronze within ranked League of leg Legend there are seven tiers bronze is the first one and at the opposite end of the spectrum this is Beni his name is BAS songong he's currently the jungler for a Chinese Esports team named vichi but he's better known as a three-time world champion with Korean t uh Team SKT he's very good at League of Legends so this is the challenge how do you balance a game where the skill differential between players isn't like 10% or 20% it's 1,000% 10 th% the pros of League

are exponentially better than people just starting out and this is me I uh started life as an oceanographer that actually isn't me but that's the kind of stuff I did dropping muddy things off the back of boats it was before digital cameras I got my start in game design working on the Age of Empires franchise where I was at ubble Studios for 10 years from there I went on to World of Warcraft where I led systems design for for about 6 years and I've been at riot games working on League of Legends for the past 3 years on all three of these franchises I either oversaw balance directly or was accountable for it I've been doing this a long time I like to Nerf things I use a crab as my avatar getting back to the Marine theme this is Ghost crawler this is Ghost crawler and more recently I've switched to the rift scuttler and this is League of Legends um hopefully most of you are familiar

with this and that's why you came but just in case it is a 5v5 MOBA uh free to play with cosmetic content we like to think of it looking like the image on the left but of course it actually looks like the image on the right I'm going to use a lot of League examples in the talk today but not everything and I'll mention um some other games as well my hope is that this topic is broadly applicable to the kind of games you guys work on and not just a story of League of Legends and this is a champion this is Leona the champion uh league has 134 Champions we had new ones every year uh Champion is a unique character with a passive and four different abilities we often abbreviate these abilities qw which I I say because I'm going to end up calling them qw that's just our short hand I'm going to talk a lot about Champion balance because Champions are kind of the basic molecule by which players interact with League of Legends so a lot of the balance considerations that we deal with are about Champions so my topic outline uh why is

balance important I'm going to cover this super quickly because I'm going to make the Assumption you guys are kind of on board with this concept already how we balance for different skill levels balance for a massive audience and then finally I'm going to spend the majority of the talk talking about how you know tactically how does Riot balance League dayto day and I'll try to end with plenty of time for questions so why is balance important I like the definition that game balance is a contract with the player because contract implies a business transaction where players are getting goods and services for their payment and it could be literal payment if they're you know if they're purchasing a character but it doesn't have to be you know real money if you play a warlock in World of Warcraft you may have 10 years invested in that character and you want some guarantee that game balance isn't going to stop you from playing her now league is a competitive PVP game we have a very very small PVE component and skill in league is supposed to matter this is what this is why players play League of Legends to demonstrate

their skill and we don't ever want the state of balance to Trump skill when you have a game that's imbalanced um it makes it too easy to solve the game and if you have a game that is very solvable players tend to get bored and when players tend to get bored they stop playing your game options are also really good for Longevity if you have a game like League that we hope that players interact with for for a long time once they get tired of playing one Champion there's a lot of other ones out there that they can try out now Riot as a company cares a lot about player trust this is something we we focus on a lot we try to be very player focused in every decision we make and if the game is in an imbalanced state for too long that kind of suggests to the players that we don't care and so it's kind of violating you know trying to hold up player trust as being so important now tragically balance is really hard and as important as it is for your game and for players the

development cost is super high and I'm going to talk throughout the rest of the talk about why this is and some strategies and tactics that may help you mitigate it so balance for different skill levels part two this is what a pro- championship of League of Legends looks like pros can do amazing things that we mere mortals cannot possibly conceive a doing and balance at high skill level and especially at Pro skill level is really different from balance at for the rest of the game and certainly for newer players and I'm going to talk about uh some of these differences that manifest in the game in any game that make it difficult to balance for extreme skill differentiation so next few slides I'm just coming at you with a list be for warn um some of these are differences you don't want to avoid necessarily but you do want to be aware of them so to start off you can break down player skills being tested in a game into a lot of different buckets things like good decision making good team communication fast reflexes and so forth

now one of the biggest differences between highly skilled and normal skilled players comes an execution difficulty as developers we sometimes balance often balance the power of an ability with it being difficult for a player to pull off so this is a a basic skill shot League of Legends is full of them rather than clicking on the enemy Target with your mouse and then just hitting a button that will always hit them you have to line up the attack and take into consideration things like you know leading them pred where the enem is going to be and so forth but it's not just skill shots anything involving timing or multitasking or often just the speed of clicking you know APM is much easier for highly skilled players some of the ultimates in League of Legends the r ability are channeled and a channeled ability you could think of the champion is like winding up casting a spell and during that time if they get interrupted it either cancels the spell outright or it might have a diminished effect this is fiddle stick our evil magical scarecrow he has a alt

called Crow storm where he summons a lot of crows but he does have to channel it and is the kind of thing that makes it a little harder to use at the pro level because or even highly skilled players because it will get interrupted you have to keep in mind when you're talking about these you know 1% of uh skilled players anything whose cost is easy to interrupt is going to be interrupted and anything whose cost is hard to pull off they will pull off it's not just fats of dexterity though skilled players understand the higher level strategies of the game as well they can pull from a lot of knowledge including knowing strengths and weaknesses of Champions almost reflexively this is Garen he is uh very powerful at the lower tiers of League of Legends and not so successful at the higher tiers he likes to hide in bushes and spin around in circles which players call Spin to Win It's very effective if you don't know how to get out of it at higher level you can crowd control them

and it kind of shuts him down I'll talk a little more about gar in a minute so going way back to my roots the Age of Empires this is one of the first balanced nemesises I ever faced two of the most powerful traits you can give a unit character Champion are long range and fast movement and the two together is amazing Cavalry archers with a ban of my existence on Age of Empires there's a reason the Mongols were so effective with them you run up you shoot you run away they're experts at kiing it makes it very difficult for other units or other players to be able to catch up with them and this extends to League of Legends we have Champions like Vay and Lucian who are highly mobile yet ranged and skilled players who are great at micromanagement are going to be able to like do a lot of damage with never having to take damage themselves now this is an interesting one most league players experience the game solo we have matchmaking we'll find you a team and we'll find you an enemy team but organized play is a very

different experience and of course anything professional is organized play these guys play together all the time they have plays they they can almost read each other's minds um you think of like a buddy cop movie where the two cops enter a building and they immediately know to like cover each other's blind spots loo on the other hand um because they don't have the guarantee that someone's going to help defend them tend to gravitate more towards generalists this cute critter here is kgma He's A Classic Glass Cannon he can do a ton of damage at range but if you get up in his face he he crumbles very quickly and as you might imagine it's difficult for players in Solo que to be able to effectively utilize them because they just don't know that their team's going to be there backing them up now highly skilled players are really good at manipulating your economy and getting an economic advantage in in any game um Can often be really powerful in League of Legends we see this as gold funneling so basically in League

you kill things you get gold use the gold to buy more items to get powerful and the idea is that everyone kind of rises in po as the game goes on and more skilled players rise a little faster but skilled players we see this definitely with Pros can kind of have a lot of the team avoid getting gold to make sure that one player gets a lot of gold early so they have this really powerful Spike and even though it's only one player on the team they can leverage that to to get an advantage in World of Warcraft we would see this with the high-end guilds when they would raid would do things like we're going to pass all the gear to the main tank or to you know a really important DPS character and this got to the point of um running alt raids where a group would run a raid all the gear would go to one player while everyone else was on their alt and they'd switch to the next player and so on and so on using different raid instances it's hard to stop it's hard to design around that but it makes balance very difficult for these high-end rating guilds AI power um we love to put

turrets in our games so here is Heimerdinger from League of Legends torbjorn from OverWatch and the engineer from Team Fortress the problem with AI power in this case the turrets but it also could be like your character has a pet um AI power just doesn't scale really well with skill level if you think of the power as a as a pie chart the AI part of that power takes up a big part of that pie chart it doesn't really change depending on whether you're talking at at low or high skill level so developers we have to kind of choose where we going to tune those turrets and if we tune them to be very very powerful at highend they're just going to crush lower end players on the other hand if you do what most of us do and make them relatively reasonably fair at lower levels at high levels they're just not going to see them get much Play Strong windowed power this is one of those cases where as developers we often make the uh the cost of like yeah we're going to give you a lot of power but it's going to last for a really really short period of time

um the more skilled players won't waste that time even if it's a really small window they can get a lot out of that this uh wormy looking monster here is Baron nashor from League of Legends when your team kills him you get a buff to sieging the enemy base which is a great way to shut down a game and at high end Baron absolutely decides you know whe whether a game is won or lost but lower skilled players can easily waste this buff they die they don't get their group coordinated the buff eventually ticks away before they're able to leverage it in World of Warcraft uh shamans have an ability called blood lust which increases group haste for the entire group and skilled teams are really good at uniting like all of their cooldowns to fire off at once your trinkets whatever your personal DPS or healing cooldowns are and I you know I'm sure this still happens the groups will all line up and pop their cool Downs at once for for a gigantic power Spike particularly at the start of the the fight when everyone's you know still at 100%

be really really careful of your early game this is a great opportunity for skilled players to get uh you know an advantage that later snowballs and makes them impossible to stop later on this goes along with the cost of but you might make a mistake because the more skilled the players are the less likely they are to make a mistake their gameplay really can kind of approach perfection in regular play someone has a chance to get back in but at high- skilled play the game may be over as soon as someone gets a slight Advantage early on you see this in real-time strategy games too like Age of Empire cares or Starcraft where a small economic Advantage earlier just pays lots and lots of dividends by the late game so I've talked mostly about game design but let's not forget we're mostly talking about software running on hardware and there are limitations we recently did a a study to look at play rates and win rates of certain League champions according to different um ping rates and we found that we found that the uh it's back um

place like Korea which has a really good internet actually allowed players to use different Champions and have different win rates than maybe somewhere like the US where um everyone's bouncing off the same servers in Chicago you can mitigate this a little bit by doing things like making sure there's a little bit of of space like instead of giving players a fraction of a second to execute a command you can extend that to you know half a second or something like that which may help overcome latency problems this is a pretty nuanced issue I'll try to walk through um tactical errors and extreme peaks of skill tend to be really rare which means that who wins can often come down to who makes a mistake first like two teams are playing perfectly somebody misclicks and that is the entire game ending right there it could be really exciting as a viewer but you're kind of left with this hollow feeling that it really wasn't the more skilled team that won it was just the person who you know didn't make a mistake so my lesson here is make sure that you have highle strategies that

highle high skill players can exploit as well as tactical considerations by tactics I'm talking about things like where you're positioning your character when you're you know using your different abilities High LEL strategy is more about do we want to have a you know an early power Advantage do we want to try to sneak in their base are we going to split Lanes things like that if you have higher level strategy the game won't just come down to misclicking so highight some of the differences in trying to balance for both highly skilled and normal skilled players but now we're going to make that even more complicated by layering on the effects of a mass audience any competitive game any PVP game is going to have a skill differential among players if you're a small Indie developer and you have a small player group there's still going to be an enormous disparity in their ability to to play your game but League of Legends has about 100 million players so that skill differential can get really large our bell curve has really long taals it both

ends and on top of that we make our lives extra hard he's saying the guy controlling the other five units almost one because in League everyone plays by the same rules this is different from say uh basketball where the point lines will change depending whether you're at like you know amateur college or Pro play or a game like golf or their different tea boxes that that different um players will use this is important to us because there there's a virtuous cycle in League of people viewing the game and then playing the game and wanting to view the game again and by viewing this could be Esports or streaming or any you know any any games players are watching on the internet if you're an Nascar fan um chances are good that you're not out racing cars on the weekends I don't know maybe some of you are but most people that's kind of a sport they participate in as viewers not necessarily as participants we think this is important because in league players look to the pros to get better if I'm trying to

learn jungling I might look at you know what are the kind of items that Beni picks he certainly knows what he's doing and there's also this aspiration of the path to Pro um doesn't apply to me because I'm old and my reflexes are shot but for younger players they have this promise that if they just get good enough at League of Legends they may make it to the big leagues and someone may pick them up for their Esports team as an aside though I just want to offer a counter example that while messy multiple rule sets do have some benefit we um we get into situations sometimes as developers where the problems are really over constrained like we can't buff this unit for the high end because it's already too dominant but players at the low end can't necessarily use it just want to throw that out there that it's not necessarily cut and dyed that one rule set is the uh the only way to go so I'm going to spend the rest of the talk Talking you know at a very tactical level about how riot balances League of Legends with all of that in mind this is

the the the secret sauce I'm going to try to be as Frank as I can about what we do behind the scenes because you know that's why we all come to GDC right so first we have a philosophy and I really encourage you to get your studio your team your designers aligned on your goals and values not just as a as a development Studio but on individual projects as well if you have a shared philosophy it really helps to arbitrate the tough calls you aren't debating are we okay with this kind of In the Heat of the Moment you already know what good looks like so then when you're missing that Target you can react it also provides uh precedence for the teams of tomorrow um game development tends to be a highly mobile career people will you know leave Studios or leave teams or or move to different teams and I feel like we spent a lot of our time doing archaeology like trying to understand why did the developers who worked on this game two years ago make this particular decision so if you have you know goals and values

that you can particularly publish someplace and make sure the whole company understands it can help to to mitigate some of that so for example these are the league design values I I won't go through them all here but we do make sure that the changes we make are aligned with these values the idea is that hopefully we're making the game over time we're making the game better over time rather than just doing this kind of one step forward two-step back thing where oh we made a change oh that was a bad change let's make another change oh that was another bad change I mean I mean don't get me wrong that still happens for sure but the intent is that we're we're making progress and it's not just design values Riot has values League of Legends as a product has values there are values for particular you know very small project it could be as narrow as hey we really need to L Nerf Leona's late game and then once you have that philosophy make sure you actually do stick with it we really like to work this way because it helps to mitigate against designers tinkering and by tinkering I mean someone's driving into work and they're like I had a crazy idea

let's do this and see what happens um that could be fun as a designer it's really pretty pretty painful for players and it also solves the mythical oh I had a bad game last night Leona beat me so I'm going to go in and make sure the team Nurf Leona um players think we work that way so instead we're trying to solve specific problems like X is too strong in the laning phase rather than hey something popped into my head out of the blue now as an aside Riot as a company really likes to work this way overall because it helps to empower our teams it makes the uh the growth of the company more scalable because instead of the same decision makers kind of having to Swan Dive Into every single discussion that's going on we can spend our time getting aligned on what the values are and then the teams can go off and make the decisions themselves without having to like reopen every discussion because we've already kind of agreed on what the high level goals are but you won't always get it right um um one of my leads the other day said

even the best design that you've really thought about and beat up probably has about a 70% chance of of hitting your Bullseye so having a risk mitigation plan can help to avoid panic when that happens um you need good kpis and you know key performance indicators you need to understand what you're going for here if you're nerfing a champion on League do we want to see her drop rate go down by 2% are we worried players aren't playing this champion because they're you know they're too complicated they don't understand the between their abilities when you have good kpis it helps that you can agree if you've missed your goals rather than debating did we miss our goals or not you immediately evaluate okay we missed our goals and then you can enact the plan for example when we make a new Champion the champion designer might say her weakness is supposed to be um that she's she's not very strong in early Lane so if she if she ends up being a lane bully Nerf for Q and that'll kind of skew her back towards the late game now we do focus on a Target we aspire to

balance for all players but being able to focus helps us avoid insanity and this is particularly relevant because we patch the game very frequently and don't always have a lot of time in between patches we do look at outliers at all levels but I'll kind of walk you through the the The Sweet Spot for game balance so if you look at how many players the changes are impacting imagine the um the bottom scale here is number of players in a particular tier there's a lot of bronze players fewer silver players only a few dozen Pro players in the world and this other axis is sensitivity game changes so if we change a Champions attack by five uh a newer player isn't even going to notice that they're much better off just playing 10 more games that will improve their win rate much more than anything that we're going to do in terms of balance and then you can see The Sweet Spot here where these two curves intersect is right around High Platinum low Diamond that's kind of a an area that we think we're impacting a lot of players still rather than the the super

rarified air of the you know Challenger Master Pro tiers but we still do look for outliers at all level if Garen or Master Y is stomping bronze will will take action there if there's a champion that's 100% pick ban at the pro level we try to take action there as well now players will assume that you should and do balance for High ELO only that's the important thing to a lot of players they don't necessarily understand why you're making changes that will help out um less experienced players but this would remove a lot of the 130 champions from those players we do have to constantly reinforce the goals of balance to to all within the community I'll watch you know prominent streamers discussing the state of balance in league and they're always talking about oh well now the pros are going to pick this or that why you know what is Riot thinking here a lot of this stuff is publishing and by publishing I mean the community you know the messaging you're sending to your communities to make sure they understand what you're goals are and why you're making the you know the wacky change to

the game that you're making now interestingly prob balance itself is largely about viewers um we don't want to make life terrible for our Pros obviously but when we make a change it's often with the idea of our viewers having a fun time watching League of Legends we will make changes if for example the game gets a little stale and viewers are saying yeah the you know it's still fun to see the endgame but I you know I skipped the first 20 minutes because it's really boring that might lead us to make a change in this case balance could be fine but we just think that you know the we're going to hurt our virtuous cycle of viewing and playing and viewing if um people start tuning out of orts I want to stress again this stuff is hard you can't hand this to one guy at the end of the cycle as a part-time job um believe me I was that guy in Age of Empires it doesn't work certainly not with the expectations for players these days it really takes a dedicated team and I

realize that may be challenging if you're a smaller an indie Dev in which case I would say you're just going to have to leverage the feedback from your players a lot more it's going to be hard for you to get tuning right so if you ship your changes listen to what players are saying and then react to that and I'll talk a little wor excuse me about that in a second so the team structure for League of Legends we have a dedicated live team these are the um men and women who are largely responsible for patch-to-patch balance you could think of them like the First Responders or the mash unit you know someone is bleeding on the curve they have to get out there and and fix them they um deal with problems that can't wait for a holistic or systemic solution they got to get in there right now and fix things we patch every two weeks since you all are um game developers you understand that that doesn't mean they actually have two weeks to make changes when you take into consideration things like build stability and QA testing and localization often they have 3 days to

decide something's a Miss make changes test those changes make additional changes and then get it out there to players our live team is backed up by a play test team of very skilled players many of these players are xpros um the play test team is critical for us to get early Impressions and data you could think of it as like being able to deploy your patch early or having a a beta even before your beta I've been able to create a play test team at all three Studios where I've worked at Ensemble uh blizzard and now at Riot it can be a hard sell when you're trying to convince the studio to hire you know in this case we need over 10 players to be able to to play a 10 player game but it's worth it when you consider that the designers in QA who probably in the trenches can't play the game all day long they have to actually like sit at the computer and make changes as well so without someone playing the game constantly you may only get two or three games in during the day again I understand for a small Studio Indie Dev this isn't going to be

feasible and as a a warning High ELO players are going to suggest High ELO changes you just have to keep that in mind when they're talking about hey I thought of a way to fix this that's probably going to fix it for them and not necessarily the you know all the rest of the skill tiers now I talked about how we have a focused team and we focus on a Target ELO but we also have a focused Target time just as with our professional game ranked play in League follows a season the the ranked season turns on players progress and then at the end of the year it turns off and we know that systemic changes or even things like a new Champion launch we recently launched Camille now her wind rate her wind rate did not hit our targets we know these changes are going to are going to compromise balance and disrupt the game for players we um this past preseason we made a massive change to the Assassin characters in League of Legends this not only involved individual Champions but their items their masteries we even changed the way stealth Works since stealth is really tied into um assassin

characters and again we do this deliberately make these kind of disruptive changes knowing that it's worth it but also knowing that balance will take a hit and during these times players are going to say WTF Riot why is your balance team asleep at the wheel when really we've made life hard on the balance team by making all these other changes to the game so we try to constrain the most disruptive changes to preseason and midseason messaging here is super important so the players understand what's going on and understand that hey guys things will be a little crazy for a while we're putting plants in the game we're trying a new pick band system whatever we know that's going to come at a balanced cost and because we patch quickly we're going to try to get things back to a stable excuse me stable place now I have a few slides here on using data We Gather a lot of data RI has a a really powerful insights team and they help us to collect a lot of analytics about the game and then help us to interpret that as

well I know I talk about win rate a lot that is an important stat for us we target a 50% win rate for Champions at all skill tiers but it's not the only stat and it doesn't tell the entire story the story here is that most players do not think that yaso is scary funny or cute but they do think he is a badass powerful unique and maybe beautiful in North America so here's an example of win rate but we're looking at win rate by skill level so on the left of each of these curves is the you know bronze players and at the right is kind of Challenger Master tier you can see a character a champion like vagar hovers at around a 50% win rate it does go down a little bit at High ELO but overall that's we're pretty happy with that that's not too bad whereas Twisted Fate here if you are a bronze player that's not a great Champion choice for you you're probably going to lose a lot while as you climb in Elo his win rate's going to get you know pretty respectable borderline overpowered there at %. next to him is Garen who I talked

about before and you can see uh just like I promised his win rate goes down a lot as you get to the tire tiers of difficulty and I also have atrox here and the asterisk is because he appears to fall off a cliff and go down to a 40% win rate which just wouldn't be acceptable but what's really going on here is the sample size is very very low high tier players just don't even play him a lot so the data we're getting from that isn't awesome this is just an example of you have to really look at the data not just make assumptions oh my God his wind rate is low we have to buff similarly here are um the number of games on a champion irrespective of Elo you could see a character here like a reli and soul your first game of the REI and soul you may lose in fact you may lose a lot of games of the REI and soul and before you really understand what's going on at this Champion the uh far right on this slide could be 40 or 50 games which is that's a pretty hefty commitment for a player on the other of the spectrum is a champion named volar after you've played

volar two or three times you've probably seen about all that volar has to offer this is not the sign of a you know a really well-designed champion and he's the he's the kind that is up for an update at some point now aelian soul is not awesome either that's a really steep skill curve we would rather them be um a little rather the win rate not be so low at the low end this is why it's so important to interpret your data if you're playing against a Le s a popular jungle Champion the player you're playing against may very well have three times the number of games that you do on that champion because it takes that long to get good at leas in so if he's showing up at all you may be playing someone who really knows what they're doing in this case if not that Leon is overpowered it's just you're playing as a very experienced player on that champion this is why you have to look at things like win rate based on games played and win rate based on on Elo to really understand what's going on here's another example Pro Champion

Choice has an immediate effect on the rest of the game once we see a pro player pick a champion we know that spike is immediately going to show up around the world it's interesting um part of it I'm sure is just oh the pros are really really skilled players if they know that this Champion's good they're probably good I also think it's players kind of hate to be accused of not being up to dat um sometimes I think players would prefer to be known for poor skill than to be known that you know they were uninformed everyone hates to be wrong about things so when you see um you know particularly silver bronze silver gold players pick up these Pro choices it's it's going to be an ineffective choice for them because they're not going to be able to pull off the things that the pros can pull off this is a froggin is known for picking anivia and speaking of anivia um she's a great case study for why you need to use your judgment to make calls we don't use machine learning to balance League of Legends we hire people that can use

their own experience as as game designers and QA to help make these decisions anivia often turns up red flags where her win rate can be as high as 55% yet we think if we nerfed her players would be like why did you Nerf indivia she's not a problem and when we reflect on that we agree with them like she has reasonable strategic counterplay meaning that you can counterpick her in Champion Select and she has reasonable tactical counterplay meaning that when you're actually out on the map playing there are things you can do to avoid her spells so is she imbalanced um the data says yes but our gut says no and this leads to another topic which is that we don't share our data publicly um we didn't do this on World of Warcraft or Age of Empires either the reason is we we're scared that we end up dictating choices to players if if they see that a champion has a low win rate they may just not play that champion and

really we'd rather players just play League of Legends and have fun and not obsessed about you know becoming armchair statisticians on the other hand as a developer or as a player it feels really flippant when the developer says well that's not what our secret data says and we're not going to show you our secret data makes it very hard for a player to argue against it and it's possible that we could provide context we could say look yes this item build looks good on on paper but that's only with a really strong team coordination that you may not have in Solo Q so I just want to present this as an ongoing discussion um we're not changing our stance on this immediately but we do discuss it a lot internally and um you know maybe it' be an interesting experiment so when you detect a balanced problem how do you know what to change in the case of League of Legends it could be the champion it could be the item it could be the the Mastery this is the you know the pregame system for League of Legends similar to talents and wow um is it the strategy players are using right now is it the Synergy with

other team with the other Champions on that team this is an example of uh poppy this cute little yordle with the big hammer we recently updated poppy it's a really you know objectively it was really good rework super happy with the way that came out but her win rate did did Spike and it spiked at the same time that we changed one of our masteries this is uh Courage the Colossus the green guy here um Courage the Colossus gives you a shield when you crowd control a champion and poppy has a of crowd control she's already pretty tanky with courage of the Colossus she would get this this shield and become even more tanky it felt overpowered we knew we needed to do something and we debated a lot was Poppy the problem or was courage of Colossus the problem we definitely needed to Nerf courage of the Colossus the question was did we also need a Nerf poppy we went back and looked at the data in this case and we saw that her win rate had started to spike even before the courage of the clus change so in this case we did change both we nerfed both the Mastery and the champion at the same time and if we hadn't done that we would have had to like go in the next patch and she still

wouldn't have been fixed I'm a big advocate of play testing your own game um the having a play test team having a lot of data from players is no excuse not to play the game yourself if you don't you you'll miss stuff you won't really understand the player experience numbers just can't tell you the entire story and sometimes it's going to be really obvious you're like oh I understand the win rate of this Champion it's because nobody can tell what's going on with their Spell visual they don't know if they're standing in the fire or not on the other hand I want to caution you not to go too far here in the other direction it can be really easy to take your experience as a player and assume that applies to all players and I definitely see this happen when as a development team matures meaning we're all getting older it's easy for us to forget just how many games and how engaged some players can be with our games so don't play one game and make a lot of balanced decisions based on that one game this is in fact how we used to balance Age of Empires we would go play a game make tuning changes go play another game make tuning changes um just

doesn't fly I think anymore I try to play league every day um I don't always succeed I've been at GDC not playing a lot of league right now but it does help to stay in touch with what players are talking about so this the best slide in the deck I'll just give you a second here I think the best thing is that he's just kind of assuming he's just kind of commenting it's like oh I just I just noticed this kind of surprised me no judgment call there you do need to listen to players we listen to our own forums Reddit Twitter uh riers who tell us they noticed something wrong Pros streamers I could give you an entire talk on leveraging player feedback um this doesn't mean always do immediately what the players are asking for you'll often be really sad if you do that but you know ignore them at your peril they're giving you feedback they want to see the game improved you do need to good solution for other regions other languages um and

again smaller Studios May struggle more with this Riot is fortunate enough that we have Regional Offices in all the countries where league is played and we rely a lot on those teams to tell us hey did you know that players in Vietnam are talking about this you know this problem in the game that may not have filtered out to other regions yet that helps us to kind of pay attention to what Vietnamese players are saying because I certainly can't speak Vietnamese like many companies we also use beta tests for the changes we're going to put out betas are really good for bugs and they're really good for Impressions so a player may say wow this is complicated or I'm not exactly sure how I'm supposed to be using that in general sweeping statement here but in general betas are not good for balance testing and the reason is because players understandably don't try that hard if they're going to devote 10 hours this week to your game they probably want to spend time in the actual game not on your beta realm trying things out that may very well never get shipped you

know at the end of the day anyway so you end up with smaller populations which means matchmaking is not awesome and so forth um I will add that on World of Warcraft testing the PVE game meaning Dungeons and raids worked pretty well on beta and that's because the highend rating guilds would want to get a jump start on the content and practice so in this case practicing did actually work out for them but the PVP balance feedback from from World of Warcraft beta was about as not helpful as the as the balance feedback from League of Legends betas I would also discourage you from trying things like oh well we'll bribe players to play our beta um you may get a short-term spike in engagement but overall you have to be really careful to not like motivate odd behavior just because you want them to play a lot of games we also just ask players we don't necessarily wait for them to come to us we do player Labs or players into the studio have them play a game then ask

them about their experience we do this a lot say for for new Champions or Champion updates or crazy changes that we'd like to make to the game we have a micro survey tool that we leverage extensively anyone who's played League of Legends very long has probably gotten one of these surveys after a game asking hey did you like did you like this change what do you think about that it's really interesting to look at how the survey feedback changes over time how it changes in different regions around the world this um last preseason we added plants to the to the Summoner Rift map that players can interact with there's somewhat of a random element to the way plants spawn and when players first heard about this they they freaked out and they thought oh we're adding too much RNG to League of Legends it's going to ruin the game this is you know sky falling and nowadays our surveys are showing that most players are pretty happy with the way plants ended up so I'm not saying players are dumb by any stretch um I'm saying that we didn't necessarily set their expectations for what the feature was going to be

so now that you have goals data and player feedback this is the basic iteration cycle that that we use um make changes gather data gather player feedback make more changes you may have to go through this cycle a few times let's take NY for example N is a champion that can turn into a panther which makes it very fast she can also throw a spear so guess what n is our Cavalry Archer and jti as with the Cavalry archers we have trouble balancing her so we'll make a change doesn't fix the problem make another change doesn't fix the problem and so on and so on spoiler she still isn't right sad in Italy sometimes problems can't be solved through tuning I mentioned before that the live team is like our First Responders they can get to the scene of an accident but they can't fix like traffic patterns they're not empowered to like rewire the stop lights to try to create fewer accidents overall we do need sometimes to go through and make more systemic changes we do this a lot through Champion updates rise is kind of

Infamous here for a champion that we just can't get right we'll make a change we'll put him out he still is too powerful at High ELO too weak at low ELO we'll make another change and this goes on over and over as a last resort you can basement Champions we hate to do this because it you know it feels like it's violating a lot of the the the player trust about being able to play the Champions they want to play but it's also not fair to players to give them a champion that is just going to ruin the experience for everyone azir is a very very challenging Champion for us to balance it probably stems from the ability the fact that he can do so many different things he doesn't really have weaknesses that you can exploit if we tried to tune him given his current set of abilities his win rate may be you know 60% at Pro and and 30% at bronze which I'm probably exaggerating there but he's definitely a a hard-to balance Champion he probably just needs an update and needs to lose some of what he can do so winding down here a little bit I do

want I've talked a ton about balance but I I want to stress that balance isn't everything and as a systems designer you can definitely fall into the Trap of looking at everything to the lens of balance Champions can be balanced meaning their win rate could be perfect but still really frustrating and here are three examples Riven LeBlanc and Yasuo um yasu's RN rate may be totally fair and enough players may think he's that they stopped playing the game which is not as a developer you don't want to say but he's balanced likewise remember that players pick champions for a lot of different reasons I mentioned the pro example with um froggin and Nivea earlier this is jinx uh she's a very popular Champion to play her abilities are cool her thematic you know she's kind of a Anarchist Riot girl really resonates with players we know that Jinx is always going to be popular it's not necessarily because she's unbalanced and in fact right now she's a little weak of anything if you look at her play rate and conclude oh people are playing her too much we we

should Nerf her you're just going to make players sad being on a balance team is hard the balance team is going to hear a lot about how terrible they are is LOL going to suffer the same fate as W because of ghost crawler burnout is really frequent you're giving this team one tool to solve every problem it's like here's a wrench no matter what happens you have a wrench to solve it players and even co-workers are going to focus it's a lot more on failure than success and there's never a break as I said we patch every two weeks so as soon as the team is like okay we've made some changes patch 7.5 now let's start patch 7.6 there's never really an off cycle some of the solutions we use are to offer side projects for folks on the team so not everything is you know can I buffer Nerf five attack damage here we also try to rotate folks off the team that's good for their sanity it also helps to add some of the knowledge they've learned about balance two other team teams as well and finally like tell them they're doing good they're going to

hear a lot about how much they're screwing up so it's great to like reinforce that these guys how important their job is because they're they're honestly not going to hear it often from players even when they legitimately are doing well so in conclusion balance matters for longevity and engagement I would encourage you to understand why certain factors affect balance at different skill levels I would encourage you to have a philosophy don't just make changes Based on data making changes against checking against your goals and iterate iterate iterate and follow through when you don't get something right you're going to screw up a lot I screw up a lot Riot screws up a lot follow up when it does having a good relationship with your community will help we work really hard on this because players will give us the benefit of the doubt when we do make a mistake they won't doubt our intentions they'll just doubt our confidence a little bit good luck and with that I'd be happy to take any questions thank you

hey uh so when when nerfing things how do you mitigate the kind of feel bad effects of loss aversion is that most do you actually take that into account making your design decisions or is that mostly a PR uh Community Management thing I mean it is a Community Management thing but we involve the people making the changes as well um I think that's a very real concern and I I think as developers we don't always pay enough attention to it it's really easy to focus again on let's make that 50% number show up for every champion and to forget how it's going to feel for players one thing to avoid is the n example where you make a change you don't quite hit it right so the next patch there's another Nerf and the next patch there's another Nerf um that starts to feel really bad for players they know every time they're going to open up patch notes they're going to see a Nerf I would encourage you in those cases to make sure you get it right if you've if you've tried a few times or you know let it ride for a little bit kind of let them recover we also work really hard on

our patch notes to provide context like look it's not that we hate NY it's that we're trying to solve this particular problem I also think Riot like many studios is a little too quick to Nerf and a little slow to buff I mean nerfing is much easier than buffing but when you when you offset that a little bit um it doesn't you know it's not as negative experience overall it's a great question uh hi you mentioned sorry I'm stealing the microphone thanks uh you mentioned minting yes what does that mean can you can you explain that that means you um your Champion is such a problem in the game that the only thing to do is just Nerf the crap out of them so that players aren't going to play them um players who love that champion will play them they'll probably still show up in in lower tiers it's a really shitty thing to do particularly for players who love that champion but you also it's hard to justify having that character in the game ruining the experience for everyone else just so the players that love that champion can do it um quick followup uh how how do you handle messaging around that is there a strategy there or do you just do it quietly and hope people keep quiet um I

my general policy is to be as honest with players as you can they're smart they're going to figure it out they're not going to buy your excuse so in those cases I would say look we screwed up we can't make this character or you know unit or whatever game you're working on we can't make this healthy right now we're going to temporarily kind of Park them and we are going to get back to it you do need to follow up you just can't leave them in that state forever we did that once with a champion named Olaf and players still call it laughing is the verb thanks uh hi uh great talk um I have a question about uh the amount of Champions you have and as The Meta progresses some of them will will be under represented so when you're doing updates like Buffs and nerves do you usually look at the underrepresented uh Champions and buff them so that they could be more relevant we look at Champions who have a lower win rate and try to buff them to stay relevant um assuming that we can do so in a healthy way um Gallo is a champion we're working

on right now he his abilities just aren't that exciting volar is another example so he'd be hesitant to like buff them excessively we don't buff Champions just based on low play rate we have Champions like Urgot he's a creepy dude we don't necessarily want to see Urgot as popular as Jinx that's part of why players like to play him is he's a little weird so in that case we're fine as long as there are players out there that love that champion all right thank you uh you talked a lot about how a higher play rate on a champion so for example if a player plays a champion a lot their skill will increase with that champion so if you take this and you add it to the fact that there's you know 130 Champions and only six bands for ranked online it seems as if you're encouraging one tricks is that intentional or is that kind of a side effect of it's a it's a side effect and I hate it and I think it's one of the biggest design challenges to fix the League of Legends overall if you ask a skilled

player how do I get better at ranked they'll tell you be a one-trick pony find one Champion play the hell out of that champion know everything there is to know about that champion um and that and that will that is effective and you will increase your your win rate and probably the tier you play in um unfortunately it leaves a huge variety of the game um unavailable to you we try to offer all these champions for a lot of diversity in fact we know that people who play more Champions tend to stick with the game longer and say that they're having more fun you can imagine burnout is probably higher when you've played 300 games with VI um you can imagine that when VI gets nerfed you may just turn out of the game rather than switching to another champion so I don't have good solutions to that it keeps me up at night for sure okay thank you so I had a question revolving around Champion tearing um which is just like a this Con this sort of construct that like other players have sort of constructed to say these Champions are tier one these Champions are tier two which basically represents like pick these first and and these guys are Niche and these guys are useless um do you Embrace champion

tiering and say like we're building this guy to be tier you know tier one and this other guy has some really Niche counter and that's okay and he's not going to be high pick like I guess it revolves around are you striving for even pick rates or do you sort of embrace the tears no we don't strive for even pick rates we you know there are champions that are really good at at particular things um Shen is a good split Pusher if you don't enjoy split pushing or that strategy currently isn't super powerful in the game he you know he may not be a great pick um we want that a player who loves a champion to have opportunities to play them it feels terrible if you you know you love this Champion you've invested a lot in them maybe you've spent a lot of money on skins for this champion and now you feel like you just can't play them that's a really terrible experience um and then I had a request from another gentleman to go back to the conclusions page I think he wanted to take some notes there we go hey how's it going hey uh you talked about not only having a philosophy but following it I'm

wondering if you can give some example actions of what you've done to incorporate your philosophy into your team so it's actually part of the culture um we talk a lot about it for starters I make sure that we reinforce with the team you we we give teams a lot of autonomy at Riot and so then when someone makes a mistake we'll retro it and we'll talk about it like did we make this change to this champion because we really thought players and embrace it or did we make this change to the champion because you thought it was fun as a designer I mean sometimes good things come out of that but it again I feel like it's not super fair to players to just use them as our as our playground like that so um we retro things we talk about what went wrong we try not to beat up our teams too much for mistakes unless you know there's kind of a repeated pattern of making the same mistakes and not learning from that so is it really kind of on the onus of the leadership team to like make sure they're hitting those talking points then yeah absolutely um I I describe my job like I do not balance League of Legends you wouldn't want me to it would be a travesty um what I do

is I I talk more about creating an ecosystem in which good design decisions get made so I try to make sure people are thinking about things in Frameworks and then you know that frankly we just hire good people and smart designers and try to make sure they're aligned with our high level values and then turn them loose and get out of their way thank you first thank you for your inspiring presentation I'd like to ask about uh the balancing of Champions in several stages like for example you Nerf one of them you see that it's not enough or it's uh over the edge and the next iteration you buff it a lot so which way do you pre prefer prefer uh do you take one Champion apply some changes and then let it let it sit for a while and come to it later a lot later probably or do try to iterate in successful succession patches until he's

perfect it really depends on the situation as I I mentioned a little bit ago I caution you not to like Nerf the same character over and over again I mean this happened on World of Warcraft as well where we couldn't quite solve a problem and so patch after patch there's another you know feral Druid Nerf or whatever um nerfing tends to be easier than buffing particularly when you're looking at a situation where we have 10 Champions that are okay and one outlier and players are always like why don't you buff the other nine to be equal to that play to that win rate um that's just a lot a lot harder to do um if you keep missing the mark you need to step back and understand why are you missing the Mark is your data collection flawed are you just changing the wrong thing are players adjusting as quickly as you can sometimes it does help just to just to step back and see what happens it can take the community a while to develop a counter strategy to a dominant strategy we do try to you know we we describe our relationship to leagues balances Cur like Riot doesn't kind of let it out in the wild and then whatever happens

happens we do try to step in when players are experiencing too much pain thank you hi uh I wanted to get some more of your thoughts on the the relationship between viewership and changes uh like one contrast to traditional sports like soccer there's no balanced changes to the game it hardly ever evolves um but on the esport SI you know every time you're making a change balance change I know you mentioned that you do keep V ship into account but you know how much relationship is there do do people uh is there a risk of them getting out of touch with the game and then not understanding what they're watching um yeah yeah absolutely we do you know we have to take that in consideration when we make changes we know that the changes we make are going to affect the pros one of the things we try to do is you know keep them in the loop like as part of the discussion like hey guys just want to let you know we're thinking about making this change you know what's your reaction we don't we don't let them balance the game for us but we do want to get their feedback they're very in touch with that we also know that making systemic changes like for example not

that long ago we changed the dragon and League of Legends to actually be five different dragons that kind of Spawn at random we rely a lot on our casters and analysts to help explain this to to viewers we have um ride is blessed with very exceptional Talent there and they can kind of help bring players Along on that journey to understand what's going on it's a very complicated game okay thank you uh hello uh I wanted to ask about um if an AI controlled Champion being good in High ELO and low ELO like how would you balance that out if it's very popular on the low ELO it's it's challenging um we have a champion named ivern who's like a happy tree guy who has an AI um a boulder creature named Daisy and he's been pretty popular pretty high wind rate recently despite the AI I think that's because not that much of his power budget is tied up in the AI he can do a lot of cool things in addition to that so if you want have a a champion that uses a lot you know a character in your game that uses a lot

of turrets or AI I would say just make sure that the power budget is is pretty small there okay I think we have about three minutes left if we run out of time um I'm going to go over to the uh the wrap-up room which is right by the escalators to the left uh different countries have uh different preferences and aesthetic choices and champions they might choose uh how does that uh play into how you decide because different countries have different play style choices as well yeah absolutely overall we' found that players are players and if a champion is cool in one region it tends to be popular in other regions as well um there are definitely Regional differences um Chinese players love big guys with swords we probably couldn't make enough big guys with swords a character like Jinx who's like you know this Anarchist Rebel that archetype just doesn't resonate as much in historically in their culture as it does say in the west um and that's fine we know there's going to be Regional differences some places are going to embrace a champion more than others we try to offer a diversity there you know we do have 130 so hopefully there'll be something for everyone but does that do

you actually make any choices saying this country may like this aesthetic and we want them to maybe try different play Styles so we're going to actually give them this aesthetic with a varying play style one strategy we use that can help there are the champion skins which change the look and sometimes the voice of a champion quite a bit so we can introduce um a champion or a line of skins that you know has to do with a holiday or a historical event in some country knowing that they may not have widespread appeal but those players are going to love it cheers how do you feel about players and fan sites Gathering and sharing data in order to uh help each other to play the game that's a that's a really good question um I mean overall I don't begrudge them from doing that it totally makes sense why players want that information particularly given that we don't provide it the challenge is that sometimes the data is wrong and in those you know as developers we don't have the bandwidth to go debug it and explain to

players why it's wrong so we end up just kind of waving our hands and say oh yeah that data is bad and then players are left scratching their heads this is why we're talking a lot about does it just make sense to offer the information out there for example a lot of the the fan sites that offer data on League only look at high Platinum which is not going to tell you the whole story because very few of us are high Platinum all right maybe time for one more question are we good all right thank you very much