Slay the Spire: Metrics Driven Design and Balance

[Music] my name is Anthony Giovannetti and I am one of the two cofounders of mega Kirk Games slay the spire was our first professional game but in college I worked with my co-founder Casey on Oh with on two smaller hobbyist level projects that taught us a lot about exactly what not to do importantly you should know that I'm a huge fan of card games I grew up playing with pokemon cards and Magic the Gathering and later on I became so addicted to netrunner that I created a large fan site for it in fact I even managed a board and card game shop oh so I really love card games so of course I decided to make one the game we ended up making is called slay the spire for those of you who don't know the spire is a fusion of roguelikes and deck building games that has players traversing a dangerous spire to reach the top every time you play you'll be climbing a new procedurally generated spire and assembling a unique deck of

cards from what you find during your run throughout early access we received overwhelmingly positive review scores and quite frankly we sold way more copies than I ever thought imaginable in terms of development time slaves barbers and development or for two and a half years before we launched and early access and then we were in early access for a little over a year before launching into one dot Oh in January of this year today for this talk I'm going to narrow down my focus and talk specifically about how we thought about balance for slay the spire some of the important tools we use to help facilitate that and just our general approach to design so as you clearly do when you talk about game design I'm gonna start by looking at one of Picasso's most famous paintings now I'm not much of a fine art person to be honest but this painting in particular has a few special properties that I think are really neat so for some context this piece is incredibly famous it's featured in something like 95% of

our text books that cover the 20th century and no other piece comes close but what really interests me about it is the process you see Picasso spent about a year in the making this piece and more significantly he has the known record with over 500 preparatory drawings just for this piece of art often I think of painting as something more spontaneous kind of like Bob Ross or you know you're painting happy little accidents right onto the canvas conversely this particular piece was a work of constant iteration after the 500th drawing Picasso had narrowed down his masterpiece into exactly the final product that he knew we wanted this process is how I think about game design iteration and testing our King okay so this sounds great it's common knowledge that play testing is invaluable so if we want to create a good iteration cycle how do we go about

doing that and are there any challenges and in particular as an indie which I am well I think a good way to start is by thinking about exactly what your goals are in this case what do I mean by balance for slate aspire by balanced what we mean is that every card should have its place and this extends to everything not just cards but the relics the events the monsters in the game but I'm kind of simplifying here note that have a place is very different than everything should be equal to each other so for example some cards can be build around me cards these are cards that like define strategies or archetypes just having that in your deck really changes the nature of your deck other cards can be staples that have a place in a lot of different decks but maybe they don't stand out in any one deck in particular the important thing is that every card is doing something and that they're all reasonable to take a certain percentage of the time in this sense I'm

not striving for the impossible task of making every card equally good but instead I'm shooting for a way more sensible target additionally I wanted to avoid content that how would I call a warping effect if something is too powerful and also too easy to set up then the game can easily become just about achieving that combo or strategy or we're striving for ultimately vastly aspire is a diversity of strategies and play styles so it is important to limit the overpowered effects in some fashion with slate aspire we actually had two main advantages one we're a single-player game and two were a roguelike due to the nature of being a single-player roguelike experience I don't need to worry as much like hearthstone or magic about a single dominant strategy because you won't always get the required cards or relics to pull it off even if you do pull it off you're just gonna be dominating the game itself and not another player who's gonna have a bad time as long as anything too overpowered as

is rare and difficult to pull off the game is still gonna function correctly as an example of this players of slay the spire might recognize the combo on the screen of the card corruption and the relic dead branch dead branch is an incredibly powerful rare relic that generates new cards whenever a card is exhausted corruption makes exhausting cards really really easy to do so in tandem they create a ton of value together and this is probably one of the more powerful combos in the game however since we're not a traditional card game like hearthstone or magic you can't just always put this combo into your deck and consistently play the same thing over and over again and say the spire not only will you often not see one or even both of these but maybe in a run you saw corruption early thought hey this doesn't match my deck I don't want it later on in the run you see dead branch because you didn't take that earlier corruption it's not as impactful because this is rare and often

the player has to kind of actively seek it out it's okay when they do assemble it and we're comfortable rewarding the player with a feeling of extreme power when they do pull it off so this is a really big advantage that we have but even with this in mind we need to recognize that we had one big downside in particular namely we're small an indie team of two two people two designers and we're trying to balance hundreds of cards and other content together especially when you start to get the crazy interactions between relics and cards acting in tandem it can quickly balloon into a near endless task as a long time card game player myself I knew that even large experience design teams working on established games with lots of resources mess up balance all the time so for us balance was gonna have to be something that we worked on at an early stage and actually tried to scale up so what tools do we use well of course we use the traditional methods

you know we watched players play the game went to events played an unhealthy amount of the game ourselves we started internal play test groups but we also made a concerted effort early on to expand our play test reach and ability to gather feedback to this end we made use of a few specific tools to greatly amplify our efforts so let's go into those the first tool that we started using was an internal play tester slack that all the play testers would use to communicate get new game builds etc we wanted to make things as easy as possible for the play testers and us so we used a bot to collect all the feedback the bug reports the ideas or anything else into one easy place that we could look to I always hated filling out surveys you know I would play test games for other people and fill out their surveys and not really liked it so I didn't want to do anything like that so instead we focused on making the

barrier to give feedback as low as possible as a result play testers could give as much feedback as they wanted at any time whenever it was convenient for them and it didn't make it didn't feel like a burden then we used my strong connections with a netrunner community to invite some of the top level netrunner players to provide high quality feedback and we encouraged them to break the game even when a lot of the systems weren't in place one of the reasons that this was sustainable for us is that basically every single day we were pushing out a new build of slay the spire with a change log into a slack channel that our play testers had access to the constant stream of new content balance changes and system overhauls meant that instead of testing something that got stale there was a constant growing experience that people could keep coming back to you we actually had individual play testers who played thousands of runs just because their interest level was genuinely kept throughout the two and a half years of development the second

tool we started using was absolutely integral for us throughout the lifecycle of the game as a data-driven person I wanted the ability to really try and test our assumptions regarding balance to that end we built an in-house metric server that collects data from the players whenever they play a run on death or victory anonymous data is sent back to us and collected for analysis we collect data on basically every single piece of relevant information we could think of we look at the enemy as the cards the relics the events what choices the players making at now at the beginning how long it takes them to complete a run et cetera this was incredibly useful prior to launching and early access as it meant that even the quiet players that just played new builds and only rarely if ever gave feedback they were still contributing their data was still useful even if their feet they weren't giving any feedback and every single run done prior to launch had merit you can think of this as passive versus active play

testing going out to events or using focus groups and watching people play it's super valuable and we did it but it's very active it takes a lot of time it doesn't scale very well and it takes resources that we just didn't have our metric server on the other hand is very passive just collects data for us in the background and then we can look at it to answer questions or generate ideas whenever we're able to this combo'd very well with our internal slack while allowing us to fully leverage the contribution of each play tester for us using metrics was a completely invaluable tool in our toolbox and actually allowed us to be more objective in our decision-making even early on however with all that said as a note of caution it's important to remember that metrics are just one tool in your designers toolbox and they should be used in tandem with other things sometimes other factors can cause the data to lie to you a great example of this since they the

spire was the card madness now madness is a good card don't get me wrong but one thing that we were seeing in our data Leon was that madness had a grossly disproportionate rate of being seen in winning decks why was this the case was madness actually one of the better cards in the game well no the problem is that most players that had madness in their deck were getting it at an event that happened late in the third act of the game before the boss so this was artificially inflating the apparent power level of the card so data is incredibly useful but it can lie to you deceive you and it needs to be thought about before you act on it it's up to you the designer to figure out the truth about what your dad is telling you another cautionary tale we had that we discovered very early on when we didn't have a lot of play testers was that we had to of what I'll call super play testers people who were already obsessed with the game playing unhealthy amounts of it and this is great you know you're always happy to see early play testers

that love your game especially when it just has stick figure art and it's it's great because we're getting a lot of rundown on them but it also meant that if we just looked at all of the data in aggregate and just took blind averages of things we're basically just sampling those two people they were dwarfing the other players who were maybe less experienced only played every now and then these are the kind of things that need to be accounted for when you're collecting metrics so with those tools in mind we launched into early access with two and a half years of development refinement and balanced testing this is hugely important as we launched with what was already a very balanced product I see many titles that launch into early access that are quite simply unbalanced buggy messes at that stage I don't think that really works very well and it's where early access gets a lot of its bad rep from our approach instead was to launch an already balanced and near complete game and early access and then just keep expanding it frequently with

more and more content so launching into early access was a real game-changer firstly aspire it greatly improved how valuable our metric collection was as the player base our metrics was pulling data from became way larger it allowed us to see new patterns that we may have missed for various reasons before we were also getting a wider skill range in our player base meaning that we could better get data on the less experienced card game players are just people that didn't play very much our release a release cycle during early access was also no no worthy and was influenced by our prior development cycle one thing that we had really learned as a takeaway from our internal play testing was that players love constant balance tweaks and changes to the game since we saw that constant engagement from our play testers with our quick release cycle we decided to do a weekly update scheduled in early access the weekly updates were beloved throughout early access and they established that things were subject to change at any moment one week from the

next this meant that the expectations of our players were totally in line with our desire to constantly iterate and tweak the balance of the game additionally we had some great new tools available to us namely discord our new ascension levels and streamers so we're going to talk about those now discord was set up to be much like our internal playtesting slack accept orders of magnitude magnitude larger to better receive feedback from the thousands of new voices we brought back feedback bought from our slack into our discord so it could easily collect feedback from the growing player community the discord continued to grow and had a large number of people providing us with a never-ending stream of feedback in fact at the time that I made this slide it collected over 18,000 pieces of feedback for us and keep in mind we basically read every single piece of feedback on there a lot we also created a beta branch that was accessible to players in early access players on the beta branch

we're getting essentially daily builds just like our old slack play testers were even though these builds were buggier and often less balanced because of how it was framed and the expectations that were set we were able to keep up our fast iterations while having an almost layered approach changes would get tested daily and the beta branch before moving to the main branch every week the beta branch was a big hit and it let players decide on how raw of an experience they wanted did they want to be right on the cutting edge getting the new stuff even if it meant for a couple extra bugs if so great provide us with feedback because we had these two different groupings of players beta ranch vs. main branch we created separate beta branch channels in our discord to keep our beta players focused on feedback without spoiling other players of upcoming changes additionally one of the things that we developed in our in early access or as ascension levels one challenge with balance is always how to handle up

differing difficulty modes we primarily balanced slay the spire around the base difficulty level but slay also has twenty levels of progressively harder difficulty that we call ascension players have to unlock these one level at a time meaning that as players skill improves so too will they climb up the ascension ladder because playing ascension is optional but it offers achievements and a sense of accomplishment players will naturally gravitate to the ascension level that makes sense for them as a result we can actually sort our metric data by Ascension levels to get a better feel for granularity in our data and it also makes targeted changes for more or less hardcore players consequently what we did is we actually turned difficulty levels from a challenge in design into a strength not just in terms of metrics and data collection but also in terms of our ability to balance for different kind of players furthermore our custom and daily modes also added additional

ways to alter the difficulty of the game with unique modifiers that greatly change up play usually to make things actually easier this time around again this gives further customization outside of the typical easy medium hard paradigm I really highly recommend experimenting with systems like this as the ascension levels turned out to be a big hit and in fact streamers in particular really liked them so let's talk about streamers and youtubers streamers are incredibly valuable I don't think anyone questions that and they're not just great at promoting your game to others but maybe surprisingly they're also amazing for keeping up to date with the community and getting feedback about your game during early access I would just often watch various stream on an additional monitor while I was working this let me directly listen to what the popular voices in the community felt about various cards or enemies so broad that is great and sometimes just

getting that strong visceral response is actually more useful the sector does a good replacement for going to events or trying to get people to sit down in the same room as me I just didn't have time for it anymore and early access when you hear that dejected streamer making yet another sad meme about some item in the game it's actually really telling can help it can help actually place an experience to the data that you're seeing additionally streamers are always talking to their audience so you're getting their internal thoughts constantly this is a very hard to pick up from random people at an event or a focus group regular people aren't used to talking their thoughts out loud like that I've often found that you get better insight from a streamer than you do from just in person watching as a streamer is not going to be afraid to say their mind or insult you they don't even know you're there half the time and you even get to see what twitch chat thinks about stuff and yeah that's bad but sometimes you get some good stuff out of it too so streamers are really

really great and you should probably watch streams of your game okay so now we have all these great means of collecting player feedback and data the tricky thing is how to holistically combine it all together into a coherent balanced whole this is where the early access model is so helpful because you can always respond to the data make a change and then see how it affects the player base with that in mind I really want to emphasize something do not be afraid to break things as an example when we launched our new daily mode players were initially happy to have a new mode new content was good but it was quickly becoming clear to casey and i that the mode had a few flaws and it was probably not gonna be very popular for very long so we simply went back to the drawing board and the very next weekly past we had completely changed the mode made it more about these strange weird modifiers that changed your game up instead of the original focus of trying to add an extra difficulty mode and compete with other players we already

had ascension of for things like that now the daily mode was becoming increasingly popular and it was a huge hit as we built upon it if we had been overly cautious instead we probably would not have taken the path we did but because we have the freedom to constantly change and iterate we could swiftly act to improve the game because we were so directly tied into our players and what they wanted between the discord the metric server reading read and other social media etc I could already tell what was desired by the player base so our approach paid off here our iterative heavy approach also started to pay dividends when we began working on our third character the defect the defect was our third character and the only one developed in early access and we thought that the defect was going to take significantly longer than the other two after all we were busy working on a thousand other things at the time we had to constantly push out those weekly patches but surprisingly for us because we had

already designed two characters at this point and also we had set up these and improved streams of getting player feedback and metric data the defect was actually the quickest character to develop we felt no hesitation at all at completely reworking cards from one day to the next day and in fact putting them pushing it out into our beta branch we were able to hone in on with great speed on something that we felt was very fleshed out and played completely differently to the other two characters this also really encouraged people to play and streamers to log in and stream the defect because every day cards were changing so their viewers were constantly engaged overall balancing the defect live in early access was a completely great experience and it was a huge success firm for our approach okay so in conclusion here's some of the takeaways we had from developing the aspire one iterate iterate iterate without change and experimentation you

aren't going to narrow in on the good stuff - don't be afraid to make changes in the worst case you're only going to get more information to act on later and ultimately change is just going help you three lower the barrier between yourself your playtesters and your audience to get the best feedback possible and - and to get the highest quality of data and finally update more often your players are going to love you for it and you're gonna make your game better as a result okay so that was my talk hopefully you enjoyed it and now we have some time for Q&A thanks [Applause] hi so you had mentioned earlier about how developing game was a little easier

in play testing because as a game that is non-competitive that a single player it's kind of more okay for certain powers to be a bit overpowered in certain situations and all that I was wondering like how did you make a decision between certain abilities being kind of overpowered in like a cool way later in the game or which abilities would be considered degenerate in which things would need to be wheeled back from being too strong since you didn't have to worry about a negative play experience for an opponent sure so so a big part of it is how common it is to get that overpowered thing it's like if we just had a common card that was way too powerful and you just saw it in every run that's that's clearly no good so one you need things to be rare hard to get and then the other thing is just seeing how people play with it so in general that's a bit harder to get straight from metrics you need that more from like feedback like people posting things in a feedback bot or watching a streamer but strategies that are very

very slow and grindy like for example if we allowed players to just keep healing up over and over and over again and in fact we actually had this for awhile you could like farm up healing powers and just keep playing them it takes a long time to do it's optimal but it's really really boring those are the kind of things that we wanted to nuke the hardest and says something where you see like a really big number on the screen you feel really powerful those are feel good and that's what we wanted to encourage thanks stuff I was wondering if you could comment on how you settled on your early access prize what kind of research you did and what factors played into that sure so how did we settle our early access price we looked to other games and we said hey dead cells the selling for $16.99 there kicking ass they're doing great these guys seem really smart and then you know we looked at other stuff and honestly that's how we did it so you have two questions the first is what were your

thoughts around having you discard the rest of your cars that you didn't use per turn instead of allowing a player to keep it so they could plan it turns a head and then my second one was kind of yeah so I asked sure so the so the first question is why do you discard your hand at the end of turn instead of in a game like hearthstone or magic where you keep your hand so that's actually kind of a holdover from deck building game it's like Dominion or ascension and board games like that we're in those games you would always discard your hand into the discard pile and I just kind of kept that model when building sleight aspire in general what it does is it allows you to actually cycle through your deck and see all your cards which is a property you really want it's if we so we actually have a relic in the game called runic pyramid that lets you keep your hand and you can see how it totally alters the play style of the game it's more about holding on to cards and thinking about how you use them instead of being able to fly through your deck a

lot sure and so it was like if you looked at the metrics around or feedback around that did players like the form or the ladder more so we never actually has tested the game using the hearthstone model of hand-holding because it was it was more based around deck builders like Dominion which is like kind of the the originator of the deck building genre and board games and in that game you discard your hand so I thought that what they were doing made sense better and like even in our original design document we had it and we just kind of kept going with that cool and the second question sorry was if you guys thought about co-op and if so if you were did you co-op how would you guys think through that we've thought about co-op maybe in a future but basically no comment for called I'm first view thank you for the great talk and my question is so slate this bar has a lot of really good enemies that challenge the different

card archetypes so how did you go about enemy design did that come after the card design or were they sort of in tandem or did you have two enemies first um most of the design for everything was kind of done in tandem like we we created an initial set of like 20 or 30 cards and then started designing some basic enemies just to like make sure everything was good like the original version of the game it just had a little single character that bobbed up and down you had a couple basic enemies that were like zombies and slimes and you know just stereotypical things and you had you had basic cards and then from there we just fleshed everything out started developing the world and we kind of just built organically Kasey and I would often meet up and we would just have design sessions where we would just hash out hundreds and hundreds of ideas and like a huge brainstorming session and then go from there so we didn't just say okay now it's cards now it's enemies because everything's so interlinked I'm just a quick follow-up how did you decide to I

guess make very predictable enemy attack patters like was that a conscious decision so how did we come up with our predictable enemy attack patterns the enemy intent system and all of the everything about that was actually a huge design process that took a long time to develop the initial design of the game they were more like kind of like a Final Fantasy enemies where they just had like random move sets and they would randomly roll and move every turn and we actually had a lot of problems with this players too felt that it was too random there's already a lot of randomness in the game with all the card elements and everything so that extra element of unpredictability just didn't work and then also you didn't know like how much block do I need on this turn I have no idea how much this enemy was attacking for I don't have a good sense of can I attack or not so we actually spent a long time and we went through a bunch of different systems on that did do the whole talk on that yeah my

questions about the unstructured feedback here collecting so it's great to see that you had those channels to collect that kind of feedback but how did you go about filtering through all that and actually being able to leverage it as a two-person team sure so uh one is you don't sleep the other is you actually get you get better at it as you as you start to go through it honestly we just had a manual process for it we thought about trying to automate that feedback but it's very difficult because people can write the exact same bug and too 10 different ways you know so you will go through and we would look through it and we would try to organize things into different slots or we'd say oh this is that bug I'm recognizing over here and we would chat with each other while we're doing it and and honestly we just did it manually and and kind of develop that skill over time but there was no easy solution cool thank you hi regarding daily challenges are you

creating those procedurally or you hand making them and how much balance are you sure so regarding daily challenges are we making them procedurally or hand making them I am absolutely not hand making them those are procedurally generated based on basically the date so we hit a time server get a time stamp for the day and then use that as the seed to procedurally generate everything and it's funny because a lot of people think that you know I'm sitting there every day like on Christmas Day I'm like alright this is the and and I never did that but because of some of the patterns that emerge players get that impression which is great because you know some people really like that but I'm absolutely not doing that and are you spending any time balancing or is just letting it go don't we just let it go and and what we do is it's one of the things that we can use our metrics on to look at retro actively and see like we ended up changing and getting rid of brewmaster which was which was a daily modifier that ended up being no fun but

that's the kind of thing where you just you test it you see how its received it wasn't received well so we modified it Thanks so you're drinking from the feedback firehose and the emotional load of that how do you deal with it because I know you know sometimes people are toxic sometimes they're not but even if you've got the same piece of feedback 300 times about something you don't want to change how did you guys manage that so right so how do we manage getting like 300 pieces of the same so specifically the emotional load of it because if you're getting a lot of feedback even when it's polite it's a lot of feedback and I know that can be challenging to hem how emotionally ok so how do you deal with the emotional aspect of games yeah feedback you just hit delete on that piece of feedback if it's trash yeah you even you even get to the point where you're like oh it's this guy again you know I recognize the avatar delete you know like I mean let's let's be

realistic so not every piece of feedback is created equally and so sometimes we'll have some people that we really recognize like they'll give big detailed pieces of feedback and so I'll be like oh it's him okay I'll make sure to pay attention to this but you just kind of get mute to the constant nonsense so it was just something at first it was a little jarring and by the 18-thousand piece of feedback you're like whatever thank you Kate okay thanks for doing the talk and really appreciate it you mentioned the brewmaster modifier and that kind of leads to my question through all the database feedback that you received what do you think well what do you think was the largest admit you know mechanic or system that got left on the cutting-room floor that you based on a database decision so what was the largest thing that got cut based off of a database decision I mean that's that's tough because there are so many things that got changed or cut but a lot of it

is more iterative the daily was the biggest thing that just totally did a 180 all at once and then the other thing happened was more earlier in designed with our intent system and how that worked out we're I mean we literally went through probably ten different versions of our intent system before we got to what it's what it's like we actually used to have a system in place where you could you would have detailed move next move information on each enemy and you would like click on them and there is almost like a sentence that would write out and have little like fluff and stuff in there and that was kind of fun people some people liked it but I had a ton of problems so we got rid of it thank you hi I was wondering if you could talk a little bit more about the weekly update schedule that you had during early access it sounds interesting but it also sounds grueling and like there would be challenges there with keeping build stability how did you get through that sure so

what were the challenges with the weekly updates and it's grilling nature yes it was very grueling and yes there were sometimes some stability issues where you'd think Oh everything's fine it was good in the beta branch and then we would push it out and then we'd have to be hot fixing at you know midnight but by having just a consistent cadence we're just every Thursday at this specific time we're doing it we had a pretty good flow going and yeah we were we were definitely working more than 40 hours like absolutely I probably in our next game I would probably do bi-weekly updates just for our own sanity sake and in particular because you do waste a good amount of time well not not waste but you spend a good amount of time writing up good patch notes finding community art and like getting that all situated but in terms of build stability that's where having our beta branch was kind of the savior of that because if we didn't have a beta branch that was getting daily updates then those weekly updates would

become a lot riskier but instead because we have that beta branch we're already going through you know like five new builds at least before you get to that weekly update so alright thank you mm-hmm oh we out of time all right we're out of time but we're gonna I'm gonna go to a wrap-up room so I I don't know which one it is but you can follow it follow me there thanks for coming