'Root': A Usability Postmortem

[Music] hi there my name is joshua grisley and today i'm going to present on root a usability postmortem so root is a board game it was released in 2018 uh and it's a highly asymmetric strategy game so in the core game you can play on one of four sides you can play the cats the birds the with an alliance which is a kind of ragtag band of mice and foxes and other small woodland creatures who are rising up against tyranny and then there's the vagabond which is this wily raccoon adventurer that drapes us through the forest and so i'm gonna be talking about uh our experience using doing usability and user experience work on root and in case you're not familiar with

those terms basically usability is making sure that people have an easy time learning the game and then playing the game using the game's components and just generally making sure that they have a good experience from opening up the box to closing it up again and so i'm going to go over some usability and user experience positives um but most of this talk is going to be about the failures uh and specifically the lessons that we learned from those failures and what you can take away into your own work uh and so i'm going to start out by talking about icon testing uh and so you know in most games you have a lot of iconography a lot of illustrations just a lot of stuff that helps orient people in different parts of the game and so when you have all of these icons you want to make sure that the icons are

doing their job you know that they're recognizable that people understand what they mean and that they get the job done and so this icon uh is an icon that used to be enroute but is no longer and the reason why we got rid of this icon was because when we did icon testing on it you know when we brought this up to people either by itself or in a group with our other icons and we asked people what they thought it was they would give back all sorts of uh weird answers like uh it's a teapot or it's a ghost or it's a monkey and uh these are all of course perfectly valid uh perceptions you know people can only see what they see but it's not the perception that we had hoped for because this is meant to be a mouse um and so because of the user testing that we did on this icon we knew that we

had to change it because when you refer to say you know a mouse card or a mouse suit in the rules people need to be able to look at the table and recognize that mouse icon as being a mouse uh and so uh here are four suits we have our foxes our bunnies our mice and our birds and here's the new mouse so after we did that icon testing we came up with a new concept for a mouse icon uh and the problem with that uh the first problem is that kyle ferran our wonderful wonderful artist hated this idea you know he said i don't want to draw a mickey mouse mouse um and to the point where now this idea of the monkey mouse icon has become somewhat of a touchstone for places where kyle thinks that something is

a bad idea basically but you know we put this new mouse icon in front of people and you know invariably people would say oh it's a mouse you know people understood what this new icon was um so it's a good change right well you know this is a presentation about mostly about failures so no this wasn't a good idea and i'm going to talk about why or i should say i'm not going to talk about why i'm just going to show you why and i think you're going to get it here pretty soon so here's our bunny icon and our mouse icon next to each other and we're just going to start making this smaller a little bit smaller you know keep going and i think that you're s you might be getting the idea it's that the icon doesn't scale well so uh at smaller scales the rabbit and the

mouse just start looking a lot like each other so even though the new mouse icon was recognizable as a mouse compared to the old icon it was doing its job there it was not distinguishable from the bunny and although like although there are lots of elements of this icon that could distinguish it the most important element in my opinion are these whiskers right here and the problem is is that these whiskers also happen to be the smallest element of the icon so even though it's fine if you just have a card in front of you you know relatively close close by the whiskers just do not show up in other contexts especially on our game board when the icon is scaled down and it has this darker background those

whiskers just kind of just kind of get reduced into oblivion and so there's almost nothing distinguishing it readily from that bunny icon unless you look really close at it so this was a big mistake unfortunately we can't really go back and change it because we need to support expansion products that use the same iconography um and so uh you can still visit our friends the monkey mouse who's on our uh on our logo um without the tail of course and so uh sorry about that kyle it's kind of a rookie mistake um but uh you know we'll we'll know we know better for next time so the next thing i'd like to talk about is storytelling and feedback uh and the reason i want to talk about this is because root as a game uh you know it's a war game but it's

fundamentally interested in telling stories about uh politics how power moves around through society um and uh basically um you know how people operate within conflict uh and so i'll tell you a little story about how we use um a few mechanics to tell great stories um and then i'm gonna tell a story about how we fail at that uh so uh this is called a sympathy token um it's a piece of the woodland alliance which is that ragtag band of creatures trying to you know rise up against tyranny and an interesting property of this piece is that whenever you remove it whenever you remove it from the map whenever you go in and you suppress that sympathy you have to actually give a card from

your hand to the woodland alliance player and so you might be you might walk into a fox clearing take out the sympathy you might have this tax collector sitting in your hand um and the tax collector is like whoa i didn't sign up for this um and uh basically goes and defects to the woodland alliance and so using this pretty simple combination of mechanics we can tell a nicely detailed story about what's going on here however a place where this falls down is in another system called crafting um basically uh what you need to know about crafting is that it lets you get the text on the bottom of the card so all that stuff and the on the paper on the bottom you can basically if you fulfill certain conditions you can get that stuff on the bottom

uh so basically the way that you get it is you just need a particular piece in a clearing of a suit that matches its requirement so if you look to the left of that box you'll see a little bunny icon and basically what that's just saying is that as long as you have your your crafting piece like your workshop uh in a clearing on the map of that suit you can get what's at the bottom so in this case we have a bunny workshop we have a bunny requirement and we can just craft that travel gear you know you can walk into your workshop and craft that travel gear um but the problem is is that there are a lot of parts of the story and only some of them actually interface with the components uh when you craft this card you end up discarding it you end up getting this boot but you don't actually do anything with that workshop

and what we saw all the time was people would just look at that workshop and then they would just say something like but what do i do with it um and this is a circumstance where i think we're missing an opportunity to provide feedback throughout this entire crafting process because the idea of crafting you know you're at the anvil you know or at the spinning wheel or something and you're doing something very physical it sort of evokes this physical this physical action uh and you're not getting that feedback from the game itself and so i think all of these questions that we got about what do i do have to do with feedback and so what i mean about feedback one example is if you flip over this workshop to show that you've used it to demonstrate that this is actually part of that chain of actions that constitute

crafting um we found that um when we instructed people to track the workshops that they used by like turning the piece a little bit or somehow physically denoting it people tended to understand the process much better unfortunately by the time we went to print we we had kind of waffled back and forth on whether we wanted to include any feedback mechanisms on these components because the downside is is that including this on the flip side means a pretty serious reduction in design space it means that you can't actually have these kinds of pieces with different backsides the backsides can only be used for this particular purpose and factions that we released later in expansions actually did make use of the backsides in other ways so in one regard it's good that we didn't include this flip side uh because we wouldn't have been able to

make those factions the way that we did but this was a trade-off um i really do think that this sort of feedback is really important for learning and so it's important to be conscious of what are all of the parts of this action chain that i'm trying to teach and how am i demonstrating that actually physically manipulating pieces of the game and this is a great example of affordances and in case you don't know what affordances are i'm just going to give you an example so we have this trash can here we have these three trash cans this trash can in the middle has a a smaller hole to put things in this the circular cutout and basically what it's doing is it's just it's very strongly suggesting that you should put things that fit in there into that trash can so bottles and cans rather than you know crumpled up paper bags and things like that the reason it

does that is it's just harder to put the the wrong stuff in there and so this is an example of a restrictive affordance where the it's actually restricting you from doing the wrong thing um and this kind of affordance is used in board games um this is an example from the game scythe uh where the player boards actually use these cutouts they're these double layer player boards with cutouts that force you to put the correct components into the right places on the player board and so it's using those physical restrictions to point you in the right direction now for root we don't actually use those kinds of physical affordances for reasons that i won't get into in this talk but what we do use are these things called affordance props this is kind of a made-up phrase there's probably a more technical word out there that i'm not aware of but this is the term that i'll use

and so what i mean by affordance prompts are places where on the interface at the point of use um manipulating a component you know pulling off something is at that exact focal point where you're you're pulling the piece off the interface instructs you to do something so in this situation we have these workshops on this track and as you pull this one off it's prompting you hey score two victory points and this is generally a much more effective method for prompting than say writing beneath the workshops track hey every time you remove a workshop from this track score two points um and the reason for that is that um generally when people are focusing on something their their focal range is actually very limited when when a person is focusing on a particular point the amount the angle

uh the the number of degrees at which they actually can focus on something is very very limited you almost lose sight of almost everything else around a very small focal point and so if you really want to get something across you need to put it at the point of use but there is a problem here and the problem is false prompts um so when you're pulling these workshops off the track you're being prompted to score two victory points and so that leads to the follow-up question of uh okay well when i put the workshops back on the track does that mean that i lose two points um and there's actually nothing in there's no intuitive signpost for why that wouldn't happen um and so a decent number of people when they come

and learn the game actually do assume that you lose points when you put these pieces back on those tracks with those the prompts to score uh even though we don't explicitly say that you lose points anywhere and uh even though it would basically be impossible for you to ever win the game if it were true um but it's still on us to make sure that people aren't getting the wrong ideas from these prompts so you know a lot of people thought that uh you did actually lose points and so in later printings we made explicit reference to the fact that these prompts don't work in the other direction because we did find upon release that a lot of people were thinking that they did work in both directions so these prompts are very powerful but they can also be pretty dangerous so i'm going to give you a little bit more another example of false prompts

this is a situation where we did actually figure out that these false prompts were happening before release so that's good uh so this faction is the corbid conspiracy it's a a faction that we released in the underworld expansion year and change ago uh they're this uh you know band of uh bomb throwing crows and ravens and things like that and the point on their faction board that i want to point you to is this crafted items box in the top right corner uh and so one thing that happened during development was um was kind of running low on space uh and so you know i did the thing that any developer under the gun does which is just to squish things down so you can fit everything um thinking that it just wouldn't be a

problem but what we found in usability testing of the the first day of usability testing somebody came in and you know they put some items in this crafted items box and they looked at it and they noticed that it was like pretty much exactly the perfect size to fit three items and they asked so it only holds three items right this box is smaller than the others so that must mean something right uh and you know when something like this happens this is kind of my expression that i i try to suppress as much as possible but it's my internal expression at least um because you know this is completely unexpected uh and it's actually the reason why usability testing is so important is because developers make changes all the time and there are very subtle implications of things that players are very apt to

pick up on uh in cases where developers aren't and you can miss these you know the these these wacky things um and this you know persisted on the next day um one person having a problem can sometimes be a fluke but multiple people having the same problem becomes a pattern and so the next day sure enough people came in and everyone was asking so it only holds three right you know the box is smaller and so i want to really drive home this point that subtle differences create the question well it's different how is it different what are the what are the implications of it being different there's almost no innocent difference out there and so you have to watch out for these so this is another false prompt the crafted items box is indeed smaller and it is indeed sized for these items

in such a way and even though we didn't say anywhere that there was an item restriction the physicality of this part of the interface suggested it and so you really need to watch out for this and so all of this has a lot to do with thinking about context and so i want to talk a little bit more about context uh so this is a famous uh optical illusions the duck rabbit illusion so you know depending upon where you're at you might see a duck you might see a rabbit and this is an example of what's called concept unsteadiness you know basically where the same thing can have multiple meanings depending upon your perception and so an optical illusion is one example of concept unsteadiness but this applies broadly throughout many different fields you know it can it can apply to just concepts to just ideas

it doesn't require specific you know sense um and a very fun thing about this optical illusion that i only learned about a few years ago was just how context dependent this illusion was so context matters in this illusion to the point where researchers compared how often people thought this was a rabbit versus a duck at different times of year and what they found was near easter people thought it was a rabbit more than at other times of year now the effect is not extremely strong it wasn't like a hundred percent of people were saying that this was a rabbit there was a small noticeable effect uh and so it being near easter is just a piece of context that was informing the way that

people were seeing this image rabbits were on the mind so people were a little bit more likely to see a rabbit and so i would like to talk about uh how context applies to reading our player boards uh so here i'm showing a very early version of the player board for one of our factions um and i'd like to point out a specific part of the player board which is the scoring victory points pane so the idea behind this in development was that um we wanted to basically give a reminder to people uh if they ever forgot like oh how do i score victory points is this faction we wanted a little reference to be available to them in a convenient place but one thing that we noticed during usability testing was that

people would look at this and wonder well is this a reminder or is it a prompt and what i mean by is it a prompt is does this victory point gain stack with other places where it's telling me to score victory points so if i have a card that says okay i craft an item and i get a victory point do i also apply this victory point from this pain so do i score two victory points do i score one victory point um and so uh very problematically a lot of people actually thought that this was a prompt um and this sometimes pretty wildly changed the number of victory points that people were scoring uh which is very dangerous it means that people are really not playing the game as intended so this was a dangerous misconception that people thought that this stacked with other places where we were

prompting people to score victory points and there's a pretty simple explanation for why people thought that this was a prompt rather than a reminder and that's because everything else on the board was a prompt you know there was basically no other information that served the purpose that this this uh this reminder did and so people took all of that context and they very rightly applied it to this part of the board as well and so what we did was we got rid of that information on the player board and we put it on a card we put it on a card called an overview card and you know these cards just basically contain a short overview of that faction and so we didn't change anything about the fact that we were presenting this information but we put it in a completely different

context and all of this context screams this is a reminder that there's nothing here that should be interpreted as stacking with other stuff this is just giving you a broad overview of this faction you're getting this card other people are getting this card everything about it is screaming this is a reminder and once we did this nobody thought that it stacked anymore so this is an example of a place where context really really matters in people interpreting your information it is not enough to use the right words you have to also put those words in the right context for them to be useful and so the next uh part of this context discussion i'd like to talk about uh i'd like to focus on a on the broader context of product design bringing together audiences and things of that nature so we designed root to bring in audiences from different parts

of the board gaming landscape you know we wanted a game that would appeal to people who are very experienced in war games and direct conflict games and we wanted to bring in wider audiences who were curious about the genre or just didn't really know much of anything at all and were attracted to these cute women creatures um and so whenever you're bringing together many different audiences you need to make sure that you include materials that are appropriate for them uh and so we did this we basically included three things in our first printing we included the law of root which is a rules reference document the learning to play guide which is a smaller more friendly example-laden rulebook and then we have a two-turn walk-through sheet which basically lets people jump into the game without reading too many of the

rules and so the first thing i want to talk about is the law of roots this rules reference document and the reason for that is because it's very polarizing or at least was very polarizing upon release and the reason for this was it is as i mentioned written in the style of wargaming rulebooks it is a very literal rulebook it has very few pictures and it is very highly structured in a way that is not the same as many traditional uh board game rulebooks and the law of root actually caused quite a few flame wars and uh really angry discussions on board game geek a popular board game discussion site uh to the point where i had to shut down some of these conversations because they

were just getting too heated uh because basically what happened was people kind of sorted themselves into two camps uh we have one camp that thought that this rulebook was great uh that it read well that the rules were clear and then you had another group that thought the opposite that just really really did not like it and i want to point specifically to the top comment here which says there are a number of problems with the law of root not least of which is the need to read it completely literally so i think that this really gets at the crux of the problem which is we have these different audiences coming in and one of them basically understands this tradition of rule book writing and that is very

literal and one of them is not familiar with it and that is absolutely not a problem with that audience it is a problem with how the aerobic is written and how we present it to the audience um to the middle statement here you know also gets at the crux of it this is a conflict simulate simulation or war game rule book not a board game one that's the problem not to judge i i think i understand so that's that's kind of the core of a lot of of the critiques of this rule book uh is that it is not written in a style that is useful for much of the audience and so these flame wars these heated discussions came from two audiences coming together from different perspectives and um not really being able to get out of their own fishbowl uh you know the famous david foster wallace speech about

uh two fish swimming along and the third fish swims by and says hey how's it going how's the water today and then the first two fish say what water so you know this is this is an example of two very different um very different groups of people coming from different traditions so one way that i like to think about this is uh in the framework of decoupling norms and contextualizing norms uh you know i referenced these uh critiques that talked about literalness before um and how some audiences are very used to pure literalness and some aren't and so these decoupling norms are kind of reading the letter of the law as the letter of the law without bringing in too many outside experiences whereas another group is bringing in contextualizing norms they're taking in ideas from other board games they've played and they're applying it to this game and

as i said before neither of these styles of reading are wrong but what we failed to do is to properly introduce people who are more familiar with contextualizing norms to the very literal way that the rules reference is written in because war game rule books are written in a very particular style that we um that we duplicated this is an example of one from a game called cuba libre it's a very popular modern war game and you can see it's very dry almost car manual like few pictures itemized numbered rules the only picture on this page actually is just this uh joke about air strikes and reprisals uh in brutal warfare so you know this is the kind of tradition that we're working in in terms of how dry the the rule book is um and so what we do in later printings

is we actually we write it up front uh we talk about um how to read this document you know uh we we don't leave it to chance by just hoping that people understand the traditions of a document like this um and we actually tell people for example to follow the literal word it's important to enculturate people into how the document is written and so this was a failure of ours that we hoped to solve in later printings and so when you bring in different audiences and you use you draw from different traditions you need to make sure that you are telling people that you're doing that so that's an important aspect of making games that bring in different audiences the other thing that i would like to touch on is the walkthrough

so we included this two page this double-sided um walkthrough sheet in the first printing of the game which basically just kind of it walked you through your turns uh you know it said okay do this do this do this and uh it basically just showed you how your faction worked um and similar to the law of roots a lot of people really liked this but there was another group of people who just really did not like it um and again we fell prey to the context problem um because almost universally the people who said that they didn't like it said okay i get what i'm doing but i don't understand why i'm doing it and so there's a large audience of people who really needed a lot more a lot more effort on our put put into um context for why you're doing things strategic reasoning how these things hook into other systems and so in later printings we actually

expanded this walkthrough into a little chat book called walking through routes and so in this we still do this step-by-step you know walking through okay you do this you do this you do this but in between a lot of these steps we describe how all of these things hook together why you're doing this what implications it has and so we give a lot more context into why people are doing this and people just really love this and so i really cannot stress enough how context in all of its forms is just really really important in writing rules effectively and one publisher that i think that does this really well in the video game space now is paradox um the creators of crusader kings and european revisalis and games like this i must admit that when i

got into one of the earlier crusader kings games i always just felt way in way over my head got really confused but the latest release crusaders kings 3 was just so much better at doing this than the previous releases and it's because they're doing this wonderful job of creating wikis youtube videos how to plays discord chats they're just really creating this thickly interwoven context heavy um mesh of learning materials that i think root does a better job of of emulating now and i think that we can you know really really uh we can increase the intricateness of the games that we're creating and introducing the wider audiences by creating these thick uh variable context uh rich uh learning materials and so the last thing that i want to talk about a little bit is

cognitive load so root is a pretty intricate game and it requires a lot of thinking the cognitive load to learn this game is pretty high all of the factions work in pretty different ways they share some common uh basic actions but they're all remixed in tons of different ways the f actions score points in different ways so learning your own faction and learning other factions can be somewhat taxing and so we needed to figure out a way to onboard people in a way that reduced the cognitive load as much as possible and so the first way that we tried this was using synchronous teaching so i talked before about these overview cards which are these you know basic descriptions of the factions in the game and so the idea behind this teaching strategy was we would give these

overview cards to their respective factions you know we would give the cat card to the cat and we would basically instruct people to you know play around you know just kind of get your bearings for a round and then at the end of that first round you should basically look at your overview card and you should kind of uh introduce yourself to the other players you know you should say hey i'm the cats you know here's what i'm about and then the bird player can say okay i'm like learning what you're about i don't like you here's how i can fight you things like that and that was the ideal right but we've quickly found out that this teaching strategy was just not going to work uh because we went out to some bars and i would highly recommend if you ever do usability work to test your games in bars uh and the reason for that is it's often a pretty bad atmosphere to hear each other in to see each other in

and just generally to learn games in it's it's almost like a stress test for your game uh and so you would go to the the bar and try teaching using this style and the first problem as you might imagine is that uh it's hard for some people to hear this important information you know not everyone is um ready to be the teacher uh in this circumstance and they might not speak loud enough you know they might not be prepared for it and so you know they might not talk loud enough or with enough projection um or they might be confused and stumbling over the words that are on the card um and so it just makes it difficult to learn um when you're kind of put on the spot like that but the other problem and probably the more important problem is that a lot of the times players just weren't ready to learn you know you could have somebody who's

introducing their faction the best that they can but everyone else at the table might just still be busy internalizing their own faction i mentioned that root is a very asymmetric game all of the factions work in different ways and so if somebody's not ready they're just going to gloss over it it's not actually going to do any good if somebody is describing their faction because they are not prepared and so between these problems you're losing out on most of the teaching um it's it's practically useless to be honest um and so for root the strategy that we settled on was an asynchronous teaching strategy where instead of one player getting their card for their own faction and teaching other people we basically decided to give the faction cards to everyone so on their own time whenever they feel comfortable with learning about other factions in more depth they can reference this card

so this accounts for the fact that people are learning at different rates and are learning in different ways and will be ready to learn about the other factions on different schedules so an asynchronous teaching strategy is much more um it's it's much more able to teach people in different circumstances it's much more reliable and resilient to situations that aren't ideal for teaching and just to people who learn differently and that's pretty much where i'm going to end it i hope that this has given you a little bit of look into usability and user experience in board games and i'm excited to answer your questions so thanks so much [Music] you