Level Up Your Game: The Untapped Potential of Roguelikes

[Music] [Applause] all right um so so hey thanks for making it out I know this is late Friday and and you know there's the temptation to go get on standby for an earlier flight or you know I saw a lot of people at w having drinks and you know they've done the learning now they're going to do the networking that's cool too maybe I'll do that afterwards but um seriously thanks for coming out and uh and choosing to attend my talk and before I go into talking about Rogue likes which is you know what you guys are are here to hear about um I'll just briefly introduce myself uh I'm Tom Cadwell vice president of game design at Riot games and um in terms of games I worked on most recently of course was um League of Legends uh where I was the design director and um you know prior to that I was the Play balance guy on Warcraft 3 expansion and uh did a bunch of different design stuff on World of Warcraft you know leading up to launch um sorry and you know these days um most of my time is spent uh helping Riot um

especially game designers be Innovative in game development and game design in particular and to do this best I think it's really important to draw upon the broadest inspiration and sources you can and you know I think that that's especially important to do that outside your genre I think you know in the past I mean I still see like these job ads like oh you know looking for 15 years of RTS experience or whatever and I think it's really critical to be broadly played um and it Riot we we all spend a lot of well I mean we're passionate about playing broad games we all spend a lot of effort trying to sample broadly and consistently and I think that um you know as developers is just is just really key not to constrain Yourself by a genre because a lot of the best innovations that have emerged from games have come from other places like for example I remember the Mbox system and wow I remember like these conversations you know 14 years ago 13 13 years ago where the source of the idea was Animal Crossing of all things right like I mean there just a lot of random sources um so um you know I also think though that that you know the craft of game design and game development are really about games not about genre so I think as

developers it's it's critical to continue to you know broadly play games and that Pursuit and and that passion um led me to conclude that Rog likes um you know tapering back into my topic here have a have a bunch of underappreciated design lessons um that can be applied elsewhere and I'd like to share those with you guys today so in terms of how this talk is structured first I'm going to give my definition of a rogue like um and you know I've read there's a bunch of different definitions and I just want to present you know my perspective so that there's you know General alignment um in the talk um from there I'm going to talk about how Rogue likes are engaging because of Mastery and how a lot of other games are as well and M by master I mean the experience of learning and getting better at a game and uh I'm going to briefly Define what I see as that sort of process of learning um from there I'm going to go into how Rog likees specifically have design patterns that amp up their Mastery potential and and drive that really long-term engagement um and I'm going to cover a couple different concepts there and and from there I'll do an analysis of some non- Rogue likee games uh and talk about

how I think they've either co-evolved or leveraged some rogy elements and then speculate on how um you know additional Rogue liness could potentially be applied to those uh games to enhance their their gameplay uh and then you know we should have you know five to 10 minutes afterwards for Q&A which my understanding is that will be done um across the hall um at the end so um you know the original Rog like is of course Rogue which I guess implicitly is Rogue likee and uh that that was from uh you know about 1980 um and in it by the way who who has played you know one of these G really early okay so it's about you know 35% is okay so um these are these are pretty hardcore games and and the basic structure of of them including Rogue the of the original kind of games of this type um where that you be it's like a character RPG term-based um where you descend through like a bunch of levels to eventually recover in this case The Amulet of yander um and then Ascend to

the surface and win and um these games are sort of notorious for like their their really high difficulty for for perade death um and limited or no saving ability certainly no ability to replay a death for sure um and um these games are like super super popular at colleges um they still are today but they're exceptionally popular uh in the 80s where you you know you go to a computer lab at a at a University at you know 2 a.m. and you'd see like half the workstations occupied by people playing one of these games um and it's been a really influential game spawned a lot of um you know mods and um inspired buys and so forth for example um you know one that I really like is uh you I really wish I had put more hours in this game than I have but I put a decent number um is net hack and and net hack is um kind of an offsho of of hack which was offshot from Rogue um and it's sort of like the I think it's like maybe the most fully featured game ever it's um like for example you can train your pet with pet treats to Rob stores reliably right I mean it's a pretty interesting stuff um and um you know they just did a

lot of world simulation really interesting mechanics um and then the the modern ones there's been a modern Resurgence I think driven by the availability of good game engine technology and and and a really thriving Indie scene um and you know these are the modern hybrid Rogue likees which are I think predominantly Rogue likee and usually have replaced the the core combat Engine with something and who's played like at least one of these games a little bit all right so yeah so this you guys all have played roog now now leave your hands up who has played these everybody everybody play a rogue like listed here hands up okay so who have you played at least one of these games at least 100 hours yeah so right like that's pretty high engagement I know a lot of you still said no but that's still like about half of you who did have done that and um you know I think that's that's pretty interesting and um you know leading to this slide you're probably asking okay well that's interesting Insight time but why why are we switching over Starcraft and angb slide and the reason is when I was in college which was um you know I'm just going to date myself here as I'm I'm slowly aging

you know um 1997 to 2001 okay um my my fraternity Brothers uh and my relatively game living fraternity um were really into a bunch of games and you know I was playing Starcraft a lot you know about 70 hours a week at one point um a lot of my frateran brothers were playing EverQuest extensively you know of course there was Final Fantasy 7 and you know Mario Kart and so forth um but then also like a couple dudes were playing in band like you know pretty much as much as I was playing Starcraft over summer break and um that was that's a lot I mean you know if you just compare these two cames together like you know it's like you got you know next gen graphics on the left from you know 1995 and next gen Graphics from 1973 you know and um so um you know what is it what is it about Rog likes that despite all the clues the hardc coreness the arbitrariness the Antiquated Graphics that makes them so engaging where you

know the modern versions which which I think you know graphically do have some of the same disadvantages compared to Broad games are still engaging people so much and I think the answer is Mastery and you know by that I mean the the like I said earlier the feeling of getting better over time and I think Rog likes provide a means for ongoing long-term Mastery that holds player interest for hundreds or maybe even thousands of hours and when I think about the most engaging games the games that consistently across some Quorum of players um Can engage that long I mean generally you're talking about games like you know Starcraft Street Fighter 2 Dark Souls mobas such as you know League of Legends um and even like if you think about World of Warcraft which I think has a lot of other things going on that drive its immense engagement um you know Master is a big part there too you talk about like getting past a boss with your guild or getting really good at PVP and and certainly I mean not all games require deep deep Mastery to be long-term engaging I mean I think Minecraft would be a good example and there's there's others as well but um the majority I would maintain that the majority of games that engage players interest for hundreds of thousands of hours

um are built on this feeling of mastering getting better um and and the way this sort of works for players is is pretty simple basically you know you you counter challenge your best just isn't good enough and that's failure um and from that failure you start to learn and eventually you encounter um you know you encounter that challenge again and you're successful and that feels much better than ordinary success because it's a success that you earned that's that's built on self-improvement um and um and that's something really special and just feel really good and and I would maintain as long as players are feeling that they're getting better and learning it's pretty difficult to actually get bored of a game you might find something else caps your interest but it's actually pretty hard to get straight up Board of the game and um I'm going to walk through Mario really quick just to talk about some examples of what I mean by Mastery of like specific things to master and this is important because I and I'm going to allude to this later but um you know to the extent that we can make our games have multiple vectors of Mastery you know we're setting ourselves up for Success so in Mario um you know the the most the most prevalent form of Mastery

is just mechanical skill and dexterity and this is this is common in any real-time game in general and this would just be things like hey dodging attacks uh jumping between platforms with correct timing even jumping it all correctly um you know killing monsters you know in the the detailed moment to moment way correctly um you know and certainly when you think about like you know fighting game and that's a key piece for sure um Mario also although less so than maybe um you know certain other modern games or certainly strategy games has a significant amount strategy and judgment this is just around like hey should I should I wait it on be patient here um you know should I even go for those bonus coins should I go for that bonus life um should I go for you know any number of risky plays um and just understanding or understanding like hey like I really need to play safe here because fireflow is generally pretty important against the monster classes here and if I lose it it's going to be a lot harder um and of course knowledge of understanding this is sort of the easiest type of Mastery to Foster which is just you know the mechanics of where

you know what do monsters do and and understanding that allows you to start to participate in other forms of Mastery or um where's the hidden stuff or what do the powerups do things like that um and uh you know conspicuously absent from Mario is teamwork which is a really deep well of Mastery that any multiplayer game will draw from at least a little bit if it's a team multiplayer game um and and I think you know when you see like part of you know Counter-Strike would be an example of a game that significantly leverages this for its replayability and um you know if if you're successful as a game designer setting up your game with a lot of different forms of Mastery where there's a lot of different skills players can improve on um you start to make possible random parallel learing and and this is basically the idea that players are sort of at different times perhaps subconsciously chosen or you know whatever they feel like it I guess to some extent um are choosing to or or unconsciously learning uh some skill and um and the idea is that you know the more you have things to master um and

the less you're constraining players on a very specific thing to master such as past the specific boss for example um the higher the chance that in a particular play session you know they're going to have a um a critical aha moment right and um you know feel feel really good about that aha moment and and know that they they learn something and and and thus feel fulfilled in the game um and um you know I also think that uh the other big advantage of of stimulating through parallel learning other than sort of increase in engagement coming from the aha moment is um has to do with sort of the natural nature of it so here we have a kid studying for a test I guess and um most people don't love studying for test unless it's a subject they're super interested in um and honestly like I've learned a lot more about some topics just sort of being inspired to go into a Wikipedia bender and you know like I not too long ago I did this giant Wikipedia Bender about like you know you know um

soil bacteria you know biochemistry and it interaction different plants which by it's going be my talk next year at GDC um just just tell Colleen if you're interested and we'll you know they'll give me a little pull there um but um but basically you know this this random parallel learning is much more like how people prefer to learn which is when they feel motivated to do so so if you remove those blockages you get there um I'm going to talk in a moment about how legue of Legends has pretty strong random parallel learning but the audio and the video is all jacked so um you're just going to see animated video and uh you know if you really care I guess I can try to get you the a one later but um anyway so um in League of Legends you know so there there's a bunch of different skills you can Master me the the obvious ones would be like for example you know if you're a jungle you might have jungle timing or um you might have like basic teamwork and and ganking skills or judgment around if that was even a good decision to be pushing that Tower or um decisions around you know timing taking objective versus pushing a tower or um you know here we have Baron and then the decision to counter gank the execution within the team fight of

you know how do I dodge the skill shots that's like you know dexterity based um Mastery you might have um you know um skill shot avoidance same thing um judgment around Rich characters should be focused fired first sort of presence of mind and not get stress Trin there's like a lot of different things like one game maybe get better at lasting another game you just get better at not overextending and I think that um you know as players master of course they're achieving Victory more but um you know I I think fundamentally that um you know mobas have been so so sticky with so many players because they offer so much random parallel learning um to players that it's just you just always feel like you're learning something getting better so um we still haven't talked about mlex very much so I'm I'm going to switch that now and um sort of articulate how Rog likes Implement um their design in a way that stimulates random parallel learning and and and Mastery in general um and I'm going to do that by just providing a few design patterns um that I think are relevant

so um the first I want to talk about is this idea of hero or zero and um you know this is sort of the name I came up with to sort of describe this Confluence of like you know hardcore consequences perade death you know you made a bad move you're dead um combined with sharp sharp in session progression where your starting character might might grow and power a 100 or thousand X by the time you're at the end of the game in terms of actual you know game Effectiveness um and if you if you do this Hero Zero pattern in a game um you you you get um Bor me give me advice here but you you get a couple um you get a couple benefits um the first is that because players are expecting failure um they start to get in this mindset of uh looking for the positive and the positive in Rogue like and games like it is that you learn something or that you made progress that you haven't made before which is related to you learning something um additionally um because failure is the default um players uh players start to remember the Time moments where they done they did

something cool or got farther than they did or or really um or or beat the game for that matter um and that that contrast is really important and and players really fixate on the positive not the negative and I'll go into that a bit more in a second here and and lastly I think for game designers uh we can be lazier game designers and no actually not lazier but you can focus your energy away from Play balance so um you know if you have sharp progression that implicitly means that as players are progressing know level to level um whatever whatever happened Play balance wise level before is kind of just erased right because the the progression is so sharp and exponential challenges are stronger your your abilities are stronger Etc um so if you as a designer make a Play balance error um you know one level it can kind of get erased the other side is that if players are expecting the game to be kind of harsh and kind of Hardcore and and setting them up for failure a lot of the time um they're more likely to just sort of forgive you and just say you know that's just kind of the way the game is rather than say like you know you design sock it's a bad game I hate this game you

know so I think it's it's really helpful um you know for design and it helps us do crazier more creative stuff as designers when we can Embrace this and um a game that I think does this extremely well is um Binding of Isaac which I I saw a number of you have played um but for those of you who haven't and I would recommend this game I think it's a a really really terrific uh indie game um it's um it's basically a dungeon crawling RPG using kind of the the original Legend of Zelda combat model with a bunch of modifications of course um where you descend through a bunch of levels steadily growing in power to defeat first your mother and then eventually Satan and it's based on true story lifetime will be airing the first part of the miniseries in April um and um so I've had playthroughs of this game where at the end of the game I had like just like nuts DPS and like I could burn down like the hardest boss is like you know in literally seconds when usually they a minutes long fight um towards the end and um you know and I've had sessions where I'm like limping

along and things are going really bad and you know I either just made a stupid mistake a diet or I got a really lucky break and got back on track and went on to have a great playthrough but you know as a player I only remember the runs where I like made an like an awesome like recovery or had like a super super epic like Victory where my character power is just off the charts the end I don't really remember like the gazillion other times where I died and I you know there's a there's an achievement I think for a number of kills or number of deaths and I I'm pretty sure I have the hund 100 deaths at this point um so um you know that's that's you know that's good as a player but as a designer like I know those negatives exist and it's fascinating that players really fixate on those positives and don't fixate and generally get frustrated the negatives once they get into the mode of these games and and that's really interesting and I think it's something that we really should understand and leverage because that's that's um it's really interesting because the high moments are what are what seem to work here and um you know to the extent that you're interested in implementing this in in your own game or or Rogue like for that

matter I think there's a couple things to bear in mind first is you have to provide players a real ultimate power fantasy and by that I mean um you know you need to be able to get superpowered in the game and then you know enjoy that moment if it's if it's kind of soft it it just isn't there secondly you should ceremonial lie things and what I mean by that is um ceremony is to call attention to same point you have a marriage ceremony to call attention to the fact hey this is important you know right so you add ceremony to games tell players hey you killed this boss for the first time that's important congratulations or you beat the game for the first time or you've never done this much damage in the game if you can add ceremony you can call it to players when they learned something or did something cool and that helps them feel those High movements and Savor them and lastly um you know I don't want to go into too much detail on this but I think um it's very tempting to mix strong out of session progression with strong in session progression so you might have like a meta character that's leveling up and then you have the in session you're also leveling up um that's typically pretty hard to pull off I'm not saying it can't be done because you know most things can be done but I think it's pretty tough um and generally

the attempts I've seen to do this well like an FTL which I'll introduce later um generally leverage a lateral progression approach where um out of session is one type of progression that's not power and in session is power or or vice versa the next design pattern I want to talk about is this idea of diverse tools and and causing players to have to make do um and um generally Rog likees players have um you know diverse challenges and they have diverse tools and those diverse tools that are often the imperfect tools so there might be an Ideal tool but a lot of time a rogue like will force a player to deal with a you know a B level or C Level or you know no tool at all uh to deal with the situation and you know um there's there's there's a couple there's a couple benefits to this um the first is that um you know by designing for this you really encourage player activity and reward you you basically um get this designer player contract going where you basically tell players look if you're a little more creative and thought a little bit more about was available you

might have been able to make it through that um and and additionally like um you know by by sort of incenting players to learn these tools by putting in situations where they have to use these tools you actually get players to sample in a way they feel excited about a larger percent of your content I know we've all made games where it's like you know you you put all this TLC in and made you know not DLC TLC right that's before released non-monetized as opposed to after um but you put all this TLC in you had all this you know awesome content and um you know and you know it's great and then players like oh um you know I'll just use this axe because it's really overpowered and I don't really care about that other stuff and you know all that effort goes to waste and players don't enjoy it they probably would have enjoyed it if they tried it well if you Embrace this virtue of not letting players have too much consistent availability of tools um or having to make do or make hard choices about the tools they do have um you know they're going to enjoy a lot more of the content and ultimately Master more things and have more fun um so I didn't have a screen cap for

what I want to talk about here because the game is text based does anyone know what this creature is close cockatrice yes very good okay thank you so yes this is a cockatrice which uh because you know it's R you know what game this is from yes from net hack yes and net hack yes and and R is turn you to Stone which is bad because generally lethal um it's always lethal unless unless for example your Stone Giant which I'll get to in a minute so you know races are really deadly and really angry and you have to deal with them and um Ned hack provides players a variety of ways to deal with this go Google it after this after this talk if you're curious but um the best tool honestly is to just polymorph yourself into something that is both stoning immune or Stony resistant technically Stony immune and extremely dangerous so you know like a stone Golem would be a reasonable Choice um of course that's a pretty powerful tool and um the first time you encounter these it's not you know that likely you're

going to have ready really high quality access to polymorphing um so you often have to make do and making do could be things like I don't know use your consumable wands which are really powerful but limited um or um use some ranged weapons some sing weapons or whatever or um you know you could lure the trce into a pit trap and then just whale on it um you know or you could take your chance and go hand to hand and just make sure you have an an antidote so like you could carry on a stone to flesh scroll or I guess if you eat a lizard corpse that counter measure stoning as well it's very strange but um the point is there's a bunch of different tools you could use to handle cockatrice and you probably won't have the Ideal tool but expert net hack players know all the tools know exactly how to use them and feel really good about the fact that they've learned to do that and um you know in terms of applying this design pattern to games I I think the most important thing I'll actually do these out of wrote them I think the first is that tools need to be limited um in some way and by that it could be that they're you know inconsistently available it could be

that you can only pick you know a or C and all of them have different uses are important it could be that there's a lot of cooldowns it could be that you know uh you know they're they're just not provided to you when you need them by some mechanic so I think that's really really important and can force players if it's done in a way that doesn't feel arbitrary and lame that feels just natural to the game it it players naturally get into this like sort of making do in creativity mode um I also think it's important that challenges inherently do respond well to multiple tools and as you go through you can think about like you know if the tool is just too powerful broadly you really should think about like is this actually decreasing gameplay potentially um but that being said I I don't want to encourage you guys to go into your game and you know over Nerf a bunch of tools because if you don't have super powerful tools you're going to undermine a lot of other aspirational elements of your game like if you want to have a Hero Zero pattern you certainly can't do that with weak tools so I think it just has to be a balance of availability that organically causes players to have to make do the next design pattern I'd like to talk about is is um an idea called

competing objectives and strategic commitment and you know I had a tough time finding a name for this and and and this doesn't maybe it doesn't seem logical to group group these together so I want to give some background on this um when I was building this talk I was trying to isolate what causes Rog likes to have such like really rewarding like sharp Rich interesting decisions so you know Rog lies are able to offer players stuff like I don't know like um lose all ability to heal uh in exchange for a really strong um vampiric effects you you do damage to heal or you know lose lose half your maximum health permanently um and get like double damage permanently and and like in a lot of games people are like that's interesting I don't really know if I want to take that on this play through but in realog likes people like jump on that and are super enthusiastic about it it's kind of interesting um and and I think Rog like set this up with two pieces the first is they offer really sharp competing objectives where the way to pursue the objectives is different so for example that might be short-term survival which is generally

you know Health Food things like that um verse your long-term power objectives which are are inherently somewhat deferred right um or um you might have like two alternating character builds like you might have two Mage builds let's say fire and ice and in between is just kind of not very effective you're just not optimizing um so what Rog leges do they they have these competing objectives and then they dangle in front of you really really really powerful stuff that often will really make progress on one objective but actively under M or come at the expense of or um you know just not help the other objective and as a player you go before you get this talized upgrade you might have like a strategy you're going to do or a preference you want to do and you get this hm am I still strategically committed to my old strategy or does this war a change and also which objective is actually more important to me and you know when players um when players you know get get into um this mode of sort of assessing like in detail um you know which objective really is more important which strategy they

really want to commit to um that's a really like rewarding kind of thought exercise to go through and players feel really good when they finally make the decision and they feel like doubly good um you know because that's you know that's strategic judgment as opposed to Choice comparison so just a backup for a sec um I think in a lot of RPGs you know you'll be going through the dungeon and you know you have your sword that does like 10 DPS and a new sword drops and it's like well it's you know slightly slower cool down but it's you know 50% or it's slightly longer swing cool down but it's like 50% more DPS yes so you know yoink I'll equip that duh right that's not terribly interesting like that feels good but it's not really interesting um these kind of like nuance trade-offs and forcing yourself to understand what you value and then make a bet potentially on on on that choice um I think just is more interesting and when it does work off it feels really really good because you feel that you were like a smart masterful player who understands the game and you know had a good measure of your own abilities and the risks of the game um and you you know I think FTL is

um has a really good implementation this and I just want to introduce it really quick for those of you who haven't played it also a game I I highly recommend playing um you know both I would play it first without the expansion then try to get with the expansion pack after you've digested some because those are kind of I guess you can disable it but they're they're different experiences but in FTL you play a ship fleeing the rebels you're the Empire it's kind of a little bit of a role reversal um to deliver critical knowledge to your Fleet um and win the war but but honestly you're just trying to flee the rebels and then blow up their mother that's really what the game's about and um as you hop system to system you encounter varied challenges um kind of in a rogy fashion and you sharply progress your ship from you know kind of a garbage truck into a really powerful Battleship so it's very rogate kind of progression and um you know I think FTL does a great job of forcing players to weigh um various goals and determine which one is more valuable any time so for example at any time you have to worry about Fuel and damage if you take too much damage uh you die and you lose right and if you run out of fuel you

incur a lot of short-term costs that end up being kind of midterm costs in some ways to deal with that um and you also have opportunity cost of exploring less at the same time though you know you need to be building up for killing the enemy Mothership at the end and and you also are trying to have a good smooth progression in the midterm um because different different ship builds have synergies so you can't you kind of commit to a direction if you if you deviate too much you lose a lot of Effectiveness um and that's bad too so um you know a situation where FTL you know leverages these um competing objectives into a really interesting decision right where you have to strategically commit would be if you're say you've been um farming a lot and you've accumulated a lot of scrap which is a game currency and you pull into a shop and at the shop you see um by way let's say you have like 10 like let's say you like five hit points you're pretty close to done and you're pretty low in fuel and you see like an awesome item for sale like weapon pre igniter which is is like a really um great thing to turbocharge a beam like a sorry a

laser build or um maybe you see you know the cloaking device which is really critical for being the mother not critical but as close to it as you can get and um you know you see something like that you really have to go through that thought process of like hm like if I pick this thing can I make it through to the next repair opportunity or should I play it safe knowing that I might not see this item again for the rest of the game and and that's that's really interesting um just as a side note one critique I do have of FTL is I think that their scrap system in which you kill ships and get scrap which that can upgrade your systems actually provides too much consistency of tool availability to the player and in some cases can really mute the variety of experiences in FTL I think the expansion um you know adds a lot of other variety to the game to offset this but that would be that would be my main critique even though this was you know probably my favorite game of that year um Bing of ISAC also does a great job of this um you know what you're looking at here is the devil room which is a shop where you can exchange um permanent health and exchange for upgrades and um so what the game would is

essentially doing here is saying hey um do you want to take a short-term risk to your Survival on an otherwise somewhat successful run um in exchange for if you pick the this red thing on the right here the little squigglies um you know a large increase in DPS and Mobility um and you know if players players choose to do that and by the way for context to get to double room it's sometimes random chance but usually it's you executed perfectly on the level and took no damage so you're usually feeling pretty confident at this point um but maybe you wouldn't feel as confident if you had a lot less Health um but you know players are basically given this challenge here and and you have to make a hard decision um so that when you do if you Nerf your health and you take this damage upgrade and then you succeed you feel really good about it but like I said before if if it fails like you just kind of forget about it and and and you move on which just this this whole pattern is very synergestic with the Hero Zero so in terms of instilling um you know this decision quality in games I

think there's a couple important things one is it's important to clarify tradeoffs so players like mechanically know what they're getting into so simil example of this is like Binding of Isaac I don't understand why they don't tool tip the items in game in some way like um because everybody just plays with Wikipedia open until they have a really high skill level in the game and just recognize things um I think that you know that kind of information helps players get in the Strategic space and out of the sort of um I don't know I mean I guess there's some mirror to the exploration thing I just I just don't really agree with that design decision um I also think that um it's important that there's that your goals are balanced and I'll I'll talk to this later when I talk a little about civilization but um if you have different objectives um that that really do compete and where decisions tend to favor one over the other that's going to tend to set you up for interesting decisions um and and coupled with that you do have to offer sharp decisions otherwise if it's too gradual players just can't really make a strategy switch and and lastly um I do think there's some benefit of the future is a little bit ambiguous around like hey Will I See

This upgrade again hey what are exactly the enemies I'm going to face I'm not saying it should be random but a little bit of variety there can uh enrich the decision space uh and the last pattern I want to talk about is this idea of uh the variety is superior to Randomness and and I put this in here because you know I ass seem like okay I'm do to talk about Rogue likes people are going to assume I'm to talk about random procedural content because all Rog lies have or most Rog lies have random procedural content um key key word there being procedural which effectively makes it less random but um anyway uh you know my position on this is that um you know players players love variety uh they love having varied experiences and varied experiences really set up you know random parallel learning and master a bunch of different things and you know keep it interesting in other ways as well um but not all varieties created equal I mean some variety is just lame or boring or um hard hard in a way or impossibly hard as

opposed to you know challenging um and I think that when we go full randomist designers we're missing out an opportunity to um basically make the game better and and I'm just going to be really simple case in point here um which is traditional you know DND so um if you're a DM and DND you know the party kills some big beasty and you know you're like well what loot shall I give and the the dungeon Masters Guide conveniently gives you a random loot table that you could use and you so you roll the random loot table and you know your first roll you get okay military pick plus two um who uses a military pick plus two very few characters um so if you gave the party that they'd be kind of like oh cool loot we can't use I guess we'll be selling that you know it's not not super exciting um you know or if you roll it you might roll something they weren't really expecting or asking for but it's interesting you might roll a rope of climbing right and there's things people could do with a rope of climbing that be interesting and I know in some versions of D and

those are very different power level items so forgive me for that but um you know the point is like that's that's an example of like using the random table and then curating as a designer to produce a result that's actually interesting to players so um now that I've gone over um what I think are some of the more interesting and obviously I didn't cover all the Rog like patterns but I cover the ones I find pretty interesting um I'm going to talk about some non- Rik games and how I think there's some similarities how they perhaps leverage some of these patterns effectively um and also just go through how one might apply um rogl patterns to them to enhance them somewhat and um you know again I I just want to reinforce like um one message I I want to have in this talk is just that being genre agnostic as a designer to a degree and really sampling widely um will make you a better designer and just to refresh because that was a lot of material pretty quickly um the patterns I did go through were random parallel learning which is you

know the idea that learning a bunch of things all the time in the order the player desires is more engaging I talked about Hero Zero which was just this idea that having really memorable intense high points um with an expectation of failure most to the time where you have a lot of low points is generally a pretty favorable pattern um next I talked about diverse tools making do which is just this idea that by limiting tool availability to players um they learn to make do and be more creative and ultimately Master more things and engage with more content I talked about competing objectives and strategic commitment which is this idea of if you have competing objectives you set up really interesting strategic choices that are motivating the players and lastly I just talked about the idea that variety is superior to Randomness Randomness is just potentially a delivery vehicle so um wouldn't be a GC game talk without Dark Souls Dark Souls players here yes that's it oh my God okay all right oh okay well it seriously it it has this reputation it's actually really rewarding game to play even if you only play it for 10 or 15 hours but um you

know dark dark souls uh Dark Souls um does a great job of of showing players they mastered things you know I remember I was playing Dark Souls and I had the hardest time with silver Knights um I don't remember the level they're on but you know they're pretty they're pretty hefty and they're like 2,000 soldiers they going to kill them oh and by the way if you haven't played Dark Souls it's like an action adventure game sort of like you know the like an Ultra Hardcore version of God of War with significantly more character customization I get maybe that's how I describe it um anyway um you know it's um so I I had to learn to beat these guys and and you know I'd fight them I lose a lot of Health I lose more health and I was having to use my healing potions and slowly getting through it eventually barely got through the section of the game with these guys and you know some hours later I you know I think I of gameplay later I think I slept in between um I encountered them again because I had to go back to that area for something I missed or whatever and um I was just like punking him down like it was bam dead bam dead like you

know no problem and um I felt really good I was like wow like I perfectly mastered this um I felt super sweet and I think because Dark Souls like has this reliable single player content thing going where you can backtrack like it really reinforces Mastery super well so um one of the features I like about Dark Souls you know you you have character classes kind of I mean it's more any anybody can do anything really but I mean you have a starting class and and one archetype you could go with is is the pyromancer um where we get all sorts of super powerful fire attacks which are um generally pretty powerful especially I mean they're they're powerful in general but they're they're pretty good early um and um and and they really can tribalize some content um on a relative basis as Dark Souls goes I mean you know obviously if you don't know what you're doing you're still going to die but it's you know notches down the difficulty from like a nine to like a six in a lot of cases um until you encounter um K which quag I guess is how it's pronounced um who is a giant fire spider who happens to be fire immune um which

is a problem um so as a pyromancer um you know hopefully at this point you've been making a little effort to learn your use your weapon you probably have um but you're probably not great at it and you're now in the situation where you have to make do with like not your choice tool like well I'd rather burn her down trivially with my pyromancer spells but I guess I'm going to have to learn to use my weapons better so maybe you go byy a bunch of consumable um you know arrows across bolts and try to fight her that way or you know maybe you just try to man mode and get really good at your melee weapon and and just learn to take her down like a normal um you know sword and board character would or maybe you know you want to invest some some souls and just buy a bunch of consumables to give your um the pine sap or whatever to give your you know weapon lightning damage to kind of burn her down quickly um but the point is like you know the pyromancer will expose you to situations where you really do need to make do and also like part of the replayability of this game is as you play additional playthroughs most players will play other archetype builds

and in doing that they're going to expose themselves to a totally different set of challenges um because a game just has um so much nuance and depth to the basic combat model that they really enable that um you know I don't I don't have um I I want to be really cautious in how I position my criticism here so um Dark Souls is anything but random parallel learning I mean that's not quite true but it kind of it's really not into random parallel learning right like you know you can do bosses of order but there's kind of a better order to do them um and and you can definitely get in a situation where the boss you pretty much need to clear um you're just not you're just not getting it and you're not strong enough and or you're just not able to get your player skill level place it needs to be and you know you just go and farm basically until you're powerful enough to pass it um and you know that that works okay for Dark Souls um because of it shtick of being like so hardcore and you're going to die a lot and all this other kind of stuff um but um you know I think it only works

for Dark Souls because of this expectation along with the fact that single player and single player games put way less frustration pressure on the multiplayer games so you know I I don't think it's best practice if you're not making Dark Souls and and there's a lot of ways I think you can make a just as fenly difficult probably more fenly difficult game um but still just as rewarding in the absence of some of the bottlenecking that Dark Souls does present and again like I love this game I'm totally cool at this I'm just saying holistically next g i to talk about again is um League of Legends and I think there's a lot of design patterns in mobas that that are quite similar to Rogue likes I mean just to the basic level I mean you have diverse tools and diverse challenges and you're kind of figuring it out right so um you know game over game players are often self-rotating their tools by trying different Champions or builds um and additionally your opponents are varying game to game so um sometimes you're going to have um strength sometimes you're not sometimes you're not really going to have the correct tool to deal with an opponent um at least the Ideal tool and um you know

when you're fighting against opponent League of Legends you have to determine both your play style and you know how you build against them like what skills you invest in what items you buy Etc so as an example if you're playing cinged on top um you know cinged is the is this um Melee character who's very tanky and like leaves Trails of poison behind him um you can do pretty well against some of the melee characters like Jack for example you do reasonably well against um but um you know you really don't do as well against like I don't know teimo or kale or something like that um and um and and additionally like so so playing against those characters you might shift your aggression level you might shift your itemization for example against Jax you you generally go armor against um Teemo and kill you you'd be looking at um I'm not actually sure the best way be kill but Teo you certainly would be looking into magic resistance um and you know all of this sort of it so also on top of this you're probably not going to encounter the same match of game over game right you fight team of one game and you play Sing top again which is not a given um you might encounter a totally

different character next game so what this means is there's a lot of parallel random parallel learning opportunities where you might learn something important from your defeat to Team One game and then seven games later try something you learn from that defeat and win and feel really good about it additionally I think League of Legends um sort of has elements of hero or Z and obviously Leal Legends doesn't have perade death but it does have sharp consequentiality to actions um and it does have have um sharp progression and um you know getting getting a really ultra high moment isn't super frequent moderately High moments are but ultra high moments aren't so you know you're always going to remember that game where you did get a pen kill and just you know Ace the entire enemy team um you know or you're going to remember that game where you know um you know you basically you basically like made a dramatic comeback um from behind because of great play on your end and um you know that that that existence of the sharp progression along with this other skill-based you know Mastery elements um really allow these

high moments to come out in League of Legends one criticism I'd make um you know using these Rog like Frameworks um is that um so you know we discovered with Aram or I mean our community made Aram which was inspired by some earlier mobile work but um you know our our our implementation Aram you know gained a lot of appeal and um the reason I think it's appealing is we provide a way for players to be forced out of their comfort zone and Leverage diverse tools and make do in a way that feels legitimate um and they would never opt into an ordinary League of Legends game and I think there's probably a lot of unexplored ground there and we've tried to do some of this with other game modes and I I really hope we can continue to find ways to get players a little bit outside their comfort zone um either in the primary game or in modes um in the future because I think that ultimately results in in more Mastery potential like I also love to find ways to have players for example you know play other roles than they're used to more often um the last to talk about is Civ and and of course I mean Civ is you know one of the greatest games of all time um

and you know certainly the basic game Loop does have some similarity um to Rog leges in terms of you know diverse tools diverse challenges sharp progression so and and so forth the Hero Zero thing going on but um you know although I think like some of the offshoots like master of magic probably embraced that a lot harder and got their appeal from that but um what I'd like to talk about on St the most is this idea of competing objectives and strategic commitment and you Sid Meer um really pushed the whole interesting decisions framework which has a lot of similarities and concept to this um and shares some commonality to the to the Rogue likes um and an example of where I mean s a lot of the gameplay is is in in the city building and there's certainly a lot of interesting decisions here around like um you know build order and things like that but a lot of them are masterful and and and I find that as players get better and better at the game um most of the most interesting decisions at the city level start to be sort of Beyond just the city placement start to be like you know Wonder strategy or or holistic military

units ver not kind of decisions um and and and the problem is that a lot of the best decisions in Civ are decisions that are easily aligning short-term and long-term objectives where look you got have more popul City with more infrastructure it's just kind of better um and it's kind of a no-brainer because you know you're relatively safe right now and um you know that that lack of tension in the decisions available between the short short and long-term decisions um I think is a little bit of a missed opportunity and I I think part of this is also that Civ by its nature just doesn't offer as many singular large decisions that could potentially nudge your strategy from like you know like a economic like a economic strategy to like a war strategy or something um and and there's a little bit of it there but um you know I think I think there's missed opportunity and I think like part of what was really great about Civ 5 um is there are some things like that that do that like for example the um you know natural wonders um if you encounter them early can potentially cause you to change strategy and you know double down on um Faith or something um you know and

um you know there's also I mean um there's couple there's a couple other things like that in general but I think that's a the the the alignment of incentives is kind of an issue um I also would say that Civ could probably do a better job of forcing players to make do with tools um and you know I think moo did this really well through a simple Innovation which was and by the way m m was a sort of space themed version of Civ um that's kind of a simplification a it's also a really great game and uh one of their Innovations was that when you're choosing Technologies um you have to unless you have a racial that allows you to get around this you have to pick one technology and some races don't even get to pick they just get one of three so if you're looking here like you know upper right there you could get an ion drive move Shi faster you could get ion pulse Cannon which is a a pretty strong late early game kind of um Shield disabling or sorry ship disabling weapon be very efficient um and shield capacitors which I believe increase your Shield r recharge rate nobody ever gets those so um so there's two good options there but

um you know that that really forces players to make do where they'll pick one and they're like oh my God I wasn't anticipating this problem I don't have the right Tech how do I deal with it um so you know of course modern sa 5 I think has developed into this a bit like the Venice city state forces people outside their comfort zone a little bit makes them make do in a different way so you have to Venice can't expand so that's a different set of challenges different set of strategies um you know addition strategic resources can sometimes do this because you can't see strategic resources like iron and coal and so forth until you have the tech correlated to them um so you can definitely end up in situations where you don't have an important strategic resource and that can be interesting as well um but in general you know I don't think um making players um make do with things as something that is fully explored or leveraged at this point so that's all I have for non-r like uh game examples and before I go into QA I want to leave you guys with uh two suggestions um um you know first to the extent that your game is based on a Mastery loop I think you can just

systematically go through things and just ask like hey is there a way we can instill more opportunity for Mastery in this mechanic in a way that's consistent with our other design values and often the answer is yes um and and and then more generally could you apply you know some Rog likee Mastery approaches to it maybe that's maybe that's a sticking point that that might um maybe that's maybe that's a mechanisms unblock yourself there um and then as well like I would ask you guys again um to play broad games like play games very broadly that's that's how we get better at our Craft um that's how we spur Innovation that's how we make you know ever more engaging in fun games um so you know if there's a game that that you know is popular and and you don't really understand why it's popular you just aren't really interested that might be a really great game for you to play and check out so um thanks again for coming um please don't please don't forget to fill out the speaker rating forms that will come to you over email um GDC needs the data to pick speakers and if if you learn thing I'd appreciate you know you taking the time to do that um so thanks

again for attending my talk and spending the time and um I think the ca will direct us for questions are we going right across or we okay so we'll do 10 minutes in here and then we'll afterwards if there's still questions we'll go after so questions [Applause] hello hi Tom how you do um good uh I have a question about um I think back when the hero and zero uh section you mentioned about being careful not to mix up uh different kinds of Mastery uh oh you're talking about in session out of session progression yes exactly okay yes right uh just just just um just to clarify my question there are certain kinds of Mastery that uh I would say are kind ofp of the game say situational awareness or um uh the ability to

process more information that that sort of thing and I'm wondering uh should we be aiming to try to recognize those kinds of improvements within the game or is it more of a responsibility of the individual or the community to sort of like recognize hey actually my discipline is you know I deal with failure better as a person that sort of thing um I'm I'm sorry um Philip you're you're asking um if there's so you're asking about the heroes your you're asking more generally I'm not more generally I guess it was just a point you brought up on that one slide but really more General it's yeah I mean I think I think certainly there's there's tons of different forms of Mastery and um different games are going to offer different forms I mean if you have a turn-based strategy game you're not really going to have executional like dexterity Mastery um so and and and I mean you could even argue um um that that there there are some very very different ones that would be foundational with their games um do you have a specific specific one you wanted to ask about or well um let's say the ability to deal with failure as a person

right you know it's like you lost and then you you could either be depressed or you could say that's a learning opportunity yeah that's something that's that if you that could be mastered and absolutely yeah I think a lot of PV games happen that as well um we should talk more about it afterwards sure um yes yeah so um one game that often gets mentioned with the Rog likes is like the Fortress mode from D Fortress uh it has a lot of like commonalities with with Rogue likes because of losing and all that um yeah and especially making do like he said um one thing though that I it's different at least from what you were talking about with Road likes is that it's not so much that you remember the times when it were really well you remember the times when it went really badly um but out out of like a positive way because it's fun to lose for some reason I was just wondering if you have any thoughts about like analysis of that or if you played War 4 I mean I I I'm actually unfortunately um that is so that so for my game that I need to go

play that I know is popular um that's that's probably my game um I know a lot of designers that Riot are are fans of it so that is on my list for sure um I would without playing the game it's a little bit difficult for me to speculate but I would speculate by saying that um if something's really really interesting that can be memorable too um and that just distinctive so the traditional rug Lakes are geared for distinctive high points but there's nothing to say that you couldn't gear a game for distinctive um Oddities or you know low points as well and potentially it sounds like they've accomplished that I I don't see a reason why it's not it just seems easier maybe to go with the high points but I'm not saying you know anything's possible and that that seems logically consistent um yes hey uh I'm wondering if you can speak any about finding a good balance between um I guess implicit skills versus explicit skills like in Dark Souls you get better just by fighting yes you could go be explicit and add 100 to your strength and just

start destroying everything yeah so you're talking about you're talking about like if I upgrade your character and that mechanically like I give you if I give you a million souls and you go spend it right so um I think I think actually a mixture can be really good because um certainly I mean World of Warcraft happs this a lot where it's like you know you're doing the boss and you know you're getting better uh but you're not quite good enough and then you did a bunch of farming and then that pushed you over the edge and that still feels like a victory like um I think a mixture can be good as a pressure valve for frustration and and actually I'd argue that for any game where um you're expecting a player to be confronting the same challenge like repeatedly with the same with a similar toolkit um You probably do want some some progression as a pressure valve thanks yep yes how thanks for the talk um I'm Jonathan Palmer um you talked about looking when players are given diverse tools they have to look for Creative Solutions because they don't have the best tools and I think you sort of missed a good opportunity to tie in another element

which is a little bit reminiscent of The Stanley Parable talk earlier where he mentioned when you don't explicitly point at things in your game and you have them hidden players feel more inventive and they feel that greater degree of Mastery or individuality when they stumble upon it and by having those diverse tools players learn your mechanics more deeply and they get that aha Mastery moment and I think that was something that that fit very well in your talk but I I I agree with um an earlier version actually did have that and I had to trim it down the example I had was um so I totally agree with you was like Binding of Isaac has this this thing called the the blood donation machine where you can bash your character in and take damage and get coins and like people played for while know that like oh if I have an invulnerability item available I could use that while doing that and take no damage to get coins like that's those sorts of examples I agree I think that kind of un deep understanding of the game world and the ways in which the mechanics and system be leveraged are really rewarding when players can get to them absolutely and I think you know if we're making deep systems we're going to we're going to provide those opportunities for sure that said and

thank you for the cockus mention it was my favorite weapon with gloves it is a it is a good weapon you can bash mobs and net hack and tourament statues but then you can't eat them unless you can eat rocks so yes all right thank you some characters can eat rocks so I was wondering about a motivating a player to play uh a game with rock like elements that that it's very challenging you you you may die a lot you you you may run into the wrong decisions you may F face consequences and at the same time making the player not feel overwhelmed especially with entry entry players because uh previously if you felt overwhelmed by a game you were like well I already bought it so either I traded in or or I be it so the motivation was there for trying to get around the game but in this case uh the game might be free to play so there may be not much motivation to continue so how will you

deal with motivating entry-level players to overcome the challenges yeah okay so the question is given that roges are really hardcore and are potentially really demotivating until people you know quote get it unquote um what do you do I I would say there's two best practices are leveraged one is um you make the early content just engaging and interesting and you that could be narrative that could be humor it could just be the game's really interesting and fun um you know the basic very basic levels I think the alternative is you offer some very soft progression um unlocks could be a really way to do this good way to do this that are that are potentially not power progression and thus don't run into some of those out of session in session dissonance um like FTL does this with ship unlocks um I've been playing uh dream Quest on my mobile phone that's pretty fun actually um they do a little bit more power progression but still like oh you're playing and you're getting unlocks and that feels good so I think you know just as long as you the ceremony like that I mentioned with tie back in if you're ceremonial ising points or rewarding points um

people will tend to you know get through it better um how are we doing on time CA five minutes okay cool next yeah hi uh do you have any ideas uh or examples about how rogo like framework would work with free to play business model um I actually haven't to be perfectly honest I would say that um you know in general a free-to-play model um could be appropriate for any game that's long-term engaging and Rogue likes are so I think that's plausible but I honestly haven't thought about it deeply sorry um next question hi I'm curious about your thoughts on horizontal versus vertical progression in the sort of out of game meta progression like a game like FDL where you get a new ship that lets you play differently versus a game like Rogue Legacy or dungeon mans where your character simply gets more powerful in a direct way yeah so um you know when I going back to the principles I was

talking about here um you know so the hero or zero pattern in particular um is predicated on on fairly precise uh control of the challenge difficulty and the player power um so you could combine them but you get challenges there um the other piece is just you know to the extent that those out of session progression fiding tools you can also run into this making like undermining your virtue of making do um so so what I would say is like you know I think Rogue Legacy is an awesome game it's especially awesome considering the timeline it was made on and and everything I really like it um but I actually think that while it's an extremely fun game it it doesn't really encapsulate a lot of the core Rog like elements it's it it encapsulates some but it doesn't really um the diverse tools you know you're kind of incented to just rroll continuously until you get your clear kind of um kit and um and then additionally um you can often end

up in situations where you're basically farming reliably because your power levels so far ahead of the content so I I think it's a great game I just I just think that you lose a lot of those the values I've articulated here when you have too much out of session progression thanks uh next question all right so I have a bit of follow-up question to the previous one so um you mentioned that we need to be careful when mixing that in andout session progression for obviously that reason um undermining the make do aspect of of Rogue likes but um I'd like to just know for example League of Legends does have a little bit of that parallel progression in and out of session so do you feel that there is uh well and then again uh long-term progression is a is a big driver for long-term engagement so do you think they're completely incompatible yeah so okay so they're not completely incompatible like all things in game design um out of session is session progression would just be a trade-off you're you're throwing against each other and there's ways to reconcile that so I think strong out of session

progression is problematic but in the case of League of Legends there's a couple things going on first um the the power progression that players do undergo from 1 to 30 is primarily a midterm onboarding solution it's not a long-term thing um the customization that emerges from that is a long-term strategy formation thing but um there's almost like background radiation right like when you're level 30 you're playing level 30s who have the same amount of unlocks as you so um it's not you know we're sort of balancing the game to that just in the same way that like hey like if we increase all health of if we increase the health of everything in the game by 20% we wouldn't be changing things that much or if we if we increase DPS and health AC crossb 20% we have exactly the same game so um you know I think you get you get a little bit of you get a lot of a pass there the other thing is you know like I said it's it's really about the amount right like and if there's a lot of out of session progression you start to risk the ability to do really good sharp in session progression but if you do

moderate or small amounts of it I think it has been shown that you can be effective with that um so that that would be my my response to that all right thanks okay sure next question uh I guess my question is actually slightly related to that a lot of the examples you have like Binding of Isaac or FTL tend to be it's kind of okay to write off the times that you fail and to focus on the times you have the Epic success but I'm wondering if you could talk about some of I guess go difficulties in games that have much higher level play such as League of Legends and how those kind of snap decisions and the elements of fun in deciding that aren't really an option you're expected to know the correct result for any situation um and how you sort of could still apply some of those design you know areas and but not have a running those sticking points um that makes sense uh right um trying to think how to rephrase your question um so so you know I think if you go back to the principles of planning like diverse tools and making do right like you could apply that by looking at a

character and just sort of asking like um you know are any of these actually a Swiss army knife because we know that's generally going to be bad design or um you know do are do I actually have B and clo plays in this frustrating situation because if I don't that's also not very interesting so a lot of I think You' normally frame that as just sort of conventional Play balance um on hero or zero you could also say like is it you know if we think it's desirable to have the ultra high moments um are we adequately supporting extreme character progression we actually had a point in League of Legends where we tapered off hyper progression too much in the beta and the game suddenly wasn't very fun um so we readded a lot of snowballing to the game because that was a necessary element for it to be a really compelling game as an example so you know I think um like I said I mean you can't I don't think you can literally interpret this stuff one to one but if you understand why it's um motivating to players in the context Rog like I think that you can then map it into into the destination genre and think about like okay well it's a master driven game and you're

kind of having this experience here and what might be the similarity so uh yes I was wondering if you had any thoughts on balancing social pressure versus um kind of a rewarding trade-off decisions so part of what makes Rog like so fun is you take this situation where I'm going to do something really risky and maybe on the outside might look a little crazy but it's all going to play out in the end cuz I'm building towards something really Niche perhaps but in multi settings a lot of times there's pressure to playb conservatively like there's a there's a known good or there's a known safe and um your risks are you know your failures are kind of much more penalized so how how can you seize on that right so you're saying that there's a lot of social pressure in games and that taking a ballsy decision wouldn't be accepted that's that's specifically sorry oh sorry about that um that's specifically is why true like just a straight Co-op multiplayer Port of Rogue likes really hasn't done super successfully yet because um you're trying to make these like Nuance

strategic decisions and hard commit based partially on your intuition yeah it's hard to get a hard to get a team jelled around that I you probably could but it's it's pretty tough um and and so I think the analogy to that like in League of Legends would be like I don't know like I'm playing jungalo deer and I'm supposed to play tanky kind of but you know because I'm I'm going Phoenix farm and kind of being tanky and you know and I I I just happen to get a couple early kills and I'm kind of looking my go tot and you know bot Lane just got a tower I'm like hm you know maybe I should get Trinity force and maybe maybe that would be okay to go damage build um so I still think there's some of those opportunities but you know like I said you can't you can't apply all these principles to every game I think the the right approach to do it is you know you take things to have Mastery and you you look for the analogy and ask if it does make sense and some it's going to make sense and some it's not because you know the Rob likes and I alluded to this have a bunch of interlocking methods that make all these work together consistently um and when they show up in other games you get partial combinations that are self-reinforcing and other substitutes

to to to bridge some of the deficiencies that are that are generated by having a different game context um we're we're out of time okay so um uh we're going to call this um we're going to go what room is it um catm 2002 we're going do 2002 um I will be walking over there momentarily