Level Design Workshop: Improving Tool Design Through Editor Triage

[Music] so hello everyone thank you for coming I'm Rob Ian storm and I'm here to talk about improving tool design who editor triage now Who I am in a quick nutshell so I'm from the Netherlands I used to be a game and level designer I worked at tech simian team 6 through games I did work on all kinds of titles like racing games to shooters to puzzle games to platformers which the most well known I was probably through and I noticed doing all that work that sort of time and money was being wasted by missing UX and you actually used solutions and level editors and I couldn't see anybody else doing anything about it at the time so I decided to do something about myself I started to do something about it myself and I became a freelance tool designer attended QA UX design for real time CSG and Sabre CC who are char to level editors for only unity asset store now I did a talk also at GDC about improving

level editors last year after which I got hired by IO Interactive and I'm now there as a full-time tool designer now so I gave that talk I compared source unreal unity 3ds max and the creation kit and I showed the good and bad sides of the UX that they contain and if you want to watch it later I leaked it on Twitter so you can grab it there there's no need for vault access it's on YouTube and my handle is at ry storm but I'll give a quick overview for those who did not see that talk yet so here's how good tools can save time save money and improve the quality of your games at the same time your tools health effects for developers probable work quality output I'm not saying tools are replacement a developer skill they are both important for example a few developers are good and your tools are ok you get a times 1 modifier on the quality your developers can output but if your tools work slowly or have many issues or they're just sort of like lagging a lot you might get a modifier of times 0.5 in which case you need to hire many more expensive and hard to find designers to make sure that

you can make something great but if your tools are fast and deep enough you might even get a modifier of times 2 which means that the your developers qualities are amplified to make your games an even higher quality and this is not some kind of new idea of course for example in 2010 Jim Brown the designer at Epic Games gave an example of what happens when you improve tools even a little bit so in this case 2.4 minutes a day times 100 employees 4 hours a day 20 hours a week becomes one year of your time on a project so fixing small things is a big deal so producers this is something important to look at if your developers are not doing it yet and level designers there's your facts to push change so the talk was very well received and it even hit YouTube I'm really happy about it but so one question kept from coming up which is I cannot hire 10 more tool programmers what techniques can I use to solve these level editor UX problems quickly cheaply and efficiently now instead of looking at the programmers I looked at the designers and that is where tool triage

comes in tool triage is how you make sure the correct features get created and worked on for your team and project for definition triage is a process for sorting injured people into groups based on their need for like or likely benefit from immediate medical treatment now we - as developers have to allocate limited resources and still get incredible results but our game developers actually triaging their tools and if so how are they doing it and what continues to use that these methods have and so to answer these questions I went around the game industry in the last years and talked to experienced developers who work or have worked at many different companies to find out what kind of triage process is they've personally used or witnessed and every single person that that I've talked to has seen or used different ways to triage tools but are unaware of what other developers are doing it looked like a mostly silent topic now these are developers from all over the place so all these logos are just a quickly identify how broad this research was it

does not mean that any of these opinions in this talk are part of these companies much like what you read a lot of Twitter BIOS I talked to them and they represented themselves now of course some people were happy to represent themselves for the sake of simplicity Mia's gonna say no one did and also for some companies such as guerrilla I did not talk to developers there but their developers talked publicly about their systems in interviews podcasts or GDC talks now in the end I found many different ways that many different companies do their tool triage but I'm not gonna say what is the best objective way to do it and the reason for that is the variables so many variables count when working to make tool triage better it's company culture country culture production methods publisher relations deadlines communication structures hiring practices etc for example on the last one hiring practices can affect what to do when it comes to to triage how many people are let go and hired affects effects that a lot like for example if you hire and let go on the two or three people per year and have your developers in a company for 15 plus years working with tools that I've been

iterating on for 30 years then doing a tool triage method that rapidly changes things a lot is probably not a good idea it's gonna cost time but if you have a huge amount of Studios all over the world with hundreds of developers coming in and out every year of course they need to find a tool triage approach that make sure you can change things fast and make things work directly when they join your company and also a thing is culture not just company culture but country culture different cultures have different ways of bringing up new ideas for example in the Netherlands we can be very direct when talking about or accepting or declining ideas we'll just say no in the US there could be a lot of talking to soften the blow of an idea not being accepted Americans might say that's interesting it means the exact same thing so this also shapes the way tool treeoche works at your company there is no one-size-fits-all approach as with anything you actually ate it it depends on your target audience in this case your target audience is your developers you probably know them best

so you know what works for you now there's a couple of problems to consider no approach is perfect but these were the problems most developers ran into when using any kind of tool triage so let's go through them before we talk about the actual solutions so we know what we have to solve first is time and money investment some approaches worked but they're very time intensive and money intensive to get a good return on investment canvassing of politics Cavatina politics can create sub optimal solutions or compromises if you have a flat hierarchy or an approach where anyone can triage you might not get the best solution because some people are simply better at social contracts than other a quiet person who does not talk a lot may have the best idea but be shut it down by someone who's way better at canvassing their ideas while those ideas may not necessarily be as good giant muscle multidisciplinary teams can create huge time investments if you have all kinds of disciplines talking to each other on a set time interval you will

lose time and simply due to haven't communicate between them programmers designers artists that serve they all have different ways of thinking and different ways of talking if you put those together and try to fight try to make them find a solution you might get there you might get not but you'll be spending a lot of time special meetings special meetings are kind of annoying as for everyone nobody likes to have a lot of new meetings so if you if you're using an approach that already that makes makes us like you have to do particular meetings every week like every month it might not be the best idea communication between non-physical teams communication between non-physical teams will be neglected over time I mean human beings are used to communicating with those in their physical circle so if there's a barrier of entry such as technology or time zones between teams then they'll that communication will be very clear at first but sort of deteriorate over time or maybe even just stop outright a faceless lymph system if it's not clear who is assigning what is happening with the tools and editor then

there's a faceless system which makes it very difficult for users to know where to request a kind of feature or a something that they want something that they want or who to talk to about problems they're having having a person a team or even just an email address that can help give a face to where you can go to if you have tool problems can really be like awesome for people a person so in the constant change a person to physically go to is better than a faceless system and a faceless system is still better than constant change now this is something that most people are actually doing which is that multiple people or different people over time are deciding what to do this changing of the gaurd creates a layer cake of different ideas and concept over time which is very unhealthy for tools because tools need to work in such a way that they complement each other that the UI new acts are the same so if a different person is doing the tool designs or two triage over every month there's a different person the end result will not mesh together as well

mid and low priority items you will almost never get to meet a low priority items the high priority items will keep on piling up and you have to continually fix those if you're using an approach that constantly quit that creates a list of high medium and low priority items then it can be safe to assume you'll never get to get to mid and low having a time investment to make so having a time investment to make these lists might actually be wasting your time and money you might as well be doing something that quickly gets rid of those and also it means that programmers are constantly seeing this usually store items going higher and higher and higher and they see how much they are behind and it just feels bad because they know they can't reach those okay so those are the problems to consider and I shored up them a little bit and added some info as well so at the bottom I have big team media team and small team so what that means is is this approach viable for a big team a small team or a medium team now let's get into the meat of the talk what are these different methods and what are the pros and cons of these methods so the ticket to Moscow approach

so this is an approach used by many companies it's really kind of the basic approach so this one can have big time investments can have multi teams special meetings constant change and fail to reach MIT a low priority priority items but can fit any sized team so in this case anyone can submit tickets and the gave it gathered in one place the tickets then undergo the Moscow method of must have should have could have would have to figure out low and high priority now of course a large amount of other priority calculation methods can be used as well but in the tool programmer simply implement Heights low but the thing is again you'll never reach low and high or low priority and high keeps piling up and it's difficult to find out who performs this priority check is it the users is it the tool programmers is it some kind of different team so it's very time-consuming to get those people together to do checks if you have a bunch of them another approach is the program our check up approach this is used by two companies that I talk to so this one can have big time investments require special meetings and can have constant change but it can fit any size team so in this

case you have weekly or monthly meetings and the two programmers and level designers meet together they then go through the levels in editor so in this case they the two programmers can check on how things are built using their toolset inside the editor so they can discuss what can be done more efficiently like for example show different approaches to use it might not have heard of like a level designer shows how they built a thing where he shows something any editor and so probably goes actually I have a way better way of doing this but you gotta go and this menu down here to this thing and then there's a thing over here that makes you do that they might not know this is a good way to find that out and they can oh and the level designers can also discuss what's difficult to create and difficult to maintain there might be specific sections of a level that are really hard to constantly redo and redo because the console it keeps breaking now they can talk face to face and solve these problems so it's sort of in-person triage the designpad Committee approach now this one is just by one company that talked to this one can have big time investments can have multi teams special meetings constant change and fail to

reach metal or pure items but again could fit any size team so in this case any requests that the users have go to a committee first and this can be tea can be made up of whoever is necessary it could be leads it projects producers of the project key tool users and programmers whatever works for you and this committee decides which requests go to the two programmers now design by committee has a tendency to come up with an average of solutions that are the best solutions and anyone on the committee needs to have knowledge of their entire departments if their leader them to make accurate decisions and of course it can be slow depending on how many times the committee meets so if there's a huge problem right now you might only come to it and talk about it when the meeting between these people happens the in the room approach this one was just by one company I talked to so in this case that you can have big time investments can have multi teams constant change and it can fit medium and small teams in this case one level designer is put into the tool programmers room or area for open for plans or slack chat channel for online

teams so if the tool programmer have question has questions pourable problems or any just curiosity for the level designer they can discuss with that level designer right away so solutions are brought up accepted or discarded face to face now of course the user has to know their user base and not make personal decisions so you have to find the right person and time is saved as no meetings or scheduled and communication is instant but it has to be done for all user bases if you want to do this not just for love designers but for other people as well like modelers lighting any other kind of tool users and of course it can also be reversed by putting a tool programmer in a user Department for a month or two so that they come back to the two programmers and talk to the programmers in their own language talking to each other in a way that they both understand the point of sam'l approach this one which is used by two companies I talked to this one can suffer from politics non-physical communication it could be a faceless system and fail to reach mid and low priority items but at the really best fit of this is really big teams so

in this case every level designer gets a limited amount of points and when suggesting features level designers spend those points so suggestions have to be balanced by the suggester can but this can result in really difficult and time-consuming programming tasks but it works well if there are many features in contention if you have a lot of features a lot of people want a lot of things you can give the limited points and find out the best stuff but it can feel like you waste your votes if you don't vote on one of the two big things if you feel like your voice isn't heard if you're one of many so and of course users might not have the best idea on what features to make there's the age-old quote that apparently isn't actually a quote of Henry Ford asking people what they wanted and they asked for faster horses he made a car your users might not always know what the best thing is for them the brute force approach now this one was used by two companies now this one can have big time investments can suffer from politics constant change but can work well with small team sizes in

this case you hire only the best of the best just 15 plus years of experience only and there's no pipeline for triage you just keep the teams or the decision makers very small and give them almost unlimited time to just figure things out now in this case developers are experienced enough to take decisions themselves but of course this is heavily limited to the amount of funding and time available and of course the loudest voice could win so the great idea great ideas might not be implemented due to canvassing the dedicated QA team approach so one was used by two companies I talked to this one can have big-time investments can have multi teams can sometimes become a faceless system it can face constant change but go work well with really big teams so in this case a separate QA team is created and tool updates are not pushed until QA verifies them so this prevents bugs from being rolled out but if these people also know a little bit about UX UI you can they can also see what is easily reachable and when it's not reachable so in this case they can make share

features all well round it before used before use but a cost considerable time and money to maintain the command center approach so this one was used by three companies this one can have big time investments suffer from politics and non-physical teams and failed to reach MIT a low priority items but fits big teams so in this case multiple studios are using the same engine and tools and especially if multiple game genres are being made on the same codebase this works really well in this case there's a single location and team of two programmers and all feature requests go to that location then the team works together to implement features and such a team can also help other first party developers like for example the ice team at Naughty Dog does if I recall correctly you've caused be adapted to multiple locations as well and there's consistent communication and if feature quests are similar between Studios you can simply copy and paste the same solutions because they all work with the same tools but it can be difficult to figure out with which parts to keep and a cut who makes that decision and it's

of course never as easy as just copy pasting there might be specific things they want but a command center can help help to make merges clearer so we know exactly like these people decide in this location the DIY approach so this one was used by two companies so this one can have big time and money investment and constant change but it can fit any size team so in this case you create a powerful scripting language and you make it easy enough for non-programmers to use if then your users implement features themselves and the features the sort of features depend on how much users are willing to work for them and the user can also prototype these features and then make it so that if it's really awesome and everyone likes it a tool programmer can make it to were a fully fledged feature but the backbone of this is really good hiring and of course it's difficult to find enough capable deaths that can apply that can do these kinds of things the focus approach this one was used by one company this one can have big time investments suffer from politics

constant change and failed to reach mid and low priority items but can fit big and medium teams so in this case requests coming through JIRA or just word of mouth and then the two programmers are asked what they'd like to work on now work on that request bug or issue can be short or long sometimes even like six months long but that tall programmer that's working at knows the context very well and that person is then the contact point for any features or requests that are clear to that so there's a personal communication point but of course it's completely not agile at all and again smaller issues will pile up the indie approach so this will be used by two companies this one can have big time investments non-physical meet communication constant change and fail to reach mid and low priority items but it can fit small teams so in this case you have a single single programmer you create a list of requests you go through them to find the severity of the problem versus time of implementation you do progressive releases so you try to sort of find out what is currently necessary you keep

continual communication with users so slack email or Twitter but there's large differences in qualitative and quantitative feedback I mean big UX issues will be brought up by our users but the small issues maybe not as much so you won't know exactly what to fix and you sort of have to make it up as you go along which might not be the best way of doing things the double prioritization approach so this one was used by one company I talked to so this one can suffer from politics can feel like a faceless system there's a constant change and you can fail to reach MIT a low priority items but it can fit big and medium-sized teams so in this case requests are submitted by users and the user states benefit with numbers like one to five and for a request then go to the tool programmer and the tool programmer states the cost again with numbers one to five for example if it ends up three to three it goes into the backlog but if it's low cost could benefit like five to one we're going to implement it right away the problem here is though it cost

benefit may change over time depending on how the project progresses maybe big things change that happens in game development so and it's hard to make an automatic system that goes through them like the old ones and sort of reassess of them and see what has changed and do we still need these features the tool designer approach this one was used by two companies so this one comparatively has less time investment s you hire a person to take these tasks on them but there is the monetary investment of hiring a new person so in this case any and all features can be requested at any time and the tool designer or tool designer team looks through all of them the tool designer Aniki me continuously communicates with all departments artists game designers level designers producers engine programmers tool programmers so the tool designer decides to remove change but also approve or combine if there's multiple things that people are suggesting that person knows oh this person requested this dispersion request said this actually we can solve this with one feature that covers all of

these things so the tool designer is a sieve in this case it slows down the process but removes unnecessary items and make them makes them digestible now so in this case two programmers do not need to think about what it looks like or how it works it's how it works but it can be difficult to find a good tool designer because you need to have a person who has the wide experiences over all those areas no minds that this should be a permanent position not someone to just put into this role for a few months another person is pulled off again because again the changing of the gaurd other guys otherwise you're gonna have that layer cake problem again and of course it's gonna be difficult for people actually find a person who is currently the tool designer who do I talk to about issues that I'm having now personally this is the approach that would recommend simply because it has it solves the issues that most people are having it has the least concurrent issues it's hard to find one but if you can find one I would say go for it it was so she helped you out in the long run and if this approach sounds interesting go buy that book it will help you immensely David love light bound the guy who wrote this is also in

the audience around here he's right there so I'll go talk to him if this sounds issuing to you and this book is also available in the GDC store so go take a look at it it's really good so those are all the approaches that I found but of course other ones out there as well for example the one that I use at ioi is the combination I am a tool designer but I also do in the room I move my desk to whoever and clearly needs my help so for example sitting with level designers to improve their system though the reason I'm doing it is because I can then over here where people are saying sometimes there's an issue and they don't know who to talk to or they are afraid to talk about it but I overhear then I go hey I can help out with it and we also use the focus system so there's a specific tool programmer working on specific issues so in this case also for me I know exactly who to talk to if there's a specific problem somewhere and of course you can make your own combinations so for example the tool designer plus a command center meaning you have multiple teams or studios but a single tool designer and tool programmer team or the committee in the room

approach where a committee of leads the site which team member should sit with who for a set time to figure out issues with tools or the point of some approach plus the double Pires is agent approach by giving the level designers and the two programmers both points to spend on what they think is important what they think can be done or some other combination or something new I mean this is only what I've been able to research in a year for those who are willing to talk about it I mean who knows what else is out there then every team of course tries to make their own way so of course so you need to find out what works best for your team how again company culture country culture production message publisher relations these are the problems most people run into so which one's of these are a problem for you or which ones don't you care about maybe you have a lot of money and a lot of time on your hands and you can do the brute-force system or maybe you don't maybe there's other things you care about but the core is this finding an effective filter is key to the school to good tool triage need to find out for

yourself what works for your team and be critical about this approach change it if it happens to not work at your in your specific team this is your team you can save your team time and headaches you want to hire the best you want to give them the best office environment so make that effort to give your level designers the best tools and level editors that they could have because it's possible thank you very much [Music] I'd like to give a thanks to these people for feedback and helping out it's awesome and also this talk was about tool triage but I'm also doing to round tables about tool triage on Wednesday and Thursday and it will be will discuss these approaches and discuss what problems you have been having come talk to us it's an hour long we can talk about it and see what works for everyone so if you're in the audience here your levels are near you think wow these problems is what we're having I'd love to have better tools but I don't really have the power to change things cool sends your producer send your

executives to these roundtables I have the figures I'll talk to them and convince them but these are really good things you guys probably need alright are there any questions yes one question you didn't mention anything about narrative tools would you view them as the same treatment as level design tools just from experience a lot of narrative design tools quest design tools in general seem to be a very overlooked part of games I and they happen very late and then it's very you know it's always a difficult situation there yes I would say yeah the same approaches especially in this case I'm also doing some looking at narrative tools at IO Interactive and it's really a thing of if you have a person doing this if you have a tool so signer they are communicating with all these teams so in this case they know that the narrative team is having a problem with the tool so if they're done in a production talk

somewhere they can go actually we might be overlooking this thing right here because there's not a narrative person in the room but I know about it so we should probably look into this because they can find out like is this a thing we currently need it's a thing we do in the future could it be part of the level design editor kit like do we need to combine these things so let me say like all these different kinds of tools whatever works for you and your team but in this case the tool design here can look at all these things and make something that works in this case does it answer your question yeah that's great thanks thanks I have a pretty simple question but I was curious since everything's sort of framed with big team medium team small team I was just curious what you consider the thresholds of each of them like how many people would you consider a small team yeah so for big teams I would say like three four hundred-plus like you have a lot of people and for medium teams I would say about like a hundred and for small teams I would say about ten to fifteen but of course this also goes like the teams within teams

I'm not saying company I'm saying specifically team if you have a really big company but working with small teams that's sort of what I mean like you can use one of those brochures that fits the small teams any other questions well I did tweet out that anyone who'd come we'll get a strobe waffle so I have a couple of them here if you like them come and get them also don't forget to fill in your forms at the end so thank you very much [Applause] you