Design Tests and What to Expect from Them

[Music] hi everyone I'm a little bit surprised there's a lot of people but I'm really happy and I hope you get something out of this talk so welcome to my talk about the scientists and what to expect from them trawling the internet and the vault I became aware that nobody is shining light upon this topic so I'm here to break that silence I'm Peter bucha and I worked at plated in Copenhagen I worked on limbo and inside and I'm currently at Studio Kobo in Brighton England where we worked on Disney infinity and we're currently working on a big devs project with Ubisoft and speaking of studio gobo and their partner company electric Square they're currently hiring so if anybody fancy a cool career at a awesome studio in England then have a look at their websites so I've been a bit more than 10 years in the industry and from time to time I felt the urge to try to look for

other design opportunities and that has left me to receive some tests from the big companies probably some of the ones that you all want to work at so I'm here to share some of my knowledge about doing those tests I did a game design up test where I got an interview I did a level design of one where I had to push for some feedback I did an RPG to sign a one where I gave up on I'll talk more about that I did a level and mission to China and scripta position where I got a job offer declined it a single-player designer and a multiplayer designer two tests at once we'll talk more about that I got an answer for no feedback sorry I see the way that tests are being used by the companies as a new tendency in modern recruitment and especially for the big companies who receive a lot of applications for their open design positions so instead of just requiring an interactive resume an impressive portfolio companies now also require you

to do a test and the test functions like a natural filter because it's only the most dedicated people who even get through that first process and it also creates a foundation for the companies to compare guys or me or whatever one-to-one because of that test because the companies have found this to be a cheap and easy way for them to create a better base for their decision they seem to forget the individual in that process and that's a little bit why I'm here to let you know so will I be telling you how to pass these tests no that's not what this talk is about the talk is not about how to deliver the perfect answer to these tests but it's more about preparing your folks for the experience of being an individual in that process and it all might come off a bit gloomy practice I actually had a lot of fun and it was really interesting to do these tests and I learned a lot from it but today I'll touch upon three major topics

that is worth keeping in mind when you decide to go on a design test adventure I'll be going into details about the format of an answer I'll be giving examples of what to be expected in terms of the workload that can be involved when answering these tests and I'll be diving into what you can expect in regards to feedback from the people who asked you to do the test in the first place but first of all I think it would make sense to just have a look at one test so that anyone here in the audience who might wonder what constitutes a real-world test that anybody in here might be asked to do one day on that note I also want to say when I present some of the tests up here there's gonna be a lot of text on the screen don't read so I'll try to read all of it I'm gonna share a link at the end of the talk where you can download a PDF with all the tests that I'm talking about so you can try it at home and fail or exceed or you can do all the kinds of stuff you want so here's one of the tests this test was done for a big triple-a company who are highly regarded

in our industry and I feel is fairly representative of the type of tests that you can expect together this was by the way for a level design opposition as you can see they asked for some slightly generic information about what games you play and what scripting and 3d experiences you have it seems to serve as a more personalized resume where you actually get to explain yourself a little bit then there's the final part which the actual test and even though this was supposed to be a level 2 scientist you can see that it requires a rather broad understanding of game design to answer it we'll have you designing enemy's mechanics and puzzles what's fairly unique about this test is that it's it actually clearly states a format that you should be answering in and it also tells you to answer the questions in two or less paragraphs and finally it states that the level scenario should fit on a two-page document this is a lot of good information that you can squeeze your

answer into but as you'll see as we continue with some of the other examples this is not always the case generally speaking tests they are answered with paragraphs of text but depending on the job type there's usually other formats to consider - for game design is primarily text diagrams and images for systems design or technical designer there's usually some code some scripts that has to be analyzed modified or created for level design it's 2d and 3d mock-ups of scenarios maps and levels but I have found that when the test isn't giving any specific indications for the desired format then you run into the dilemma of wanting to make sure that your ideas and answers are fully understood and that the reader isn't left with the feeling of you leaving things out but at the same time you don't want to intimidate or bore anybody with walls of text at

least this has been one of my greatest concerns when answering these tests so what can you do to a probe approver approach this format the best thing you can do is to make sure that you read and understand the entire test I know it sounds simple but as I'll show in some of my examples what my approach was that you don't always do that and then make sure that you feel that your ideas reach well you could potentially have interests savvy friend or just somebody who's better at grammar than you read through it because they might have some input for you on how to formulate yourself better of course if you're lucky enough you'll be able to get information about the exact persons or the teams that will be reviewing your tests and by from that try to pick an appropriate format but remember often these tests they are sent out by the HR department and honestly they rarely have a clue about the actual tests its context or its precise purpose so let's have a look at a test in question and how I decided to approach the format this was this

test was for a game design a position firstly the assignment itself you'll be creating on design specification for a granade system used in an FPS then the test States a suggested structure and this is really cool because they mentioned before that it was a design specification and that can be very different from company to company so apparently they have structure for that then yeah it mentions format as you can see here but what's a bit funny about that is it says said you can do anything you want as long as it's easily reviewed but honestly what does that mean a sentence like that does not help you find your format so the test I just showed you was a rather technical game designer test so there was some some tech questions that came with it but because they were most straightforward I'm gonna kind of dive into it but for reference I just go over there quickly

the the three steps there were there was stuff about explaining a scripts behavior then there was stuff about fixing errors in the script and finally it was about suggesting a new script structure for a set of new requirements so I ended up with twelve paced page answer that featured eight images and seven diagrams and so back to my design of I did decided to do what I call a brief feature and context analysis which is something I've gotten very familiar with will while working at studio gobo - and of course there was eight additional pages to answer the technical test so I want to briefly explain my way of using feature in context analysis since it's a design tool that I've used quite a lot for these types of tests but also in a more professional capacity the basic idea is to look at how similar features are treated in other games but also look at the context that the feature is supposed to be used in so in my case with the test you just saw I would look at other grenade

systems and I wouldn't just look at first-person grenade systems because the first person approach they're asking for I might be able to pick some cool things up for the third person person approaches that is out there so I put that into my test but for context I looked at other games from the same studio there's a way to see how I would approach my design so it fit within their way of thinking but if the feature is for a more specific scenario then it could have been looking at the narrative where it's been being injected or the mechanics that surrounds it or it has to fit in with or it could be something completely different so based on their previous games I acute that a normal create system wouldn't really cut it it would have to take some sort of twist to the mechanic so I suggested magnetic grenades that when you threw them they were somehow get influenced by the middle that was in the world from there on I followed their suggested format which made a lot of sense to me because

it was those type of questions that they you need to be answering in a professional capacity anyway at Studio go what we call that a creative brief when we try to draw those kind of outlines up but in this case because I wasn't there to to present it I thought it was okay with a little bit more text I ended up oh sorry so what happened with his approach I ended up getting the interview where it clearly felt like the people in the room had only skimmed my answer so I felt I could have created a more elegant answer and that my answer might have been too long and we didn't even touch upon the technical tough enemy stuff in the meeting which meant I didn't get any feedback on that and I didn't know if I had done it right or wrong or or anything so I'll bring another test forward now this test was made for a company that does a lot of open-world Triple A games and it was for a level and mission designer /script opposition

so the actual assignment this is as simple as it gets design a new mission for this then it states some requirements that is where the mission is supposed to be injected the location in the game and some narrative considerations to take into account yeah and then it also has a suggestion for the format but as you can see it says you can do anything you want and it even says there's no limit to the quantity of pitches so what does that mean do you do two pitches Oh ten pitches or do you do one really awesome pitch I don't know I did one really awesome pitch I hope so like the first test I presented to you this test was also threefold the first pad had the meet of convincing them that I can't assign in the way they want and

it fits with their games the remaining pan path seems to be gauging what I see as quality where I get my inspiration from but also my ability to reflect on my own design ideas and other analyze other people's designs so like with my first example these paths seemed like a more elaborated part of a resume or cover letter if you want because I wanted to illustrate that I liked and understood their games I did another basic context analysis where I looked at the place they asked me to inject the mission for me it made a lot of sense to consider what the player had learned and what the tools what tools the player could access at the time and also what the immediate mechanics and tools were after the mission because I felt that this would be designer consideration that were I would do that it if I was working on that in a professional capacity so at the same time I also wanted to give our a narrative wrapper so I wrote a narrative outline for that to show them and understand that narrative is

important but by the time that I added up all the relevant screenshots from the game the illustrations of my map layouts I ended up with 30 pages not 30 pages of text but 30 pages with a lot of imagery and whatnot and again you ask yourself is this showing what I have to do if I get the job when I need to do my designs or does it say this guy is really thorough and he knows how to make himself clear when he has to and you're also thinking can they even be bothered to read through this I could probably have explained and tried to sell the idea of the mission in one to three pages really but I feel that when answering these tests it's important to show who you are and what type of designer you are being a designer requires you to be able to approach designing in many various ways and the way you structure your answers shows a lot about how you get your ideas if you're good at understanding context and if you're good at communicating your own

designs so I was aiming to try to show that I could design almost anything for anything and not just answer like one specific question but of course you guys have to find your way of doing it and something you feel comfortable with so on an inter note about formats let's look at some of the format suggestions that existed within these tests it says whatever mixture of mediums that delivers the best impact efficiently detailed design documentation any form you require as long as it's easily reviewed as you can see here you will encounter encounter very open-ended tests and you have to ask yourself how you want to proceed to be perceived and see if you can find a way to rationalize what they might be expecting some tests do have a stated length limitation which really helps in understanding what depth they want you to be answering in and at the same the same with our suggestion to the structure can tell you a lot about what they like you to focus on so we

touch upon the format but as you can figure for my 30 page example another important thing to prepare for is the amount of work that has to be committed to a test from its received and you send it off the majority of the tests that I have encountered I've been told to answer within seven days but I've also had four day tests ten day tests and I have the odd pleasure of receiving two tests at a time where one was a forty-eight hour test and one was a seven day test and I had to do both of them in seven days so I feel it's a important to take these timeframes into account when answering the test but you will find it really difficult to assess especially if you're studying or working while answering you'll be hit with the feeling of having much less time and resources than other potential candidates who could be spending 10 plus hours a day and you'll only be able to squeeze in a few hours every night and you can maybe you can imagine being lucky enough to send out more than one

application at a time and then you receive two tests at the same time this is of course a luxury problem but still so the short story is that when answering these tests it can feel a bit overwhelming let's have a look at some of the tests that I feel have had a big workload connected to them as part of one of my examples for workload I'll bring forward a test that I actually gave up on let's go over the tests so first of all it has a statement for the assignment design on paper and implement using this unity UI tool that was on the job a crafting system for a futuristic RPG followed by a design brief detailing the context for the system and some function requirements after this we see some more specific requirements for the crafting systems features and its content and then there is a list of required deliverables in terms of documents and files furthermore it

states some generic requirements for the answer and finally it concludes by listing some topics that if included would be considered a bonus no pressure so what went wrong for me with this test firstly I didn't do my homework I didn't consider all parts of this test before I started it this meant that the first thing I did was research crafting systems and sources of inspiration before understanding the exact requirements for the implementation of that prototype the result was that when I finally had a solid design idea that I believed in then I found out that the free unity UI tool was a bit deprecated and a bit broken and I didn't have the time to find and fix the box secondly I also realised how much work it would take to implement all the desired features I felt that it would be near impossible for me to deliver something that feature-wise would live up to my own quality expectations but also that I would be under delivering on the requirements stated in the test let alone the bonus

goals so with three days left on the test I gave up I throw the towel in the ring reto oh sorry and this left me with a feeling of not being good enough for the industry retrospectively I've considered that part of the test might have been to test how I would react in a situation where there were exceptional high requirements and sometimes I regret not actually trying to submit something for that test but honestly at the time I just felt like giving up and with companies throwing these tests after applicants without much consideration I'd say it's okay to let go and give up on some of them you might not even be the right fit for that position if it feels bad to do the test so now you'll all at least know that if you end up in this situation you're not alone other people have been there and maybe you can sleep better at night with that so another example of big workloads it's when a studio decided to ask me to do both a multiplayer and a single-player

test at the same time expecting the answer within seven days since this might be a slightly unique situation I'll just have a look at the multiplayer test and go over it because it has plenty of work in it as I'll show firstly it won't they want you to design a gun recoil system then they want you to fix a really hard technical problem of over-the-shoulder aiming and a third-person shooter the third pad will have you designing multiplayer skill balancing system and surely also a full will to player mode and then after that broadly outlined design ideas for an engaging progress and progression system and finally present ideas for new mechanic so as you can see the base that tests basically have want you to do five different designs and solve a difficult technical problem I like these - chests I like answering them but it felt like I could dive down and use seven days on each of these questions I really wanted to after squeezing these

six entries on to 15 pages with imagery tables diagrams it still felt like I might have been over delivering and somehow should be able to create a more elegant answer I feel it's a really good example of where you as a designer have a hard time guessing where the focus should be applied and where to elaborate and where to cut it to the bone as you can see some of these tests seems to be requiring a lot of work and a lot of work can mean long answers which in turn can be conceived badly because the recipients don't have a lot of time to go over them so you want to make sure that you cover all the relevant aspects but also keep it relatively digestible which of course comes back to the format concern here I want to add that I have asked for time extensions on these and the companies are usually very open to this if they know that the engagement was something you had before you even receive the test so keep that in mind if you ever feel like you're out of time so when it comes to feedback you would

imagine that it was simple do this test you say sure it's cool here you go it was an answer and they say thanks then they think about it and then they say no because of something or they say yeah let's talk more about it but sadly the current reality of things is different and since the companies have introduced this test to save resources so after spending all that time worrying about the format of your answer and doubting your ability to even complete the test on time you managed to pull through and deliver something you're truly proud of and it cost you quite some time now what can you respect a expect in return be realistic absolutely nothing should you accept no feedback no I don't think so and so if you don't receive tangible feedback you should try to ask for it and if you all do that maybe some of the companies will realize that they should be treating this a little bit differently but now let's have a look at

some of the feedback that I had received over the years so let's first start with the 7 and the 2-day test after finishing I received this we wanted to let you know that has been reviewed by our leads and unfortunately they have decided not to move forward with the interview process at this time and you think it s you can guess this feels a bit off because I pour my heart and my soul into a test and then the leads sit down and actually review it and they don't give me any feedback or insights into why they choose not to continue with me secondly I want to bring forward an example from interacting only with the recruiter not the actual company I honestly think there was a great mismatch between the my profile and the profile desired by that company my theory is that it was a recruiter who was aiming very broad and the candidate if with candidates for the position since it was a recruiter who asked me to do the test I felt there was okay to require the recruiter to also make sure I received feedback for that test they

should be protecting my interest - right so after having asked for feedback quite a few times I got this we have reviewed the test and we feel like the test is incomplete or vague at best the level design proposal is way too high level and not detailed enough and you might be thinking this seems harsh or it's really short but as my other examples show this is actually good feedback because they're actually letting me know what part of the test they didn't like and also that it's too high level which meant that I learned something from doing this test I also have examples of situation where it seems like the companies forgot to check my profile before they asked me to do the test from a test the test I answered with 30 pages three days later I received this thanks for your reply Peter we just found out that a u.s. work reason would be tough to get in your case so that just hurts and it it leaves you wondering how to do better in the

future and the last one in the same category we were impressed with the the effort you put forth in the scripting test while your background is impressive we were going to focus on candidates not requiring sponsorship we apologize for not recognize this requirement ahead of time so but at least they recognize they made sort of a mistake and of course it's also reassuring to receive positive feedback in regards to my answer my point about feedback therefore is that it can cause a lot of frustration not to be able to learn from doing these tests and if you feel comfortable enough then try to push a little extra to receive some of that feedback I hope that these concrete examples will help you prepare for similar situations and let you rest assure that you're not the only one to have dealt with these results so we're here at the end of my talk and I just want to quickly recap some of the major topics from the talk the scientist comes in many shapes and form so prepare to be an agile designer even though you feel specialized within one single design

area read understand and consider all parts of the test before you start answering it C key oh sorry you don't have to sit with the test completely alone it's always good with inspiration you will find yourself wanting to design entire worlds and games to make your ideas and passion for it come true so prepare for the test to take time and utilize that time as well as possible don't do everything you can to adapt you want to leave some of you and your answers as long as it's easily readable of course lastly and probably most importantly prepare to do a lot of work potentially multiple times and all that without receiving any feedback at all but don't give up so that was it for me I hope that this helps you guys find helped you chase the perfect designer

job here's a link to the tests I hope that it will also help you prepare for any future tests that you might be doing I was told to please remind you to do this surveys at the end of this but I'm really happy you all came and thanks for coming [Applause]