Recruiting Like A Pro | Games Industry (Product Philosophy #10)

hello everybody and welcome to philosophy friday although we may be changing the name of the series soon but this will be the practical implementation application edition today we will be discussing one of the most important topics for the success of a game studio especially for startups and those working on new game projects zero to one and that is recruiting so with you as always you have your hosts myself decriminator aka joseph kim brett novak and with us today we also have a special guest you've heard us reference him before on this podcast the deep thinker of the games industry vp of product at a thinking ape taper boyer hey everybody hey welcome uh and tabor before we jump in can you tell us a little bit about yourself and also about ata yeah sure so

i'm vp of product out of thinking ape and we've been making mobile games since 2009 one of the first companies to make games for the iphone in fact when we first made our first game there were no ieps and so um in order to you know be able to sell things in the game we would sell um premium versions of the game that would unlock currency in uh in the main free game so in the back end of our code there's still some ieps called unlocks and um yeah so we started as a y combinator company building a chat engine and very quickly found that game companies and gamers were using the chat engine to talk about games so our founders built a proof of concept game which was you know with a you know a strategy game that was like sort of thinly veiled over top of a chat engine and it exploded and um pivoted into make a games company and they moved up to vancouver and i started working with them in 2013. before that i worked

at eastside games on the hit title pot farm which was a viral sensation on the facebook platform yeah and you basically uh were out in the woods and it was like farmville but instead of growing vegetables you're growing marijuana so uh yeah it was really fun project and yeah since then i've been working at ata for almost 10 years now all right and i think there's been a bunch of folks that kind of came through esg and like the canadian gaming ecosystem so that's that's great so maybe kind of diving right into our topic i thought we could start by talking about problems when it comes to recruiting and maybe if we could go ahead and start with like some of the biggest problems that you guys have seen from a recruiting perspective as you guys are trying to hire for your teams and taber as our guest maybe we could start with you sure yeah so um i think problems you know with hiring really start at the value proposition of your company

um i think if you can't get that right you're really going to struggle so what is it that's unique about your your company that is going to give people um you know a great career um and that's kind of like that's pretty high level stuff that you know we're not going to solve for this podcast but i think it's really important to figure out what that is so that when you talk to people you know what it is that you're offering to people that's going to make them want to work for you but when it comes to like the real practical stuff i think you know on the design org side something that we've been working on a lot lately is getting our take home right so it's a really deep topic but we do take home tests and i think we're gonna talk a little bit about that later because you know i did do this linkedin post that went viral but i think one of the biggest problems in the take home that we've had recently is we've discovered that there were some questions that were really fun they're fun for us to ask and fun for people to fill out but they didn't actually tell us anything they didn't tell you tell us if the candidate could do the core job that they are being asked to do so we had this really fun game theory question

essentially what would you do if you had to remove scissors from rock paper scissors how would you rebalance the game and still make it fun and in the spirit of rock paper scissors so i can tell you a lot about how people think about game theory but it turns out that we never really use that to hire somebody good like if somebody failed that question but they did really really good on let's say our spreadsheet part we would kind of be like ah that's okay let's still interview them and so over time we realize it's not a good question also somebody did a gdc uh presentation i think from movies ubisoft where they talked about using that question they gave all the answers away so then it became hard to use that question because you know it gets out into the ether um and that's another thing it's like i always wonder sometimes if like you know we use a similar test over and over again if people are like giving it out to each other and like you know giving our feedback to each other so that they can answer it better that's always a risk um but yeah so that's like i think one of the big practical problems that we have

um and that's something that's changed for us in terms of problems is we've always had a limited talent pool being in vancouver canada it's expensive city it's in canada which a lot of people like but if you're trying to hire the best gaming talent a lot of it is in the in the us um and other places in the world but going remotes really changed that i think so that's something that we're trying to figure out how to you know be an amazing remote studio and you know i've learned a lot from brett on this topic but um it's opened up our talent pool that's always been a big challenge for us um right yeah so that's just some of the stuff and then last but not least is culture interviews what the heck is a culture interview how do you do a culture interview what does that even mean there's a whole bunch of stuff there again huge topic brett what about you big problems with recruiting oh there's so many problems jk but i'm going to keep it short the first is hiring too heavily on unrelated or unimportant characteristics or skills right taber mentioned some of that i couldn't stand when we'd spend a ton of time

talking about if the person was going to be great at happy hour for example i mean who really cares we're here to make money create awesome games and build a wonderful business not talking about happy hour the and it's fine to talk about happy hour but if you waited it probably should be about one percent right the second is for on the hiring side spending too much time or being too costly it's a very costly experience the interview can be up to an hour the review can be a long time then you have like a meeting to review that maybe even a follow-up interview and you have to do multiple interviews very very costly for the company and for the individual particularly if it's a manager or lead or someone high up ceo and then the third big thing is the inaccuracy right so missing on the hiring process you hire somebody you spend all this money time on them and it's often very inaccurate so those are

kind of the big three things that are probably the biggest three things that i think are wrong or hard about the hiring process can i ask you a question about that brent um so we have this thing where we try we would rather lose out on a good candidate than have a false positive but in your environment you actually um have contractors and you can let's say fire somebody after three months if it's not working or even sooner so are you more willing to have a false positive because you can you can you know quickly fire somebody definitely and that's basically one of the cores of our interview and onboarding process is that our belief is that we are inherently inaccurate and that no interview process is very accurate so we optimize the system around being inaccurate and not accurate therefore we're very willing to take a sort of stab at somebody give them a chance and let them go or you know vice versa like try someone we think we love and then also let them go right yeah i think as we do the opposite

we just try to raise our bar so high that um we eliminate a lot of false positives but then of course it makes hiring way slower and harder so yeah we i think we've talked about podcasts we just don't believe whether or not it's possible to raise the bar higher like we yeah we disagree on whether or not that's possible right um yeah and then from my perspective i definitely agree with you guys and some of the problems that you guys are seeing maybe taking a little bit more of a process focus on this i do think that one of the big problems i've typically seen is just like not getting everybody coordinated together it's like how are we doing this oh is it this is it that just like having a well mapped out workflow that everybody understands this is how it's gonna work this is you know these are all the steps and so everyone's clear on what that looks like in their own specific role in it i also think that sometimes what happens in terms of the coordination is that the scope of the role you know what does the jd look like what are we hiring for in

terms of responsibilities and then also what skills are necessary in order for that person to be successful at that role and when that's not properly agreed upon or aligned against then you're like different people may be testing for different things and then who's testing for what and so one of the things that we do is we have we can go into later is what we call a skills evaluation matrix where for a role we'll kind of we'll kind of have a pre-meeting that suggests these are the skills that are going to be required to be effective in this role and then we also try and get like two touch points like either two people or or a person and a test that will test against that skill but just going back to problems i think that the other thing that happens are just expectations for interviewing right like some people have this notion of like well you know for me an interview means i'm just going to talk to them and to brett's point you know are they going to be good at happy hour do i like them and so like one

person's idea of an interview maybe hey do i like this person are we having a friendly conversation and another person might you know might actually you know be really testing for the for the skills and so like what is the expectation of the people that are interviewing is something else that oftentimes does not get uh coordinated correctly and then also in terms of like the philosophy right so taper for you and it's interesting that you raised this point about you know how do we test for somebody well they may have had a great spreadsheet but then they or a model of some kind but then they miss something else and so you know just the philosophy in terms of do we need somebody who's pretty good at a bunch of stuff or do we hire for strength instead of lack of weakness right like that the agreement of like what what approach are we taking in terms of how we make a decision on hiring for somebody that

also seems to be something that's not well coordinated oftentimes at companies and then um taylor you uh had mentioned not how do you actually test for some of these skills that's always a even when you agree like being able to kind of come up with the right test the right questions to unearth that stuff and then also just having like a quick and efficient hiring process and brett i feel you when you talk about the time that's required and so for us you know recruiting and interviewing is a really time intensive process but we also because we aren't typically hiring superstars at based upon our situation and where we are we're often having to hire people who are smart and who can learn fast um the time invested from us is really really significant so we really do have to be much more careful about who we hire uh probably compared to other companies on a relative basis

and then um probably the other a couple other points would just be not keeping track of metrics right like oh we're just gonna we're gonna advertise we're and we're gonna try to promote on these channels and then we're gonna try and hire some people but then you know are you tracking what channels you know which recruitment agencies what were the candidate wait times where did we get good candidates from how long did they wait at each step are we you know [ __ ] up on our process are we making them wait too long before we give a response to a written test or before we schedule something but not having metrics i think is also a common problem that i see and then i i would say the final thing is just simplifying the recruitment process for candidates and giving them a good experience because i think ultimately you know this is your customer touch point when when these candidates interface with your company you know this is the process that they're going they're going to kind of

evaluate the opportunity you know working with with you is going to be like and um i i kind of put it in the notes but there's actually i i took tesla's um application form and and kind of stuck it in the notes here i don't know if you guys had a chance to look at that i'll put it in the youtube channel if you want to see it but their application form is so simple it has name email what exceptional work have you done in software hardware or ai and this is for their their ai roles and then it's there's a button to upload your resume and that's it and you know when we first started like you know our hr person had had a similar form but it was like like what's your phone number what's your linkedin profile it's like why are you having these people input like 50 different fields and because you know because this is also your funnel and if you make a really [ __ ] hard thing

to to actually get into your recruitment pipeline you're going to have drop off at that funnel step and so i definitely think that that's another issue problem that comes up which is you know then simplifying the the process and making it a good experience for for candidates that are applying to your company but those would be some of the uh some of the biggest issues that i see and so maybe moving on to the next topic unless you guys have any thoughts on that uh i thought we could talk about interviews specifically and i know that all of us have a preference towards doing both right live interviews and written interviews but in terms of the live interviews what what do you see as some of the problems with the live interviews and why they may potentially not be as good as having both or you know kind of weaknesses compared to a to a written interview

taber or brett i mean i can start so yeah you know interviews are well documented as being a poor way to make judgments on on candidates i think the biggest pitfall that i've seen people fall into is doing unstructured interviews which means like you're asking you know a bunch of different questions you ask different candidates different questions people you don't have the same people in the same interviews like anything to do with an unstructured interview tends to um lead to really bad results and it's it's well documented there's lots of studies that show this and yet people still still do it and i think it's because i don't know what it is about i don't know north maybe it's north american thing but like we've always been taught like trust your gut you just gotta tr go with your gut there's horrible advice right um like what does that even mean it's a terrible thing thing to say in fact what we do and what i try to do personally is anytime my gut is telling me something i

try to disprove it so if i'm in an interview and i'm like yeah i really like what this person is saying i have trained myself to recognize that and then i say like oh i'm going to double check if they actually know what they're talking about that's a signal for me that i'm being probably being biased and i need to dig in and give them a little bit of a harder time ask deeper questions um and the and the opposite is true sometimes i'll find myself in an interview and i'll be like oh man this person's just like doing a horrible job and that's also a cue for me to say like wait i need to dig in and make sure that it's not me asking bad questions or like you know doing doing a poor job in the interview or am i just biased so yeah just like don't trust your gut make sure you're asking the same questions of all your candidates make sure that they're scoring for the questions you have a clear framework for like what answers you you're looking for and then everybody's trained on the questions so that you're at least getting some semblance of standardization between between candidates that's what i would suggest anything brett i think you know how much i don't like interviews jk we don't do any of them

they're just brutal i think they're biased inaccurate costly one idea actually that my father-in-law taught me about it is he would have people actually speak the answers into a video and send the video because he didn't want to actually have to listen to the whole thing and that idea got me thinking about alternative ways of doing it because it saved him a lot of time although he could like skip around the video but yeah in general i i find them to be um very just uh costly and inaccurate and actually if i were to kind of summarize something that we've taken advantage of that i don't want others to but it would be in your best interest to do that is we hire a lot of people who suck at interviewing but are great at work and i i mean that in the nicest way and if these a lot of these people were who on my team were here they would

agree with me i mean they're bad interviewers i have stories the guy had like a cat coming through the desk and he i mean he was like in his parents house and it was just like everything about the interview was terrible and this guy just told uh celebrated his one year here and he's been killer he helped build our whole impact portal which is an incredible tool that i think every gaming company should have but doesn't um yet and um terrible interview like like the whole time i was thinking this this is awful and we have lots of instances of that so i think that's another thing about it is it like is optimizing for someone to be good at interviewing and potentially good at their job but like if jk says i know you love matrixes it's like good interviewing and good at your job that's what you're going for what we look for is good at your job and potentially good or bad at interviewing right so that expands the talent pool that you can can hire from yeah i think for us one of the

advantages of the live interview however is that at least what we found is that there's a lot of people who actually don't seem to be doing their written interviews by themselves and so it you know it helps us wait how did you figure that out well because some of the answers were the same and you know when i asked some people during the live interviews about some of their written tests they don't know what's in in it right they're like they're like oh uh yeah uh yeah it's in it's like okay clearly that was not your work that was somebody else's work and i i mean i don't know you know i think that um the other thing with live interviews is i think that it really helps it it helps for me anyway i think it's better at understanding how people think and so by to your point tabor about depth

like having i think one problem is like not everybody has experience or knowledge about testing for depth so somebody could say yeah we increased um arp dow by 50 right and then like a poor interview would be like oh yeah that's great this guy's done so much stuff and then he's like whoa how did you do that what did you do what was your role in that right and then you go deeper and deeper and deeper and then you really find out how how this person thinks what their actual impact was what their actual participation and contribution was and that's not something you can get from the written but for me as well i think both are critically important i think if for us if we just did the live interview i think we'd be in trouble and i think if we just did the written i think we'd also be in trouble but i would say my recent experience uh maybe because we're a little bit more high profile in india that a lot of our stuff seems to be getting leaked so i'm

i'm constantly updating our our written tests a lot i i just rewrote um most of them yesterday again so uh so for me i think both both are important but definitely i think uh you know there's i think on the live side well and the written side you know there there's certain there's certain skills that you need in order to um effectively execute those in in in the proper way in my opinion all right um and so i thought maybe we could actually just be a bit more practical here and just give people an idea of exactly what the recruitment process for each of you guys looks like like on a step-by-step step basis like what do like from step one through through higher what does that look like and again maybe we can start with youtuber yeah so for us it really depends on the role but on design how typically happens is you know we get

the role approved and then we'll post the role up on our website obviously we have job posting it goes through all the platforms that we use and then we'll post it up on a bunch of different websites and then depending on the level of experience required we'll get incoming applications so more junior roles we tend to get a lot more applications and then the more senior it is the more we have to do active sourcing and this has changed a lot over the last i would say three years and especially in the last year where like i got to be honest like sometimes it's like you have to i'm asking for a favor just to get somebody to apply to the role um and i think it's just because competition in games has become really fierce and so what i will do is i'll find good candidates that i think have potential and i'll reach out to them on linkedin or find some way to message them and i'll pitch them on the studio if um if it's a high profile role and spend some time just getting them to apply and

convincing them that you know it's a company they want to work for and from there they go through and get a take home that's the first step like we don't we don't do any interviewing none of the interviewers talk to the candidate until they've at least assessed their their skill in the written interview and then we do a full loop interview and then we try to get back to people within two days um with the decision all right right yeah we well we're currently reviewing our process but in general we post on craigslist or other platforms we're expanding that i was just speaking of analytics where we've been looking at our analytics for a hiring process and gonna actively try to improve it but i spent ten our company spent ten thousand dollars on craigslist over the last 12 months which is um seems like a lot to me considering it's uh but anyways so we post there and then it's all remote

test-based so they'll have a very simple early test to respond and then our chief kind of chief of staff person will review the first test and second test and the tests are created by the manager the hiring manager and uh so the the candidate will fill out the google form it'll go in a spreadsheet the chiefess staff person will manage the first two tests and then if people pass to the third test the hiring manager will review those and then just put in higher or i mean or sorry pass or fail and the hiring manager has templated response about each step which is all through google forms and then if a person gets to the third or past the third or fourth step depending on the test then we will onboard that person they will get a email they'll onboard completely virtually they won't talk to anybody and they'll start working pretty soon right

basically right after they spend a couple days reading the onboarding documents which they'll get paid for as well got it and then they're also just the last part is they're kind of notified that in the beginning during that process when they get on board at that the hiring process extends it's a paid hiring process after that and that within the first month or two they're really getting seriously evaluated as well and that that's part of the hiring process and we we do like you know actively let go of people who don't kind of make it through that part but they get paid for that time period but we're very upfront about the fact that like for the first two months they're still sort of in this like hiring process phase got it okay and i think for us we we start step one with like agreement on in terms of the role the scope of work skills required a line on the jd so we kind of meet about that then we as i mentioned before we have something called skills evaluation matrix where we

list out the skills and then we basically just have two checks okay are we going to cover these skills in the written test or through live interviews but we we basically want to have dual dual coverage either whether it's with written and live or to live interviews and then step two would be we then create the written test for the roles and for some rules we actually do have more than one round of written tests just because i think for some roles for us like systems designer for example i think it'd be really a lot of like just considering the applicant themselves we don't want them to go through like this huge amount of work and then ultimately you know not be qualified so we try to have like an initial test which isn't too much work which can qualify disqualify a big percentage of the population then go into deeper depth from there and then we post with our jd's that when we post our jd's we have a submission link that's tracked

and click up and and tied into a click up form so that's what we used for tracking and then we and it also sends out company and culture information about our company and stuff like that and then from there if they um from there you know we then have like live interviews and for senior roles we are considering um having something even deeper than just like a live interview having like a live case study where we have a panel people and then we send them um a number of questions that they'll have present to the you know um to to the panel and maybe even go through like live uh case studies together and then the last step is like the decision making part where we kind of then meet up talk about the person and then determine whether we want to hire that person or not um and i think um one thing that we also do is we do try to provide feedback either for the written tests um in click up we it's you know we we have it relatively

automated so that like if we're rejecting someone we just change their status field and it automatically fires off an email that says hey sorry you know you didn't make it but you know it also gives them an email signup form so that if they want to see what we consider to be the you know quote unquote correct answers for that test then they can sign up and then they'll later get an email once we've closed the role about what kinds of of answers that we were expecting and then for anyone that we do live that we generally do try to provide some some feedback to them and uh yeah that's that's basically our process isn't this why you have to continually rewrite them yeah so you're you're i've written a bunch of content really rewriting the test because you're giving out the answers that and also just because it does seem like some of our stuff leaks out

it doesn't sound like it's leaking out it sounds like you're sending it out well not not well for some of the roles where we have we're still looking and we haven't posted it does seem like some of the stuff is leaked too so i got it but but you are giving the answers to people who we will yeah ask for it right yeah because i also think that they are theoretically posting it online they they could be but you know for me that's okay because if i ask a similar question and they use a similar approach like if they can eventually have the skills to do what we need them to do i mean that's fine and i i think for us also this is kind of like a way of having a relationship with these people so that if they get to be good enough over time let's say you know three years from now that's great you know and i want them to know how we're thinking what we would expect so i i you know we've we've had people who have applied like five times you know it's like and for different roles but you know i i

think that for for us if they can get there somehow then that's that's great uh so the way we do that um is our questions um we give the same question for the design role to everyone into all levels of seniority but it's scored in such a way that like a senior person can really like score really highly without even necessarily doing more work and so we have a scoring setup where we can figure out like where somebody would fit in our in our career tracks in seniority level based on how they answer the question so when we give feedback it's it can be more generic we can say like we would expect people for your level of seniority to answer the question you know this way without giving out the answer because there's many answers to the question um so that's the one way that we kind of get around it is that we don't actually have to give out the specific answer and even if you had the question in advance there's like so many different ways to answer the question that you there's no like right way and so even if you had like a right answer but you weren't able to explain it we'll be able to tell really really

quickly it's almost like you have to show your work yeah i that's a huge thing about the interview process that i i love that you're doing is asking the same exact questions right because then you're comparing apples to apples and when people ask for feedback for us we if if they're really far along one thing that we i will do is send them the person who who got the jobs response for the same reason because it's not really like you said right or wrong a lot of times it's like we're comparing 10 people and one person's just really standing out and we can kind of send it to the to the other candidates and say like hey this is it's not that you aren't good it's that this person kind of was the best and here's their work like do you think yours is better than this yeah you know actually usually that person is like exponentially better so it's like pretty obvious you know like you look at like a graph or something

it's just so much better it's like clear you know they're like no we're the same it's like okay well then we have to go and like point out the things that they did wrong or whatever yeah for tests two and three are typically have keys you know for us they're very like yes or no they got nine out of ten so the hiring so that the chief of staff person can easily manage that to save time for the managers because i don't want the managers doing spending a lot of time on it so that's why like test one and two or like here's the test and here are the answers you know only nine out of ten right passes it through and then test three is that like more um the apple's apple's comparison style test where there's it's not really like a obvious answer but it's obvious who's the best you know that's actually kind of an interesting approach something i might want to consider which is like if for anyone interested in getting the answers like whoever you wind up hiring have that person run through it with with those people i think that might be an alternative approach that maybe i should

put a little more thinking into but i i think that's interesting well we don't have that person do it we just send them that we just send that person's answer oh i see i see we just say like this is the person that won yeah look at their you know excel like i'm thinking of one for the data analysts we just had sent them a packet of data and said like build a tableau dashboard based on this data that you know does a bunch of things that does x and it's like oh i think i should have gotten this like okay well check out this tablet dashboard and kind of like you know now tell us you think you should have gotten it right like this is what we got from this guy or gal all right and uh and there's like oh yeah that's that's pretty good you know or they say no it's no mine is still really great and either we have to debate with them or like to your point tabor we can say okay you know we'll onboard you see how you do although we don't we rarely do that

all right well maybe we can now talk about the controversy all right taber so you posted on linkedin about written tests to a flurry of different discussions debate controversy so i thought we could talk about this the three of us may be a little bit biased because it seems like all three of us are supportive of written tests but maybe can you talk about your posts and maybe some of the feedback or the um or some of the folks who kind of what what some of their major issues or complaints with was with that please tell us a little bit more about that yeah sure so i started using um linkedin polls to try to showcase some of the the work that we're doing out of thinking ape um you know we have really highly successful games are highly profitable and we're really well known in vancouver but maybe not in the rest of the world so i was trying to showcase you know some of the you know more interesting you know systems design problems that we

have and so i was having fun with polls and i posted a pretty what i thought was benign updates saying hey i think everybody should get feedback on their test if your company doesn't get feedback on your tests i think it's a really good thing to do it's the least you can do and a person responded and they were very angry they were full of vitriolic internet fury and responded in a reactive way and basically what it boiled down to is they said that take-home tests are immoral unequivocally and i was so just blown away i was completely you know blindsided by this opinion so i tried to ask more questions like i tried to understand the point and because they were just so emotionally angry they were being a little bit of a jerk uh so i kind of agreed to disagree but i thought like maybe i'm just missing something when somebody disagrees with me that much i often think like maybe i'm wrong about this i need to like think about it more i need to research it and i thought well there's a pretty

easy way to figure this out linkedin's full of you know recruiters and hiring managers and people looking for jobs let's just ask linkedin so i did a poll and uh it kind of went viral a and you know i think like over ten thousand responses or views or something like that and um i just asked people are take-home tests um unequivocally immoral um are they fine and you know other why or why not and it just exploded and people from all kinds of industries started responding and i think it boiled down to a few issues that people have with some of the take-homes and um one of them that i found completely immoral and terrible is that some companies apparently will use work product from the test in the company so they get someone do a test and then they'll take that work product and actually use it to you know make a profit and that's completely that's you shouldn't do that at all sure like that's just silly um and so uh the other thing is that some people think that you should get paid for taking a take home which

i don't necessarily disagree with though i think there's some weird incentives there like i think as a hiring manager then i have to probably give it to less people and that gives like maybe some people less of a chance because obviously it'll be expensive to you know send a test to more people if you have to pay everyone that does it so the other part was people said um they can be really bad if they're too long which totally makes sense you have to find a way to make your test reasonable um and um but the vast majority of people were like yeah it's way better to do a test than an interview oh and then some people just think interviews are better i don't know if those people are higher probably those people are not hiring managers or not experienced ones in my opinion um so i did a few things i changed a few things based on that response so i shortened our test a little bit and i removed a question about one of our live games so that even though we would never use work product there could be no chance that somebody would misinterpret you know maybe we launched something similar to an idea that they gave us and so i wanted to eliminate that because i

thought that was pretty valid but yeah it was really insightful for me i learned so much from that post just hearing everybody's thoughts and feedback on it um but i was just so surprised i had no idea people felt that strongly i mean i just i just thought it was a given that everybody was doing them so yeah that's what happened yeah i remember i caught a little bit of flack on that one for for some point earlier but uh yeah i thought it was interesting um brett what's your i remember did you comment in on on that post as well or what were some of your thoughts are reactions from from taber's survey posts well i can't remember if i i mean moral is a strong word for anything to do with sort of like the that part of the process yeah but i have found and we have found that companies must do this where they they like post what you were talking about where they immorally post jobs than they use their someone's content because we actually have had people that we've rejected

get annoyed and say oh you're just doing this to get our work product and it seems like there are comings out there scamming because people have actually accused us of of doing that and we've actually in our initial job post like have paragraphs dedicated to that now that explain that this is a unique hiring process and we you know aren't using it we're it's legit you can go to our website here you can go to the ceo linkedin's page here and so unfortunately that is uh fairly common in the market probably particularly in for contractor roles the um the second is i mean uh i think that if you ask them to spend as much time on a written test as they would for preparing for an interview i think that it is not immoral at all right because theoretically they would spend time to prep for an interview although as all of us i'm sure can agree there are a lot of people who don't prep for interviews and uh you know a quick story of one

thing that happened when i was in blue shell games is i interviewed somebody who didn't have a job who hadn't downloaded the game or our biggest game and it was a 30 minute interview and i think we were about five minutes into the interview and i ended the interview uh in about six minutes in i was like you just didn't download the game it was a product manager role and i thought that was basically um that was a reason to stop the interview well that guy ended up posting in a forum because his whole company got laid off that i was a jerk and that um he got offered but he was turning it down because because i was so terrible and that got back to my boss who ended up having to like call me in and be like why didn't you know why is this guy saying this in this forum and i told him and he said okay well you should probably just interview for 30 minutes and then just hang up so you know unfortunately people can take advantage of it um and the last thing i would say is that we do pay for the last test so that does

i think help and we have found that it sues the rejection for example because we do ask them to do like five to ten hours of work and we're upfront about it in the beginning that they will have the third test will be paid and it'll be five to ten hours i think we have changed that though because we were spending a lot of money on that and i think what we ended up doing is shortening that third test and then just onboarding people but we did see that and when we have had people get really pissed that they are rejected because boy people have their egos and think that they are the greatest of the great at everything uh or some people do i don't want to sound too salty but we get some of that we do end up paying people and kind of saying like here how much time did you spend uh it'll be like 10 hours and we're like you know looking at the work thinking yeah right probably maybe three but we'll swallow our pride we'll pay them the 10 hours at the hourly rate they would have been hired for and we pay them and that basically shuts them up and gets them on their way and and kind

of proves that we're legitimate and if they're claiming we did it for their work then they got paid for their work and they kind of move on so that's a way around it if you get any flack for it and they're they're up front with you about it it's like basically say like okay how many hours did you work on this what's your hourly rate you know here's 10 times whatever your hourly rate is and like get lost you know i mean don't say get lost but you know that kind of ends it kick rocks get out of here yeah get out of here yeah yeah i can't i can't believe though people i've had this happen to me too right where people come and they haven't played your game it's a free-to-play game company it's free to play the game the very least you could do if you're listening to this and you're applying for jobs at free-to-play game companies download the game and play it like how do you know if you want to work on that kind of game what blows me this is product product management role the product manager role like you you can't download the game that would have taken you five seconds and you and you guys have known at this point like i don't really uh i'm not too uh afraid to just speak my

mind and just get it i was like all right well this interview is done it was like five minutes in you know i mean it was a very short time well he didn't like that i'm really sorry you know um that was for anywhere it's not even product management looks possible even if you're going to be an engineer any role you should play you like use the product of the company that you're applying for at the very least i'll give a counter point to that is and i was just talking to someone who's been at my company for nine months i've never spoken this person until this week um and the people they love our hiring brothers because because they're the people that are really talented who've been overlooked because they weren't you know cool enough or weren't really good at happy hour or whatever you know or weren't um whatever the case and they're like dude i love your hiring process like this is amazing because they love the fact that it's based on skill right like it's based as much as it can be on skill alone and so the people that are complaining saying it's immoral are probably the ones who aren't very good

who want to [ __ ] their way into in a into companies you know no comment i'm not gonna yeah i thought that you the post raised the number of and you know of important issues that probably don't get discussed enough i think the issue on whether you pay or not i think that's a that's a delicate one right we have a mixed policy where at least for like product management or design we don't really pay for the tests but for artists we often do it's like if we're trying to see like we could there's a lot you could see like from an artist's portfolio but then it's like well can you hit the style or things like that then we'll we'll generally pay artists but we won't pay the other roles and then i do think that um one of the issues that came up where i changed my practice after reading your post was you know there were people who were who are complaining about well you know you could steal uh somebody's idea or this

or that so then instead of so i used to have questions that were more relevant about the specific role and then so like for like let's say for a producer instead of asking them to create a staffing model i changed it and said well okay create a model for a candy shop we've got hershey's oreos this that and then i changed it and then but you know like the models i got back weren't really as specific or relevant and so now i was just like you know what i'm just gonna forget about all these like you know like i have to do what's best for the company what's best for the company is seeing if they can actually do the actual work so i mentioned i rewrote all of our tests last night and then basically my my thinking now is i really don't care what those people think i'm gonna do what's best for our company write questions in a way that's very specific to the work and i mean realistically like i don't i don't know who these people are i do i do agree though that there are some people who are just crazy and they're going to think that you took their idea or whatever

i i can literally count on one hand and i actually remember the names of the people who i actually learned something from during an interview and it was never a written interview i've never learned anything from a written interview but from a live interview i remember the two people because it happened so rarely and it's not like i even used what they you know their their insights on a product at all so i don't know i i do think that this is a sensitive topic some people are going to get upset if you ask specific questions like you know we're a shooter game i'm going to say for you know how would you think about battle royale how would you think about this and and i mean you know there are going to be some people who who are going to be like oh i'm so smart i i'm going to know i'm going to like put in a response that that they think i would have never thought of but guess what i i really really doubt it if you do i'd probably hire you so anyway that that was my thought on that um

i don't know i don't know if you guys have any more thoughts on that or maybe we could next move to how do we actually recruit in terms of channels brett you mentioned you're pretty heavy into craigslist i don't know if you've tried or thought about trying anything else or external recruiters or things like that but maybe we could start there uh and with you brett yeah well before you go on i do think i want to stress that i love the the paid i would even say paid tests like for the artist i would for example we um i would just pay them to do the work and i would just pay a few of them for example we were hiring someone to build our dashboard design and i just looked at a bunch of portfolios and hired three people to actually build the dashboard and it was costly in terms of a money standpoint but it was extremely um inexpensive in terms of time because i just wrote out like what i wanted once and then i sent it to three different people and they just built the dashboard design all three of them and then i picked one i mean in terms of my time it was extremely low it cause i had to pay

three people for a dashboard design but that person was fast and amazing and we got a finished product so i i think for positions like designers and things that are very like tactical paid did not even test but paid projects right up front is a great way but that's that's a little it's a little trickier when you have like because usually there's like an um an ip agreement as part of your employment contract so when you're interviewing someone from like a competitive company like uh you know zynga or ea or whatever like having them actually do work that you use i mean that opens you up to all sorts of ip and legal issues so yeah yeah obviously with that kind of stuff it's it's sensitive but i mean if you're hiring for something that's like a contract role or something like that okay right right right those type of work yeah for that type of work i would i mean again i i'm making recommendations that are bad for my company because i'm gonna be more competitive but i think for that type of

work i would highly stress going contract direction you know the type of work that it's very like specific and you're going to be just like creating work product you don't need to be like you know sort of like in the company culture as much it doesn't need to have like a dynamic personality or a dynamic skill set it's just very much like a you know build art or build code or these things like that i think that contract work is just like a much better way to go but you know yeah i don't i don't want to talk put words in um our vp of creative's mouth uh tyler schrader he's our vp of creator and i believe that he's gone away from doing art tests because in a he thinks that you can see if an artist can do the work based on their portfolio the question is always like how much were they involved in that work like sometimes people put something in their portfolio that maybe they're working on project and they put the art from that project so you have to figure that part out but i think artists are much it's much easier easier for them to show off their work than it is let's say for a designer to be like look at this amazing spreadsheet i built they could just we don't it's not here's my spec

yeah that's just my design yeah that's a great point we actually did that with our writers is like we just have them submit writing samples now because it widens the funnel right it's like well okay you wrote this so you don't have to do a lot of spend a lot of time on it so i agree with that yeah if they have previous work and this is why people should think about creating their own blogs and things like that because the more of your writing the more of your work that you put online you know there's less evaluation on on the part of the the recruiter so yeah i definitely think people should also think about just doing that for themselves personally as well it's on my list of things to do yeah yeah the dashboard people we hired was just looking at their previous like tableau dashboards and stuff for dashboard design right it was like yeah i can see all your work this looks pretty similar and sweet yeah so your other question is platforms yeah i mean we do craigslist and we're examining a whole bunch in fact we i mean if you ask me this question in a

month i probably have a better answer for you but we just built a spreadsheet of how many people we get i think it's between three to five dollars per applicant on craigslist and as i mentioned we spent ten to ten thousand dollars in the last 12 months on craigslist for our jobs and we're going to be testing linkedin instagram indeed and a whole bunch of others and looking at the stats for that so if you're really curious about it about in a month two months time we're going to have a lot more data on it but um yeah we've been craigslist has been great for us and you can you can get a wide variety of people from people with ivy league backgrounds not that they are any better than anyone else but and people with no college degrees who are excellent workers as well and uh you'd be surprised all right what about channels for you tabor yeah so i think you know we post on like all kinds of job boards it really depends on the role um and like i was saying earlier their seniority level really determines how much inbound we get from those different postings so the more

like the less technical or less skilled the role is the more applications that we get from job boards and then for more senior roles it's all there's a lot of active sourcing and you know i don't think i did a good job of maybe selling the company at the start when i was introducing myself but you know we have been making you know games profitably for over a decade we have you know an incredible studio an amazing work culture and in vancouver we've done a really good job over the last decade of becoming you know one of the destination studios and so now that we're remote first we have to do that around the world and that's you know part of the reason why i'm doing these linkedin polls and coming on a lot more podcasts and trying to get our name out there a lot more is the you know the pool is a lot wider but i do a lot of active sourcing for people and i think that it's really useful for me as the hiring manager because i get to talk to people and understand what's what are they interested in like if i pitch the studio to them i can figure out like what things resonate with people the most so we can tailor a lot of our marketing to candidates around the things that people actually care about because sometimes there's things like you know a long time you know there's always the

the you know joke about like culture isn't just like a ping pong table and a beer fridge right you really have to talk about what it is that your company does that makes it unique and and so figuring out what parts of that resonate with people is really important um for example one of our core values is have wonderful arguments and i'm always surprised how many people read about our company and they wanna they wanna know what that means and talk about it and and for us it means that you know in order to make good decisions we have to be comfortable disagreeing and we have to be comfortable um you know having our arguments but we have to do it in a way where we show each other that we care about each other and we're not doing it in you know the brilliant jerk kind of way so we follow the radical candor model and i think people are really interested to hear about that and just talk to you somebody who actually works at the studio i mean recruiters are worth their weight in gold okay they're they're amazing but i have a much higher hit rate when i add somebody on linkedin and say hey i want to talk to you then a recruiter does if the vp of product sends a message to a game designer is

like look i want to talk to you i think your experience is interesting that's always going to be better than having a recruiter yeah or you have your recruiter use your link hey that's a good idea i should try that you can talk to me offline i can give you some tips anyway um yeah as far as like my own personal experience i i think that we've had the most success through linkedin either direct outreach or through linkedin ads and then we tried a number of boards but without much success so we tried indeed you know we tried hot jobs we tried uh i forgot some of the other ones monster and then um we're recruiting in india so we tried some of the local job boards as well as like now korean things like that and we just haven't had a lot of luck i will say like the other source where we've had better luck is angel list so for us it's been mainly like linkedin and angel list um and then i would also say just uh

probably the the third source is um you know we we have like a so i i have a newsletter the game makers newsletter and then just people just coming in from the newsletter and you know as i post jobs there that that's that's been helpful as well but um do you do you hire people directly to a company that you operate in india or do you contract them through like subcontractors yeah so like the way we're structured is we have like a u.s delaware corporation and then that company owns the it's called lila private limited the indian indian entity and then there's like a there's like contractual agreement between the two but like it's almost fully owned besides one share because of some indian legal restrictions but we also give like um stock options to the us entity to all of the employees in india even though they're part of the indian company as well so you know it's um

you know for us i think it gives them um you know a little bit of uh the upside in terms of the overall company as well that's really interesting yeah this uh i wanted to add and i think this is a good illustration of it is that we're talking about the hiring process in isolation but i think it's it's so important that the whole company kind of supports the hiring process and it's hard for me for example to talk about my hiring process as someone at another company because it's like talking about my arm on someone else's body right it's like the whole structure of the company has to kind of feed into that hiring process and for example your company legally has like an indian entity to help you hire in india and our company for example is completely remote and doesn't have a lot of meetings which helps us with much more contract workers for example and i just thought of something while i was going p that uh where all most of all the great thoughts come in the the

bathroom um which is that it's extremely difficult to hire these higher level rock stars right like these managers these product managers these art leads and things like that and i've really struggled with it too i think our system's really good for hiring contractors who do very specific things and our business is built to have a lot of specific things but it's very difficult for me to hire the leads and we either get those for referrals or they level up in our organization but i honestly don't have a very good solution for it but the thing that came to me in the bathroom is that what i think is the case is those are just very hard to hire for they're very hard to find and they're very risky to hire for so i think in terms of an org structure what i'm going to do what i think is the best way to do it and that's why i'm going to do it is and i drew it out is that you know basically you want to have a you know like leader and then these leads but underneath the

leads you don't want to think about it in like sort of this typical sense where you have you know a couple people under them right like a lead has like three seniors or whatever you almost want to think of these leads that you find that are it's rock stars as being these just like super powerful like humans in those like in those robot bodies that make them be able to like jump and everything where they have like 10 15 20 different contractors to support them so you basically make them become these like super super employees right because they're so hard to find and they're so risky to find it's like make them a hundred times more powerful with like support of designers and contractors and excel spreadsheets and editors and chief of staffs and you know whatever and just let them do their thing as opposed to trying to find like a lead who's really good two seniors who are good two associate three associates that are really good like it's just so hard to find those people and

that's the direction we're gonna go in i think so did you draw that in the bathroom too no i drew that when i came back i'm not that just checking yeah i can barely i can barely just pee but okay no when i came back and you were talking i drew this with my uh yeah but speaking about superstars that's the other thing i wanted to talk to you guys about because we well i'd love to hear your first your thoughts on that taber because i know you have some questions about it before you go on a superstar um yeah so i think that that model can work and actually this kind of ties into my response about superstars in general is that that i think that model um i mean that's the model that ea sort of popularized with producer like they stole the idea of producer from the music industry with this idea that like a rockstar producer can work with any band and make them successful right um but i think that there's some problems with that for a sustainable organization

and that then your superstore superstar is sort of like a single point of failure and you give too much responsibility and power to a single person depending on your org i think that that can be uh risky it works for a lot of people like there's a lot of uh you know game studios just built i mean look at how many press releases come up like from the team that brought you halo and halo 2 and now they're making a game a lot like halo it's called destiny like that stuff works and those people know how to do that and but i think if you want to make an organization that is innovative and um and you know like less fragile if you want to be anti-fragile i think that the approach should be to find those people and have them coach and create other mini superstars um but to try not to rely on just the superstar model because you just end up with yeah that single point of failure end up with like everybody's a supporting cast of that one person and if they're not happy or they're not performing or they get you know hit by a bus or something then you're you're

screwed dude that was so spot on you just threw a huge wrench in my process idea you're completely right you don't want to have those people you don't want to have that because you're right like they're rock stars they could ask for a massive raise they could walk out on you or whatever like they you know they got a big ego then they want to do whatever they want yeah forget that system i mean you in in essence i've built you know let me just draw redraw what i've what i've kind of built but it's more like this okay this is my i mean it's it's more just like a straight it's just me and then just like 50 people underneath me um and that's sort of how i've built it in some ways although we do have leads and seniors and things like that but i agree i think ultimately you don't want it's relying on any superstar even yourself that's what i've been trying to solve for the last year as i've been trying to basically replace myself but you will write you want to replace yourself with

non-all-stars you want to replace yourself with people who can like like positions that can be done by lots of people so you really want it to be like i don't even know what the picture is but really just like a like i'm almost thinking about a globe with a lot of different dots that can be filled by almost anybody and also filled by multiple people so it's like you have redundancies in the company which is something that we actually do a lot of here a little bit off topic but yeah that's interesting you know in that topic what we talk about a lot is like building the machine that makes the machine so i think even if you do hire those rock stars your goal should be for them to make a bunch of mini rock stars and to extend that knowledge out into your organization not just through coaching and mentorship but also like documenting what they do and what they know make them thought leaders right so this idea of playbooks that we've taken on where people write playbooks for the things that they're

really good at and really passionate about so that information can be disseminated across the studio and they become living documents that other people can work on and that's been a really effective way for us to try to do that you know process of it's making that information and that skill something that everybody can learn from and we also do things like cross-team um designer meetings for example we'll do workshops where somebody who's really skilled at something will do like a workshop on what they're working on or even like pair up with a junior designer and have senior designers paired up with junior designers and like help them with you know hey how is your spreadsheet set up for this let's look at it let's like make it more efficient and so it's all about you know training and coaching and and that model means though that you have to give people a lot of responsibility and you have to give them space to fail and um that doesn't work for every organization we have an incredibly stable live ops platform of games where we can give people varying levels of responsibility and risk to learn from in a way that doesn't negatively affect our

studio if they fail because we can sort of like toggle and control the level of risk that that they're facing but like joe for you like you you have everything on the line for this product and so like you can't afford maybe to have that kind of uh that risk tolerance so probably you're going to need a lot more rock stars like people that have proven experience that you can trust because this is this is it for you right yeah and i but i think to your point about why do we seek these superstars or rock stars it's because they have specific knowledge or experience and i think there's talk about it in two ways right so like one kind of talking about superstars one kind of superstar has very specific experience right so like for us like for for a shooter game it might be it might have to do with netcode it might have to do with like how do you design the feel of a gun and the kind of tools you'll need to build in order to optimize that so that that's very specific knowledge or like level design for a map or something like that right and so we're looking for very specific knowledge that's one part of a superstar the other kind of superstar we look for

is just skill and capability so like i think one of the things that i have realized is that that talent is not linear right and so you can have an engineer who is literally 10 times better than the mediocre engineer not superstar engineer and actually not even 10 times it could be infinite because this superstar engineer could quite literally do something that the rest of the team just cannot do so that and so to some degree that difference is infinite but um to your point though the reason why we see these superstars and this is something that i learned working on king of avalon f1 plus was that we had something which i have now come to refer to as the leads based model which is that we took a relatively inexperienced team but then we we've found a lead from the best you know we started from

the it was always like okay who's the best person in the world let's work our way down and then that person would join the team and then to your point teach disseminate the skills so that the rest of the team over time levels up much more quickly than if they were just like well okay let's try to figure out what the best practice is let's try to figure out let's kind of iterate through this and no no it's like you want to like for those superstars if you pay 2x 3x 4x 10x it's probably worth it and so my question to you guys then because we're just starting this this program right like how are we gonna find and and get these superstars like how how would you guys go about trying to attract these superstars and i and i may know some of the answers that you're going to say taber but the one of the ways that i've seen it is like quite literally these you know some of the people who are recruiting these guys will try and get into like their game clan and play games with them for like

hundreds of hours and then like half a year later you know try to pull them over something like that but i don't know what do you guys how do you guys think about that pay them ungodly amounts of money he's like yeah sure like if i was building a shooter game i would be going to like the destinies and the bungees and the four nights and i would find their best person i'd be like what do you want what would it take yeah what what how much money do you want how much money would it take and there's there's gonna be some people were just like what get away i don't wanna do that i like what i'm doing but there's gonna be somebody there's gonna be people who are interested in one a novel problem and starting something new and two um just ungodly amounts of money that's just the truth if you want a superstar you got to pay for it right it's like that you know the old saying it's like you know the the bus doesn't wait for anybody on the basketball team except it waits for michael jordan like michael jordan doesn't show up that's going to wait for him and say you know it's the same same thing when it comes to salaries like if you have that model where you need that person just you have to pay them yeah and you know it i think it was google

that was one of the first to do the study on employee compensation and what they found was that the way that most companies think about compensation is linearly meaning that you know they'll pay people um let's say 10 more even though that person might be doing 50 percent more work might be might be three times more productive but the talent isn't linear right i mean you have you you have a steph curry you have a lebron james in the industry in work as well and so if you're gonna try to pay them ten percent more you know it doesn't make any sense and so yeah i mean i think that's something that some folks at least in the industry will need to wrap their minds around is that for that superstar talent you gotta pay them more um i guess i i don't know any other ideas besides just paying them a lot of money because for a lot of these guys i mean they they will get the offer with lots of money i think you need to be building

something that's interesting right yeah and that's hard too because like one problem that we have is we have really interesting problems we have really interesting engineering problems but because our games are relatively small compared to other games in the mobile industry even though some of our user metrics are quite competitive is people don't know about it so that's one of the reasons i'm trying to get like hey these are the specific problems that we're working out i'm trying to get that stuff out on linkedin and just say like hey have you ever thought about this problem how would you answer it and i do a poll and just try to get people to think about hey here's a really interesting problem how would you solve it and then that at least showcases some of the the some stuff that we're working on right and then of course you're doing an amazing job of putting yourself out there and talking about what you're doing and doing these podcasts and doing videos and that's a really important way because now people are going to look to you and they're going to learn stuff from you they're going to see you talking and they're going to say like i want to work with this person so i think that's a really good way i don't know if like gaming with people is necessarily like a sustainable approach like if it works so it works like we had a guy that we played hockey with who he just sort

of like signed up for a hockey team and he was a candidate we wanted to hire and we played with him every you know we played hockey with him he played on our team and just like hanging out with him all the time we'd always just be like hey so when do you come work for us when are you going to work for us like no no no and you know you shower with a guy enough times and eventually they they want to work for you my hrt would be very bad if i said that it's just a joke it's a joke just kidding okay um so there and that's actually how can she recruited me i'm sorry not in the shower um by uh coming out to i was doing meet-ups in vancouver and he started coming to my meetups and i asked him to be in one of the presentations i was doing about day analysis and we just kind of got to know each other and over the years he just was really nice to me all the time he anytime i had a question he gave me the time you know even when i was a junior designer and he was you know the ceo of this company he saw something in in me and he always gave me he always talked to me and every time i was at gdc would get like you know some private party and he would come over and talk to me and just over time built a relationship and i wanted to work for him

i you know just because i really respected him and he took the time to get to know me yeah it does seem like with superstars it's more about the long game right like being like just making sure you keep in touch with those guys once a quarter or something like that and um and when the time is right because i think everyone experiences some change in their lives maybe maybe that's the time to if things align then you can try and pull one of those superstars as well as to your point you know financial compensation as well as things related to the mission the problem you're working on things like that but and work culture too is really important to senior people like what is your work culture um you know i had some feedback from one of our senior people that you know one of the things on our work called in our company culture is best ideas win and he gave me some feedback that like you know that's a really powerful um you know core value but it cannot but for really senior people it might also make them think that that means that everybody you know gets to have an opinion about every decision and so we

should be really careful about how we articulate what that means at the studio and there's a lot of things where like if you have senior rock stars have them go through your entire hiring process again and be like what's good about it what's bad have them audit it for you right i think also as a as a business owner even though you may need a rock star and this is my bill belly check in self talking here jk all right you want to either after you hire that person or even before start figuring out ways to reduce your need for a rock rockstar because building a business around rockstars as you brought up taper is very dangerous and very costly and so i think as much as possible you really want to be de-risking yourself for or removing the necessity for rockstars and i think we've chatted about how it was pretty inspired by the movie the founder which talks about the mcdonald's and how they built a large business off of like operationalizing the cooking experience

but um yeah i mean as much as you can you wanna and in that way you can almost again go back to contracting it's like distill down what it is about the rockstar that you really need right in the case of shooter it might just be the concept of the game or it might just be some some small aspect you might not necessarily need that person and it's like well how much can you pay that person to give you an idea probably if not that much you know for an idea that might not work right i mean would they take fifty thousand dollars twenty five thousand dollars for a bunch of ideas on what to do with the shooter market probably if it was like a weekend's worth of work or something like that i mean so there's i i gotta be thinking about uh elf it's christmas time but you got me have you seen the movie elf yeah yeah right do you remember the the whole premise of the movie is they can't come up with a good book idea so they hire that writer to like come and give them an idea just

for one pitch meeting and he like comes to the border with these ideas and and he's like you know it's that concept you probably could find somebody who's like really really good maybe that maybe they're between jobs they don't have a non-compete clause and just have them come in and tell you like how did you how do you do this like i i know a guy who i talked to one time he was involved in shooter games anymore but he was one of the guys at ea that had designed the math for um the the part of auto aiming when you're aiming in a shooter where it like you know sort of like pulls your your um your your target into the target or whatever the the cross you're into the target and uh like i could hook you up with that guy i don't think he's like working on any shooter games he probably he loves talking about this stuff you could just like get him drunk drug start talking to him about his like you know his work and he would tell you about it so there's probably a lot of people out there like that too yeah yeah i mean i i think to th that's probably like a whole other podcast on its own in terms of you know how do you because i think i think there's always going to be a trade-off between risk in terms of somebody leaving and

then how do you building in redundancy around that person leaving or not and the reality of the current situation that we're all in is that most likely you're not going to be able to keep everyone forever so um i i do think that thinking about how you um you know not not necessarily like in a mean way but you know for the in the interest of the company making sure that the company can survive loss of any single person is obviously something that we're all going to have to think about and usually i think earlier stage startups probably take a little bit more risk where if certain key people left there'd probably be a significant impact but then i would say over time you start building in more and more redundancy for sure yeah and i i do want to highlight one part of this so that we're touching on that's key to our hiring process and i think could help a lot of others is that really understanding what exactly it is you need that person to do and weighing that thing way more heavily than other

things right and to your point of linear versus exponential in the hiring process i think we wait or before i created my own hiring process and without other companies they would weight a lot of different things fairly equally right but when you're thinking about the rockstar the rockstar that you're talking about it's like the idea that they have about the shooter is maybe infinitely more times let's just say a thousand times more valuable than their ability to get along with the team right but when you think about it through the bias interview process a lot of people weigh those things fairly equally right and so if you really distill it down and push yourself to be like hey we really just need this person to give us the like best uh you know auto aim system right and that's the most important thing by a thousand x then your interview process is much more accurate i think about if you should bring him on or not if he's a huge [ __ ] it's like well we're willing to deal with it for five months because we just need this like shooter accuracy right and so

i think that's really important that's helped us a ton right and i think those are all the questions that i had but one one comment you made paper around you know how do you even test for culture do you guys want to touch upon that and whether you guys actually do or not sure um so we spend a lot of time recently thinking pretty deeply about culture interviews um see that's why you think that we think deeply because that's how we answer every question we've spent some time thinking deeply about this um i did a quite a bit of review of even some um like business journals trying to understand like where where is you know even the literature in on this and what came to the conclusion is that um i think a culture interview can be useful to um like raise a red flag about a candidate like if you have a culture interview um i think i think that there can be scenarios where like this person you know has you standardized questions first of all um with like reasonable answers where you figure out you know if the person's going to like be a really

really bad fit so a good example for us and um hopefully i'm not giving anything uh away too much is like if somebody doesn't like free to play games they're not going to be a fit at our studio and you'd be surprised how many people apply where they're just like i think free to play games are predatory okay well people free to play games so do you you really want to work here probably not um but that's an example of like we've actually had a lot of people at the studio who become like kind of toxic because the truth is they don't want to make free-to-play games they took the job because they needed a job and we want to try to eliminate those folks so that's a good example of a standardized question it's not about like hey are they going to be fun at the christmas party it's about like do they believe in our business model um but like it's i don't think that you can have a somebody who fights to hire somebody because of culture right so a good example of this would be like engineering let's say because it's pretty technical if somebody fails your technical bar but has an amazing culture interview are you going to hire that person probably not and i think where that type

of thing happens is when you're dealing with like let's say a manager or a lead and you're trying to hire them because i think a lot of folks believe that management and leadership is about culture but it's actually a skill that i think you can test for and so in every role i think you just have to be careful that you're not hiring people for some kind of implied culture fit but rather you can use culture to make sure that they're going to fit into the studio for a more standardized set of things because just like anything else culture interviews are probably far worse than a technical in-person interview but it's something that i still think that there's some usefulness at least to like make sure that you're aligned on the same same incentives as well like asking somebody like hey this is how our conversation is set up this is what our bonus structure is how our bonus structure is set up like what do you think about that you can probably learn a lot about how somebody how somebody reacts or um like work life balance what do they think about that like maybe you have like a really a studio where you give people two weeks off and everybody that works at your studio loves that because they all love to work and there's somebody who's like yeah i take two

months off every year like that's a good thing to know um you know in our studio we have we have we we believe like work-life balance should be a very personal thing so like there's some people that want to work all the time and hate vacations some people like need a month off to you know to focus and that should be specific to the person uh but i could see scenarios where those questions can be really powerful to like understand somebody's motivations i guess and um but they should be standardized like 100 standardized and everybody should be trained on them so that you're not getting you know just like a random person coming in like i don't think they're a culture fit like why like i don't i don't know i just didn't feel good about them like why i don't know i just did it and then now they're committed to that answer they don't have an answer it's just something they said that you know didn't make them feel good and it's like we you need to be way more specific about that stuff that was kind of randy sorry that's great read any thoughts well we obviously have gotten around that with the contract hiring because we

get people to be on board for a couple months and we see if they really fit so that's one way again we've we've we being me have chosen this contract model because it gets around a lot of the issues that have to do with full-time hires right i'm not just doing contractors because of whatever like these are the reasons i do hire contractors it gets around a lot of the issues with the hiring process which are very detrimental to companies but i think in terms of the culture we use the you know couple first months to determine it also our hiring process is very similar to our culture we don't talk in our in our culture although some people are apparently slacking more than i thought and we don't i'll talk to you during the interview process to your point tabor i think one thing we've also found is because we are so clear about our culture and we're fairly extreme about our culture right a lot of times we actually have the candidate opt out

because the candidate doesn't like the process so they may be like well i'm a little uh overwhelmed with this like hands-off e-culture or whatever and we're like okay great you know or they'll be like hey i'd love to just jump on a call with you and uh hear a little bit more about your company and i'm like fail you know because it's like that's the opposite of our culture right like i'm not going to spend an hour talking to someone that has a very low likelihood of being hired at my company um and so and they don't understand that right away then they're not going to be a good fit so i think the other thing probably the last thing is we're hypersensitive about that stuff like when you see just a little sliver of it generally like it's only going to get worse after they onboard so if you're seeing sort of the slivers like well good examples we'll onboard people with an email and if they ask like questions that are written in the email like there's probably a 90 chance they're gonna get fired with it or like let go within the first two weeks like i've just seen it so many times where they'll ask a question and you know it's like on the

fourth line of the email or the way down the bottom of the email it's like yeah this person's not to last right like if they don't have the ability to be diligent on the first email it's only going to get worse from here as well we might as well just let them go pretty soon and that's like kind of their first red flag we see anything else they're they're gone so people self-select pretty much yeah people self-select but also like we're really hyper sensitive about little stuff you know and i think that's something like you have to be careful of an interview process in particular is your brain just likes to convince yourself of not seeing data points right oh you know yeah he answered that question weird but um he's kind of a good guy right it's like it's like this what's your brains doing over and over again and what i've trained people to be like no you see that one little thing right like this they they had a little error in their email or they didn't read that one line like we just had something the other day we had to resend the like google invite a second time right like to me that's already like that person is probably already not gonna make it

right like because yeah what went wrong like we don't i don't understand how you couldn't get that like what does that mean for you handling much more complicated things right um and you can kind of see that early on but it's like our brains are so trained to be like no no it's fine dude like you said this person was good like overlook it overlook it overlook it right and it just gets worse you know it never works out right when you take that chance on on somebody that you that you have like not somebody that you have a gut feel isn't good like you shouldn't trust your gut but when you have like evidence that they that they aren't going to be able to do the job but you do the opposite that's yeah that never works yeah the one thing on culture that we have found and i don't know how to solve for this i'm trying to figure out a good interview question that can be standardized for this is the biggest culture problems that we've had are people that we we say they have a distortion field thinking what we mean by that is like you give them feedback or information comes in but when it comes back out it's different so like you give them feedback and say like you know you

um have been interrupting people in meetings a lot and like we'd just like you to spend some time thinking before you speak and what they hear is like oh um i'm really good in meetings and nobody is listening to me and i need to like talk louder and you're like what no that's i said the opposite and i and it's wild when you have someone like that like i've had i've seen it maybe like four or five times in the last 10 years and they're incredibly disruptive because oftentimes they're like yeah okay i'll take that feedback but in their minds they processed it differently and they go and do something completely different and it takes a while sometimes for those people to get picked up because they say the right things but then they do the wrong things over and over and over again and it's hard for a manager because they're like no he he gets it he's going to work on it or yeah you know like they're they're really they really understand but then it just keeps happening because they don't they can't process information in the same way as everybody else yeah that i i call that the difference between someone that hears you versus listening to you in terms of feedback and i think one of the things that i've done

against that is um just you know is i i think you can't give the feedback just verbally in my opinion so what what i've done is actually given them the feedback verbally and then written it down and then have it in a written form and then we go over it together and then it's like oh but i don't know you so you may try that maybe that helps maybe that doesn't that's probably based on the person in terms of like testing for culture i personally do like to test for cultural fit i just because we're pretty particular right like um and to very similar to brett like i kind of feel that life is so short and like wasting people's time is something i really hate to do both on our side and their side and so we generally do try to like bat signal as much as possible like this is our culture this is what we're like you know we we aren't like if if you're if if you're nine to five and like work life balance go you know you do not want to join us be somewhere else

because yeah because i think it just saves so much time if you if they can like see what kind of company you are and then just self-select out that you know just makes it a lot faster and more efficient for us but then we also try to test for that as well now i think the the one tricky part would be like you know um [Music] for younger recruits who don't actually have a culture who are trying to figure it out and then i do think that there's kind of like this perspective around frame of reference and as an example you know it's it's i i think people can be the way that people view the culture of a company or whether something is good or bad often in my opinion comes down to frame of reference and how you're thinking about it it's kind of how like james clear in his book atomic habits talks about you know you can think about work in one of two ways that it's either work you have to do or work that you get to do

and it's you know and i you know i'm reading blog posts about people who who talk about well you know you can't overwork people they're working too hard you burn them out they get tired you have to give them breaks you have to give them rest it's like oh yeah yeah yeah that makes a lot of sense you can't overwork people can't do that and then i see um you know and then i watch a video interview with like where joe rogan interviews david goggins and david goggins like man you just gotta like get the [ __ ] done just work on it stop being a [ __ ] and you know just like get get [ __ ] done it's like yeah you should just you know and so like the way and you know i'm the same person right but the way that it's kind of framed and how you view it i think that also is part of not only your culture but how you interpret the culture and i think especially for younger people who don't have a strong point of view that that can actually just be well they just don't know and so and they're still trying to figure it

out so i think that's a tricky part with um with the cultural interviews as well you i have thoughts on that yeah yeah go for it so um i think for young people one thing that i tell young people is that when you're young in your career you have to make a choice what you don't have is experience and there's only one way to get experience and that's to spend time what you do have usually when you're young if you're lucky is a lot of time and you can choose to do what you want to do if you want to spend that time working and getting experience you will become experienced if you don't want to spend time getting that experience then it's going to take you a lot longer the best thing that you could do when you're young for your career is to work a lot to spend that time you're going to be inefficient but you need to get that experience the only way to do that is to put the time in when you get a bit older sometimes you become more efficient and maybe you need to spend you can spend a little bit less time on work because maybe your job actually requires a lot more i don't know deep thought than it does actually like sitting down and grinding out work right and the other piece too is i am so

surprised how many young people ask me now in interviews about work-life balance this is a new thing that's happened maybe in the last three three or four years and in games i think it's because there's so much focus on crunch that it's become this thing where everybody is like well the games industry is full of crunch and i don't want to crunch crunch sounds bad i don't know what it is but it sounds horrible and i'm like well it can be absolutely if somebody is forced to like work all the time and you know it's a horrible work environment and working 15 hours a day you know that that sounds terrible but most of those like a lot of people have a choice not everybody does and everybody's lucky to at our studio what we do is is we say like look get your work done if you want to do it however you want you know when you work best you want to work all night on something work on work all night and sleep in i don't care as long as you work hard and you get your work done and it's high impact that's the focus you're you might have to like spend more time at your desk when you have low task based maturity right that's a popular phrase from andy grove's book high output management

task-based maturity you need to spend more time on it you need to get that experience there's nothing wrong with that there's nothing wrong with working hard a david goggins on the other hand like that is he's like a genetic freak that guy like i would not be trying to compare myself at all and like i don't know if if you if you do any um like sports like i do a lot of weightlifting and one of the things that happens as you get older is you just don't have the recovery that you used to have and fatigue management becomes really important and actually becoming really efficient about what you're doing is way more beneficial than just putting in more work putting in more work until you're just like broken your knees don't work and your all your joints are broken and i think that's true of you know working in in in games or in any other discipline is like you have to figure out how to be efficient the only way to do that is to do a lot of stuff yeah i totally agree i mean dude i could go on and on about this stuff you know i think about it all day long but i i mean another thing i really love about

contract and i think when we go to full time which i think we will for some people in 2022. wait what yeah i know well because part of my vision for the company is provide excellent medical and and so i think we'll have to do that to do it and the only reason we haven't is because it's very difficult for small companies to provide excellent medical in america but we have anyways we're figuring out a way a little bit around that but anyways um to be able to provide great medical but the um the point i was going to make is the hourly pay we probably will keep in some format because one of the things that i find is that when you pay somebody a hundred thousand dollars let's say and one week they are working at a rate of uh let's say 70 hours and the other week they're working the rate of 20 hours it's a it's a really just a bad agreement right like if i it it's it's a over the maybe the year you're averaging it out but who knows and so hourly is

much better because it's like well if you have to do more work on this on the weekend or whatever you're going to get paid for it right it's not like extra work like i just assigned somebody to build our retro template right and it wasn't part of his job but to him he's getting paid extra for that it's not like i'm assigning extra i am exciting extra extra work but he's opting in to work harder and what we found i think the and what i think is the best option solution jk i mean obviously everything i recommend i think is the best solution but um is that we basically have a opt-in time frame right so some people will actually tell us who are rockstars like hey i just want to work three out of four weeks we had a video person it was amazing that's what she said another person wants to work 50 hours a week right and so we just built a system that allowed for both and the person who works 50 hours gets paid 50 hours times their hourly rate and the person that wants to work 20 hours gets paid 20 times their optimal work and so

it the nice thing about it they were too is that like you said work-life balance is becoming much more attractive so the the employees and the potential candidates are weighing that much higher in their value scale for picking your company and we do find that we beat out companies we beat out nike and other companies that we're basically saying like listen you know you tell us how many hours you want to work and a lot of them the answers are surprising and they'll be rock stars who say that they're rock stars who are like you know what i want to work class i want to work 20 25 hours a week i want to travel to spain it's like okay cool you know i don't care right i get you for 20 hours a week so be it right as opposed to you going off to nike or wherever else you want to go so the personalization system i think is really really key a really great uh to getting yeah i think to your point brad there's probably ways in which you can kind of make your culture a little bit more flexible by having opt-in systems like that i know lilith for example i

don't know if they still do it now but if you work the weekends they just pay you an extra day or two of work if you work both weekends you get two more extra days of pay that's an opt-in system yeah but um you know and i know we're starting to go a little bit off topic here but i do think that like the you know this this notion of about you know people wanting more work-life balance i i it's it's kind of a bigger there's all sorts of theories and stuff we can talk about but like you know i this is something i've been thinking a little bit about in terms of the millennials and um you know why why why why they're a hedonistic and there's a you're not going to drop some simon sinek are you oh man no no we almost made it through the podcast without you crapping on somebody oh you got it in dude you got it in the last the last five minutes you [ __ ] on the millennials dude well no i'm a millennial so be careful here yeah but it's i think i'm like right on the border too

there's a perspective uh that peter thiel speaks about about how if you view life as deterministic versus random and if you view life more as random you become more hedonistic and like more likely to you know value these things like work life balance and things like that when even when you don't know if life can just end any moment now or if you think life is just more full of chance but i think at this point we're going way far afield from recruitment [Laughter] talk about elon musk oh yeah bro what was your beef on elon so i love elon he's my he's my hero i was just watching a video from his uh his uh first wife and it wasn't that positive and that's partly why i you know be careful about who you've said as your heroes you know i i remember when i first started my company i used to listen to how i built this which is a great podcast of all these founders these amazing companies it was just kind of incredible how many of them told

me about their broken families and their you know broken marriages and all this stuff and i was like okay so you made a hundred million dollar company and your kids don't talk to you and your wife divorced you and if you live in the state of california you just gave 50 million to your spouse um so now you have kids who don't talk to you who don't like you and you have half half of what you made it's like it doesn't sound like a really great plan you know and so uh i don't know elon musk i can't speak to him but the video was not very positive from his first life yeah i've seen it yeah but i mean like i i would say more from the perspective viewing him i mean i have no insight into his personal life but as in terms of his accomplishments as a ceo you know kind of the stuff he's done i i think for me i i i think there's a lot of lessons to be learned and here's my beef with elon musk i don't know if you can see that yeah i've got a model of a truck here this is a a classic toyota pickup truck

i have have one that i'm rebuilding in my garage yeah and you'll notice that it look if i show this to you like what kind of vehicle is this you build it's a truck and i don't know what kind of monstrosity asterization idea of a truck the cyber truck is but it's offensive to me as a truck but this truck thing is a disgusting monstrosity no i mean obviously his accomplishments speak for themselves in the business world i just you know yeah yeah all right well guys that i think that's the podcast unless there's anything else only an hour 45 minutes there needs to come on to the to the to the next one or the next one we're going to talk about i don't know what we're talking about but yeah dude i i really enjoyed yeah we've got a couple of additional good guests yeah uh building in redundancy into your

company compensation work life balance we there's a bunch of additional follow-on topics we can we can talk about so yeah totally i think brett should tell us his plan for making a new game and how it and we should we should evaluate how his culture will change i wanna i wanna know are you gonna have meetings my culture will not change because i'm gonna go drop it on the [ __ ] gaming world to go see i made a [ __ ] game with my culture because the amount of times i've heard someone go well you can't do that because there's a gaming company it just pisses me off dude it's like my motivation for doing it but no meetings no meetings still if you're making a game no no no tabor brett did have a good idea for for a new game which uh he pitched when i was at sega and uh there's research behind that you should go pursue that brett oh you know what i'm talking about yes i do and uh you know i am thinking about it and i want to actually do it

not that game but a game um i would i actually thinking of maybe going through the process publicly as like a you know an exercise of kind of showing but um yeah i want to do it to kind of like strengthen our research right like what would we do if we were really making a game right like what would we do so um yeah i don't know though because the gaming industry is really competitive and i don't know if i have really competitive advantage for it whereas i feel like i have more of a competitive advantage in the research business so yeah that's what's held holding me back a little bit all right well i'd love to come on anytime you know i hopefully i don't have contractors working in the background next time but i'm in yeah this is a lot of fun please do all right guys well after an hour and 45 40 minutes there we have it catch y'all next time bye everybody bye [Music]

[Music] you