Producer Bootcamp: High Performance Producers

good afternoon it's 120 good producer start on time right let's get house DDC so far awesome and the volunteers do an awesome job you guys should give him a hand some of you might have been volunteers once I had an opportunity a long time ago now back when GDC who went to GDC when I was still in San Jose anybody thank you I'm glad I'm not the only one we got a few awesome it's grown a little bit since then today we're going to be talking about high performance producers we're gonna be talking about production generally if I hope you're in the right place if you're not it's your last chance to get out what heat you don't have to sit through an hour of this if this is not what you were planning for please make sure that your phones are off you know all that jazz and at the end of the talk

we always ask that you'd rate us we love being reviewed and especially as producers I imagine most of us in here in production it's sort of our trade right so I'd appreciate getting some kind of review at the end preferably better than worse but we'll see if I earn it or not so I've been I've been at gearbox but 10 years now I've been in industry about 20 years I was at origin and in the late 90s was on the team just when we were launching Ultima Online I worked in flight combat Sims right on the tail end the Wing Commander seen a lot of cool stuff but I tell you what some of the coolest stuff I'm seeing is happening right now I mean we birthed a lot of things industry wise between emmos and had the benefit to go up to Dallas where shooters were created and working some of the best shooter developers out there right now but the

stuff I'm seeing in the indie scene is knocking me out how many of you here are teams of like less than 50 let's say fantastic there's so much exciting stuff happening that you guys are doing it inspires me so please keep doing that I know everybody's ambition while people yeah seems like we're doing well in Indy space so I'll let it go with that there are 44 slides if you're counting hopefully you're not counting too close let's get started so let's just clarify I was like to define things we're talking about high performance production so you know I rather than just have assumptions let's check those I'm saying that when I talk about high performance producers I'm not talking about just any producers and I'm not talking about people who are in easy jobs not that productions ever easy but there are some that you know are you kind of you slide and buy this is not for people who are just getting by this is for people who work in high-pressure

environments where time is critical where speed is key I would say that's the case for everybody it's it's not you know it's it's more true for some than others and if you feel like you're under the gun all the time there's just a few techniques that you can use some of what you may already be employing but some which I use and that I just want to talk about here with you guys you know operating at greater speed or greater power than others things are the same kind when you are out there in the industry you want to be noticed as someone who's extremely valuable and production someone who when you're added to a team you are a force multiplier and it's because you have tools in your toolbox that not everybody has now high-performance producers are very active but they're not hyperactive and sometimes we kind of go too far in the other direction right sometimes we

forget that we also have to be the voice of reason a lot of times or we have to be the ones reigning again feature creep or you know everybody's great ideas that the thing about games everybody has an idea and we need producers to be mediating those so that things can be done feasibly and on time some characteristics of hyper active predictors just so you know when you're around one or if somebody's mentioning it to you maybe you see this in yourself impatient now it's okay to be impatient about a lot of things in fact I tell producers a gearbox sometimes it's okay to be unreasonable about things when you need to be but if you're characterized is constantly impatient then everybody just goes tone-deaf on you and you don't want that right they have difficulty focusing and you're gonna see this mirror is actually very closely with ADHD which I think can be a gift in game development but it also presents a lot of problems when you're working in a team

environment and this is one of those first big ones that it presents a problem with is in building trust and stability when you have someone who's bouncing off the walls all the time or always seems to be like you know crank three notches up more than anybody else while that could be you know good leadership sometimes if it's a constant then again people tend to go tone-deaf we don't want that and also this type of person who many of us probably have worked but tend to favor quantity over quality they believe that you have to go wide early one of the reasons for this at least in my experience is because they don't focus on one thing at any given time and need a lot of things to think about they tend to put a lot into the pipes and so you just kind of have to watch out for that and if this person is say your manager you can do a lot to help the project by just you know pointing out to that person this is what you're seeing in them and they also attempt to over centralize control because they get so many things in the pipe that are all in their head this can not just happen of producers but with

anybody on the team if you see these characteristics of somebody and command and control and in this idea of low self accountability that they're pushing a lot of decisions out but they themselves aren't holding themselves accountable again you can point out to them this is something I heard and the GDC talked you sort of fit this mold let's talk about it well I don't imagine anybody's really seeing that but if you are you know it's good to have that chat so high trust teams when we talk about high performance how do you get to high performance how do you get to speed and how do you do it so that things are being in an accurate and precise way both and minimal error right that's kind of the Holy Grail at speed well the first key is high trust and we hear about trust a lot but it's not like this nebulous thing it's not this thing that just you and your best friend have or you and your family have right we have key things that we use tools that we employ to build trust with people in our teams

and it surprises me actually how infrequently I see producers especially trying to employ these methods to build trust rather than just assuming that it's something that will develop always more than one way so it's grade that now in the description of the talk this is given be a bit interactive okay and I'll tell you right now that the last slide is an example that I hope all of you use to grade yourselves I want you to come away with a custom report card for you to use much like you might how many of you are employing agile and scrum in here and they're using burned down charts and enjoy using that data it helps us out a lot well we should use that same methodology you're already employing it

let's use it for evaluating ourselves also to help us build a lot of credibility with our teams when that's why are we doing that so let's grade trust a little bit first what have what have we done to earn the trust of our managers we don't tend to think too much about earning the trust of our managers you know other than making them happy and personally I don't really want the people around me trying to make me happy I want them doing a good job if they're doing a good job I'm happy so I would imagine that it's the same with you if you're doing a good job you hope that your manager and your manager manager is hearing that and is happy but specifically what are the things that you've done to earn that trust and how about if your team if all they ever see you for is because there's a touch point where they are tracking something right if you are asking the people on when's this going to be done how long is it going to take all those things well they already expect that right I mean but that's that's not really the way to

earn trust that's the way that they know that you're gonna be there and that you're basically doing your job but it's not much beyond that it's like what good are you to them other than pulling data from them you need to go a step beyond that ask yourself just do you feel your organization trust you does it trust you enough for you to make these decisions on your own when we do our performance reviews at the end of the year at gearbox we have a question very much like this in in Europe in your reviews do you get asked a question like this in your performance reviews very very few sad to hear that I would wish that everyone gets asked this question so that it's more transparent to your leads how you feel in the responsibilities that you have in the job that you're doing so think about this think about it sincerely have you been trusted to make decisions beyond just the role that you're in are you ever asked into someone else's office who has responsibilities that are

maybe greater or different than yours and asked to consult on something are you a person who's gotten into that kind of seat yet and it's not why might that be and what sort of conversation can you have about it and very often this is a key point and why that may be if you're not getting asked to have those conversations a lot of times as high it's hard for us to reflect on ourselves that we are exhibiting that exact same behavior to other people how often do you invite others to help you make critical decisions and how often do you decide that you really just need to make that decision yourself I mean that could span anything and do you give others that opportunity to earn your trust you can ask these exact same questions to your leads and if they are a good lead and manager they should be very happy about having this conversation that they likely don't have very often and what opportunities can you create I'll tell

you that gearbox and everywhere I've worked anytime there was an opportunity for me to do something beyond just what was right in front of me I jump at I love learning new things I like doing new things I'd imagine a lot of you are the same way what can you do to create those opportunities and also do it so that they're not a distraction but are adding on to existing efforts a lot of times the first easy one to do is in morale I've challenged producers blocks recently to see who can come up with the coolest morale building events and I'm really looking there's some amazing things happening right now I mean I found out that we have a what is it like a Destruction room people can go into and just break everything and get it recorded on video and now we're using that for our effects team I had no idea it's amazing I want to see what else people can come up with and then no I'm not worried that that's what they chose to do now LeBron James I don't know if any of your sports fans in here LeBron James has a special handshake he

does with every member of his team in the Cavaliers he is Bill incredible trust with his team and when we talk about high performance teams and high performance people I mean you really can't get better than LeBron James and the Cavaliers now I'm not saying everybody in here needs to develop their own special handshake with everybody on the team but the fact that he's doing it and he's certainly better paid than I am and taking the time to do that with his team you know that really says something about the level of commitment that even a star player like that somebody who we might think of as like a rock star in his industry who's willing to take the time to do something like that with his team you better believe that team likes playing together and that they have a lot of trust with each other let's talk about another area of high performance sales culture this is not one that we tend to be comfortable even thinking about in the games industry outside of those marketing people who do marketing

stuff right and this talk is more targeted towards people who are in development you know we developers we do the hard stuff but the fact is that people in sales do really hard stuff and there was a period when I had decided to leave origin along you know when was that in early 2000s I knew what I wanted to do but I didn't know how to get there and I knew that even though I'd been in the music industry before that I didn't know how to pitch to people in the tech industry so I took a few months and I read up and I spent some time with people in sales and asked them for literature to give me what what could I do to kind of bone up and in sales and I learned quite a bit about sales culture and it wasn't what I expected it to be so a point I'm going to make here is there's a lot that we as producers can learn from that kind of culture that has to perform that has to deliver something all the time all the time they're measured and hard numbers

we while we you know kind of peripherally get measured in that way we're usually observing the team it's rare we get measured in the way that a sales person gets gets measured and I wonder what it would be like if we were so certainly demands high performance very results driven it is very customer focused I was surprised at how team oriented say people who work in a luxury car dealership or any sort of specialty luxe areas how team oriented they are and how much sharing there is between people and sales cultures it's a good thing as a producer to have in your background but of course when we think about the individual who comes from that sales background and then putting them into a team with other game developers you know very often I think this is what we think of so I just did a simple search for kind of the ideal salesperson and believed it on this was one of the

top things that popped up I thought it was kind of funny just what would happen if I put this person into one of my teams and I think you were to make it very far into the building actually you know but I liked where his heart is that he is determined disciplined he thinks about the impression that he's going to make he thinks about how he's going to interact with his customers or her customers and then I think about what we really need is that in somebody like this this is rod for those of you who don't know this is somebody I respect a lot this is Rod Fergusson who runs coalition his background his gears he's making gears for he has all those same qualities but without that sales edge and so you don't notice it in a great producer they have those qualities but you don't think about it that way so

you need to think about what your goals are ok how are you going to be measured this talk is about you grading yourselves how are you going to measure yourself and you need to know what you expect of yourself first ok so another basketball forgive me for the basketball references but here we have Michael Jordan arguably the best if not one of the top two basketball players and you know athletes ever okay we can argue that point if you want but I always thought he always visualized what he was going to do and thought about what he expected from himself alright are you doing that and I liked having a Chow in fat picture because I was looking for this Confucius quote anyway so this is a this is a critical one when we think about evaluating ourselves again it's the you don't know what you don't know problem right but there's wisdom and realizing you don't know stuff so and you've probably heard

if you were at other production talks today and just in life you're told you know always ask questions and tell people when you don't know and it surprises me again how often I come across people and it does tend to be nor people who should be saying they don't know more often who are so afraid to and so we need to tear that fear down so that we can mentor and have better hype more highly performing junior people in the industry one of the difficulties in preparing for this though was what is a game producer so I looked around I did I did a bit of a internet search around and I like to hit up the same places so I went to gamma suture I found this really interesting couple of competing articles and I'm love for you to go and read these yourselves after if you haven't already this this works this first one from Ernst and Bosch who was at Blizzard is about the kind of traditional role of producer in the

industry the one that comes from Trip Hawken when he brought in producers from the music industry thinking about the ANR the artists and repertoire process and managing talent and how that competed with this notion of the project manager the producer is project manager so one of the difficulties when we talk about measuring ourselves as producers is that there's no standard still in the industry of what even a producer is we have producers and development directors and project managers and development managers and there's at least a dozen different roles that all amount to what could be the same thing some other company now I don't know even if we're going to get to a point where we have standardization for this but it's one of the hiccups that we have when trying to figure out how to explain to people in other parts of the industry what it is we do they might think you're one thing when actually or something else and it's also why I said I want this to be a customized

evaluation of you for yourselves because only you know can know all these things of what you expect to yourself what others expect of you what your role how it's actually defined but there are some basics that everybody needs to know and here are some questions you should use to evaluate yourselves in a project lifecycle and product lifecycle management are we all familiar well at least the way I'm using these terms who's not familiar and I know it's always hard to hold up your hand when I'm asking who's not but I'll help me target in a little bit have are you familiar with project lifecycle management that concept are you who is not familiar who's not had like a formal course or something in the project lifecycle management okay thank you thank you it's a fair number of us that's good to know and then and so I will assume then that product lifecycle management also is enough talk a little bit not too far in depth we don't have time for it but I'll talk a little bit about it I won't say all

these questions out loud I'm leaving them here and they'll also be up these slides are all gonna be available to you online later I don't want you to hear me say you know repeat these things you'll all be asleep in ten minutes we don't want that okay but these are important questions to ask yourselves for both the project and product side of the house okay and I'm gonna try to leave them for questions after as well absolutely though this should be a baseline all right so on the top left there we have kind of your typical requirements gathering waterfall approach to project management and on the top right there the agile lifecycle which as you go into these product increments are you know your sprints essentially and then you have a larger arc for development with

both of these it's not reflected as much on the waterfall side but there's some iteration there also although I have to say I'll never forget my first day at origin I walk in and as I'm walking around I go through the James combat simulations group and so these are all project managers well the person who is managing that project at that time we'll just call him Brian because his name was Brian he had printed out this Gantt chart that literally took up the whole wall and I mean this thing was probably 12 feet high by 18 feet long it was an amazing piece of art and they stayed on that wall for a while and it was never updated and it was monolithic just like that thing on the top-left is a key thing about either of these how you participate in managing your teams through whichever project methodology use is this one that's kind of front and center here this is one that actually

gets ignored a lot of the time and if you can be someone who masters this part of what happens in project management you're going to be head all right and I want you to evaluate yourself on how much you've participated in this part of it it's about costing it's about the cost of making a change depending on where you are in development and you can see how cost skyrockets about the time that you're hitting beta there right I also have the reference here now this is applied from broad software engineering it's not specific to game development but it is from a design background I haven't actually measured what it is at gearbox but I bet it's not too far off and now here's the product life cycle as opposed to the project life cycle when we talk about wanting to have a long tail this is where you're launching your game and how long it

lasts in the marketplace and how quick you are to respond to launching the next product that comes on the heels of that one but since our industry is so diverse now how many of you are in mobile or in mobile development awesome and how many of you are in say more PC focused like Steam development and how many of you are in console triple-a see where it looks split like in thirds to me and there's a different product lifecycle really for each of those and now even the segmentation within those whereas it was more static and console before so different now we see console actually looking a lot more like mobile with some free to play games in console so we have to be really adaptive as producers where we could again I said you know there are those who could kind of more coast not that our jobs are ever easy but if you don't have to apply say this mobile product lifecycle to a console game you

know what we thought used to be hard in console development with just releasing patches is getting much harder and so we we have one of the reasons teams are growing why it's getting harder to manage these things I see somebody who knows that intimately and here's another one about why this needs to be so customized for you what type of management style is expected of you now you may not know you might not be aware what kind of organization you're in you might have some inkling of it and you may not have had a formal way to really think about how you're being treated in the organization and how your boss expects you to treat other people but it's good to know the difference between these you probably know it innately right you probably have a lot of assumptions about it but it's good to be able to put more of a formal focus on it so in organizational behavioral behavior with management there's basically two kinds of men's but we have

transactional management we have transformational management and often you'll see if you have I'm I happen to be in kind of a matrix structure as far as my reporting goes my bosses Randy Pitchford the owner of the company and then I have another person I report to use and kind of a different part of the company and they have completely different styles and I have to shape the way I approach things differently because I always want the best result for everyone but you can't talk to somebody who has a transactional management style as if you're talking to somebody who has a transformational management style you're going to hit a brick wall so knowing how to shape both reporting up and then how how to work with the people around you to adapt your Styles really important you need to add those tools to your toolbox and you need to practice it constantly and one of the things you need to do is pick up on who responds best to what kind of feedback and that goes for the people you're

managing directly up down across everything what I'd like you to do for yourselves in your evaluations of yourselves is provide examples about where you've needed to employ each of these and they should be very specific examples not like you know sometimes I have to go see somebody you know I have to go see so-and-so on the floor and tell them that they're gonna get a bad review if they don't get something by the way you should never do that anyway but you know if that would be nice and while that would be nice and specific I I want examples where first of all you think in that case you think you're keeping things moving but also that you're improving performance of people right so I want you to think about instances where you've employed these where you've needed to employed them and the result was good for everybody

high performance means you're going fast how many of us have heard of an ood a loop a couple okay good I'm glad that I'm introducing this to the majority of you from one of your missions I hope from this point forward is you're going to master this concept it's not a particularly difficult concept to grok but it is difficult to master one of the hardest things about it and I have a few notes here where I'm going to touch on it is ensuring that you're constantly employing it that you're being mindful and employing this process it will make you a superior producer and a superior person as well so if you want to be fast okay you need some framework to be fast within because you need to know how to match what you're seeing with what you've seen before and remember how do you act now that you're seeing the new thing but what you also have to do is avoid some serious problems I'll mention

what those are what the lingo is for those in a second so the OODA loop is observe orient decide enact observe observing is each one of these things by the way while it's easy to say the word is hard to do observing we see things all the time but as you know from the concept of flow we know we know the idea of flow from cheeks of nuo Yi it's sort of like where you get lost and what you're doing and you can daydream and get a thing done or you're driving and you forget that you're driving but you get there right what's hard to do is be an active observer once you've noticed something in your environment what's even harder to do though is to orient yourself to it you might observe something and see something that doesn't occur to you that it's out of place or that you need to do something about it so that the will the decision to do something about it is is very hard especially if you're in an organization where you feel like you you

might be punished for deciding to act where it's not part of somebody else's decision right where you have to use your influence to do it and the will to act can be the hardest thing so you as producers you need to stamp out that lazy in action I'm going to talk about in action a few times the best way to do that by the way is with the positive using the carrot which is to always reward people who are not afraid to act when they see something and to especially do that if it's if it's quick and a smart but if you see people who are who are moving fast to act on things that is one of the you know if you are mentoring people and you see that they are willing to actually take the initiative to do something reward them for let's skip ahead this is what it looks in the more formal cycle you can go online and find this as well I wanted to show it to you here since this would be the first introduction for many of

you the OODA loop is what got us through some asymmetric warfare that we had to get through we when we go into Iraq for Operation Iraqi Freedom we don't have time for that for that talk right now but when you when you approach a new situation in a new way and you need a new frame of reference to deal with it you have to orient yourself to that new thing in a new way this gives you a framework in which to do that so that that kind of orienteering orientation is you take that new information you apply it to some pattern you already know then you intentionally break that pattern you apply a new you come up you use your imagination to think of other reasons why that thing might be happening come up with a plan for how you would deal with each of those now you have a war game for how to approach it

you notice here in shelter constantly you've got who's this do we not play shelter yeah the stranger he pops up you have to notice that he's there when you have a you know a big vault it is pretty easy to miss this you dilute is basically what you're doing if you didn't have the sound that clued you in that he was there you always need to be doing this to be a master producer to be truly highly performant you've got to master this you've got to get you student noticing things that other people are not noticing you know when you come in in the morning and you see somebody who looks like they may be a little off of that day make sure to stop

by their desk you noticed it probably somebody else didn't notice it their immediate lead may not have noticed it pass on information to other people around you anytime you notice something that may be slightly out of the ordinary if you feel like it's been a day too long since you know some task was reported well I don't like to over focus on tasks you know everything you have to be the one to catch all those details it could be indicative of any number of things it happens to be this is very native to agile and you will run better meetings for agile if you're employing this as part of it okay exploring what the possibilities are outside of what the people who are in your scrub meetings and in your sprint planning sessions I do this a lot I even when I know answers or I feel I know the answers I still ask the questions because it gets people thinking using

their imagination outside of what they believe is already the answer outside of the solution that they already want to pursue and I try not to do it you know too often where people pick up on it and think that you know it's just a distraction but anytime you think you should trust your gut anytime you think that a solution is being pursued because it's maybe the easiest and isn't the one that's going to work best for everyone for the team for others for the people who are going to pick up that code after you need to do something about it you have to have the will to act you're not going to improve if you don't increase your self-awareness and I say that in that I am always trying to increase my self-awareness okay I'm not making an assumption about where anybody else but this is something that's a life long pursuit okay we have to pick up on in ourselves when we're being overly cynical when we're steering

away from pragmatism when we're not using our executive function to tasks which okay when we're getting too stuck on something so it's important to take time if you could every morning every evening whatever think about are you doing this evaluate yourself and how often you're trying to do directed things to improve yourselves and then situational awareness now going back to the who dilute you have to learn how to filter right so every room has a baseline this room has a baseline you know the hallway has a baseline your studio has a baseline you have to notice anything that is perceptive outside of just that background noise and normalcy bias is the thing that prevents us from acting when we see something really weird now a lot of you might have already heard of normalcy bias right this is where you know oh I'm I'm used to if I'm walking in London you know I forget that cars come the wrong way well I look you know to me I look at the

other way I walk in the street boom I'm hit by a car well timaya was normal that there were no cars coming from the other direction right Tamiya might be normal that I hear a couple of beings outside in the parking lot or something right but I should still go look and see what it was I hit on this one before around discerning in action the best producers and you need to notice for yourself when something should happen and it doesn't and it kind of itches at you you have to immediately follow up on this so if you want to increase the speed of your teams in a what you're doing notice that things aren't happening that should be happening and don't accept that they're not happening don't accept that the meeting didn't happen that was supposed to happen don't accept that the asset that was supposed to be on last week isn't in this week you can't accept those things you you have to seek out the reasons and be a conduit or be an

agent have changed for those in action as a decision is important to think about for high performance so have you ever talked about in action with a member your team the fact that they didn't do something it's so easy for us to talk about wrong things people did or that we feel they did but it's so difficult for us too a lot of times even notice that they didn't do something they should have this I think is a pretty good length especially because it's John Cleese it's on the same basic topic and just going to the bold part there that most people who have absolutely no idea what they're doing have absolutely no idea that they have no idea what they're doing if you're just starting out as a new producer in the industry you might not be at all

aware of the habits that you're developing that are affecting other people around me you might not be realizing which habits you're even picking up on you know one of the reasons I like bringing new junior people into gear box is because we can more shape them right haven't developed all those bad habits somewhere else but we also also have to be mindful of what people are picking up on from us especially you know if you've been around a while you shortcut to a lot of things it's not obvious to the people around you why you're doing what you you sometimes you have to stop and take someone especially junior person aside and explain to them what they're seeing you know and a way to know that you don't know is that you're getting surprised a lot so think about when is it that you're getting surprised I won't linger on this too much right

now but I want you to actually think about this take stock of write down notes here's the last few times that were surprised because of something that happened other people might notice that you're getting real often there are some cases where I've had to pull people aside and my organization and say hey you know it's like three times now that you got surprised that so-and-so didn't show up or that you know seems like you're having a hard time getting out of the door well you missed that meeting that didn't get called because they just assumed you'd be here for court hours there are a lot of things like that as well as technical things that if you find this happening a lot it's probably you filling in those dark areas of the picture that you didn't know we're there were there before these are pretty straightforward okay detail so important in game

development so you know these are gonna be I'm going to skip ahead to the next one but you're gonna see these reflected in this kind of report card structure that we have if you get the chance mentor as often as you can it makes you better it builds trust with the people around you it helps you check your assumptions and you get to answer a lot of good questions you should if you don't already have mentoring as a formal thing in your company and I'll have to admit gearbox we really don't have it in a formal way though I do encourage it many of us encourage it and we try to connect as you would anytime you're you know in organizational management you're going to senior and junior people so that sharing can happen but any opportunity you have for it you should engage in it grade yourself for how well you think you did when you were mentoring and write down any specific

measures you can imagine these are the same measures that you use in your end of your reviews or any other performance review one thing that we might not do because we're afraid to is ask the person who mentored you how they think you did a lot of times were nervous to do it they're much more senior than we are we don't do it we should and you can only get better if you do it frequently enough that the memory of the lesson lingers with you if there's one a fleet that's better than LeBron James right now might be Americans although American Fela Pharoah hasn't raced in a what a year and a half or so now strong mentorship is like a great gene pool okay so American Pharoah siring you know

very high dollar rental fees as well as American Pharoah's dad actually is doing even better for the owner but if you want to have a really strong structure of trust and to be able to go fast in your organization mentor a lot also if you want to go fast you're learning lessons now this talks not on post-mortems and after-action reviews so if there's a different talk you can go to you and post mortems and after-action reviews please do it and there's a lot of literature online how many of you have engaged in formal post mortems before okay a lot of you awesome they tend to vary in terms of quality and they don't always have actionable results and that's the only thing I really care about with post mortems is where they're actionable results or not I don't just want to see everybody and get around have a cathartic moment where we all

complain about what happened on the project well that can be fun over beers and so we can call that a post mortem that's not a post mortem all right there's not a post mortem unless people have actually learned and you've put in measurable measurable outcomes okay an action plan that can be followed again this these slides will be up for you this is the after-action review process is applied to post mortems okay and then walks you through what you a lot of what you already know there but again you you have to have these measurable outcomes you have to know how you're gonna measure them and you can practice them you don't have to wait till the end of a project so that's why I like experiments and that's why I like doing what we call it mid mortems right but you can do an after-action review at any time and then go the next step invent things where you can have post mortems in do it in ways where they're fun and hard that helps you

battle complacency with the team right it's easy to get in the rut you got a lot to do on a project come up with stuff that can be measured that isn't necessarily obvious to you already and in terms of speed when you do those things and it doesn't have to be something new that you made up I mean you can do this with some activity you're already employing on your project practice increasing the tempo don't give everybody as much time as they would usually have cut in half cut in two-thirds make it a less risky thing so they know that this isn't what's going to get their neck chopped if they don't get it done in time okay but get people practicing is just like speed modeling for artists get them practicing moving faster in less risky ways I love talking about Musashi the only thing I'm going to point out here from going no show which is a book of five rings is this one of know the ways of all professions so cross-training is so important if you

want to be a great producer who's fast you wonder every understand everything you're hearing and when I talked about separating out from the noise suffering it out signal from the noise one of the best ways to do it is to increase your knowledge of what other people do I'm going to go through a couple of analyses that I did with some software a friend of mine is building and this is not a fifth by there's no cost to this so I'm not pitching a software really you can go do this as soon as this is over if you want okay it's called skills ends and and is he's building a machine learning platform for skills analysis for things like job postings and helping schools develop curriculum that gets people jobs as soon as they get out because they're very targeted to what people need to know to do their jobs and you can put a resume in here and see what you get I'd love to get feedback by the way if

you do that with them job postings from from your companies now we don't have time for me to go in detail over all these all right so I'm just going to kind of point out some high-level things first of all all of these I pulled in job descriptions from some of those different jobs we talked about earlier Development Director man a development manager producer I have an EP posting in here as well and they all tend to focus around the same basic skills okay some of which we don't measure in ourselves that often which really did catch my attention and they only vary in terms of the skill you need to use very slightly when you go to the executive producer domain where you're dealing more with budgets and P&L and product lifecycle as opposed to project like life cycle for producers but we can I pulled some things out of here to tell you how I built the categories for this report

card you're gonna see in a second all right if I look at the top things so if I go back one if you look at the top knowledge areas for this one which is Treyarch ap okay this gets grouped I'm gonna skip it bina but these were all active isn't a sledgehammer Enochs skip ahead and I'll show you even though these are all aap rolls the way that they were correlated to other industries and other jobs were all a bit deaf a bit different right which also tells me all of you again need to understand the context of the job that you're doing because it's always getting be a little different even in a organisation like that that you would think have standardized roles more it's not as standardized as I would think one is very art heavy and creative how are

you going to evaluate yourself and being able to talk through our issues and you know having requirements of you being more aware of what's happening on the creative things in the industry one is much more technical one is more general measurement oriented and here is the which is funny by the way dev director and dev manager turned out to be exactly the same except for years of experience which even coming from ei didn't know that this one turned on its head a little bit critical thinking over things like information gathering which should be you know I think probably should be obvious but since there are so many EPS who work two ways up from APs you know be needing to notice those things that are missing becomes so much

more important when you get to the EP level right noticing the things that aren't there because you have to be the ones to construct the ways that other people are doing their work and what the goals are for the project let's get the head sales and marketing was one of the top four hits for EP all right you have to pitch you have to understand the sales cycle you have to understand finance that was for ep okay it's to time I want to get you guys out of here on time so took all that information all right and put it into this table and my thinking here is when you're writing down your notes for your examples of when you yourself have

exhibited something from what we've just seen what's just been presented write down those instances all right keep notes of these and if you do it periodically right if you do it in a way like you're running your ads all teams you can now track this you can track the data just like you would your sprints or your mouse nose and then jot down how many instances would you expect of yourself and the number of years you've been in the industry and your role and the job you have how often do you think you should be exhibiting this so for example if you're responsible for a P&L and know you're gonna have a budget meeting once a month and you can really kick butt and having all your numbers together and be able to explain how well the teams doing that's you know basically once a quarter you're going to be expected to have a great instance for for that topic right think about it for yourself and I hope that you can build your own customized report card out of

this if you have any questions you want to talk further on any of these topics any of them are gray please get in touch with me talk amongst yourselves and next year I want to hear if you've employed this and if it's helped you at all thank you [Applause] we do have nine minutes if anybody wants to ask questions otherwise okay we got 101 please don't forget the the reviews yeah shoot all right so gearbox has a great reputation for your internship programs and I wanted you to talk a little bit about what you mentioned is the carrot what type of rewards you implement both for your core team members and for those more temporary team members sure so we we've had a

program since before I was at gearbox called the cogs program contractors of gearbox which is our which is our formal internship program and we give when we run that program it's not constant you run it quite frequently and across different disciplines we have you know basically interns come in and work on our games typically in some area that's fairly safe like I talked earlier if you want to do kind of rehearsals in areas and try things out you doing it in a safer way so we tend to have interns come in and work in teams where they're paired up with a mentor and it might be sometimes to try out some new concept that you know we wouldn't have the time to do otherwise very frequently those interns are hired full-time and do fantastic work where actually they are

executing on you know R&D things that they were doing so so for the intern it's pretty much the opportunity is the carrot what about for the core team members well for the core team it depends on which what you mean with the when we do the internships or just generally know for your full-time team members your leads you're sure well I mean that's okay so it's a little out of the domain of the topic but I'll I mean I don't know if you want I don't know if everybody wants me to give them like the gearbox pitch or something like there are what I'll say is it's important you know that the people who on the studio came from the trenches as developers and worked their way up so they always wanted to be fair to everybody else who's in the studio so it's really important to them and and to us who were an executive leadership in the company that everyone in the studio be rewarded for the work so they all you know everybody in the company is full-time C's back-end and it's in a very fair and flat kind of way well it's

in a very fair way tonight it's in a calculated way thank you sure I don't do there's anybody who happens to be from gearbox have anything else they want me to add to that or did that sound ok all right yeah next question oh wait hold on we get them on the side yeah ok you mentioned something about people have an initiative and working on things that you know what if those initiatives aren't on the like from the backlog and not from the sprint and stuff like that like they did something that wasn't supposed to be done at the moment how would you I know that situation yeah so as long as well let me answer that one person and that's ok if it's like good to part so if people do things that aren't specifically assigned to them in the sprint if they get the sprint done no problem the assumption is if they're going to do that then they are going to get their work done that they've committed to you and the sprint and then if they want to do something beyond that

will totally support them I have people who have had ideas to do different things and I've told them to go for it so I'm talking about a situation in which they didn't meet the oh well I mean you know it's between them and their lead ok right so if I've never I can't think of a time that a lead was surprised in a situation like that and I and I also can't think of a time when somebody didn't get something done but was spending their time on some other thing so I hasn't that I don't think has really come up yet but I would expect that buddy be comfortable just talking with their lead about you know how to basically negotiate their time on that thing the baseline is the you know they're gonna get done whatever it was that they've committed to and the other one is you know perfect world do you

think producers are you know would be phased out if everyone is alike sales of self-sufficient or something like that would you still need for the perfect let me just rephrase to make sure I understand I think you asked in a perfect world would all producers be self-sufficient no if the all team members are self-sufficient would like answers and what and don't need producers let's see I think actually that's a it's an interesting question I think it's a fallacy because the best producers are not observed that often I think the best producers create a flywheel effect and that's why wheel doesn't slow down and they're there to protect the flywheel and and once it's going fast and you've built the organization have the right people in place and processing all those things your job is to care for it and then you can turn your attention to other things that aren't as a observable to people within the team thank you sure so how do

you feel about like the the frequency of reviews that you conduct like be it like one on ones bi-weekly with artists or doing like yearly reviews and sitting down and doing goal settings what how do you feel but a good frequency for doing this is there is there some subtext in there having to do with the frequency of how often you have reviews yes that's a good question it's a hard one I mean not everybody is Lisa Simpson and wants a review like every day and know exactly what grade she got on every you know every class we try to be good about having an informal review process just making it very clear where everybody stands at any given time with that said with that said I wish that we were doing reviews more often you know we we have one to your performance reviews and on occasion we also do 360 reviews

which we haven't done in a little while we need to get back to it it has to do a kind of democratization in when you have a social organization and that's intended to be a meritocracy you want people to understand you know where they fit amongst their peer group and if are they earning the merit of being there with their peer group and that if you don't get that kind of feedback very often there's really no way for you to know and there's in fact dangers of having a culture like that if you're not providing reviews fairly often so we encourage our leads to ensure that they are interacting with the people they work with too so that's always very clear but you have to balance it all right I mean you have to balance it with a formal review process versus informal because it can be a distraction and it takes a lot to gear it up so I don't

think it I don't think it's optimal today but at the same time it's provided I think decent conversations for us to make it clear you know where everybody is and they get again they get that touch point from their leads form you know as often as the lead does it so really it comes down to making sure leads are doing it more often thank you what qualities are good mentors looking for in good mentees that's a good question open-minded fast learners who can develop technical skill quickly if they need to you know I had a five minute wearing before am I out okay up I know we have a couple more questions if it's okay I want to answer your questions but can I do it off the stage is that all right for about to you'll

all right thank you guys you you