Autonomy, Mastery, Purpose - Building a Hyper-Engaged Team

[Music] okay let me get and please make sure to turn off all your phones electronic devices anything else that might be making sound and you'll get a wee speaker form coming to your inbox after this might go to your spam box so if you could fill it in that'd be great the GDC folks really like it when you do that so my name is Alex mole I'm the chief technical director at criterion games and we're a small EA studio just outside of London England where we're currently having the best time we've ever had and we're making the best games we ever have and so I'm here to tell you how so I'll start off by talking a little bit about our back story and and then I'll move on to how that led us to come up with a new philosophy for how to make games better and I'll then talk a bit about how we've proven that this actually scales to triple-a production and why we're

confident this is great and then I'll move on to some specific practices that we use day-to-day to implement this new philosophy and finally I'll sum up so first of all where did this come from so I hopefully some of you at least have heard of some of our games we're really proud of these we've got a a rabid fan base who were very very happy with and you're always talking to us about them all of these games are characterized by at least my memories of them are characterized by two things so one is a very high Metacritic so I think the lowest Metacritic on that slide there was about 86 and I think it goes up to 94 Metacritic at launch huge critical success very popular games and and certainly I didn't work on burnout 3 but I joined the studio because I loved it so much but the other thing that those games were all categorized by is really low team morale as I know there's a whole bunch of expert area folks in the audience here and they're all smiling

and so I think this is true and not just for me so there was a bunch of staff turnover there was an awful lot of unhappiness and frustration and an awful lot of crunch a crazy amount of crunch and when we when we looked at the reasons for that low morale really a lot of it came down to and this this reason so on a medium-sized team of maybe 70 to 150 people there wasn't really anyone who had a chance of making a good decision that would stick that led to much of the team pouring their hearts into features that were just always destined to be cut and that was actually compounded by a constant feeling of being behind relative to some ideal game studio who was much better at it than us and that feeling of behind meant that we were always scrabbling to build things as fast as we could rather than taking the time to figure out how to work efficiently and effectively and so we wasted a huge amount of time and effort and the upshot of that was that much of

the team was under motivated despite being incredibly passionate and we had people working really crazy hours and making themselves properly ill long-term relationships breaking up all sorts of really horrible stuff that people were willingly putting themselves through and despite all of that passion they were still really under motivated no one really wanted to to do this stuff and the other part of it was most of the team just weren't empowered to make decisions despite knowing everyone else on the team but that doesn't make any sense because on a small team where you know everybody you should just be able to walk over to the right person and ask for whatever information you need to support making the decision you're trying to make well that didn't happen what happened instead was we would brute force our way to great games but at very high personal cost so in early 2014 there was a big change in studio leadership you may have seen it reported in the press as the death of criterion if anything was quite the opposite the

new leadership myself included seize the opportunity to do things better on a new game we believe there would be possible to make Triple A in a sustainable way and so we took our inspiration from the best of other triple-a studios and indie developers and then found places where science backed up and codified our approach and I've got to be honest I'm really grateful to the EH Studios leadership who gave us an enormous of encouragement and time and space to try these crazy new ideas and and ultimately prove that they really worked so personally I was very inspired and still I'm very inspired by Indies I love coming to GDC and hanging out in the indie game summit and and one thing was very clear to me was the small teams where everyone really knows the whole game that they're working on and they're able to turn on a dime just don't suffer from these really horrible issues that we were struggling with so we figured well it's definitely possible for small teams so surely we can get this to scale to big teams too so we came up with a

new philosophy given what we'd learned we said about turning all of that experience into a coherent set of practices that would allow us to make games better and over time we've managed to distill the down to something coherent but we started off with this question if we just assume that everybody's here to make the best games they can what would we do differently and we figured we do two things differently so the first thing is treating people as talent that means letting people make the right decisions for the game using their own skill and judgment and expertise because we hire great people so obviously they should make great decisions and the other thing we do is actively eradicate harmful dogma there's lots of examples of harmful dogma and one includes the expectation of crunch so research has proven that crunch is only beneficial for 48 hours and is detrimental after as little as a week or two so it's really

in my opinion nothing more than legacy and fear and some macho posturing that means that we still expect crunch it's you know science tells us it's it's a waste of time but it's also things like expecting to use a centrally distributed list of tasks maybe like a hand software a JIRA type thing to determine how the game gets made because the person with the best understanding of how to fix a problem is just going to be the person who will be fixing that problem it's not going to be someone off in another room typing tasks into some big spreadsheet so it's really the studio's job to facilitate the team to pick the best problems to fix and then really help them and guide them on how to fix them so we have this as a little aside we have this poster up in our office and I really hope that this just seems totally weird and obvious to you unfortunately big gains companies and I'm pretty sure all big tech companies often use the word resource to mean person now I'd argue that's always

inappropriate but certainly for any creative endeavor endeavor its provably inappropriate because a creative collaboration between the right people won't just get you the right results sooner it'll generally deliver your result that wouldn't otherwise have been possible so the trick is really to gather the right people and then help them to creatively collaborate so a criterion as leaders we've stopped telling people what to do because that presumes we're more experts in their field than they are and instead what we do is we take time to sell ideas two teams of creative collaborators and then when those little teams and those creative collaborations engage with those problems they're able to take them into delightful new directions that we would never have expected and I wish that I could tell you some more about the game we're making at the moment because I have some videos that illustrate this just perfectly but sadly we're not there yet and if nobody on the team wants to work on a feature wants to engage with a feature then one of two things has happened either we've not done a good

enough job of selling it and maybe that means we don't understand why is why it's truly valuable or maybe it's just not a very good feature and the team is tacitly telling us I've been that one move on there's better stuff which is really great feedback for us to have and when when as a leadership and as a production group we noticed that someone's made a decision that surprises us rather than telling them that they're wrong we'll speak to them we'll take the time to educate and be educated we'll share the bigger picture context that maybe they don't have that led us to believe they'd make a decision differently but we'll also listen to the detailed context that they've got on the ground and and understand why they made their decision in the first place and either a better decision will get made or we understand why that was actually a better decision in the first place and either way both of us have come away from that with a great of trust and and general happiness so what we've done isn't just hand wavy

nonsense as much as I love that there is actually some science backing this up we've we've looked at a bunch of research and our main inspiration for a lot of what we do is this book which is actually summarized in a TED talk which Dan pink gave in 2009 titled the puzzle of motivation I highly recommend it it's a great TED talk so really his thesis is when it comes to motivation there's a mismatch between what science knows and what business does and he lays out a compelling case why existing ideas of motivations or things like financial incentives and status rewards like job titles and corner offices really don't have a big positive benefit on creative pursuits that the great news is that science actually knows what things do have a big positive benefit on creative pursuits and so really what we've done at criterion is just the project management equivalent of taking a seven-year-old SIGGRAPH paper and getting it working in videogame so this Dan pink says as soon as people feel

that they're being fairly compensated for their work money stops being a major motivator that said it is crucial that people feel that they are being fairly compensated and that means that they're they're being paid what what looks fair relative to the larger industry but also what looks fair relative to people doing similar roles within their studio but as soon as that's out the way and I'd really urge everyone to get that out the way as soon as possible there are some big drives that continue to motivate the first one is autonomy over team tasks and technique the second one is mastery of a skill that an individual values and the third one is working in surface in service to a purpose greater than ourselves so I'm going to talk about all three of these in a bit more detail I'm going to spend a chunk of time on autonomy because it's the most different from traditional project management and it's where we've seen the best results so a rough little definition autonomy is

having controlled within project parameters over what you work on who you work with and how you approach that and that's been shown to be hugely motivational so we've strived to come up with a way of working that supports all three of those big pillars so this is the guidance we give out everyone on the team so everyone on our team decides where they can have the biggest positive effect at any point in time originally we actually tried using this as the only rule and that worked actually pretty well until we got to about a dozen people and after that it got too hard to share enough context and communicate enough to make sure everyone's actually working on the same game so now what happens is as a production group will provide a handful of big themes perhaps five to seven big things with suggestions of some priorities within those themes and then the team just takes it from there we've never ever had a situation where nobody wants to work

on something that's really important so I'll say that again because we've given this talk if you build similar talks to this a few times within EA and this is always a question and it's a fair question but we have never had a situation where no one steps up to a plate to do some even dirty boring ass work that's just never happened a couple of times we've had a scenario where no one's turned up to work on an entire theme and that was surprising to us but when we dug into it it turned out that the team had actually noticed that there's some big dependency that meant that that thing couldn't really usefully be worked on anyway so they've gone over to work on delivering that dependency before working on and the thing that we'd naively assumed could be done right now so the team is really wise right there's all this wisdom out in the team and now we get to use it which is great the most surprising problem we've actually had with this approach is that we've had people choose to work on things that they don't actually value

but they feel like they kind of ought to and that's a subtle distinction but quite an important one and it can lead to people being quite ineffective and miserable where their skills don't really match work they're doing and it's actually something that we constantly vigilant of now like double-checking are you working on this for the right reason like you actually are interested in this right so context becomes very important for us and in a more traditional project management structure the big picture of the game is held by a relatively small number of people on the team and most people turn it into chunks of work that get passed out and distributed out to the main team and generally that keeps everyone working on roughly the right thing but it can lead to this feeling where some folks on the team just don't really understand why they're doing the work that they're doing and it can end up to a lot of that lack of decision-making and really that feeling of helplessness and that huge amount of

waste that I described earlier so by embracing autonomy and pushing decision-making out across the whole team we now have a situation where really everyone on the team needs to have as much of that big picture as possible or they just won't be able to make good decisions and I have to say as a leader it's much much more effort to push that big context out and keep refreshing that than it is to just do our tasks it just is but the payoff is huge as I say it allows the experts on your team to make all of the easy wins that service your game in the best way possible and also warn you and guide you away from the big expensive moon on a stick rabbit holes that producers and designers tend to occasionally for the best will in the world lead us down and those just don't really happen anymore the context does change and fade over time so it is a like a full-time job for several people in the studio to keep making sure and double-checking that everyone knows what game we're making and that that's kept up to date bilities

as I say just astonishingly valuable so that was the first kind of autonomy that was autonomy over tasks the second is autonomy over team so as I mentioned the feature teams on our in our studio form themselves around high-level themes that the production group has identified and those feature teams will stay together for as long as it makes sense and then they'll disband and go work on other stuff but it is vital and this is true for actually any studio but it's vital that any team or sub team is viable which means it has the capability to deliver what it's trying to in the time that it has available and that's simply because a viable team is greater than the sum of its parts whereas a non-viable team just isn't one of the most common mistakes we've made on this journey is starting work towards a goal without all the viable or without a viable team in place and it can be surprisingly hard to spot when that's

wrong the most common cause of non-viability x' and missing skillsets but there can be all sorts of other subtle things like it can be not having access to time from an important reviewer and we've seen cases where a team will kind of spin its wheels for several days like gilding the lily honing something that they've already kind of proven all the important stuff on just waiting for a review to tell them how they can move on with it and so invariably there's there's two ways of fixing a non-viable team right there's the obvious one of add more skills to it and and there's the other really importantly valuable one which is change the goal posts change the goals that they're trying to sign up to if you don't have a 2d artist well you obviously can't deliver anything that needs a 2d art but there's still probably tons of useful stuff you could deliver in the same kind of direction so that's that's very important and so and what happens in the studio after the teams have initially formed they then figure out what goals they'll be working on and we then kind of rationalize and

make sure that the teams are all viable by adjusting goal posters necessary and the third part of autonomy is autonomy of a technique so as I said each team is responsible for creating the goals they'll sign up to you but they're also responsible for deciding exactly how they'll meet those goals that includes the exact deliverables that they plan to create but also improves includes their daily process are we going to work in like an agile scrum e-way are they going to work in a Kanban and iteration style they decide that and actually we give each group a process expert didi in ei terms I don't know other Studios coulomb and each group gets a DD to really help them coach them on which process might work for them and then coach them on how to use that process throughout the the milestone so all of this autonomy has the effect that everyone on the team kind of needs to get a bit better at selling and this is a surprise to us we didn't realize this

would this would kind of be a sticking point for some people but designers and producers need to get better at selling things to the team in order to inspire them to come and make them because just saying well because I said so doesn't work anymore right we don't we can't do that there's a producer from criterion laughing ruefully here because he used to maybe enjoy doing that and there's also experts on the team need to be able to sell the value of like long lead technical work in time for it to be started so that it can deliver value at the end and they need to be able to sell it to people who can prioritize it within things for a Mustang and so as a leadership group we we need to be conscious of this and we are and so we try and solve it in two ways both we coach people on how to sell better when we notice that there's an opportunity to coach that but we also actively try and signal boost quiet wise voices so that we get the benefit of their wisdom across the team so that was the end of Dan Pink's first big drive that was

autonomy so moving on now to mastery so mastery is everyone mastering skills that they value themselves and for everyone to do that we need to they need to feel that they're supported in their favorite ways of working that we can have a team structure where an engine coder and a gameplay designer both feel supported and able to to grow and we spend a bit of time trying to figure out how do we how do we balance this out how do we even understand why those people have different needs and and then how can we satisfy them and we came up with this division of types of work so across the horizontal axis you've got how well-defined the problem is from very well defined clearly briefed all the way to who knows and then on the vertical axis you've got how well defined the solution is so is this an R&D project or is this and something that you've done a thousand times before and it's about applying

knowing solutions to it and so we've come up with some names for those types of work so crafting is the process of taking a well-defined problem and applying a well-defined solution to it so craft across people are really motivated by excellence of execution these people will take a brief for a bit of work and then give you the they'll use their and their skills and their judgment to give you the best whatever it is that meets that brief the next group is experts experts really like a well-defined problem that nobody has ever successfully solved before and they'll go away and do R&D and loads of hard sums and then come back to you with something amazing then we have explorers explorers who really love uncertainty and who will take the debrief of it seems like jumping should be fun right is jumping from can you tell us how jumping fits into our game and why it's amazing and and that's you know that's that's the Explorer mentality and then the last one is what we call connectors

is folks you take well-defined solutions to problems that the team doesn't know they have yet so there are people who often talk to other teams and so when the team comes up the idea of yeah so we're gonna try and put all of these badges into a box they'll say gee you should probably go talk to Dave over there because a few years ago he tried to put later badges into a box and now Dave doesn't has a face so maybe you want to learn from Dave and the last the last big drive is purpose and so damping describes purpose as attaching your desires to something bigger than yourself and let's be honest in the video game industry the purpose is typically pretty obvious I mean really not many people are in this this gig for the money right this is borderline vocational what we do which is kick-ass a criteria we've actually tweaked and added a little bit more to our purpose that actually makes it even more motivational for certainly for me and I know for a few other people in the studio which is we make incredible games in the best way we can so we explicitly

care about the people in the process as well as the product that were ultimately building and happily results in a better product as well as happier people so that's pretty kick-ass so those are those are the big ways that we treat people as Talent the other big thing we do is we actively eradicate harmful dogma and we just do this I mean it's pretty simple we don't do things because that's how they've always been done every process and plan we have is just the best idea for it that we have right now everything can always be challenged and we actively look for ways to get better one way we do that is that at the end of every milestone each feature team will run a post-mortem on the last milestone to understand how they can improve for the next one and then the leadership team runs an aggregate post-mortem to figure out what large areas the whole team can improve on for the next milestone so hopefully that sounds lovely to you it sounds lovely to me does it actually

work I mean this sounds kind of hippie and weird and like don't know so and yes of course it does work otherwise it won't be here and so the early signs were really good by the end of 2014 we're going to be a team of like 25 to 30 people working on a new game franchise and in terms of quantity and quality of what we were making we were more productive than any team I've ever worked with we had executive reviews where the feedback was I can't believe how much has gone in since the last time I reviewed this I don't know if you've ever had an executive review us and he never had one where there was feedback like that until recently and prototyping on a new game was consistently turning out really fun new features that all work together as well as hundreds of thousands probably that didn't that we all happily Bend and because we were able to make those decisions as a studio and the other big thing is within EA every six months we have a company-wide Team Health Survey where everyone rates themselves on how confident they're feeling in the game they're making how well supported they are in the studio

and criterions consistently been reporting the highest levels of satisfaction content confidence and hope of any game team in EA so I think we're totally hitting those two big pillars that we're aiming for with we're delivering great stuff and we're doing it in a great way so that was the end of 2014 and then suddenly the old world came King so we had a request from help for help and we were asked whether we'd be able to help another EA studio ship the best game they could which would involve committing a large chunk of our team to spend nine months working on their game and at first glance I've got to say we were quite fearful and we were concerned that this would involve a return to a much more traditional form of game development at least for that big part of our team and having seen such a promising new way of working and starting up we really didn't want that to happen but we had a great conversation and with the support of the other team and with the a leadership we managed to use as an opportunity to try out our new highly autonomous way of working in a production environment to

facilitate this we did build a very small interface layer to smooth over process bumps because the other team was obviously working in its standard way and you know there was some jiggling needs to be done there but most of the Criterion team was just working in a new way and it totally worked right so we built the cop experience for Need for Speed and we're really proud of what we delivered and we're even more proud of the way it was delivered so despite zero overtime from the team we delivered an experience that was repeatedly called out by reviewers as excellent the only the only times we really hit snags in in our process will really places where our way of working butted up against the main team because they were scheduled in a more traditional way and weren't capable of fluidly moving their priorities as much as we needed and then it actually happened again before we were even done with that engagement there was another EA studio who asked for our help on a feature that they felt would benefit from some of our special

criterion source so we took the same approach and the genesis the Need for Speed thing was going pretty well so we started working in our new highly autonomous way with a team that was all self selected and the team decided what to build and how to build it and yeah that went pretty well too we delivered the speeder bikes for Star Wars Battlefront as well as some of the indoor speeder bike training mission and the fans and the critics loved that stuff so there's this quote from Eurogamer that I really liked the two-minute thrill of an indoor speeder bike chase is as pure - star wars joy as we've had since Ataris legendary arcade sideshow in 1983 I feel pretty good about that so once again we delivered that on-time on-budget without any overtime and with very happy people both in our studio and with the main game studio so since 2014 we've racked up 65 staff years of using this process in pre-production and 18 staff years actually both of these numbers are higher now and 18 Staffie is

on titles that have actually shipped so we're pretty confident that this this time scales to triple-a production which is why I'm now comfortable standing here and talking to about it so have we actually put this into practice so we've experimented with a whole range of different time structures for working from completely freeform chaos which was fun but didn't Rachelle - everybody swarming on a single feature until it was done and there were good sides and bad sides to both of those but really we've actually ended up coming back to something a lot more traditional with a regular seven-week cadence for and milestones and then six weeks of that solid development with work chunked up as appropriate right depending on how the team's working if they're working in a scrum style then maybe that's three week three - expense if they're working in an iteration style and maybe that's one six-week block that's totally up to the teams but the big thing we've added

is this big review and planning week between the milestones and this is really our primary means for sharing context and pushing that out to everyone so this starts off with a review of the previous milestone the whole team comes together for this and we all review the previous milestone and then we start looking at where the game backlog highlights some themes for where we should go for the next milestone then individuals will join a major theme to agree what can be achieved in that milestone paying careful attention to team viability the teams then physically move all their desks together so we have a big chaotic 20 minutes of desk moving and then the feature teams spend the next six weeks sprinting towards the next deliverable with the reasonable cadence of intermediate reviews as appropriate we also have a broadcast this is a team meeting which happens on Fridays Fridays are also our formal dress day for some of the team which is why this is our technical director Christian Holmquist wearing a bowtie and this isn't normal

attire in criterion but so every every Friday afternoon we have a team meeting where any group that has something cool to show gets the opportunity to stand up in front of the team and do it there's no Q&A it's not super intimidating it's a high-energy celebration of the great things that have happened that week aim to bring everyone along on the journey that the whole team's going on and then actually talking about reviews so historically in criteria now I'm sure there'll be some nodding on this point as well software reviews have been a very very fraught experience and they've often ended up being quite painful and actually quite ineffective as a result but we know that software reviews are crucial to the success of making a great game so we said about figuring out how we'd make software reviews better I think it was another talk about this earlier which I missed Nestle we we came up with the concept that there are actually at least three different very different types of review and if everyone goes into a review with the same understanding that this is what this review is for then an awful lot of

the fractious nurse goes away so the first type of review is just an update review we don't really need any feedback from you this is just so that you know where we've got to so you can talk to other people about how the journey is going and we'll get back together next week and show you again then the next type is a steering review this is when the teams come to a point a fork in the road and they really need some guidance from someone external to that team to really say okay maybe we should go should we go like deep on this one thing or should we try doing like another four examples of this and see what what breath looks like instead and that's a that's great to have an external view onto onto that and then the last one is a quality review so this is the team saying we think we're just about done on this feature like is this good enough should we stop investing in this should we stop working on this and move on right now and so by starting every review with this is an update review we don't need steering here we just need you to know what we're doing or whatever

then the the reviews go way way better so as I mentioned we also have frequent retrospectives and we actually and we've co-opted the agile start stop continue process and which is quite simple is everyone in the group writes down the things that went bad and the things that went good in the last chunk of time then the facilitator grouped those into themes that are related maybe these ones about communication maybe these ones are about some tech that we're using and as they're sticking sticking the stickies to the wall if anyone wants to discuss something if something is contentious they just shout flash and you draw a big old lightening bolt on it and then the team once they're all all up then the team goes through and discusses the flashed cards and agrees actions for the big themes that have come out so things were going to start doing because that will make it better things were going to stop doing because that'll make it better or things we're going to make sure we carry on doing because they're actually really working for us and as I

said the twist we've taken on this is that once the teams have done this the production group then does the same exact thing at a higher level with the outputs of their of the the team's starts up continues and that allows us to come to the team and say right so for the next milestone we're going to attack fixing this this and this so the whole team can see that the process is continually improving and that we're really listening and making it better so we have those retrospectives in the morning of the Monday of the planning week and then in the afternoon of the planning week we found it was pretty demotivational for the team to stand up so we have a big team meeting to review review the milestone but we found it quite demotivational for the teams after a morning of critical self introspection to stand up and say yes oh we missed this goal because well it was really hard and dave was sick and well we missed this goal because of this unexpected dependency and well we kind of missed this goal just cuz we're terrible human beings and like no one

really liked that and actually we we felt we'd already got all of the the main learnings out in the morning so instead what we have is a highly motivational celebration that we call the wall of Awesome so the feature teams demo the great stuff they've done and then the big highlights they stick up on a wall and they get a big round of applause and the whole team gathers around and it's all it's all kick-ass and it's way more motivational and also helps really sell the story of where the games going so everyone's really got a good a good feel for how developments going the next big thing we've really changed is how the space is laid out in our office and this really is to support the mastery phases the mastery pieces that I mentioned earlier as much as anything so we partnered with some workplace consultants to design an office space built around the concepts of explorers crafters and experts and their different needs and we came up with this idea of an interruption gradient so explorers on the right left

on this end or that they really like chaos right so the Explorer teams embrace chaos and like lots and lots of stimulation and lots of new ideas because that helps them find fun in whatever they're working on and allows them life stops them getting and drawn into little ruts then in the middle of the office we have the craft area so these are the feature teams that have agreed okay right these are the goals that we're working on we're going to crack on and just kind of will collaborate tightly within ourselves but we don't need to talk too much to any other teams like we can just crack on for six weeks and that'd be fine and then at the far end of the office of the experts you tend to be much more and lone-wolf mind Palace type people who really really hate being interrupted because it'll take them hours to get back to the same state they were in before and so here's a photo of the Explorer to craft area of our office and hopefully you can really see the difference just in this one picture so

on the left-hand side of this picture you've got the explorer area where everything's kind of chaotic where we have big two-foot cubes that we build little meeting rooms out of as needed and we've got big open tables that are that never have the same people sitting at them twice and lots of game playing lots of musics playing lots of good stuff lots of good distractions there and then you can see as it moves too right the much more ordered groups of craft groups so those are the feature teams working disappearing off around the corner there and then in the foreground you can see the big kitchen table we've got and we actually have a bunch of these big open tables around in the office and we much prefer them to meeting rooms for almost every case there'll onic meeting rooms in that a bunch of people can get around them and collaborate really effectively but have the huge advantage that they're super low friction to use so if there's a couple of you sat at a desk talking about something interesting and someone

else wants to to jump in it's virtually no effort to stand up and walk over to one of these tables and maybe grab another person on the way past that you know we'll have some good input you don't need to find a free room and then check their calendars and send the invites and all that jazz you just grab a table there's enough around the office and they're cheap enough and easy enough to use another big thing we've got is a number of people who work from home or work remotely for some or all of the time and so we've invested in a bunch of tools to make that more effective and the most obvious one is this it's called the double robot and essentially like a mini Segway with an iPad on top and it sounds like just a stupid toy right it sounds just ridiculously exorbitant but it's really not so when you have ever worked remotely you can feel super isolated so you can feel super like you're hanging on the end of someone else's agenda someone else's I am you just sat there waiting for someone to respond to you and please I'm totally blocked I can't do anything until you answer this message and so now you can

just walk over to someone and say do to figure email and it's amazing like it's astonishing so it used to take three days and cost about $15,000 for us to rearrange our office and because we've designed a way of working where we need to consist like constantly every few weeks rearrange all the desks that cost had to disappear or else it would have bankrupted us so we were inspired by valve obviously and so we developed mobile desks that required only one powerup and one Ethernet plug for anyone who hasn't visited the UK the thing at the bottom is a UK mains plug and they're designed to be or painful to the better than Lego and they'll have you down like a sack of potatoes it's amazing so we also have console dev kits in many audio racks on wheels so they can just wheel around the office as needed and then we have banks of power and network points in the floor along sort of well-designed lines which really helps that craft area be kind of

ordered and grouped and and fast and so now as I said we rearrange the whole office in under 20 minutes and it's a very very funny 20 minutes as well like everyone loves that and so that and the last thing we've built is we've gone back to school and we've got a special cloakroom where that keeps a lot of clutter off the floor but it's also super fun to come into the office in the morning and just hang out in the cloakroom for a little bit and talk about what you were doing last night and that's super nice and actually makes a surprising difference to how clean the offices so to sum up then how do you build a hyper engaged team you treat people as talent realize you don't have resources you have talented individuals who just want to make the best game they can give them all the context in the alignment you're able to help them to set up those powerful creative collaborations and then just get the hell out of their way and let them do amazing stuff you use proven drives to motivate your team don't rely on management thinking from

the 1960s use autonomy mastery and purpose that science knows really work and don't accept a harmful status quo actively seek ways to get better let your team tell you how they can be more effective and listen to them don't accept any dogma and let everything be challengeable so a criterion we're now confident that we're building games in the best way we've ever heard of and we're actually hiring so if you want to come and work with us then head up these URLs we've got a few places open at the moment we'll be opening a few more over the next few months so that's everything thanks for much [Applause] and other any questions just one question clarify it was a great old things and when you mentioned that you relocated people across this space based

on their type how do you actually get their type to test some more yes so the question was how do we actually move people so this is literally we have wheels on the bottom of our desks I mean that how do you determine that this people this guy is Explorer and this is expert I see so this this really comes down to the theme that they subscribe to so typically there'll be will have a core tech theme and by and large the people in core tech will really want to be experts so typically the core tech team will just choose to locate themselves at the choir end of the office just as themselves right yeah there's like a big conversation because obviously you can't have everyone deciding on the same exact space so it's a bit of a negotiation but that's ultimately how it works is the teams choose how to make themselves as happy as possible right thanks I was wondering

if you could say something more about supporting autonomous teams I've been part of a organization that said go autonomy and put a group of people together they thought was a viable group and say go and you know come back six months later and not really support the team during that time and so I heard you have DDS is process experts and I don't know if that's throughout the entire process but what's support to do you see is necessary for an autonomous team so that not the idea that they can just go off in the corner and succeed by themselves yeah that's a great question so really you mentioned teams going off for six months at a time and I think that's too long right so context will change and fade within a couple of months so you really need to bring the whole team back together far more frequently than once every six months and the other part is absolutely the the the DD the process coach stays within the whole way through the process to lend of the Mustang okay I've got two

questions one when you have multiple autonomous teams that are working together have you run into difficulties with all of them aligning on the overall project like you potentially have two teams that are doing great work but they're in different directions and aren't gonna overlap and my second question is what do you what do you end up doing when an autonomous team is failing like have you have you ever had to have a situation where you have to actually stop an autonomous team or break it up or give stronger direction to the team in order to help them succeed yes right so you trying to remember your first question aligning multiple word right yes so that really is you're absolutely right we've absolutely had that problem so when when we first started we didn't give much guidance to the team at all we'd like we'd have constant shares and constant discussions about the design of the game but really it was not everyone came to those not everyone that's involved and we did find people with diverged an awful lot like like you say you'd have everyone's doing great stuff but it's

all different great stuff and we found that started happening after about a dozen people and and I don't know if there's something around that number where a dozen people do too much work for you to just absorb what they're all doing and so we that's what we have to do so we have the broadcast every week for that reason and we have constant big team wide shares actually so one of the other things I didn't touch on is several times a week there'll be like an hour-long slot where someone will present like okay this is the new design for this vehicle or this part of the game or whatever so we try to actively keep teams more aligned the whole way through and we found that the balance we've got at the moment really doesn't result in that and divergence anymore so we've we feel like we've got that one nailed the other one was about teams that are failing and we absolutely have had that case a few times and that's always been because the team wasn't viable so either they've signed up to

unrealistic goals or they didn't have the right skills to meet the goals and it always takes longer than you'd hope to recognize that that's the case but still hey we're all human beings you know it's not like anyone's perfect but as soon as you noticed that you can catch it in those terms right of hey that your team doesn't look like it's viable right and the team kind of knows it's not succeeding so it's not like you're terrible people for not doing this better and it's like hey you aren't able to succeed at this so what can we do to help and we've had a few cases where we've recognized that oh well you obviously need a couple of engineers in here to help you prove is that part of what you're working on or or maybe you're missing 2d art design or maybe actually what we need to do is have a big serious like to our review of the work you've done and then figure out the direction for me to progress in and if you like for the next part of the Mustaine that's awesome thank you cool

are there any tools you use to to manage project wide high level tracking and predictability yes so predictability is interesting we've been talking about it quite a bit recently I'm confident in predicting based on our past performance that we can deliver something kick-ass at certain points what I can't tell you is exactly what will be in it because the team needs to figure that out along the way and so that is that can be a bit of a nolle thing and we need to work with that and that's just constant conversations we do and we do actually use roadmaps so the team it and it's the team that comes up with the roadmaps it's not us and driving it but we do have roadmaps to figure out okay well we know we need to be releasing something at this point in time so let's have a talk about what's reasonable to expect to get in there and and to align across different teams we don't have any specific it's not like I have a silver

bullet tool it's basically a big bag of conversation and alignment dewey's thank you cool hi Alex thanks so much my question is around scalability along the two dimensions of size so how many people you have in complexity how many things you're juggling and the first part is looking back what were the inflection points where you really to adjust your approach because it was breaking and then projecting forward is is there a limit that you're hit and this will break down like if when you are throw the space or whatnot you're hiring and you win will you hire this forever or are you going to cap your size so yeah it's a great question is how scalable is this so we've spoken about this quite a bit so we hit a big inflection point at about a dozen people for sure and which as I say I think that's probably about and if you've got more than a dozen people you can't all sit around one table you can't all keep

up to date for free with what everyone else is doing the next big inflection point was kind of around 25 to 30 people and that was the point at which we needed to start identifying teams as a production group so less than 30 people we were able to share enough good context that people that the team could just make the right calls about what they should be working on but going beyond that we needed to to do that going towards the future like we don't actually want to grow to be a huge team we don't because we all like being on a team where you know everyone that's it's a super nice thing and I think as we've showed with the Need for Speed work and with the white shell work actually what you can do very effectively as effectively subcontract out a chunk of your game so I think if we needed to scale beyond a point when it when it got comfortable in our space what we do is we would have a second team in a second space working in a way that they wanted to which would hopefully be similar to the way that that we work and then feed the two

together thank you it's my questions about prioritization and we need to give your team's complete freedom and autonomy like how do you how do you know they're not chasing shiny objects you know they're already working on the textures of the wheels before they really even built the engine to the car you know that kind of thing it happens a lot you know in my team so I'm just curious how do you how do you keep them working on the right things yeah that's a that is a great question and interestingly I think we've had less fewer problems like that since we've gone more autonomous and then when we're less and I think that's parley it comes down to the bigger context right because the the decisions that they're make the people are making when they're spending too long on shiny wheels is just a bad decision but then maybe they don't have the bigger context of dude you've got like 200 other cars to make I mean that wheel is good enough at this point and so sharing that context constantly either is a big help but I think also I don't know it seems

to be that people take on so if you're told yeah you're making a car you have less ownership over making that car than if you're asked hey well you know we need to make some cars and you then say oh I could probably make a car Ryan right so your question is more of like you know let's say we're making a car racing game you guys make car racing games how do you how do you make sure that they're starting at the right spot you know just priority wise in general you know step one we should probably make XY or Z you know I mean are those just constant communications you guys have as a team and pretty much you're just talking about context and making sure where it's and aware of like what we're trying to do and I guess there's conversations that break out of that if you know well maybe this makes sense for us to do first you know yeah exactly I mean so I remember at the start of the the Need for Speed work what happened we've got some really great creative direction from the main team from ghost and we then got the whole team the you know engineers designers and me and other people it's

not like I do any work right so but we've got the whole team around one table and we sat down and we and planned out on like the old cards and all sorts actually what's the story of a cop chase how how does this work how does this all come together and because everyone was there everyone totally just got okay right so that means this thing's important and this thing's important and obviously these things are relatively you know more important than these things that makes them make sense yeah it just sounds like you guys communicate a lot more which is great anyway so yeah thanks it was great talk cool thank you I'm curious and sounds awesome I'm curious how and maybe if you could gel this with a traditional like work-for-hire or publisher relationship where you're having to design to spec or come up with features and commit to features ahead of any of the development on those things yeah I mean so we did that it's a fine question for sure and I'm fully aware of that I'm The Criterion are actually very

lucky that we were in this position to be able to do this first of all certainly I think not every team is as lucky as that and some teams will and won't be able to go as full autonomy as we've gone that said I think I suspect most teams could be more autonomous than they currently are and certainly my experience so there are within I was careful to put on the slide of ala Tana me within project parameters so perhaps if you're working to spec your project parameters are well we have to deliver all of these things by this date but then it's up to the team to figure out how we deliver all of those things and who delivers what bits of those things does that answer your question I think so yeah cool thank you maybe a junior person on the team they're excited at first but in the middle yeah

I mean that so happened I think that's I can't think this example I'm sure that's happened so we have what at whichever way you're working pretty much and you'll have a daily catch up with the rest of the people on your feature team and our feature teams so our Dedes are very supportive and will notice that kind of thing happening but a feature teams are also very supportive so a lot of the time it won't even be that person kind of freaking out and retracting a little bit would be someone else saying hey did you want to handle that I've got some cool ideas which is yeah I think that yeah yeah this'll Virginia people need to be working amongst more experienced people right so the more experienced people are typically good at supporting the Virginia people and saying hey it

looks like maybe that's not going so great and John a hand cool hi great tuck thank you I was wondering if you had any experience trying to go faster with this process so like I work with a mobile team we're like 20 people and a seven week sprint it's just that that's a beautiful luxury that's like it's like seven live pushes for us so sure yeah have you have you had success doing this with a smaller time scale yes so and not with a team of this scale and but when we were a team of like a dozen people what we'd actually do would be we'd have the whole thing compressed down to maybe three or four of that whole cycle within one day so we'd have maybe three big build reviews a day and then everyone would leave the build review with notes okay I'm going to fix this I'm going to fix this and then come back in a few hours later we'd go again and that works really well that we were flying at that point but it just unfortunately didn't scale to bigger projects I like bigger teams and bigger

timescales also we had to put more structure in but yeah all right good try it out cool my question is almost identical just where do you put the pressure of the of the time schedule does that stay with your leadership or does that actually go straight down as a just a parameter for the autonomous teams it's just a parameter so we've actually am something I didn't mention is we've we've really recognized that there is a crucial difference between state targets and date targets so date targets tend to be immovable but what you're releasing on that date is typically moveable whereas state targets is a vision for your software that you need to get to for it to feel like you want your game to feel like and you just have to be comfortable that those two things are different so what goes in a release is more or less of what you meant to put in that release and sometimes you have to be flexible and change stuff around you've got quite a

big team and I've seen it was an interesting way of just dividing everybody by space and their working mode that's a real good idea actually to implement how did you do that how did you define the personality types for few teams and how much time did it take you to define that and put it in action so we actually we've never so we had the conversation where we came up with that grid and that perhaps took like a week of conversation between lots of people I've you know over the course of a week to to agree that this actually made sense and everyone kind of fitted in it somewhere and we've never done like an official hey you're that kind of that kind of person you're that kind of person but because the teams choose roughly where they want to sit and they choose the way that they want to work it's kind of as a I think that what we need to do as studio leads is make everyone feel comfortable that hey yeah so if you're if if you're an expert right if you

really want well-defined problems where we're not going to change the design on you all over the place and where it's really R&D work that's cool and you can succeed and also if you're if you just want to go play in a big open space and then kick out amazing bits that are fun then that's cool as well and so I think because we're explicitly saying that that's all cool and we've got this rough rule that the office runs like this from highest interruption to least interruption like the team just does the rest thanks cool I realized that I was thinking of your crafters and your experts just in terms of engineering I was wondering there's QA there's art as so does everybody I was wondering if all the work or how much of the work fits within this team context so surprising amounts of chelating sorry yeah design QA art that's engineering this is

actually my bad so I noticed this week from some other talks that not every studios and as cross-discipline as us we're just crazy cross disciplines so each one of our feature teams will be composed of whatever it needs to be composed of so maybe that's a designer maybe that's an artist movies an audio engineer maybe there's QA maybe there's whatever right so we have all of those skills on the team and the people will subscribe to a feature team but so we've got a like a vehicle's team that has a couple of vehicle designers and then a implementing vehicle designer and then a couple of hardcore physics engineers and like an audio engineer and producing the DB and and QA on it right so the they are really really well self-contained feature teams and that's really important for them to be viable and succeed having worked with something else like this one of the problems that we ran into was for example audio we

don't have many audio people and so a lot of features would have leftover bits how do you guys you know kind of pull together those leftover parts like a feature might be complete but it doesn't have audio and how do you keep from stretching one person to thin across multiple teams yeah and so that's where that's a great question we also don't have many audio people I think we're hiring at the moment for an audio designer so okay oh yeah we only have one animators as well and yeah we're hiring four more of those but so we absolutely have that same problem and really that comes down to making the teams viable and as I said there's two ways to do that so one is to change the skill sets sorry one is to add more skills and you can't do that past a certain point like you say if you've only got one audio engineer only got one audio engineer and we do sometimes split one audio engineer across multiple feature teams like people do drift between teams as I say the teams don't

necessarily stay together for the whole time there they do drift around but the other thing you do is you just change the goals so you accept that hey so do you know what this time we won't get we'll get to good handling feedback on this thing but we won't have the audio pass yet and then next milestone we'll make sure that we've got the audio pass ready and so that that becomes a question for producers to kind of wrangle with and work with those dependencies and and talk to actually the audio designers and say hey so what do you think is the highest priority to to deliver that yeah that's true as well so we have as we get to a release we'll have a chunk of time set aside for the hardening and that's much more iterative much more and less blocked out so all of those little pick ups will get picked up then is the hardening is that like a Polish yeah penalty or whatever you wanna call it yeah thank you yeah it is okay it looks like that's all the

questions and I think I'm about at a time anyway so I'll be over in the wrap-up area just around the corner and for the next little while with some other criterion folks if you want to and talk to us about anything you've heard yeah thanks very much [Applause]