Creating a Development Culture that Embraces Change

[Music] hello everybody and uh there's a lot of you today this is good thank you for waking up this morning and thank you for coming to join me today this is a talk on how to create a development culture that embraces change uh for your awareness there are exits at the back of the hall and my goal for the for this is that nobody's gonna use them for the next 30 minutes um we also you're going to get a survey at the end of this or a questionnaire and it is vital that you do that and it actually helps in the talk as well feedback is the core of being able to change and adapt and your feedback to me allows me to improve and change how i present and improve over time and that's why you're all here right we're here to improve changing for the better and that can be difficult but it doesn't have to be and actually we're always changing that's a concept to take into what you're working at here every single day you'll do something a

little bit different and what we're going to talk about is how we turn those little changes into something a little bit bigger and make it more comfortable for everybody who's involved what we're going to do through the talk today is use runescape as a series of examples on some practices that i'm hoping that you'll be able to take back to your own teams and i suppose this is a great chance for me to actually introduce myself my name is connor i'm the game director the director of game development at jagex game studios and i suppose like most of the people in this room i love games but i also really love making games my goal really is to better make better games so we're going to continue to improve over and over again i got the pleasure and privilege of being able to manage and lead runescape for a number of years and for anybody who isn't aware runescape is a 17 year old veteran mmo it has lived through the rise of free to play social media and streaming and for all three of those it's managed to embrace them and make them a strength for it rather

than a challenge we use runescape structure as a template for how we build service teams at jagex and we continue to build teams within the studio on that model jagex for i'm expecting uh fewer people are aware of jaggers than they are runescape the game kind of leads to the company um where our goal is to become the home of living games and we see living games being the next step after games as a service are live service games something that moves beyond a hobby and becomes a lifestyle for our players and that means that we need to build things that are evergreen and ever-changing and can always be there for our players that adapt to what they need and adapt to the market it this demands our teams to be flexible and adaptive and our goal is that with that we can change with the market are preferably ahead of it to make things better i'm going to cheat and i'm going to let well i'm going to let you guys cheat i'm actually going to give you the takeaways at the start so if you do end up having to use the doors you can at least take something away with you

at least you get some value from today so the first part we're going to look at is talking about how improvement is not immediate and that sounds like common sense but the number of times you're going to try and make a change and immediately expect everything to be perfect would probably cause a challenge so taking that into mind is important we'll be talking about pillars for your development process that can support the change process we'll go through how smaller teams can help that and help that adaptation and then for me the most important thing is distributing decision making down into your teams and giving ownership to your developers and then also because i'm not going to completely throw everything away there is a super secret end of the talk takeaway that if you do wait for the 30 minutes you'll be able to see what that is okay so change is hard right we know change is hard so why don't we plan for it when we are going from today to this bright and shiny future we expect this lovely path of continuous improvement and getting everything better all of the time i mean the real fact is it actually looks a little bit more like this the expectation that you bring something in and it will immediately cause your teams

to feel a little bit uncomfortable and it is change and it is different the habits that people formed before will no longer be useful when you go into this state going into the process understanding that there's going to be an impact means that you can look for how to improve how that's going to cause the improvement in the future you can look for small wins and look for a turnaround you have to get over that first kind of the stumbling blocks i'll give you an example from runescape there was a time fairly like i'm kind of ashamed at how far into the development runescape didn't have source control we had 100 odd developers all working from a network share right that's crazy right actually it has a lot of benefits you have you've accessed everything everyone has done and is doing right now the level of sharing and collaboration is incredible the level of flexibility that you have is amazing you don't have to worry about being able to or what's your branching strategy or where you're going to grab a particular asset everything's available to you immediately sounds great except for the fact if anybody breaks anything on your login

script the whole team is taken out so that's not good so we decided to change and we looked at how we could roll source control into a team while continuing to run a live service i'm hoping no one ever has to do this by the way it's an awful experience also you have to train everybody on how to use source control because many of them never had even seen that knew it existed and when we did make the change that was difficult suddenly all that flexibility wasn't present we had artists who were like well there's a texture that someone's using for an update that's coming out in a couple i want to use it you know well that's coming out after yours so we got to plan around this now and we start to put in all these processes and structures to improve things and eventually it got better we looked for those quick win our those early wins and look for those small changes to suggest we were going in the right direction we had an easy one um the login scripts weren't being changed and knocking the entire team out every every couple of minutes so you know that was good but looking for those kinds of wins uptime inside the development team release cadence becoming stable all that kind of structure coming into place demonstrating to the team that the

improvements were happening allowed us to continue to improve to today where it is just our habits are formed again and everything's fine if we had gone in without expecting to stumble and without uh expecting everything to be rosy and perfect at the start that turn around to bring the team on board would have been far more difficult and i'm guessing a lot of people in this room have experienced something similar in a different type of change in the past but with this some of the ways that we'll try and be able to support those kinds of changes in the future is our development environment and process itself so i'm going to go through our development pillars saying try and go pretty quickly give everybody a bit of time to ask questions at the end so uh if anybody knows scrum you'll see that we rip these straight from scrum they're just the three pillars but actually they're really good you're talking about transparency being open with information and open with each other on what's actually happening inspecting because being open is pointless if nobody's actually looking at that information you're just generating random useless data and then adaptation because

inspection is pointless if you don't actually make a change you're just sitting there having a discussion and then going away afterwards so the three of these build on top of each other are in the core of how we work and let me go into each a little bit more in depth to give you an idea of the level of transparency that we have at jagex and why i say you can always be more open our personal development and our objectives for each individual inside the organization are open for me all of my my objectives for the year are available for every single member of the company to see everyone from player support qa design marketing and my own direct reports know exactly what i'm trying to achieve and they also know if i am achieving them my progress and everything is available too in terms of our products every piece of data is available to anybody who works at jagex you can delve into our current conversion rates our retention rates all the monetization detail and down into the individual little details around engagement in particular pieces of content that exists in runescape that's that's the level of transparency

that we try to get to it's important that people are able to see how those things work to try and get to this level of transparency it's something that you can take on to yourself if you are taking and putting any piece of information together for us the default is to be open and closing down access is the exception for yourself ask yourself why you're not sharing something wider why you're only choosing a small subgroup to get involved in something you don't have to tell everybody you're doing it but don't lock down their access to it you can also think about the time you're looking for information and if it was difficult for you what's going to happen when the next person looks for that piece of information how can you make it easier for them to be able to find it and grow that culture of transparency inside your organization and then transparency is really tough it's it's difficult to be open with everything that's going on so being vulnerable yourself first will show those in your team that that's kind of the way you want to work to give you an idea of where we've used openness on runescape

uh let me go through a a tr a tough harsh truth that we went through at the start of 2016. we launched a very significant update trying to kick off the year we brought out a new skill and for anyone's not aware runescape has a number of skills that players can progress by engaging with them adding a new one is a significant amount of new content for somebody and when it went out we noticed that the day one was really good reception was strong people were enjoying what was there but then we noticed the progression was way too fast they were getting through the content very quickly we half expected that it was driven by wealth people could just liquidate their wealth and get through the skill we thought people would do that but we had to look deeper now runescape gathers about two terabytes of data every day that's driving some of our heuristic analysis and a lot of the metrics that are given to the team it meant that we had a lot of information coming back but you'll see that when something bad is happening a lot of people jump on assumptions and guesses you'll have red herrings flying around having the level of transparency and

openness in the information allowed us to cut those down and go directly to where the problem was happening how are the people who are progressing fast really engaging with the game what exactly are they doing and that information allowed us to see the see the cause identify a fix and roll it out in an incredibly short time frame which is immediately dropping out of my head right now uh i think it was in a week or two so the kind of cycle that we're doing on how we're altering the game is driven by the fact that the information is available to everybody but of course that required us to actually delve in and inspect the information and data is worth nothing if it's not observed and understood anybody can look at a page of stats but understanding what those stats mean is the most important part so another example of where we've used this in runescape why it's important um the most important part for us in sprint cycle is the retrospective that element where you come together and look at what's worked and what's not worked and how you can improve over time

we rolled out a marketing initiative it's called premiere club it's a year's membership it has perks you'll be familiar with it for a lot of games and one year it went really well the stats came back they were strong um but we felt like we could do better the process internally could have gone much better there was some confusion some conflict so we ran our retrospective gathered everybody together we talked about what worked and what didn't work we documented it made it made it open had a some good honest open conversations about what we were going to do next and then we documented it and put it away and the important thing is coming back to that inspecting it and understanding it the next year we brought that back out when we were about to run the project again it was like a letter from the future which was just telling us all the things we were about to do wrong allowing us to make sure that the next session was much better and it was the development was much smoother there were no conflicts there were no issues people knew exactly what was going on the communication lines that we needed in the first year were much better in the second year and our performance was up as well

so we saw tangible change in the performance of the product not just the performance of the team so making sure that you are inspecting information and inspecting any information that is there to make your future efforts much more valuable but then of course the most valuable part is actually adaptation so like i was saying retrospective is the most important part of our process for us we try and make sure that change is an expected part of our day to day i'll give you an example a really tiny example uh not all changes have to be big we do free fruit um inside in the studio it was getting a bit stale wasn't was really exciting it's really weird fruit wednesdays to try and liven it up a bit small little change make people feel a little bit more excited it didn't work nobody ate the weird fruit so we've gone back to the other one but you know not all changes work out a much bigger change that we went through and this level of um saying change is one thing but actually doing it is completely different a lot of people will have experience transferring a an established team from

kind of a waterfall step-by-step method into an agile framework like scrum and we did that on runescape looking for ways that you can make small incremental improvements or small changes or pilot those changes creating a movement for your change rather than an immediate dictation of altering can be very useful and we did that with runescape by creating a small scrum team gave them an isolated area of the game and allowed them to continue to improve it didn't work out immediately so it's a good thing we didn't change the whole team and we altered how that process worked because we went through our cycles and we were open with all the information we inspected it and we added we adapted as we went along it allows us to refine that process in a small section without impacting the whole team as it was improving we rolled it out to a bigger and bigger group the people who were involved at the start created a movement that allowed the overall team to start to change now eventually we did actually have to go for a big like you know that last big bite of a burger and trying to get that last thing done so we had to do that and it was a little bit more difficult but at least we knew we were going in the right direction because we had tried and made an example of this already

so we make sure that we're always changing and we make sure that our process is designed around it but what does our actual structure look like and how we make sure that we're making those changes happen so i like to think of this as if you think of a large development team runescape itself is uh over 100 people involved so trying to move all that can be difficult like steering a tanker well we cut it up when we try and make it more like a fleet so each of this fleet have their north star the direction of where they're going they understand the general direction they're trying to go but they have the flexibility to be able to move and adapt and adjust as necessary each one can be an individual one ship having a problem doesn't mean that the whole fleet is in trouble you can continue to adapt and cover people's changes we started this with one small team not the team that did scrum a completely different team inside in runescape and we called them the ninjas i actually did a talk a couple of years ago about creating these autonomous smaller teams and ninja featured heavily

in that we give them a complete autonomy and control over what they're trying to do they understand the game they don't need to be told what needs to happen next you give them the ownership and the capability to be able to deliver improvements into the game over and over and over again constantly improving but they know the direction we're going so if you look at the leadership structure of broadly how the team is working you kind of have these six groups and they all have their development and their art and their qa each of them are building out their content um and if an injury or one of those they're given guidance by that product leadership group but actually one of the most important things that we found was that trying to put that ownership into the team was difficult so we actually built a small little structure within those teams which are allowing each of those development teams to operate as its own group they're small enough that they're almost telepathic in how they communicate so even though you have that single focal point everybody knows what everybody else is doing

but they feel like they have the control they don't just feel like they have the control they do have the control they can choose how they're going to do something the leadership group decide what generally needs to happen how we achieve it is all down to the team that is trying to develop it to give you an idea of the level of control that they have one of those teams was taking on a fairly major release for us it was god wars dungeon 2. godwars dungeon is a very old dungeon in runescape we launched a second one of those um also in 2016. i promise not all of my examples come from like the same time period um that team took ownership of what they were going to try and develop everything from the visual identity of that project in keeping with runescape as itself being given guidance by the leadership group the communication plan marketing plan and then also the release rollout like how would we do the lead-in over a number of weeks and how would the actual event of the launch feel for the players that control was given into the team now they got advice and they got guidance there were suggestions coming from

leadership we made sure that everybody knew what we were trying to achieve and they worked with the other departments inside the team our publishing and marketing groups our community group to make sure that they had everything they needed for a positive launch and making a significant dent in player behavior and a game that's been running for 17 years is difficult i'll tell you that from firsthand experience but they did it and it was by allowing them to have the ownership over what they were trying to do and lay out what needed to happen gave them that level of control but sometimes that can be difficult right handing that level of control into a team so i'm hoping i can give you something that will help a little bit pure freedom it's probably not a great way to approach this especially if your team hasn't done it before so if you think of the concept of freedom within a frame set your boundaries set the space that somebody can operate within some examples would be depending on the size and the level of awareness in your team you have maybe one of the boundaries is the cost of the team or the cost of the project you're trying to do you have

an amount that you can spend there's an expected return i mean these are pretty basic ones uh the quality that you're expected to deliver all of us have are expected like please don't ship a class a bug day one and then the satisfaction and the expectation that you come from back from the players those four antagonistic expectations um allow the team to have complete control within that space not telling them what the update's going to be i'm not going to tell them what the creative direction should be or i could one of them that might be one of the boundaries that we're going to set but really it allows that team to feel like they have that control the borders create the constraint that allow them to feel the freedom that they need as that gets better maybe you can extend out those boundaries but that's up for you the boundaries that work for a team depend on the team your own teams themselves so let me try and bring all of those four ideas together as a single concept we're going through something right now right so let's go through these

our pillars for change first of all we have been looking at how runescape has been performing we've noticed that there has been some ways that we could improve some little bit of stagnation in some part portion of the community that we need to energize we need to really do better so going and looking at that we took a lot of player feedback a lot of feedback from within the team a lot of open and honest and good conversations none of them ended up with people arguing or shouting or screaming at each other i promise but we made sure that we also interrogated that data delved into it really understood what it meant were we just seeing thing or were we seeing things that weren't there was the data backing up what our expectations are as a team and it was so we went a little bit further we look at the fact that we can change so we considered whether um our small changes were good doing enough and the fact was we were always changing so the problem was there so obviously the changes that we were doing weren't enough we need to go bigger we need to do something a lot more bold

so we spoke very closely with the team that distributed decision making this isn't one leadership group dictating what's going to happen this is a collaboration working with the team and discussing what people are expecting from within the team we were looking for more control within the developers i've already told you about a team that had felt like they had control we wanted more of that those teams are very close to the player base so being able to give them that that level of ownership was really important and they wanted to be able to have uh more awareness of the direction of the game and make sure that that was really clear people felt like it was starting to defuse a bit looking at the players they really wanted good releases very regularly i've asked our our data analytics group about how we should approach releases and they gave me the very helpful advice of they want more bigger better faster and more oftener so i suppose we have to do all of those things so we will try so with that discussion after happening we decided it was time to change and i realized i said small teams but we

went smaller so we're looking at giving that ownership almost to the developer level we work in pairs with qa support as well but really growing very very small consider it to be more like a swarm instead of having six fairly large teams we now have like nine or ten possibly eleven depending on the time of day uh each of those teams has a feeling of identity but the important thing is you can bring them together on a larger project or spread them apart as necessary it's that kind of flexibility where you see people bringing together bringing together developers for major features and splitting them apart we're trying to create the team identity but also have that flexibility and i may be trying to have my cake and eat it right we're right in the middle of this change so maybe i'll come back later on and tell you if this actually worked we're trying to give them as much autonomy as possible but still giving them that north star from the leadership team to give direction and so then it's understanding that improvement is not immediate like i said we're at early days on this

but i'm hoping that if you consider this graph from earlier where that's yesterday and the future is off there in the future and i'm not going to go along that lovely dotted line i've to accept that i'm going to cause problems when we start the first change i feel like we're about here we went through a lot of difficulty when we started it hit the teams quite hard trying to understand what was going on and trying to get that ownership into a smaller group making sure they're all aligned that was difficult but we knew what we were getting into and we understood that there was going to be a challenge and we started to see winds in the past couple of weeks even shortly before gdc this slide was today was a little bit further back along that line we've already seen a couple of updates come out between middle of uh february up to today and uh we launched a large new quest in the past week or so and it's going down very well players are seeing that ownership and they're seeing that the change in our direction our development teams are seeing their impacts we're not there yet we are still below where we were we still need to do better

but we know we're coming around the hump and we're seeing major changes every day and it's not just me it's the whole group can see those improvements starting to happen and we look out for them when we celebrate those small wins and then to just kind of really confuse ourselves runescape's actually going to be coming cross play to mobile later this year so looming on the horizon for that swarm is a fairly major change in approaching uh cross-platform development so we'll have to go through the process again so it's supposed to wrap everything up and give you a little bit more detail and of course give you that super secret under the talk takeaway that everybody is still sitting here waiting for the first thing is improvement is not immediate you need to become familiar with the change before you see major benefits and that's okay that's the same for everyone the smaller the change the smaller the dip but it's still going to happen be aware of that when you see those wins make sure that you celebrate them celebrate them as a team it'll feel better you're going to have your resolve hit by the first impact

especially if you're doing a large change the tiff the development processes that you're using and the pillars that are there to support them what are the core concepts that you hold dear as a development team we make sure that we embed change at the core of our process to make sure that people are constantly adapting and constantly improving changing for the better consider whether small teams work for you i like this one as you can see we went even smaller so that's the thing i like may not work for every team to consider whether it's the right thing for you for us having a fleet rather than the tanker allows us to spread our bets and consider where we're going next it doesn't work for everyone but i encourage you thinking about it does it work for your uh for your group like i said the most important one absolutely is distributed decision making understanding that you can set what is going to happen but that how being in the team and allowing them to creatively approach a challenge giving them the ownership and the empowerment allows them to enter into any large change

with more investment and so i encourage everybody to consider how that could work for you once again this doesn't mean just tell everybody to do whatever the hell they want there needs to be context there needs to be control freedom in a frame is a good way for us to try and communicate that to people and then i suppose i should give it away overall i'm hoping what you've taken away from all of this is that change isn't really a process it's more of a behavior how you work and how you approach your day-to-day should be something that allows you to alter and adapt and that is changing a creating a development culture that embraces change and thank you very much for listening and i believe we have five minutes for questions if anybody has any oh that's easy well thank you very much hi hello uh i know you're still earlier in your rollout overall process of

improvement how what changes do you think you anticipate you would need to make if you had to scale this out to two to three x the current team size um what i'm hoping is that by so for anybody who didn't hear uh what would i do to try and scale this out to two to three times a larger team so to give you an idea we're talking really the the core development group is about 70 people so are we talking like two 300 people um i believe that the it becomes the coordination of the multiple smaller teams trying to split them into other smaller areas trying to create that hierarchy of groups or creating that ownership may help the clearer the clearer your direction and [Music] vision for a particular product is the easier it is to disseminate that information to a larger team and if there is any weakness in your vision the more groups you start adding in the less effective that they'll become

so i would look at like clarifying exactly what you're trying to achieve so everyone understands and then also looking at ways you can improve communication between teams because trying to get well if we had like let's say six before we go to the really small groups but if we say six if you added like if you had 12 or 24 of those then communication inside in that room even if you only bring that leadership group is communication between 24 people and that's a challenge so think about layering a higher hierarchy inside there my question is how small were you able to take the teams before you got to a point of diminishing returns uh well to clarify points so the question is how small can i take the teams before we see diminishing results there is a benefit and so this is very much up to individual teams and how different teams are working on runescape the structure that we've built over multiple years has all been around rapid iteration rapid development

so to give you an idea once someone writes last line of script to everybody in jagex actually being able to play the change they've just made is about 45 seconds so that level of rapid iteration means that we can go very small in our teams we also make sure that our development group um are as as cross-discipline as possible so those scripters are writing both scripts they're also designing puzzles gameplay and narrative design they have support in that and that's vital just letting people do whatever they want in any direction they want can cause challenges but giving them that level of control and then allowing leads for a narrative or for conflict combat story and puzzle to be able to support them means that they can take more and more content onto themselves but that's a big challenge it's been 17 years of trying to get that right and i wouldn't say that we're perfect so we got down to two in a qa

uh hello so my question uh regards uh scrum philosophy uh i just i'm curious how much do you took from that and if how much your your approach varies from classic scrum i mean for example the roles like scrum master product owner and the team division scrum events and i'm just curious what what you guys did actually to make it work for your team uh so we um i was told once about a way to approach learning new things and the first thing is to do exactly as it said exactly as someone says to you so i suppose just to clarify the question is um how do we vary scrum and how did how do we come about where we are so we started and we had um all ceremonies were working exactly as you'd expect we had sprint reviews uh sprint planning retrospectives backlog grooming daily stand-ups and we have most of those already are still today we actually hear very closely we do have scrum masters inside the team i just got the boxes started to get a bit messy um

what we try and do is the first thing is approach it just do it as it says it in the book don't think that you know better than the person who's actually laid out a framework and designed it because they've probably gone around the mill a bit uh once you are doing it and understand that everything are you're doing everything as it says then understand why you're doing it don't assume you understand it really understand look at what it's actually achieving and then once you have both of those cases learning are doing learning from what's there then you start making the changes because you understand what impact is going to happen from making an alteration so our sprint reviews started to get a little bit shorter our retrospectives got more important our product owner scrum master role started to diminish when you have uh swarmed and you don't really have enough people to have things so we don't even meet the criteria for a minimum scrum team but we make sure that people have the support that they need but it came from doing it by the book for a period of time to understand why great talk uh you were talking about

giving ownership to the teams but how often do the teams report to the management and how formal is that uh so the question is how often do the teams report to the management and how performance okay we have depending on which layer of the organization we talk to the teams themselves are regularly catching up day to day and week to week as a leadership group so that product leadership group so that that team are like as a title people are very aware of what's going on as you move larger in the organization so to give you a context i have i have um runescape and we have uh old school runescape if people are aware so that was we launched that about five years ago and there's another two teams at jagex as well so each of those will speak to me at least once a week but i'll be in touch with the leads for each of those groups on what's going on fairly regularly in informal discussions and then we have at least once a month a relatively detailed breakdown of where we are in progress what we're doing in the future and what direction we have and we go we're looking at being more

thorough in uh quarterly chunks but depending on how close it is so the rate reduces the further away you are from the customer thank you thank you hi thank you for speaking today um my question is how do you maintain uh cohesion across your product when so many different teams are making decisions is that the leadership team the product teams who's responsible for that so the question is how do i maintain uh cohesion i suppose the easy answer is i don't um it is a lot of hard work from everybody involved in developing the game and a lot of communication back and forth between them the along with the retrospective we do a daily review every two weeks or not daily review sorry uh a day of reviews uh every sprint cycle and that is streamed within the studio so everybody can see what's uh what's being changed and all of the teams can see all of the other things the teams are making that's that's one

of the first points we then have the leadership group that layer that i had at the bottom that group is aware of everything that everybody else is doing and their primary role is to i like to think of it as like nudge and guide people in the right direction but when necessary try and come in and a little bit more force to try and bring people back in line and some of that is clarifying the vision like i was saying at the start like if people are deviating it's not necessarily because they just want to do what they want it may be a case that your communication of direction and vision isn't clear isn't clear enough to bring people back in thank you i'm kind of doing we are out of time uh well i'll do one more can i do one more i'll do one more i'm sorry i'll ask you uh how do you uh organize communication specifically communication between those teams uh is it like single point of contact one person represents the team talks to the representative of the other team and in case of conflict between them how do you mediate those conflicts

okay so uh i'll try and do this very quickly so i'm sorry if i don't care everything but this is a communication between the teams and resolving conflict when it arises the way we make sure the communication happens between those teams is there is a like a producer who represents multiple of this in the swarm stand up that there's somebody who represents multiple of those teams and makes sure they're aligned and then that person speaks to somebody who represents the other swarms of people uh that's how we make sure that people are speaking between each other but also the team isn't large enough that we have massive breakdowns of communication they're all on the same floor they all speak to each other fairly openly anyway the other part that we try and do to mediate conflict is open and open and honest uh conversations are not only good for transparency they're also good for airing conflict and resolving them i'm not going to say we're the best in the world at dealing with conflict inside in an organization and we could be much better but we do try and hit it head-on when it arises so i'm going to ramp up because i've totally gone over time i want to thank

everybody here for your time inside in the room and i will be in the wrap-up area at the other side of the hall thank you