Hutch: Fantastic Mistakes and How to Avoid Them

[Music] um this is hutch fantastic mistakes and how to avoid them thank you very much for coming i'm andy watson and i'm the ceo of hutch and we'll be talking about the mistakes and learnings of hutch focusing on the last five years that have led hutch to becoming a profitable studio so you don't have to make the same mistakes that we've made the reasons behind this talk i wanted to talk about hutch because i think the growth profile is very different to a lot of other studios i think most studios have a really big hit or significant amount of funding whereas hutch has had a very modest success in the beginning and have grown quite responsibly after that there's some common themes that i'm going to keep coming back to lean culture like for us lean isn't about building cheap and ugly games it's about building for design efficiency once you accept that designs are an assumption and you can build the core value of the game the mvp and but focus on the viable

you can increase your speed to learning to understand the value to players i'm not talking about the soft launch though i'm talking about before you've committed to making your game break one rule this is an internal term we use and it's about understanding your references um understanding the risks you're taking and only innovating in a way that adds value to your players and to the game and invest in your team this seems really obvious uh but you'd be surprised how many studios don't do this the team are the ones making the day-to-day decisions so sharing positive and negative performance with them and never taking learning for granted just a little bit about hutch we're a studio of a company of 60 people spread across two studios the london studio focuses on the creative aspect of games development allowing our canada studio to focus on tools technology and ports uh hutch was founded in 2011 by five x sony developers they've been there between five

and ten years each were working on console games but they'd never actually managed to get a game released and they wanted to go through that cathartic exercise of getting something into players hands so the studio was really born out of a sense of frustration so all their experience was that sony and sony is all about innovation so they saw an opportunity to innovate and make a mobile first racing control system so they made a paid game in smash cops and they're a little bit lucky with featuring and they made about a million dollars and in 2012 they saw the market evolving towards free to play so they integrated some light free to play mechanics into smash cops to create smash cops heat and they found it quite a humbling experience and realized they're only just dipping their toes into the water of free-to-play they made a little bit more money but they realized if you want to make the most of free-to-play you need to build something from the ground up with free to play knowledge and really invest in bringing free to play experience into the team and that's when we've built um smash bandits so i joined halfway through the

development of smash bandits and i'd worked at glue playfish and ea mobile on some really big successful free-to-play products but also some massive flops so i understood how brutal and fragile the free-to-free to play market could be but there was this emergent culture that you needed to innovate to succeed was a pillar at hutch as opposed to learning from successes at the app store and it was creating this reliance on featuring so we're always swimming upstream and we're building great games but we weren't building great businesses and every free-to-play game needs to be able to exist as a business and champions of gameplay was another pillar and champions of grain play is great and it's something we still do but you cannot do that at the cost of building a deep metagame and copying and referencing other games were almost dirty words and that was founded in the desire to innovate and that really affected the development of smash bandits and that's the first game i'm going to talk about in any detail

but this is called fantastic mistakes so i'm going to get straight into it smash bandits is a great game it's just not a business what we wanted to make was an endless runner casual accessible mass market with short game play sessions and a deep meta game with significant monetization potential durable um and consumable content what we actually made was a highly skilled core gameplay experience not casual or accessible at all with a light meta game that didn't create enough play enough reason for players to monetize and incredibly difficult to operate as a service we weren't lean at all if you look at the screenshots it's full of great gameplay it had a procedurally generated open world environment that went from cities to water levels to canyons which is insane when we look about at the ba look back at it it had no soft launch we used smash cop's performance as validation for it and we indulged the studio's desire to create and be creative and innovate ultimately we built the game for

ourselves and we broke all the rules um smash bandit there's nothing like smash bandits in the market it had a new gameplay system for free to play a new meta game system there was nothing in the market that players could reference they had no familiar experience and we didn't understand the model we were trying to use it wasn't accessible it wasn't mass casual and had really long session times so when we look at the revenue and the downloads you see the shark fin revenue charts which grow a little over time that shows there's a little bit of learning but every update was focused on innovation and skill based features there was very minimal meta game stuff we were really just skirting with free to play and the company's always been consistently able to get lots and lots of installs but not been able to monetize them that meant our ltv wasn't growing enough with the updates so our ltv to cpi ratio was never enough on smash bandits to be able to do meaningful ua and if you look at it we've got this console mentality of

these big updates so big releases so we weren't we weren't understanding our references and it was just feeding our desire to innovate and all of this could have been avoided if we'd understood what players really wanted and created value for them so it comes back to those core themes evolve alien culture smash bandits could have been validated in two months rather than waiting the nine months to release it and then we would have built a better game or a different game we didn't break one rule we broke all the rules and had crazy levels of innovation but we were starting to invest in our team we were transparent with what worked and what didn't work and the team were beginning to learn from these mistakes and you can't teach that seeing a great game that doesn't work as a business is an incredible experience for the team and you can't buy that experience and the team were beginning to understand what risks they could take so we moved on to make mmx racing mmx racing we made in nine months with

nine developers and it cost about a million dollars and today has 25 million downloads what we wanted to make was a drag racing game with monster trucks we felt there was an opportunity in the drag racing space we felt all the players were focused on the rev counter and the gear changing mechanics and we wanted to focus the player on the truck and the environment so we stayed really close to our reference points drag racing games are simple and accessible and we ensured that mmx racing was simple and accessible and magnus racing was a really successful title for us and we learned it a lot and we supported it for 18 months post launch it cost about a million dollars to make and to date has grossed about 10 million back so we stayed close to our references but all hutch will always innovate but that needs to be justified and you need to understand the risks we did make mistakes with it so we did a big multiplayer update it the whole game of mx racing took nine months multiplayer took eight months after launch to make and we guessed what we thought the core value was and built it and we didn't

validate with an audience and it was nowhere near as good as it could have been and ultimately with mmx racing supporting it for 18 months was too long and we kept the team on it for too long when you look at the downloads and revenue charts you'll see with this massive launch and this is just on ios we did four million downloads on ios alone but we didn't make the most of it because we didn't validate early we did multiple updates with wwe car crashing fusing systems chat systems and multiplayer but you can you can see there's no real significant up impact on the revenue post launch because we didn't validate with a live audience and you and you can see that on those charts the one big update that we did that had the biggest change was we changed the tutorial and that increased our ltv by 30 and that's what pushed us over the ltv to cpi ratio that allowed us to do user acquisition and it coined this term in the studio that featuring is for a week but ltv is for life and later on that led

for us to make top drives and i'll talk about that a little bit later but the big so because that biggest change was down to the tutorial we could have made that earlier if we'd been lean and we'd validate it early and we would have had a better product for it so we'd improved in some ways we'd understood our reference and we'd built out that core experience and we've done it without over-indulging our sense of need to be creative but once you accept that all your designs are an assumption you're really only guessing what players want and if you don't validate you're gambling and you're assuming you're right and you just continue down one path until you launch it and then you find out but what players actually want can be really quite far away from your initial assumption so by validating early with a live audience you get one step closer to understanding every time and improving that product and the earlier the better and there's more scope for improvement and it this all seems really obvious but not many studios do it or even believe it's possible and if you wait until your soft launch period to do that it's already too late

you can do this stuff before you commit to developing your game so you know you're developing the right game from the beginning so we wanted to solve all these mistakes going forward we'd released mmx racing in 2014 and the team was supporting mx racing until late 2015. so after that massive launch um and our ua understanding it approved we were quite confident um with our free-to-play strategy going forward we had the opportunity to work with this amazing external team the team behind csr classics and the product owner from csr racing now csr is a 200 million dollar franchise so that's great expertise and it was only 50 miles away in brighton and with this great idea drifting is massive gymkhana has huge views on social media and drifting has become more prevalent in popular culture thanks to the emergence of ken block and there are great examples of drifting games doing well in

asia with games like drift spirits so we saw this opportunity to make the player feel awesome with an accessible drifting mechanic so it felt like the stars had really aligned for race kings and we're all really excited and we were going to validate early in a way that we hadn't done before and we were going to do it before we'd made a commitment to the project and we're going to build a business as well of building a great game and we're going to invest in understanding references so race kings was meant to be 8 ball pool combined with racing rivals both huge titles both top 20 grossing 8 ball paul had this accessible multiplayer betting structure and racing rivals had this deep metagame with a player focus experience we were going to combine those to make a competitive synchronous multiplayer drifting game so we set out to make race kings with nine developers in a year at the cost of about a million dollars so very similar to mmx racing and our teams are an autonomous mixture of art design production qa engineering and analytics and that's a model that's

worked really well for us in the past what actually happened is it took an average of 12 developers three years and 3.8 million dollars to build race kings and it was only downloaded 6 million times despite having the best featuring we've ever had so the main banner on the ios store and android stores worldwide to put that into context our other titles have been downloaded 160 million times so it was our most expensive development to date and our worst performing title so what went wrong uh we were misaligned with our on our with our development with the external studio we wanted to empower them to make a great game as they'd done before and we knew they were capable of and we wanted them to benefit from the learnings we'd had on smash bandits and mmx racing we wanted to justify our investment in drifting by validating early but we'd underestimated how necessary the process we'd been through had been to our learning and it's in very difficult and sometimes impossible to teach that

the product lead had come from csr racing and he was new to hutch and we weren't fair to him we expected him to represent our culture and our values but he'd come from a 200 million dollar franchise and he wanted to make a 500 million dollar game we had come from a 10 million product and we were looking to take that next big step to be a 50 million product so his desire to make this massive game and innovating across the board so race kings was built as if success was a foregone conclusion and he was addressing the design decision uh design problems that a 200 million game would have and this created a real cultural issue for us because our culture is about empowering the team but they'd only worked on massive successes and they were prepared to innovate everywhere because they had so much confidence but they didn't understand how fragile free-to-play could be and we as a company were running before we could walk we were only a company of 30 people and we weren't geared up to run another studio

so we had this ambition of validating early but it's not always that easy the business value for race kings was set in this high visual benchmark licensed content and synchronous multiplayer and that takes time and time increases your risk and if that's the value how can you create an mvp so instead we did user tests focus groups player diaries and expert reviews all really useful but none are a replacement for early validation getting that quantitative data early and from a live player base before you've made that commitment to building it now a lot of studios have talked about validation about the mvp and they say the audience is too sophisticated now that the market has progressed and a lot of studios get bogged down on the m the minimum of mvp but at hatch we focus on the viable nature not the minimum so towards the end of development of race kings we took two people off the project for two months to make a prototype and you can see it on the right and that had no tutorial it had two tracks and five trucks taken

from mmx racing so those two people in two months but a game that had better attention better engagement longer sessions and more sessions with players progressing further in the game than the 12 developers did in three years on race kings it's because we started down this road on race kings and we didn't take those steps of validation to get closer to what players really wanted so while that was happening and uh um all happening externally in our london studio we were ready to move on to something new we split the mmx racing team in half and after a few game jams we started on top drives and mmx hill dash mmx hill dash we wanted to make hill climb racing with great social features so we had a team of five developers for five months and a 400 000 budget and today is made back 12 million so it's back to those themes again reduce risk with a small team and a short timeline and validate early break one rule so strong reference came from hill climb racing and we understood

what risks we wanted to take and where we wanted to innovate and invest in your team we were really transparent about the problems we're having on race kings so the team was always learning so we were very focused on the early validation so six weeks after um the team came together we had a prototype live on the google peter program so it had three cars and five tracks and what we thought when we launched was one week's worth of content but it had great metrics and it really and it justified continued development there were players playing every day for 30 to 40 days and from that we knew where to put the value in so we put the value in the some of the graphics to help with user acquisition we could improve the economy which improved our monetization and we had a better for tui and ux improvements which improved our attention so when you look at our revenue and downloads you'll see the ltv ltv to cpr ratio is the best we'd ever had to date it was scalable uh we could scale user acquisition for the first time at a major level in the company's history and we had this content treadmill that was really cheap

and we could keep on going for a long time and that's facilitated at making 12 million dollars the mistake we made is we didn't appreciate it at the time we thought it was going to be a mass casual game we weren't expecting to be able to do user acquisition hill climb racing was justified by having 500 million downloads and we only had about 20 million on hill uh hill mm-hmm and you can see there's a big drop off after launch from uh in the revenue and it was only picked up later so that was a missed opportunity for us by not understanding the model or what we had at the beginning at the same time we were making top drives so top drives was made at a slow pace over a long period of time we wanted to move away from those shark fin revenue graphs and we wanted to validate early now top drives the design called for it to be an always online game so we built the client first and we faked all the server and we put it live and we validated it and that prevented a wasted effort on an expensive server stack that we might never have needed

and then when you look at the top drives performance top drives performs unlike any of the rest of the pro the titles in our portfolio it has grown since launch with our greatest revenue days coming months after launch like hilldash we've been able to scale user acquisition but this is based off a different audience profile it's got a smaller audience with a much longer ltv with a low cpi which is generating 100 000 days and can be serviced by a team of four people so again it comes back to those core themes of lean validating early before you've made that commitment to your product managing your risk creating the value for players getting closer to what players really want and our team loves doing this they get to work on multiple products and see how players respond and they know when they're working on a title they know it's been justified because we've done that early validation and break one rule understand your references know where to innovate and what risks to take and where to be creative we have this term a hutch it's called

being hit driven and not dependent and we're trying to get that next 250 million hit but as we're always lean and agile and we're always validating early we can take lots of shots at goal without risking the company and we're focused on getting two games a year and every title is increasing our revenues and we're learning and stacking allowing us to be hit driven and not dependent and we've built that into our culture so our culture is player-centric everything the players at the center of everything we do we're data informed rather than data-driven and we're entirely transparent with our team we present everything back to them they get the board back poured pack back they see all revenue user acquisition and performance data and our growth and insight team present back to them on a weekly basis on any topic they want to hear about they're empowered they're autonomous and trusted to make the right decisions and our process is lean understand what compromises you have that what compromises you have to make and our business is mobile free to play

racing which provides this clear sense of focus on what we're making so we've made some fantastic mistakes but we're proud of where we got to and these core themes have been honed organically through our learnings and allowed us to avoid making those mistakes in the future there's a few more minutes for some questions if anyone has any uh do you want to come up to the mic okay when you when you mention about the the budget for all the production is this including also user acquisition and also can you share a little bit about the channels that you use for user acquisition as well no those budgets aren't inclusive user acquisition they're just development costs and content channels for user

acquisition uh depends what game we're talking about it's usually facebook we work a lot with some we have some good partnerships with some specific ad deals uh dealerships that we've made that are bespoke to those games as well okay hi um i'm just wondering if you could elaborate a little bit you talked about um it sounds like you you put out the games super early maybe even before they're done like what is what does the actual process of validation look like uh you know what kind of things are you looking at to determine whether or not something's worth doing i guess worth finishing uh that's right also shout out to your canadian studio i'm from the same town thank you very much um so the question was what's our process to go through for early validation so it's a combination of things we first we look for something that we think that we can make internally and that what the team can really get behind there's a lot of it's really easy to justify different types of genres of games but there's certain games i don't think that hutch would ever make very well i can't see us ever making uh taking on

clash of clans with that style of game so it has to be something that works for us as a team we build it internally and it has to um has to be something that grabs us and be fun very quickly within a couple of days of the game jam and then it has to be something that we believe we can validate early there are some games that can't be so once we've done that we basically just build a profile of what is the core experience of that game what is that core loop what are players really getting grabbed by we build that in a maximum of two months and then we use usually google's beta program or early access systems or we just soft launch it in specific territories that we know represent certain performance profiles and then we look at retention engagement session lengths um and that in relation to what we've done and because we've done this so many times now we've got like a we know we've got benchmarks of what we know we can get at that sort of level of development because obviously you can improve some of those metrics over time

can you elaborate a little further on how you how you act on that feedback you get from those early validations you talked a lot about listening to players and what they want and in the experiences i have that mostly amounts to removing friction from the experience and not necessarily figuring out how the game should change how do you try and and get that out of your users and make those decisions to make changes or improvements to the titles um it's a great question so we use a combination of qualitative and quantitative feedback so there's the metrics and all the stuff that you're looking at when the game's live and then there's the talking to players through the kind of the the google beta process is great for getting feedback but we also do a lot of focus groups user tests and stuff like that where we can physically watch people playing a lot of that is great for kind of understanding pretty small stuff quantitative feedback can give you an indication of where players are getting stuck or where they're dropping off but you don't really know exactly why so understanding that feedback it

depends a lot of it can be emotional like this is too hard and i want it to be easier but then when you watch their actual data it doesn't back that up like a lot of players if you look at communities you'll see there's quite vibrant communities and we have this on top drives where some people feel like this event's too hard i don't like it but then they're the they're the ones who are winning that event and they're pushing really hard and they're driving it and that's a real incentive for them so it's it's understanding what the true nature of your game should be what you expected from your audience and where it's differing and then trying to gauge whether it's an emotional response from them or of whether it's something that's tied into that commercial value that you can kind of move forward with does that make sense yeah i'm short uh so first of all thank you for having this uh as see ceo and talking about failure it's not often in in companies or teams culture to be able to say that so huge kudos to you thank you very much uh that speaks

volumes of your true culture and i guess my question to you is could you elaborate a little bit more of how do you see that playing out within your team and organization to really set your team up for success and celebrating wins as well as what failures look like so people are more approachable with it so it's not this dark cloud and people would just come out front admit it move on learn from it and how do you build that safety net within your org it's a great question um so it's a lot about making so there's no it's about creating a sense that there's no shame around failure the free-to-play market is incredibly brutal and the vast majority of games that go into it don't work and we've had to be incredibly honest with our team by accepting that we're all incredibly fallible and we all make a lot of mistakes so when we're sharing this data on a weekly basis with the team they're understanding what's working and what's not working so that they can be empowered to make those decisions but it comes from right from the top so

our ceo who's sitting there with the silver mane of hair like he makes tons of mistakes all the time and he won't mind me saying that um it's worrying actually how many mistakes he makes but he's he's incredibly open with what mistakes he makes and we're just very open with that and once you accept that and it goes back to the kind of validate early that every design is just an assumption you're just taking a guess and really i don't think there's any games designer or any studio really on the planet that can set out and just hit that mark of what should work and what shouldn't there are no developers here that there's no studios in the world that haven't had multiple failures if you look at supercell they kill a lot of their games you look at king they've not been able to follow up with a better game than candy crush and that's an eight-year-old game once you accept that and you talk about failure and you just make sure that it's it's okay to be a part of that and it's almost celebrated in some ways because you're learning and celebrate that learning then you get a truly quite vibrant

culture around it and the team can really start learning what risks are worth taking and where they need to innovate thank you thanks any other questions brilliant all right well thank you very much