Better together – How to build a mobile game with help from your community

[Music] I should point out ahead of time that this is a taught that I could probably talk about for about two hours so this is going to be very much a whistle-stop tour through it so if there is anything that doesn't make sense then please come and grab me afterwards so yes so the topic of this is better together and how to build a mobile game with help from your community so really what this 50 minute segment is going to be is kind of making the case for why you should do that and involving a community early in development and we'll touch on kind of four major areas why we decided to do it how we put our planning to action what we found out there as a result and when to use and when not to use this so first of all I just want to caveat that early here we're not talking early access or

beta those kind of terms are now used for kind of pre launches and that process is already really well defined and there are lots of talks about that we're actually talking super early in your development process and when I'm talking about community for us that's really a well-defined group of players that kind of exist with an existing brand that can be your brand or it can be someone else's brand but really what we mean is any kind of group players that can be identified by kind of core likes or something that they share so first of all like who are CCP so we are an Icelandic game company we are most recognized for our sandbox MMO couldíve online which has been running for about 15 years we've recently been acquired by pearl abyss which is a Korean MMO development studio which creates black desert online and I'm here because we established a London office in 2016 and the focus of that London office is primarily to look at mobile development and that kind of task is to bring kind of the e brand to mobile so why are we

kind of why we talking about engaging with the community so early so I just want to kind of be upfront that there was no kind of psychic vision or clairvoyance involved in this we we took a punt on it and it turned out good for us we didn't go into this expecting how good the results would be so the major two areas why we wanted to go into it was community buy-in first of all so EVE Online has a huge community already and we wanted to make sure that when we're moving into a kind of a new product line that we're building trust around that expansion when it comes to mobile there's a lot of speculation around business model and the impact the impact on the MMO so we wanted to make sure that we were kind of coming out ahead of all of that and having the conversations with our community upfront and also our community is also now what we consider our brand experts they've been playing this game for 15 plus years most of us who work at the company haven't been there that long so when we when we're doing something new we like to go in there and gauge their input and secondly to that we want a kind of trial innovative features so we're kind of really looking here for a qualitative validation of the whole kind of approach that we're taking to this

game we were kind of creating a game with a with a social strategy kind of feel to it but we wanted to go really heavy in on that social and so we wanted to kind of identify how you know how far we could push the social elements right from the very first session with the game and if that was too high potentially for a mobile audience so I think it's also important to talk about things that we weren't looking for so at this point in an early play test we weren't looking for KPI measurements at this point in the game the the systems are just too underdeveloped to be able to take proper KPIs out of it and also the community that we are targeting is not representative of the full kind of mobile marketplace so soft-launched really remains the best time to gather this information we also weren't looking to kind of have an overall green light as a result of this test the breadth of the information that we were getting back from our community wasn't going to be wide enough for us to make the call whether we should go forwards or not go forward to this project and really enough development time still remained that if we did get bad sentiment back we could actually turn the project around or do something a bit different with it and last of all we weren't looking for

kind of feedback on balance bugs part and polish again development at this early stage wasn't focusing on these these particular areas of the game however we did know that this was going to be a challenge not to get our community to focus on this because generally players are most comfortable providing feedback on these particular areas so the kind of key takeaway there is it's really important when you're doing these tests to kind of manage expectations with all of the stakeholders both internally and externally so kind of moving on to how we actually did it so the first thing that we did was to kind of identify our audience we had an idea of who your game is for is kind of super important at this stage and it's ideal if you can back it up with market research but if that's not possible and we know that those resources aren't available to everyone then it's possible to to kind of evaluate existing audiences out there and say okay well our community is a bit like the community that exists in this game it's also important to be able to local to locate the community hubs and

understand where your audience gathers well we actually used a Vegas which is our fan gathering that happens in Vegas every year so he's the physical kind of gathering location but a virtual one would work just as well and we made sure that up front that we were communicating our expectations really well we kind of spoke to our audience and tested the water a little bit before we went ahead and did this and made sure that there were enough people who are generally interested in partaking with it and then the last thing I think is important is that with any kind of play test early on it's important to be respectful of people's time you're asking them for feedback on on a game there's very unfinished and ready unpolished so it's it's kind of easy to slip into place where your flippant trying to to kind of say well what do you know you're only a player you're not a game maker but really you have to take that step back and understand it they're there to help you make this thing better in terms of preparation we made sure that we were collecting emails super early so having an understanding of the volume and how many players we're going to be in the play test really helped us to to kind of understand what our limitations would be

we had that contact list then that we could then send out updates on the play tests and we had it for an emergency as well in case things went wrong but as with all gathering a personal information it's important at this point to remember that since the end of last year gdpr is a big consideration and you need to be able to understand what you can what you can't do with that data so knowing the limitations of what you and your development team can do is really important we plan to invite the players into waves in the game because if you invite everyone together there's the potential then for a crash if there's overwhelming interest in the game and having that bad initial experience will really hurt the engagement with it and put a bias and a spin on everything that that you're gathering after that point and if at all possible have playtesters be incentivized to to kind of take placed in your play test if you create an event with your game has kind of an achievable goal that really works it can be cooperative goal or it can be a competitive role we were lucky enough to be partnered with Samsung at the time so that they provided us a few phones that

we could give away as tangible prizes and so that was a really good kind of way of getting people's interest piqued right from the beginning with this protest for platform distribution we we made the choice to use peat everyone's personal devices this wasn't a kind of a light choice to make but we decided that by letting people use their own phones we would maximize the engagement across the people who are at the event but we did do that with eyes wide open knowing that people's experiences would vary depending on their phone but we did want to players to be able to play the game while you know having their own phones and using it as they would be using a normal phone we didn't think that would get a realistic piece of feedback if we were inviting players just to play it for 20 minutes on a bank of phones with other players we used both iOS and Android again because we wanted to kind of reach as many people as possible and it kind of it really adds up to making your project feel inclusive right from the start if you tell half of your player base that you can't partake in a player base very early on then you stand up setting half of that that user that user

base and they might then kind of react negatively if future play tests and you know if you're using a Unity engine or if you're using a Unreal Engine this is actually a lot of the heavy lifting for this has been done for you so it isn't too much of a stripper both both issues working at the same time and so in terms of distribution we use TestFlight in google play beta there was you know those systems now are quite developed and they they do a lot of the hard work in managing to get the build out to players if you've got their email addresses but the initial set up can still be pretty complex so I definitely recommend practicing kind of internally distributing your build through these systems before you go ahead and do the actual play test in terms of collecting feedback we set up social channels we had in-game chat but that's kind of too momentary so what we decided to do is that we use discord a lot of people are familiar with that now so it's a natural place of people to go and share their feelings about a game and also means that the moment-to-moment reactions are recorded and we can go back after the play test and kind of study exactly what

people said as they were getting to grips with the game we also created a feedback survey using tight form and that was kind of asking sentiments about the game itself but we also asked a lot of questions about the player because it always helps to have context about the kind of players that you're talking about so when you're reading why they like or they didn't like something you can tell if they're part of your audience or what segments of your audience you need to change to make those people happy we also distributed that towards the end of the event just before it finished so that way the game is very it's on people's minds still but they're not kind of making early judgments to it before they've had a chance to get to grips with the game so I think it's important just to mention at this point that feelings all the data that we're looking at from the feedback the feelings kind of outperform hard data at this point we're looking for emotional responses and even though some general kind of behavior or data can be useful like session ants really or engagement with specific features what's most important is that you're getting that general sentiment and that feedback towards your game as a whole I think in

terms of the build that you're providing I think having a coherent experience is a bare minimum for anything that you're putting out there and it's an intuitive core loop should be present if your game is just bits and pieces then players aren't going to get it without a lot of hand-holding and documentation and that's not what you want to do for a play test should also be trying to minimize high profile bugs and crashes and also areas where you're just not going to have time to finish creating builds I recommend using wireframes better than low quality assets and the reason for this is that players obviously won't know themselves what is there because it's finished or what is they have just because it's not good enough and you want players to know where you haven't finished something or when extra polish is coming so wireframes do a much better job than that than doing kind of a low quality version I'd also highly recommend a game that has a kind of a social toolkit in there so we had kind of chat in there and a push notification system and so that allows our developers to actually be in there in the chat and talk to the players and provide some very early first-time user experience to players without actually having to develop in the game which then kind of increases

the speed at which you can bring this play test to the community and also what we found is the players actually assist each other with issues and that where there are problems or there are things that aren't easily understandable players actually then kind of group up with each other and that kind of built a really nice virtuous circle within the game in the community itself I also highly recommend building an emergency response plan this is an early play test after all that will go wrong we we did actually investigate having kind of out of our support for it because we we were running the play test in Las Vegas and the development team was in Europe and we kind of had planned for a graceful failure strategy so in the event that something catastrophic went wrong and we couldn't continue the play test the build we had a switch that we could flip that basically presented a message that said you know we're sorry the build has gone down at the moment we thank you for your input and then push them towards the feedback areas this this picture here is actually at the top of one of the ferris wheels in Las Vegas and I put it in here because the

emergency response plan we actually had to put into action while we were all on the top of this so there was like six of us with our laptops out not looking out the window at all trying to fix this play test build so that's kind of how we actually did it running like on - what did what we didn't expect and that we actually got out of the play test was we were really amazed at the kind of the actionable data that we that we generated so we got crazy amount of edge case QA examples players are being players and the EVE Online audience being the EVE Online audience they discovered crazy fantastical issues that you could do this picture here is how everyone who is playing it decided that all spam won one place exactly the same time it put an insane load on the servers and actually brought out loads of systematic flaws with our engineering approach which we hadn't imagined before which then resulted in a far more robust architecture going forward as we develop the game it also gave us a sneak peek at scaling so as I said we had way more concurrent players than expected and that created some performance issues and minor down times but what we did get out

of that was learnings on how to spin up for the kind of cloud services in real time and also we got performance data on real devices as we said we used everyone's real phones and so the breadth of data that we got was much wider than we expected and we saw that in some cases we had a kind of significant battery drain and overheating issue which then we kind of put into our schedule to make sure that we were focusing on performance we actually added a system in there where the game would automatically dial down the graphic settings on kind of lower devices one of the really interesting things that we had back was that player came to us and during the test and he said he think thought thought hid after the test had happened he thought I'd solved one of our performance issues and we've are great he's a coder he's going to provided some really interesting insight into our code but actually what it did is just mounted the phone onto a heatsink and solve the problem that way which it's not a particularly scalable way to solve our over power issues the other thing that we that we got which didn't expect this kind of enthusiasm at a large scale we got massive buy-in from hardcore PC

players we didn't expect that we got even lines kind of hardcore player base to ask if they could import their entire corporations in one go we had some real-life confrontations over the loss in the game and they were all pretty good natured with kind of banter and teasing kind of happening at the event itself but also we found out the players were kind of meeting up for better coordination of how to play with each other we got a mass request to extend the play test so it was just three days to begin with who ended up running it for about nine months and we got had kind of internally we had a really huge level of internal development support so without us asking the entire team stayed at for 48 hours to support the build and that kind of created a really good camaraderie between the team and also it gave the whole wider team a much better understanding of the players that they were aiming this game for so why did we keep the game running for nine months so the benefits were still massively significant we many players were really engaged with the game and investing investing a lot of their time into it we had a positive feedback cycle between

the developers and the players on discord they were talking each other on a daily basis about what they liked what they didn't like where their problems were and that stream of QA data on client performance and service stability was still in there so again really useful data as we carried on managing it was also should say that it was also very manageable to support this game we kind of we had updates the play tests every milestone internally so that was once every three months we updated the game and spending time on pre-production on technical debt made this possible because it made that really that there weren't issues that we had to be fixing at a regular regular cadence and also we maintained a population of several hundred we didn't actually go above that at this point because we thought as soon as we start having thousands of players in here that's going to negatively impact how we do it and one thing that we did do is we regularly had status feedback sessions on this to make sure that everyone was still aligned knowns time was being drained on working on the play tests and we all understood that if at any point in time we felt that the play test was negatively impacting our ability to develop a quality game then we would stop the play test so kind

of a I've talked about a lot of these advantages in in kind of separate areas but really the main advantage is that these these are all kind of indicators of the more holistic benefit of having the early play test which is uncovering your golden cohort so knowing exactly who your golden cohort is and by that I mean that the players that are your super enthusiasts the one that are going to you know put hours and hours in every day who are going to talk about it who are going to be the real champions for your game once you know that it allows you to kind of accelerate your marketing reach when you actually go live with the game it it promotes a organic virility by having them talk about it and have a social media footprint it also reduces the time that you get to social critical mass which means that if your game is social based and you need a certain amount of players for the whole loops to work if you have your golden cohort in there from the start and you can get to that point much quicker it also allows that the fooled player development cycle to be in place when you as soon as you hit soft launch if the play test is still running or you've got a group of people who know how to play the game from day one and your soft launch you're gonna have a group of experts in that

game who are going to provide aspirational goals for your new players coming into it it makes the game world feel alive if you've got a kind of a mobile game with an MMO flavor and it also kind of breeds your social leaders the people who are going to start your guilds and your clans and going to be inviting people and educating people what it also did was provide as mentors we actually had players coming to us if they could ask to be part of a mentoring system and what that was really good for was plugging gaps in the first-time user experience where we didn't have time yet to implement the full thing which then goes on and reduces the churn out of your your game in soft-launched due to potential confusion areas and the mentoring itself gives those players who are your golden cohorts a sense a real sense of feeling of belonging in your game which will then keep them around longer as well so I just wanted to we talked about how to run an early play test I just want to talk very briefly on how to ruin an early play test so this these are kind of four areas where you should not take this process it kind of under consideration if you're not

willing to do these four things so first one if you don't know who your target audience is you're really going to struggle with this this is not a system for finding who your target audience is you need a reasonably good idea of where they are to begin with if you don't know that then the feedback and the data that you're getting back from this system might not be valid you might not actually be telling yourself the answers to the questions you want so number two if you're not prepared to act on the feedback you know there is still kind of healthy skepticism in some areas or unhealthy skepticism I should say that none developers can provide meaningful feedback to gang development and also that you know some people with their game are just fully committed to following through a vision that they've got in their head through to launch if that's you and you're not prepared to be willing to act on the feedback you're getting from this play test then perhaps this isn't a system that you should be using likewise if you don't have the resources to act upon the feedback if you've got hard time constraints on your project or your team is small enough that there's just no bandwidth to support it you should really think about doing this because if it's going to have a heavy impact on your development then you could be kind of shooting yourself

in the foot before getting any value out of it and then I touched on this last one a little bit but if you're expecting this to provide full validation for your game this isn't this is what's going to happen you're going to get some qualitative information back that will help steer you in the right direction but you're not going to get a green light you're not going to get KPI so it's important if there's people expecting that from you that you kind of don't go into this with your eyes kind of closed and realize that you're not going to get that information back from it so kind of just providing a recap of everything so in summary like what I'm really trying to get to is that if you engage your community early in development you can you can get some benefits in some real areas so number one you can get you can make smart informed decisions you can provide your diet designers with qualitative feedback in time to make meaningful changes to your game you can verify that you understand your audience and that you're targeting the right people you can better prepare your team for the realities of running a live game essentially while you're doing this process you'll be doing development and running a live game at the same time now

even though you're doing it at a much smaller scale that process gives you a lot of insight into into how you're going to split your team and when issues happen on one side or the other how that you you resolve those and where you where you funnel your users it also improves the kind of the awareness that the team has of the audience they're making for and usually there is something that a PM has a strong kind of understanding of but we really found a benefit when the whole team fully understands the audience in the game that they're making and it also provides a lot of familiarity on the release management process which can be complex to set up and again it's much better to kind of go through those those kind of pains when you're in a much smaller audience and the last that's one of all is being able to jumpstart your games community so as I mentioned it breeds your your mentors and your super enthusiasts that are deeply invested in your game and also helps to kind of minimize any negative sentiment with a lack of product information so there's especially important for companies who are moving into the mobile area from kind of PC or console sometimes players will have a

con preconceptions about what mobile is and the business models and free to play or pay to win and all those things you can get them out of the way quite early with with a game play test like this and again it will support and multiply the impact of any marketing activities that you you engage in from that point onwards as I said super super fast Whistlestop tour of this I could talk for this for ages so if there's any other questions about it in detail then please let me know I'm kind of running slightly over here so I'm not sure if we've got time for questions or yep yep exactly and that's that was kind

of one of our bigger worries kind of coming out of that and we've got all of this good information and all of this optimistic feel about the game however that kind of question was always in the back of our heads as well is this just you know how big is this girl growing in cohort and is this just a very small number of people who are very invested in kind of EVE Online who are getting into this so through out the front we took this information and then we kind of did a bit of market research of our own and we we kind of spread that out and we used expert reviews from other people in the industry as well to kind of try and back that up so it's always a hard thing to do and to be honest you're never really going to get full clarity on that until you soft launch the game and you get that really wide scale of imported players but there are kind of some mitigation steps that you can do and pretty much any any other data point that you can get whether it is experts or other companies that have done the same thing or whether it's widening your bracket of the play test a little bit will help you kind of be more certain every time you do that

[Music]