You Need to Know who is Going to Play Your Game - Teppo Soininen, Metacore Games

[Music] [Applause] [Music] thank you [Music] I'm now going to welcome my first Speaker stage from metacore please please come up and and take me remove me from this get out of here take stage off get out of here I'm gone [Applause] okay hi everyone my name is I'm a game leader at metacore games uh we do mobile games our biggest hit game at the moment is uh more emerge mansion and I'm here to talk a little bit about how we do concept testing at metacore games so let's get started I wonder how this

works press it maybe does it work no can I click here no yeah cool so why do we do concept testing the idea of concept testing is basically to try and predict if a game has potential so game development is quite costly it costs time and money and time is probably the usually the more scarce resource because you can do only so much in so short of a Time All of Us game developers have a bunch of cool ideas that we'd want to try out we think that this is going to be awesome this might be awesome and that might be awesome there's only so much time I think if you think about your whole career you can probably make tens of games instead of hundreds of games during your career so you got to pick your battles in a way I have way more ideas than I have time to develop so

we want to try and predict if a game would be successful so here's a rough idea of how a mobile game could be developed you first develop a game concept and then you've created a playable prototype you soft launch it you get some retention metrics marketing metrics and put the game live and then you get a bunch of money usually at this stage you do a soft launch and usually around that stage you will see if the game is successful or not you'll start getting some proxy metrics like retention that should is considered to be a really good good kind of predictor of financial success for a game but the problem over here is that it can take you months it can take quite a bit of time for you to get a game ready for soft launch you need to create a concept you need to create a bunch of art you need to program the whole thing you need to design it that sort of stuff usually if you soft launch a game in two or

three months it's considered super fast sometimes it might take a year or even multiple years so this is the reason why we try to predict success already at an earlier stage these are not foolproof methods or anything like that but it's still something that should help you at least avoid the biggest pitfalls so at metacore we try to do this already during pre-production and the key to making successful games is of course trying to figure out what the audience would like and what they wouldn't like so again this um you can see this pipeline picture over here we've changed a little bit to kind of fool around with the terminology and that sort of stuff but at Medical we start thinking about the game Concept in the earlier stage kind of more widely with the fantasy of the game where in the entertainment the industry and uh a

big part of selling our products is to sell some sort of fantasies to the players so the fantasy in our case me is basically he tries to Encompass the type of promise that we as game developers give to the players this is the type of experience that you will be having this is the type of entertainment that you will be enjoying when you start playing this game this is how we form the fantasy you can think about it as some sort of a trailer for your movie or I don't know like the description in a can bag of candy it's like it's gonna be really sweet and sour and whatever that sort of stuff that's a promise for the person buying that entertainment product what what they will be experiencing when they actually start enjoying this thing so we start already testing the game fundamentals in the early stage we try to figure out whether the game has potential already here instead of the soft launch phase

this is mainly because it is at an earlier stage we haven't invested that much in the game and if we find some fundamental issues with the game we can cut our losses earlier of course we have to accept the fact that at an earlier stage we are going to be more prone to error this is not going to be super accurate or anything like that the accuracy increases as we move forward so in the earlier stage when we're testing Concepts the accuracy is going to be low we're going to be basically able to root out super bad ideas and we're going to be able to maybe detect some ideas that are better than others but we have a huge margin for error over there we're gonna have to deal with a lot of uncertainty at this point we move forward to the next stage when we start doing a playable prototype we get more accuracy our tools are better since we have an

interactive product at that point it'll get more accurate but even at that point we're not 100 sure we go to soft launch and everybody who's soft launched the game will know that there's still plenty of room for ups at this stage also so you can basically make a game that has awesome retention doesn't monetize at all it's not really workable at that point necessarily or you can have bad retention and kind of good monetization metrics or whatever but it is still possible to screw up at this point but anyways the point being that we increase our certainty confidence as we move forward in the pipeline but we try to do it at every single stage so what does this mean in practice what are practical tools that we use for this the key as mentioned is to try and understand what the audience likes and doesn't like and something that we've been thinking about is kind of

maybe if you're trying to pitch something to your friend or family or whatever for example let's watch a movie this afternoon I'll go to my daughter and Pitch her a movie like hey daughter let's go watch a movie and then she's like okay what kind of movie do you want to watch us like there's a lot of the Rings thing it's really awesome man it's like got Wizards and swords and that sort of stuff and she's zoning harness oh God ball ring that sort of stuff that already that small pitch already tells me that she is not interested in this thing then I pitch another thing well let's watch something else that's there's like a Marvel thing where there's superheroes and they're flying around and shooting a bunch of stuff it's like yeah I like superheroes let's let's do that okay now this really really simple pitch already has taught me that my daughter prefers superheroes over fantasy stuff and that is something that we try to try to utilize a little bit so

we try to figure out what the fantasy would be what kind of fantasy would be uh roughly suitable for the players so what is player fantasy in in gaming context It's a combination of a lot of things it's basically the somehow tries to Encompass the experience the players will have once they start playing the game it's a combination of game mechanics it's it encompasses art it has the premise and all that stuff it's basically what the players can expect when they start playing playing your game and we try to make sure that these players we have an audience for the type of fantasy that we're selling so we're not trying to sell a Mech game for elderly people or whatever that sort of stuff some things are just complete mismatches some things you'd know intuitively that my girlfriend is not a fan of pirates and that sort of stuff I wouldn't try to sell a pirate game for

her and that's kind of I know it because I know her relatively well but some things are not as intuitive as you would imagine sometimes you find find out things about your audience that are unexpected or actually quite often you find out things here's an example so here's two games that you could describe as zombie shooter games kind of grimy dark zombie shooter games but when you look at those images it's really obvious to everybody that these games offer a vastly different fantasy from each other you can see that both are kind of dark you can see the gloomy colors colors over there and both of them are have zombies well Last of Us you need to know that they're zombies but anyways it has zombies I can assure you and then both of them have Shooters they have guns in the images but when you look at minigorous image it's way more wacky it's more playful and that sort of stuff and it tells you something about the type of experience this game

will give will offer you and look at Last of Us it's clearly more dark and more grimy and that sort of stuff it clearly offers you a more cinematic a more serious type of an approach to the same type of a thing even though they're both zombie shooter games so this is basically roughly you can see that there's a different type of a fantasy these games offer so how do you go about it let's pick a starting point for us a good starting point is usually the premise some sort of description short description about what the game would be what what is the type of fantasy that this game would be offering in just text format and why text format we chose to start writing text because it's really easy and quick to make it is super quick a person like me alone can write two or three premises a day text premises a day and then test this really quickly

the next stage if you go into visual testing you start creating visual images art assets maybe gameplay images or let alone videos that will take you way more than a day to make a visual representation of a game maybe AI will help us in the future but at the moment it seems to be still pretty hard to describe some sort of a visual image to an AI and make it look exactly how you like in a day or so so we'll start with test a text only testing all right a bunch of premises text premises some a few sentences describing what the fantasy of the game would be I pushed them to an audience using play test Cloud you could use any other service latest cloud is just something that we're familiar with it might be super skewed there might be issues with it but it is still a service that gives you answers SurveyMonkey whatever use anything and then we ask audiences to rate them

once the audience has rated these Tech surveys these text premises I mean we pick the best ones and we discard the worst ones usually in this stage we ask players to give some sort of feedback why did you like this thing why didn't you like listening that sort of stuff that'll start giving us already a pretty good understanding of what the player base enjoys and what they don't like we keep the best ones and we discard the worst ones then we repeat this process we can as you can see here the best one is still included I write a bunch of new premises and I repeat this process I repeat this a few times and all the time I'm getting feedback from players I like this I don't like that I like this I don't like that repeat it for example 20 times each time I write five different premises I do another round I repeat this process and eventually we start finding out themes

that do work for example in in the case that I previously did I found out that the target audience is really enjoying Mysteries they like realistic stuff stuff needs to make sense historical stuff seems to be cool altruism helping others not being selfish seems to be a big thing for them and then relationships Supernatural stuff comedy and apparently dead bodies are not relaxing also that was a surprise but it might be that I'm just bad at writing comedy or love relationship related stuff that sort of thing but then again if I'm making the game that's something that I need to take into account also so these types of learnings you can have when you're doing it this can guide you moving forward as you're developing your game so now we somehow created with after multiple rounds of testing we are capable of figuring out what kind of a

premise the game could have what at least some sort of a scope of these are the types of premises that could work for this audience that's important then you can go on and test for example visual style I don't know exactly how this could be done effectively this is super costly because going from text only testing a single iteration can take you about a day or two days or whatever we're talking about a few days but when you go to visual testing we're talking probably about weeks for one single visual concept for a game so anyways is something that we haven't really cracked we don't have a silver bullet for this it seems to be at the moment that visual testing just requires a lot of work we're looking into AI Solutions but that's something that we haven't really figured out yet but b as it may once the game concept once you've tested all the aspects of your game concept the visual style the premise some sorts of game mechanical uh gimmicks if you want to try them out and so forth then you have

a playable prototype and you start testing that playable prototype and the um we've extended kind of regularly you'd use latest cloud or similar services that the playable prototype just for ux testing and that sort of stuff what we try to do in this enthusiasm stage already is to try and see if there's some sort of wow factor for the player in a way um oops some sort of wow factor for the player over there basically the players if you pay the players to play for 10 minutes and they end up playing 10 minutes and they complain all the time is your game is probably not it doesn't have that type of a pool that you'd want to even though the ux might be perfect they might just like play it exactly how it's supposed to be played but the ux mine uh the the enthusiasm is not there that'll tell you already something and in an opposite example if you pay them to play for 10 minutes and they play for

50 minutes then you probably have something in your hands something that's quite exceptional if they're just willingly extending their Play Time quite a bit but that's a story for a different different time it'll take some time but anyways once you've got to the stage where you have your game fantasy wrapped up you have a bunch of premises for example that would potentially work you should already know at this time that the premise whichever premise you choose out of these ones should not be a complete mismatch for the audience that you're developing your game for and yeah as mentioned testing the enthusiasm is the next step and as I described you could already start thinking about these kind of things on how to how to gauge for enthusiasm players who play longer than you pay them for they're usually enthusiastic when you listen to players play the game that's quite of a quite a good marker for enthusiasm if players are constantly

having these moments of like wow yeah this is cool that's awesome and that sort of stuff that's usually a good marker for enthusiasm if your players are constantly complaining it's like I don't know like there's like ah this is totally not for me and that sort of stuff that's probably a bad sign you should probably do something about your game at that point so to recap what we've been talking about the idea of this method that we're using is to build confidence step by step there's a lot of inaccuracy and there's a lot of uncertainty in the beginning I get I have lines of people outside of my room at the office telling me that yeah this is not statistically significant that this is not that and this is not that and that's not the point it's not trying to be an exact science it's trying to give us leads he's trying to kind of point us to the right direction and avoid obvious pitfalls like trying to offer sci-fi games to my mom wouldn't work at all and these types of pitfalls we try to avoid and uh

doing multiple rounds of testing I think is key in this type of methods because a single round you do I don't know 30 40 100 answers of whatever your survey might have it's not really going to be super important the numbers but once you do it multiple times you do 20 rounds you do 50 rounds or something like that and you see a pattern repeating you see certain Concepts that are constantly being getting good ratings you start building up this confidence As you move forward and it's not an exact science you don't can't expect dark answers you can't expect these kind of definitive answers that this is definitely going to be a hit you should invest all your money into this thing there's never going to be anything like that in your hands but you'll get more confidence that's the whole point and statistically significant data I have mathematicians telling me that there's no statistically significant but it's still better than nothing so zero people playing your game is

probably the worst case 10 is way better than zero 100 is better than 10 and so forth and we haven't as mentioned this is pretty experimental for us at the moment we haven't ironed out all the details we do believe that the direction we've chosen is the correct way but there's a lot of kind of uh uncertainties in how we should write text premises as I mentioned how we should test for visual concepts and that sort of stuff these have not been ironed out we're working we're trying to kind of we're making this up as we go but this this far it seems to be producing pretty good results for us so that's it thank you [Applause] thanks to me not him we're must be right does anybody have one quick question before we move on there is one we you have to ask the rest out there do we have a a mic we can just give this this

chap if you have a roving mic no yes okay I'll repeat your question how do you define your target audiences so I think that is something that we just start with we Define what kind of a game do we want to make for example uh let's say King does a new Candy Crush game and they want to make a follow-up for Candy Crush that would be something that they pretty much already know what kind of an audience Candy Crush has and that sort of stuff or supercell does a not new Clash game or something like that so that's where we usually start we try to stake our claim at some point we Define our we already know what audience we're going after and that's key because we need to focus these tests on some sort of an audience you could potentially maybe go the other way around also in a way that you just make a concept and you try to see what kind of audience this resonates to I haven't tried it out myself but I

would might I think it might be a valid also valid approach to the same methodology cool thank you so much I'm sure there's many more questions but you can grab Supple outside thank you so much Tempo thank you great start thank you