Let’s Work Together! How to Improve Your Game With Your Players. Cooking Diary Case Study. PGCD #2

our next speaker is uh diane from my toner uh and diane is going to talk about how to improve your game with your players and using cooking diary as the case study so diane are you there hello how are you doing i will hand over to you thanks can you hear me well just to check i can hear you perfectly okay great so hi everyone my name is diana and today i would like to share with you one of interesting case on how we improved our game together with our players in cooking diary but first let me introduce myself my name is diana corkina and i'm

business development manager at maitona i've joined maytona in 2017 and if you have any questions or you want to chat or anything you can contact me the email is here so this is a short plan of what i'm going to talk today at first i will introduce my tona and cooking diary so you will have a little bit of context and then dive into details on ways on how to improve your game together with players through different tools available such as analysis playlists questionnaires and reviews and show some practical examples of how we did it in game maitana is a developer and publisher of mobile games and one of the largest game development companies in russia and cis now we have more than 900 professionals working all around the world our office

is located in saint petersburg ivanova yakuza develops singapore and in auckland new zealand hey guys now quick question do you have are you are you trying to share your slides because they're not currently showing oh wait if you look at the green button at the bottom um you can you can do a share screen sure can you see my screen i can now perform okay cool sorry and yeah and with covet 19 we moved to 100 uh work from home but before the pandemic we also had a distributed team working from different countries such as brazil ukraine and kazakhstan so it was very easy for us to move to remote work our you could scoff is actually moved in a day but this is a whole different topic here you can see how the company

developed through time the company was founded in 2012 by twin brothers avanasi and alexey schnitzky in yakutsk this is where i'm actually located now and this is one of the coldest inhabited place on earth just a fun fact we started to create games back in 2000s at first it was pc games and in 2012 we moved to mobile games the first game that we've developed and published ourselves is seekers notes the secret seekers notes is a game in hidden object genre and it helped us grow open up new projects and offices currently we have three games on global and two games on soft launch in our portfolio there are seekersnotes a game that for five years already is entertaining people around the world cooking diary the game i'm going to talk about today hill another game in the hidden object

genre but with a mix of match 3 brittle side a mastery game and a detective setting and our first mid-core game monostorm arena of legends so let me briefly talk about cooking diary in the game players become chiefs who have to serve their clients as fast as possible in game we have a lot of different restaurants with different styles and cuisines players can fully customize their restaurants and characters by the way that's me i think i look quite familiar players enjoy the game because of its vivid and colorful art style and high socialization the game is popular all around the world and in deep blue you can see our key markets from where the most players are this is united states united kingdom

germany japan france south korea canada and china so the game is popular in the west and in the east also the game was featured several times on app store and google play and we've also received the game of the day in more than 200 countries but one of the biggest achievements that for me personally that i believe is like outstanding for the game is uh winning imga and webbie awards in people's choice and people's voice nominations this means that not only the honorable jury selected our game but it was our players who love and enjoy the game who voted for us to win this award and how we did it you can see from the chart the top grossing chart that we gradually climbed it there wasn't a moment when we had a rapid spike up or down and we hope that this sustained growth

won't stop and with our players we will steadily continue to grow we have one rule that drive us that is always there is always some room for improvement and we never stop developing and improving our game but how to make players like us and what we do more i think this is a question that haunts us all and the answer to this question is a data driven decision how to get this data to know what to improve in game is another question and the answer is through analysis play tests reviews and questionnaires i've highlighted these keyways that you've tried in our games but you of course can find a lot of other ways that can suit you as well

so first one is analysis i won't talk in great detail about analysis as i'm sure it is obvious and the most common practice and my colleagues on this tracks will talk about it in greater detail what i wanted to highlight is that analytics team in a game development companies has two tasks first is to find a weak spot in game and to find a way to deal with it they are like detectives who investigate the case and try to eliminate it to make game better with the help of data let's move to play tests there are several kinds of them first one is uh internal play testing when you can test a game with your colleagues friends or family external player testing when you invite people to test the game

here you can select your target audience and outsource play testing here i decided not to show examples not to advertise companies here but you can google it yourself these companies can help you find your profiling audience in a different countries whereas with external play testing you're only limit you you're limited geographically there are many outsourcing companies who provide an outstanding service and with internal plague tests it's very easy to run and ask your colleagues to play test the game but because all of us are working in in game development and we play a lot of games ourselves our behavior can differ a lot from our target audience and our feedback can also be biased that's why of course it's better to run an external play test and that's how we did for our cooking diary we invited people to play the game before it even soft launched

so the player playtesters played one of the first builds of cooking diary and we selected all sorts of people they they were different by age gender profession background and the main focus for our play test was the first session the top of all finals and we wanted to know how the introduction of the game begins uh the better the first session the better the whole lifetime of the player will be we closely looked at the tutorials uh if you're going to playtest it here you need to know that many people nowadays play games and a long and very detailed tutorial can bore players whereas a very light tutorial can also be harmful as they won't understand on how some features work and churn and

so balancing the tutorial is crucial gameplay uh so it's very common you just want to know to what extent player understands the goals of the game did they understand how they play the game do they agree with the rules of the game do they like the games generally and like the dynamics of the game yx ui of course is very important um here on the photo you can see how players position their hands closing some parts of the screen and this is actually not really good because it might prevent them from seeing some user interface elements some gui and it's important to keep in mind on where the buttons are located and how comfortable they are to use and of course balance at first your balance is based on mathematical calculations on hypotheses

but during play tests you can actually run your balance life for the first time you will see the first live data whether your balance is engaging and understanding and of course the most important thing that you will receive after playtest is hypothesis the theories by uh understanding on how players engage with the game you can build theories that you can later test during the soft launch let's now move to reviews reviews are the tools that most of us use during aso optimization good reviews attract more players the higher the review the higher are the chances that play players will convert and download the game however that's not it there's a notion of useful information that you can get from players reviews you can check the dynamics of reviews there are more or less of them after the update this is

a big actually signal for game that something has changed and a big advice here is to segment those reviews with their positive negative or neutral as certain reviews will help you understand whether the update was successful or not whether a new feature was liked by players also you can segment uh reviews by words which were written more often and the other way around and the best way is to segment them by topic for example you receive mixed reviews and by segmenting them by topic you will understand for example that your players like the design but they don't like the monetization that much this helps you to find weak spots in game and works towards solving them so you don't need to change the whole game just beats and pieces and of course reviews help you to identify technical problems

all of us have quality assurance but sometimes it's hard to identify rare bugs but with millions of players who use wide array of devices and have countless of sessions in few days you can identify those bugs which even bug truckers can't and solve them also um reviews our customers user customer section uh there is a lot of material in internet and how to work with customer satisfaction and i think it's great to check it out and learn about it and players feedback is always important in our company we have a machine learning team and this is an example of on how they help our support team to categorize their views by topic the machine automatically places reviews according to the words in review so like so if there is like

words such as guilds or upgrades or chat or help this means that we have some technical problems and so on this tagging helps to save time time and optimize support teams workflow of course we don't stop here and we always try to find more ways on how to optimize our workflow with machine learning but it's a whole different topic and we can discuss it in another lecture and the last thing that i wanted to talk about is questionnaires you can create questionnaires in game or if you have a very strong social media you can also do questionnaires there like social media presence what can you learn from questionnaires it's a net promoter score whether the players would recommend the game to others you can get a valuable feedback from

them again it can also help you to run liveops and i'm gonna show you an example later on how we did it in the game and questionnaires also increase user loyalty so with the help of surveys players loyalty towards the game increases players understand that their opinion is valued by the development team and this is a very important aspect in building this long-term relationships and yeah so about the service on how to structure them how to place them you can find a lot of them in the internet and find the one that use you modify it and test it this is an example of the survey we did in our social media in different languages we asked our players who they like to see more in-game from our npcs chief adam

or delivery mandate adam won the survey he's my favorite character as well actually so i'm very happy and we included him during our food truck event as a helper and players liked it a lot they left a lot of positive comments and we're happy to see their favorite character by learning that adam is liked by the majority of players we included him in our other events such as sugar rush where adam is the main character so you help need to help him to build the cake by not losing levels the engagement rate of the event turned out to be quite high and everyone is happy second example of the survey in social media is when we asked our players to create the outfit which they would like to see in game and our community went crazy there were lots of

like art created with different styles and players voted a lot which one they want to see in game and the winners are here on the left is the winner by our internal world development choice developers choice and on the right is a one that was chosen by players so we recreated them in game i think it looks pretty similar and we decided to sell those outfits in offers and players liked it a lot because it was our combined effort to create them the conversion of the offer with this outfit was five percent we didn't stop here and few days ago we launched and another questionnaire and we asked the players to to draw the outfit and actually out of this three the one who won was actually the right one

and uh yeah i it received like lots of likes and everybody just had fun and we are now uh trying to figure it out how it's going to look in game and to summarize the opinion of players is sometimes bigger than your developer's opinion and you shouldn't be afraid to ask your players do ask them what do they want to see what they like and what they don't like in game and work with data and make a data-driven decision thank you that's all from me for today thank you very much that was fantastic yeah i mean i am a big fan of net promoter score i never used to be and i suddenly got it with one of our clients um it's a large corporate doing a learning game

but they want to do it as a live ops and they're obsessed with that promoter scott and i suddenly it's it's worked for me this idea of somebody asking someone a simple question like would you recommend us i know i know that sounds stupid but the fact that you mentioned it really i think just shows how powerful this this question is um and i think also the point you made about um if you ask people their opinions you're going to naturally affect their uh desire to to work you know to play your game more the one question that raises though i think is how how does that affect the reliability of their responses because if you're building up a rapport with someone by asking them questions is that going to mean that they become less useful to ask them more questions the reason i'm asking is because if they're biased towards you you're not necessarily going to get an honest a truly honest reflective answer that the rest of the community are not

as bonded will say does that make sense yeah and i think i understood your question and the answer is that you have to segment your players as well so uh if it's a new player who left a review it goes in into one cohort in one final and if it's like the player who plays like for a year and like pays regularly this is another one this is another group and uh as as long as you receive those feedback from your players you can categorize them and decide which one is like which feedback you should take into consideration and actually implement in the game yeah and i think i think you nailed it on the head there and the key thing for me is you know understanding how we you know which which players we're talking about you know how do we set up cohorts and data how do we compare like with like because i think it's very easy for i mean i know it's a stupid thing but the

simplest thing i learned was just ranking people uh in terms of the number of days since they first played together because if i if i was looking at a snapshot across the board it's meaningless but if i know that this person is in day three of playing as opposed to that person over there is day 100 of playing that simple simple measure is so revealing in terms of what their answers are and it has so many layers of information that go with those simple cohorts so give me tips for people about cohorts and why why they're useful well actually you have to do please use cards you can this is the the simplest advice you can you can do it any anyhow like they imagine just your imagination is limit you can just like the how how long they played did they try this event or did they not

like when did they they made the purchase or like the limitless possibilities on on building the corporates yeah exactly i mean i've been asked to do reviews for um some uh chinese studios uh in the past uh few months and um one of the things i've been doing is um i've been doing breakdown based on their retention so you know day one day three day seven users day 30 users day 90 users that kind of thing but i also found it interesting to break down in terms of spending cohorts are completely different uh here's non-spenders here's people who spend the smallest 25 of audience and then there's of you know of those people who spend is the you know the the highest 25 percent well you know that quartile model and i found it's really fascinating when you look at you know behavioral data over time versus behavioral data by spend so any kind of combination of how you can slice and dice your

players i think really is powerful um so again i think it's really interesting is like observing how they're in how you engaging with them changes where they are in those cohorts i think actually it's one of the things i find fascinating um anyway we've worked on too much on cohorts uh marcus here uh on uh mark sternland um he was saying it's observed that um quite a lot of youtubers in games um do reaction videos um and his impression and i'm not sure if i agree or not i'm still my jury's out for me but i think there's a there's some truth in it um their initial reactions are often very honest especially where you know they've not been paid by somebody to give it uh because of course they're an authentic spokesperson you know influencer they have an audience have you looked at using that kind of reaction as a source of data

actually we don't receive that many like youtube observations because this is a like this is a very casual game and uh like the reviews are often on youtube often left uh from players who do play like mid core and hardcore games they they do more and um i'm not really sure that if like if we received a little number of it on like whether we actually even took it into account but actually i've seen some just videos that show the gameplay of the game but they were not actually commented that much yeah no and actually i think um where i have seen youtubers doing commentaries on uh specific games um and i think that that can be really powerful uh and even casual games uh but they tend to be people who've been playing the game for a while um because this isn't to me i mean this

isn't classic you let's play fodder this is a a game that you play and you engage and you build and you build and you build you're not going to play time and time again and restart and i think let's plays need the ability to restart to be able to get that kind of entertainment because i don't think you get the i don't know maybe it depends on the youtuber but anyway i suppose the point i'm saying is it's really interesting seeing how you get this different types of audiences reacting in different ways and we use whatever data we can get but i think probably the biggest issue you've highlighted there is that you can't guarantee that any any influencer is going to play your game um and the only way to guarantee it is to pay them and then you've ruined speaker who's oliver kern who's a fantastic guy who knows all about live ops um uh he's asking here how do you remove positive bias in play tests for example um if you ask friends and family or if you ask someone in the in the industry for

example who knows you you're going to get them having couching their response because you that you're going to affect they you know they're going to want to not annoy you they've got a relationship with you they don't want you to feel depressed how do you get them to give you the brutal hard vicious responses that you need to make the right choices that that that's why i'm not actually um advising you to rely on the internal play test it's always better to do the external it's always better to actually even ask an outsourced company to do it for you because they if you with an external playtest you can actually ask somebody like i'm based in russia just a russian person to come in and play the game but uh the geographically russian players behavior is completely different from the from your target audience which is the united states and yeah and other countries so yeah it's you you always have to take it with a grain of salt

and just uh just try to categorize it again so if um many of your friends said like the game is great the gameplay is great but i actually uh just played it for two minutes that that's also a signal that's yeah yeah there's a whole bunch of techniques that i try to look at so first thing i look at sentiment analysis uh in terms of looking at what people have said but i compare it what the data of what they actually did and that's a big reveal but also i i look at asking different questions at different stages so if i am at the mechanic design stage i'm asking questions like is this fun if i'm at the context loop so they're kind of like the purpose and progression stage of the design i'm asking do they stay playing it if i get to the metagame i'm looking for do they scale so i will i will involve people who are close to me in the first stage but i'll generally try to get people who i know

understand games and trust me enough that i won't hate them if they tell me my game uh i've got a couple of friends of mine who are really good at telling me my games are and i love them for it we all need those friends oh you need those friends so badly um i mean particularly for me it's like it's the sort of heartache it's that it's that's holding truth is a painful process uh but we're gonna learn to love it we're gonna build up the calluses for it and it's heartbreaking every time it happens but i i know we need it so that that matters to me and so to me it's about breaking up and asking what is the question i want to answer and is that audience that segment the right one to answer it i don't know if that answers on his question but um at least that's that's my take on it i know he has his own views on this by the way so um he'll have an opportunity to talk about that in a few seconds so um so i suppose the last little question for you is

what was the one bit of advice you'd give to people who are going to take their game forward into live ops what's the one thing they should be considering make a data driven decisions sounds good to me and on that note i'm going to say thank you very much diana that was fantastic by the way i loved the curve you showed it was to me the perfect live ops curve of how a game should sustain as a constant gradual growth um which is so rare and so hard to achieve and congratulations to you guys on doing that you