Custom-made games with machine learning and big data

so hello everyone it is a pleasure to be here today Seattle today I'm going to talk about the data the data the players generate and what we can do with them to to predict their behavior through machine learning models so just wanted to introduce myself myself my name is Africa pianist my background is theoretical physics I have a PhD mathematics in a similarly since 2005 2006 I've been working in data science in in different fields I start my career after in the particle physics elevator in Geneva and since then as I said I mean working for academia and Industry predicting

different kind of behaviour from coca-cola energy also working in research in recurrent neural networks etc but it was in 2015 when I was in Japan working with satellite data in an operational center when when I discover game data so where a game studio silicon estudian a spin-off of Silicon Graphics that were asking for that the scientists to understand and predict player behavior so then I talk to the CEO and I proposed to okay let's analyze those data but but what about creating a platform able to provide this kind of intelligence and knowledge to other game studios so this is how everything started was founded in 2015 and we are in keywords we were acquired by keywords studious last summer what is what is our main purpose is push the frontiers of general behavioral machine learning so this means that we

use machine learning models that are able to adapt to any data distribution to any player to any kind of game so our models must generalize well no bias in a way that we are able to predict players when they are going to quit for instance for very casual games for MMO or RPE is completely different as you can imagine and sorry we also want to we were in one direction that I think what we're seeing that is the right direction that is happening almost every field every data field every industry it's towards personalized games what we are doing with our models first we are able to predict to open those databases normally very big that almost most of the game studios are not able to to strut that information because basically the data are huge so I will so - we

focus on unpretty behavior and - to provide actions so okay we predict behavior of players but we want to do something with them so we want that that game studios are able to retain to engage etc so and also the totally produce it's very very important for us to understand what's going on to understand your place how what they like what they don't and also not in cohorts or groups we want to produce in individual level why because the quality of the data is so big is so good that allow us to do this how we are doing this first of all we have one of the best teams in the world in terms of data scientist machine learning engineers and that engineers these three blocks are extremely important for us we don't do classical data science reporting and do presentations we do all own production so that the scientists the very sophisticated models and machine learning engineers tried to how we can

put this and in scalable way and that engineers are able to do everything distributed from the from the data upload from the feature engineering from the results etc whatever what are our fundament so we have had very strong relationship with academia we also publish our papers and we also have entrepreneur mind so we try to execute very fast we we put in production actual scientific research so we we are not developing our own algorithms from a scratch we try to focus on different fields different scientific field or even industry fields that they are using models robust for years sometimes decades and this is what we put in production we are about to solve the business problems or the business challenges that this industry have and also interdisciplinarity because of the

nature of our way of working we we have employees from all kinds of backgrounds the only diversity in in this way I mean also background so we have vertical physicists mathematicians computer scientist engineers etc this is one example of the kind of outputs that we try to focus we focus on long-term predictions if we predict that a player is going to leave tomorrow you almost don't have chances to take any action so we normally are able to produce mainly with variability of games with two months and then six rows ahead when a player is going to quit which state and in what level this is going to happening or in what the stage is more likely that every single player is going to leave the game how many hours is going to take forever player until get bored how much money they're going to spend it to the game we have a very sophisticated lifetime value prediction we also have

incorporated in our platform and also connected into the game an item recommendation system why will have this we want I mean we do this because okay we know our players going to quit in a couple of months how how we can make it to retain this person to obtain this player then we we know every single player what are the items are more likely to buy and the word the ones that they want to purchase the most so with our model we know they buy they play a right player what are they interest in terms of will to light or actual items for every single player so then we have you can't decide automatically and can be fully automated if you want to give a reward to engage them again or you want to give 80% discount or a 10% discount to increase their inner purchases and also to engage them again into the game how are we doing this this is just a schema

of our item recommendation system one of the challenges that we have is that the dimension sometimes the dimension of the items is huge the dimension of I mean how many items are in some games is to be so then we apply an unsupervised unsupervised learning model to reduce the dimension of it and with the groups so we apply machine learning models to understand which are the players that prefer different kind of group of items then we do we do circle of our collaborative filtering to again be able to move from the from the group of predictions of itis of items to the actual item that every single player is within to get nest our results are are really really good with this item recommendation system we we also focus

not only in in applying data behavioral data of the players we also include in our system external information to include water information we do event information Christmas Chinese New Year sports event even the mean if it is the final the Super Bowl or the final of the Champions League this this kind of models are of course localized because it's not the same effect in every part of the world so so what we do with these is to be able to to recommend what is the best moment to apply a marketing or promotion campaign and try to maximize this effect I mean sometimes doing a campaign I mean right before Christmas is a great idea sometimes it's the opposite so all these old models analyze the past information and recommend you what is what is the best timing for doing those events this is for marketing a promotion event and also from game events it is

good idea to put my best game event together with this promotion and precisely and enough for games like fifa in the final of the Champions League maybe yes maybe not so this kind of others are able to recognize those correlations and as I mentioned before we focus on personalization we understand that predictions cannot be in another way as individual only and this has this is give us space and move forward to personalization of games to give rewards to give recommendation of actions for players and even be able to that the game as long as the as the as the player is evolving into the game so we focused not only in how players play they behavior up special how do you play also external information like how do you play and come in under 30 events how do you play when you go to work when you come back or when you're on vacation

be able to recognize all those effects and try to adapt the game for your needs for instance as well personalization in a way - which is the best plan in a class of clans that is going to make maximize a fund depending on the time and with whom you should compete against the mariokart these are samples of the personalization that this is already done in some in some areas and some industries but still this missing in video ends and it's very surprising for me why we su do this why will soon rely on data why we so look forward and be much more data-driven in video game industry well basically I mean I already Amos burski and Daniel Kahneman I mean that won a Nobel Prize in 2002 say it - a deterministic device thrown into a probabilistic universe what does

it mean it means that it will have a model a good model just to predict behavior it is much better that we use it because we are very badly in predictions so I hear a remark this sentence are from Amos verse keen so I like them I mean like people predict very little and explain everything people often work hard to tell information they already have and try to bow it always near knowledge and and I will I think everyone see this on daily basis so and everything that has already happened must have been inevitable well in this is always the world this is an example this is an a scenario of how much money you can make if this is a very conservative action area okay this is a big game a big title mobile game this is just an example of how you can make using it they insist in a conservative

way that the science to improve and to improve retention to improve the development of the of the game so for instance if we managed to increase just for one month that your VIP players in a 10% 10% of them for one month it means an increase of 1.3 million dollars so if we increase the number of pyou because we have a feature that tells you who are the non paying users who are potential paying users and try to to to provoke this flip this would mean four million dollars a month in sales increase and if you improve those levels because we also a spot which are the part of the game that are not likely that are making your players leave if you improve those critical may be there normally we spot 10 those those those levels it means 8 million a year and this figured a median a year because of

improving the development for casual games and for in the games is much bigger because normally they don't spend so much time in the development of the game how are we doing these so as I said we'll have created a platform and we focus on operational play behavior prediction so every day the data fit in every day we update the models and every day we are everything is Rufus I mean 24 hours 7 days a week and we also focus a lot in visualization of the results because we want that this information will go across the whole company from the secretary of marketing game planners for of course developers in order that they support the decision the decision making process the data which model we use with everything that works we are not only doing the learning we do the learning but also a number learning and other statistical learning techniques that are

robust and work well and also a scale with big data this is very important because the data are massive we are working with games that are huge with millions and millions of lonely active users and historical data sets from 16 years so similar name is I mean I want to go very deep on this I mean but rather than running only what machine learning model we run a cloud of them and rather than take in a 1:1 result we are taking on a statistical unaesthetic of the results that we obtain so normally it per semester under summer learning that we use but normally gives very robust result and it are able to be paralyzed this is really important with the amount of data that we have we use as well deep learning as you know rather than using only one layer of classical artificial neural network approach we use several layers to make the non-linearity much bigger on the other hand we are not able displaying what is

behind something that we can do with a sample learning but in our cases when we choose it is because the curacy is very high for our LT in life and body model we use convolution and they were nervous for instance one of the advantages that why we use this is because we can import directly raw data into our models and at the end it was working much better than other models so this is what we have in production so so for instance we want to predict with how much money they are going to spend every single player since today until they leave the game so in that way we can we can give proper words we can try to engage them again the ones worth we're going to be finished to finish the in terms of lifetime value etc what is the challenge I will have I mean this is today and here we have the dates I mean

here we have all the different players so we have some players with the tougher main long historical information and others just started the game and for all of them we need to predict how much money they are going to spend with them with the maximum accuracy and also we don't know that your day so at the end what we did is connect our Chum prediction model together with CNN and it turn out very robust result to be able to be in production as I said before we we are connected with academia and we publish our own papers why we are doing this this is just some marketing campaigns well with the marketing in the consolidator so we we try to show that we are doing serious work and we really put research into production so you can take this is in our website if you want to have a lot to the models that are in production in our platform and this is just an example that we participated in

2017 in the first competition of PDO games by computational intelligence and game hold in New York there were two tracks the data were from NCsoft and we had to predict which players who live the game I mean that the binary prediction who is going to live in a certain period of time and which day this is going to happen and this was a very useful it was very useful in terms of benchmark our models with other teams with other I mean companies and academia researchers so we wanted to tracks with quite mainly in the truck too there is the most difficult difficult with a significant difference normally data science competition there is not such a gap so it was yeah I would wait to measure the quality of our models and as I said I mean yes I want to put one slide more

about the data they be data architecture and infrastructure so as I said I mean we need to be fully distributed from the data upload for the future engineering for the data preparation and from the solution of the machine learning models so this is an ax schema of the architecture that that we have so we have spark cluster which is Cassandra as a main database then we have the computer servers that for small computation witches are or even Python 5 PI spark and the platform is Alice in Java also PI Church what about this complex which was natively in Java for instance to make it I mean fully distributed and then we have the part of the recitation of the results that I would like to do a very quick demo just in the one minute or half remains

okay this is the shape of our platform that again Colet to the back end of the to the service of the games so the action that they that you can take are also reflected into the game so this is just the dashboard which are the levels that makes your plate with the most I mean you can go through these and understand I mean what is going on in your game this is the sequence of game events that maximize Saints in the next two months or that optimize the maximize play time so I said how many players are going to quit in media play in ten days in one month etc so going very quickly I can show you for instance the the life expectancy model so here we have place we're going to quit in and as I said before in one month three months etc and

we have play by player we showed this in order that you really feel I mean that we are doing individual predictions so for instance this player is going to quit in sis 630 date this is an RPG I mean Japanese very very low deal game and on to the level the last numbers is going to achieve is 207 so this is an example here we have I mean the most loyal want the one who have a shortened life expectancy and here is a summary for instance the guys who are going to leave it in 30 and Nate ninety days everyone is in the different stage of the game some of them they were for two years so there's four I don't know one month but they're going to quit rather this soon so then you can take directly for instance a push notification if it's a mobile game or you can literally send a reward so for instance like a discount item okay so we can say 10% or you can say 90% what is the item that is they're

going to receive the item they're going to receive is based on the item recommender I mentioned before again for player by player we know what is the most likely item they would like to purchase nest so SOTA matically this is the reward or the discount I mean the reward that they will that they will receive and then everything can be automated so and you can follow up and you can do a/b testing out automatically as went to evaluate I mean it's better than I put 10% is better than I gave a food reward or this 50% is enough so this is how you can follow up a reaction that you take time is over and well we can also do simulation of game events and marketing campaigns so you can just play around okay what about if I do this super game event and I run my marketing campaigns at the same time whatever and you run it and you observe how then then the future sales are going

to behave how is going to fit in terms of of the played time so thank you very much so yeah I mean obviously this huge amount of time but it's you can see there's a huge amount of potential when we use this kind of technology one question I have is how much time or amount of data is required before you can start getting some level of accuracy because I suspect that you need to have quite a lot of data to get the level of Accra so you're kind of talking about is that fair yeah it's very long we have a first phase of research that maybe takes 3-4 months then we post we will pass the next step that is production and then sometimes okay look guys I mean that the scientist you're great but it's impossible to put into action okay and the engineers the the machinery engineers try to get through this similar they come back to the data scientists and back again and finally we

got a version that we are happy more or less there is a always a commitment between the three teams and then we start doing all the sky level that the feature engineer the model etc presumably designed still plays an incredibly important part of that because but you need to know that you're not just making something which is an item that's pushed to somebody you want them to actually be engaged in something is that