Design versus data in game development

[Music] so it's my profession now I'm going to get to ask Africans come and join me we never know what we call a fireside chat which basically a nice conversation about the role of data science this is design and ok it's a bit of a false argument but it's worth as having a bit of fun with it and so let's just introduce yourself and so tell me a little bit about your background so you your head of data game data science at silicon studio yeah but you also have a history of all sorts of things pretty wonderful thing going string theory CERN what why do you end up going from that hardcore physics to game well I mean the first painter was moving as a

theoretical physicist to a lot of scientists yeah so that was my first I mean decision I mean I took em in long time ago and yeah I mean maybe suppose to do more apply things I mean I use them in my mathematical and maintaining a background to do things on matter more to people and in a string theories to options different marvelous we said they we I'd love to know more but like you say something that's provable by that is always entertaining and particularly to have effect online and I think you know I've heard earlier with the the composition of the gate Isis is so amazing it's where you guys showing just how much an impact games have on people's lives and what I find really interesting is that somebody with your background with obviously maths being the sort of fundamental William basis of that but also particularly the principle of evidence coming into games only what

were the initial challenges that you had well I mean we have many challenges I mean that the most of them are not maybe so science related but what we are doing and in a sense I mean from Silicon escutia indicas narita that is the unit Reilly is to apply I mean actual machine learning models I mean to produce understand player behavior I think I mean we are filling a gap I mean that is still and missing in the in the game industry that I'm very very surprised with that because the quality of the data I meaning that in the game field is extremely rich I mean it can be useful I mean to understand I mean a human behavior so and I mean cyber scientist I mean we are we're I mean allow us I mean to use I mean really sophisticated models and to produce them in a great tradition I mean so something that in other fields I mean that they are also I mean that are driven industries I mean they they don't they got I mean so much advantage because they don't have success showing

I mean quality of theta I think it's a really what my good I think back to when I was working at PlayStation home so Sonja thing can be quite a bump but it come from a telecoms background in mobile operators and I work for BT before that so I'm used to being an environment which is hugely data rich yes you can see what people see it how much time the engagement and how much is spent yeah and be able to analyze that and look at patterns and go over that's how we make a division that's another Sony thinking okay that's how we work nothing no data capture at all that were just starting at that time I want to be fair I'm being a bit unfair but do this they just started to use heat maps okay so they were what with killzone and actually trying to think about how where people were when they shot gun and then where people were when they got killed mmm-hmm in the gate and that was at the time with the Communists we're talking about 2008 that's really really new relatively you

know I mean it's only 10 years of actually people's paying attention to data but this is new elsewhere as you say yeah I mean I mean as I said I mean I work as a data scientist in many fields I mean industry and research but yeah I work in insurance I mean there is nothing look like a super I mean a catenation insect or I mean or I mean but I mean they really I mean to a strong smuggling or also I mean like telecommunications I mean in the same and it but when I moved to games I mean surprisingly I mean they didn't immediate even do some in Los Cabos models I mean they're using other industries or even I mean synthetics at first so it worked three years ago so and then I mean I mean I had portunity of queuing I mean given our data and I said oh my goodness I mean there is I was no one doing this can I believe it and I still I mean I mean we we just release them in two weeks ago I mean and we are the first of doing a mean individual player I mean predictions I mean for games and

I think that's an interesting kind of way to look at it because we're looking at here we did up the patterns but can we look at a specific motivation a specific behavior or not I think one of the things when we first started talking and that it was clear we wanted to kind get across this question about that this is the place where data it's important and actually I think the fact that game is our digital the fact that we are are now increasingly connected at the mobile particularly absolutely all about being connected so you could argue that means that we kind of excuse anymore and yet I still hear some designers who are nervous about working with someone like self or getting a machine to tell them what they should know but it's almost like and I don't it isn't a la dice thing I don't think it's like you know people are denying the technology but I think they're in a tension between data which can tell you what people could did and being able to know what people will

do and if you're going to predict people's individual playing behavior how do you break that and you jump that gap mm-hm yeah so the picture how you look at that yeah I mean like a subtly I mean instead of treating I mean analyze a time in a sort I mean we just analyze player by player I mean so currently I mean with a deep learning or and some learning techniques I mean is very easy a quick coming we can say I mean like integrating a big data platform so and it's possible I mean and and we do it I mean an hour allows I mean I really I mean a high accuracy prediction I mean as you say I mean we track every single movements of the players I mean we know when they play I mean where they get forwards I mean which part of the game so we also I mean no I mean when they're gary boards I mean the very beginning couple I you're doings again I mean we know I mean until which slab of acquainted if I mean we are very predictable I mean so but it's very difficult I mean even if game designers has a strong intuition I mean and we

don't want to compete with that I mean they can really take advantage of this in the data and you think I mean I mean algorithms that are able to fabric relations that for human beings is really impossible I mean maybe when with joint we are talking about on so many more vehicle uses so by the I mean when when we tell them okay so I mean the most successful part of the game is start and this is the kind of player that can in enjoying the most and is the less and this will mean personalized promotions and so on yeah I mean the some of them I mean or most of them I mean they are very elliptical and they don't trusted terminal or I don't like they don't trust if they are afraid of someone you know what visitors over to that kind of question about you know where did the line draw and I think if you look at on the historic attempts to women think of particularly you know I had a very big focus that the data can solve all the problems of course they didn't solve all the problems and I think this is where the maybe it's worth sort of set Stephan

back a little bit in thinking about the definition of what the designer does which is I think we actually were a little bit you know kind of vague about game design and what it means and it I don't think it's just about kind of like coming up with mechanics I think it's actually about problem solving and to me if you're a problem solver you want information and so you want to be able to say okay here's the flow of the game here's how a typical player path will look I can look at individual players and compare them to that model and then make predictions based on the data of how they're behave but the data won't tell me what changes to make necessarily although or if I would invent a new leveling system what impact that new leveling system would have but it is shortcut that process certainly it certainly I mean of course we cannot say I mean it's difficult for

us I mean to say I mean it mainly I cannot play all the games I analyzed I mean it's impossible so so but I guess tell them I mean ok the level 50 I mean is try again I mean a lot of churners I mean I don't know what is going on there but you know I mean as game designer so this is something because these those players I mean that they spend a lot of money in your game I mean I Gary imported difficult I don't know I mean but they are very discerning yes yeah it's just say it I love it when you put when it was its simplest simplest thing to do is just to put down players activity over time based on the number of days that they've been acted not that's not a date the number of days are active and awe when I first in and this is like what back in when I was a telco person over more than you know 1719 it was a 617 area and and it was a magic thing that happened and because I could see this curve appear and it was exactly like the Gartner hype curve yes and it was like

bang ok now I understand there's this anticipation there's this little disillusionment and then there's this you know leveling up to the normalized behavior and when you see that just popping out I even that's simple in um was an intelligent way of just pulling data into a pattern made a transformational difference the way I manage my service I was running yeah yes so when you can do these much more sophisticated models than that it can give you much more detail yes you know example um I wonder and then maybe you can help provide some examples of what the kind of peculiar variables might be like for example I'm wondering how long some it takes to get through the menu screen if that changes dramatically over a period of time is that a point to worry about that player it depends on the game but yeah I mean it butters I mean it's a variable but you see I mean very significant and also I mean the the tutorial I mean is an illegitimate

totally bad teachers are not having to with my own game with that but you hear the scary is yeah yeah yeah I mean I saw the beginning I mean like we focus a lot in features that the strain the behavior at the beginning okay to very in a very early stage meaning what week we can classify those players that are going to be on in like very loyal or very I mean guys I mean we're going to be doing very soon and both I mean there are going to be in a high spenders mean that doesn't need to be the boss loyal ones mean this is another point you kind of need a critical mass we can really make this work yes so where does the designer need to start working with their data scientists or at least or some data framework in the beginning I mean this is very important whatever because it's for the beginning and I agree with you but you know it's what is it that we're trying to do yesterday just question really so I mean in order to took us to half a minimum curacy best inaccuracy I

mean in our collections I mean even for time serious or I mean like item recommendation I think it's month okay is the minimum we work a lot I mean with big data but also with a small data okay it is very important I mean that because many of our I mean the companies that we work with they are very interested in testing I mean experimental I mean game so they want to see I mean how they are going to behave them so we are I think that's what is interesting if you take let's say superstar always going to do a longer period of time I mean that's as it don't I it's been a long long time in sophomore and it's interesting how the bigger companies are the bigger budgets you said a lot of new things and yet you take a game at a Mike Graham game and they will do a prototype in two days they'll put it on the market and they look one and they'll do that for months and then when they find the one that sticks but for when they're going to develop and so the data analysts work that they're doing is really just looking at app

store data they're not necessarily looking at like each decision point within yeah and what I'm thinking about how you map out the data model for a game and so do you have any kind of particular advice do you have any particular advice for how people should think about their game so it's useful hmm I mean every game is of course different I mean but taking the information of the data I mean for efficient game development I mean is is very important and accepting the surprise but the data are going to give you because many I mean developers I mean designers I mean when they say something basically something that it is strange for them I mean it's not what they expect I mean they're more afraid than ever whenever so this is the moment I mean that you are learning and discovering I mean parts of your game that doesn't work I mean you thought the were successful not the other way around I mean so learnt I mean with the data and be developer the sort of thing I'm

thinking of because I also as a designer I want to think about the decision points of the player mm-hmm and so and it's a technique I kind of picked up where I looked at the whole process as a designer acting as a building a data model is a design choices in many ways and you could just try to okay every point where there's a variable let's save every variable at every duck time stamp them I'm actually a bit messy yeah so instead I've tried to look at it using a method called asset which is a thing for the food industry so hazard analysis and critical control points yeah I mean we work with us on it so I think it's really probably best I got I just have just stopped the food issue a long time ago and it was really interesting because if you think about it like this so the premise is basically what happens are you like to encounter and this is something that was devised by NASA and we have an ester and tildes bridge to make the food for the

Apollo missions and you know the idea is you work out whether hazards are in the process to deliver the food so you know you don't know how a little bit the metal in it so you have to do a metal technically you don't want it to go off so you need to make sure it's sealed in the right stage you need to make sure it's past or and whatever it is you work out what the hazards are but then you work out where are the moments where it's appropriate and valuable to test so that's what you're testing for metal the beginning of the process yes because the machinery that you build this media crate the food with could drop more it could wash it into the tooth and I could spend ages all about it but the point to me is is looking at player behavior as a flow for me is critical to that is that fit with your yeah I mean it's for you and I strongly agree I mean in deepening we do the same I mean I strongly believe come in is what we do I mean in our team I mean in interdisciplinarity so we are not something created new always I mean what we are we are doing I mean it's adapting

algorithms that are robust in other scientific fields in game data what do you say I mean they think kind of algorithms that they are doing some inference I mean Hospitality survival analysis that we use I mean we are up one algorithms that is used I mean in biological research on the medical research to determine when a player is going also is going to die yes indicator so Evans I mean I mean time for Evan to happened the same I mean when you go in to support chasing the same when is going to do the next in poor taste so this is I mean exactly what we are there a limit down to statistics and because if you've got a pattern of events and you know where those events happen in the life size to live an average player you can see where the disturbed deviations of that are and then make decisions based on so it sounds like it's easy why don't you get a son this is not having videos I mean look at them in two years I mean develop this platform and also something important is that I mean the the algorithms that we use the Manor I mean scale with big data so we we do

something for instance for channel prediction the survival analysis but put in an assemble learning model that first time in a scale would be data and second add up to every data distribution that we fit in there in that way I mean excuse me for every game data in different kind of games that we put into soon yeah I think I mean for me I hope we're getting across I mean the crux of it is that you know we need to look at players and their flow of engaging in your game and then once you've looked at that and you map that out we can then use models or analytics to basically assess what the differences are between players create you know archetypes within that code warts are than that allowing us to work out where we are and how we're performing but what this is one of the questions I really have is I can kind of do that to a certain extent but really the joy that's going to come out when we can compare like with like I

don't think we're in a stage I mean I don't know if you feel this way where we can see teams collaborating mm-hmm but even if they could would it be useful so if you have two games relatively similar yeah could we benchmark that in a way that's useful yes how are the machine learning and you know are our librarians learn and even if we I mean if we have different companies I mean at the end our voice yeah yeah I mean yeah and hope would subpoena and then I mean like for instance I mean like a we have a some models I mean that's running I mean for Japan yeah okay so I'm all our models are localized we can say so localization is not only language or I mean currency I mean is beyond that I mean so the same principle do the recommendation of what I suppose I mean and in-game events I mean sequels often game it and modernist month and the same sequence of events I

mean maybe it's valid in department where we are based I mean but it's completely different from the same game published in America so what we do I mean is learning I mean and it's different from every culture that behaves could be different and also we introduced I mean other variables like temperature are my holidays so I mean it has the story in science I mean no remote yeah I've had that part really fascinating when you when you look at the other variables that are not the sort of thing you'd think about then this impact that can have on on play performers like you know and you know when Bingley gets knocked out the World Cup it has a negative effect on all sorts yes that happens for far too often yeah that's really what that sort of kind of sum up because we're about sort of if anyone has any questions please quick put your head around got long but please do and what's the lesson really here is it is it that this machine learning process is going to eradicate the need for a designer or

is it but this is something where every designer needs to embrace and learn from you think I mean of course I mean designers are extremely important and the but the lesson I mean it's like the sooner they embrace I mean they die that Draven's revolution the better this is something that it came something is going to stay so the sooner the better I mean there are thousands of games I mean like the published several years for the competition everybody's competing for the same I mean players so I think it's very important to my competitions not gonna get any easier is it yeah so it's like it's dangerous to go alone YAG data thank you very much thank you [Applause] you [Music]