Turing Tantrums 2017: AI Devs Rant

[Music] I've somewhat ranted a little bit about this in the past and that I think does a design and AI are very linked I think that what I what I want to rant about is is sort of codifying that a bit so oversimplification is bad right I've even had other rants where I've said AI that oversimplifies how it looks at you is pretty stupid if it reacts to everybody in the same way that's stupid and likewise if companies don't have a rich enough model of me I think that's pretty stupid too and it doesn't matter what model they use right I think a lot of simplified models are pretty dumb so the problem is humans love to bucket things they just love it we're we're classification engines and it just sucks programmer artists designer producer these are buckets and they're not very good ones especially because a lot of us are mixtures of these things um even games do this though we have tanks and DPS and whatever although a lot of the better rpgs recognized that the combination characters are some of the

more interesting ones to play right but in real life you have dedicated career builds and you have a severe penalty for respec and if you do do quests outside of your main line you're sure as [ __ ] not gonna level up so what I'm saying is that there should be a big v of developer skills and I'm not talking about personality wise I'm talking about actual sort of skill set based spec of what you do as a human being and in the thing so now programmer designer artist producer are sort of buckets of skills and instead of an AI program or you're a mix of them all and I'd also recommend having some optimal specializations within each so if I am rated p7e I database that means I'm a seventh level programmer with AI and database specialties d1 means I don't really have much design space in my head yeah the audio means I'm probably the a audio guy for the company and pr6 remote means I've got really high producer skills and I've specialized working with remote teams so let's do this here's an AI generalist now this graph instead of pointing at an entire team points at a single person he's a p7 AI D six a two

PR one it means he's an almost 50/50 programmer designer split he has some good art tendencies and he has a slight notion of organization I would call this guy textbook senior AI programmer this is kind of what I look for when I'm interviewing a senior AI programmer a eye specialist here's a different mix p-8a ID four systems a 5 animation p3 this guy has he's a really high-level programmer but he's also a pretty decent artist who specialized in animation and he's also a systems designer and he has a quite a bit of organizational skills he probably loves working in heavy AI animation systems especially with AI K or mocap and he even has the organizational skills necessary to handle the heavy asset notions that you have to do when you're dealing with that kind of stuff and I know all of that by reading that thing I don't even have to like look at his resume damn-near doesn't just work with AI programmers here's a bunch of tech artists p7 graphics D - aap - is probably a shader programmer just by looking at that I can

tell you he's probably a shader programmer here's another one P 3 D 6 UI a 7 this guy's still a tech artist right he has a a 7 artist just like the guy above them had an 8 8 but he's half the programmer but he could probably design and implement the entire UI system for your game all by himself here's an exact similar guy except he doesn't have that UI specialization my guess is this guy's probably a lead artist of some 4 minutes had to wear many hats over the years or he's a prototype or an indie who's by simply not having a lot of people around has had to have a lot of hats on and has had to do a lot of things so stop the oppression of the pure disciplines it's not helping us [Music] what the [ __ ] is a technical designer or a creative coder or an advanced experienced developer those are meaningless garbage ways to try and show a blended discipline skillset and they don't give me any actual information as

to what you do I spend the entire time talking I could spend two years talking to you and not exactly know what a creative coder does because 90% of people have a different definition of that so the other thing is is that we should have these different skill buckets so that if you do improve other parts of your skill set you actually get recognition by the company and you can actually get raises and bonuses and level up based on any given skill set like if you need to improve your art skill sets for a particular game or your design skill sets for a particular game you get the recognition of doing that as opposed to like I'm sorry you're a programmer and the guy who's reviewing you only knows how to review programming so tough noogies you didn't get five more programmers under you know no cheese for you and finally I think this should really help start breaking down the discipline walls because goddammit we are all a mix and just like in real life stop the oppression of purity let's go for it thank you

I didn't expect going after Brian so my slides kind of suck and since I'm Kennedy and I want to apologize I'm not talking about about any one of you if you feel targeted see it that's motivation if you don't but know someone that should be targeted try to mentor them so basically I wanna bash with gameplay programmers that let themselves go so let's start with a little story of a little boy wanted to be the very best the world has ever known Ash Ketchum a lot of programmers start their careers trying to be the best and want to learn so many things but as they work on games and go from milestone to milestones I feel like a lot of gameplay programmers let themselves go where as engine programmers try to be a bit more structured so I wish employ grammars would continue using the broad range of data structures algorithms and focuses

on efficiency and there are designs a lot of time we take the shortcut saying oh but I need to be flexible and revise my game because the game game designer wanted this change or that change I feel like those are excuses a lot of programmers stick to only STD vectors or the equivalent and there are game layer that that's kind of crappy and I think we owe it to ourselves to at least know when to use other containers and if your gameplay layer doesn't support them I know a lot of game engines only support like dynamic areas for a gameplay programming take the time try to learn how to extend the language and create new containers if and you need to also keep an open mind there's not only utility systems or there's not only finite state machines try out stuff try behavior trees planners whatever you need to try and experiment with them

lastly I'm trying to go fast for the other guys here's a quick book recommendation because yesterday res said then that maybe 50% of juniors you'll interview will have a hard time during the part of the interview process and that 10% of the more experienced programmers you'll be meeting will have a hard time well keep in mind that it's very likely that distant person is already working for another company and that 10% might be your coworker so you can suggest them like a quick shortcut a book like this it's like crunching through algorithms and data structures so you can really quickly practice some computer science concepts that you knew when you came out of school that you might have forgotten over the years so just try to be a bit a bit better with that and honestly read books but by the game AI pro by AI

wisdom just try to be more curious also try tricks there are so many things that these books give you that can be implemented in like a half a day just to play with concepts so do it and try to find better ways to to improve these tricks that you read about also if you think of something great contact Steve and write an article also most of you guys studied this so just try to work on it and gameplay code it's just an excuse when they get when the code sucks you guys can do better so focus on being better programmers and the also premature optimization is evil but doing things right is it's what we should aim for umm yeah game designers now try being independent I've seen so many game designers telling me well I have no idea

if my game design is fun or not I need to see it in the engines don't you should try to be able to prototype your own stuff there are so many engines now like unity even unreal that you can just download and learn how to script quite easily and if you can prove me that your concept is fun with a cube it probably won't be fun with the super expensive mocap animation and it's also 2072 2017 what are your tools Word PowerPoint and Excel come on you can do better and with Excel it's quite easy to learn vbscript so you can automate some of your spreadsheets and try your concepts within Excel and this will allow you to have quicker iterations and it's easily shareable with the other people on the team and like I said there are so many game engines that are easy to play with and to script so even some stuff as simple as blockly or Prime Minister in Canada program the

game it really sucked you can probably do better but it's there and it's easy to do also don't be scared to use paper design and you can play like Dungeons & Dragons your game and just say that they add a table and turn use turn Bates turn-based methods to test out your gameplay so just do it all right hi everybody my name is Mike and I have a beef with magic we're all probably fairly well exposed to science fiction here so I'm sure you've already heard this cliche quote any soup any sufficiently advanced technology is indistinguishable from magic this is due to a certain arthur c clarke and since brian did me the courtesy of already breaking the f barrier in this rant session [ __ ] you arthur c-- clark i

hit quit hitting the wrong button there magic is [ __ ] and i'll give you an example of why i'm saying that so we have here a couple of high school graduates they're deciding what they want to do for their institute of higher learning and they're trying to figure out where to go so where are we going to send them well we have options in the united states colleges like MIT or Caltech and i don't know how a lot wall in there we're sure as [ __ ] not sending in the Hogwarts and the reason for that is because we study computer science algorithms are not magic okay solving bugs are not magic this remote apparently is magic [Applause] software is a science and that has a very significant implication for us which is that superstition is irresponsible here's another quote for

you that I'm sure we've all heard at some point or another I don't know let's run another build maybe that won't happen again my opinion is that this [ __ ] has to stop so this year is an interesting screenshot this is a debugger Microsoft Visual Studio in 64-bit mode debugging the disassembly of the Gilberts 2 game client and I have one question for you does this scare you because it shouldn't be [ __ ] scary this is our bread and butter this is our lifeblood this is the tool that gets us our job done never be scared to keep asking questions because science itself is predicated on questioning and that means that if you ever stop asking questions you're basically dead computers don't run on magic which has an interesting implication our questions have answers and we should go out and find them so here's another inspirational quote which crafted the

ignorant simple science to learn this is due to Lee Brackett in the 1940s there's a much less popular quote than the arthur c clarke one but i think much more reflective of how we should be thinking of our practice so this guy here is a magician and this is a computer scientist let me leave you with one question who do you want to grow up to be thank you [Applause] so it's like we are doing kind of the same rent here some my rent is called garbage in garbage out so yeah I'm a programmer I might be putting myself in a bucket here I take random designs and turn them into computer code and you know what computers don't process very

well well they don't process nonsense they just don't care what you feed them with so you might have heard of Charles Babbage the father of the computer himself but guess what he knew about this somebody asked him once mr. Babbage if you put into the machine wrong figures will the right answers come out I'm not able rightly to apprehend the kind of confusion of ideas that could provoke such a question but at least I suppose that's how Charles talked because that's so ancient that there's no recording of his voice like 1864 so some people still don't get it I said you said no French sorry about that if you don't man so here's an example this is from the developer of this suite titled about being a top model and yeah I worked in pretty

awesome games we had basic dialogue scripting system with from some variables for good measure I remember telling the narrative designer to go easy with that system we have no budget no time to check for consistency what do you think happened we let our players to options in the story you wander around in luxurious gardens and at some point when dialogue lets you date the gardener because everybody loves romance involving gardeners and there was another dude you could date later a taxi driver that would drive you around to your model stuff so that made a total of two choices two choices there's two possible combinations for which we had dialogues or throw in the end you could either be dating the governor or dating the driver recovered right right well nope provided with these options the computer performs very advanced computations and the juices for possible outcomes a granite

you can be dating the governor or the driver but you can also date nobody or worse be polygamous because the computer just doesn't care about morality so we discovered that we were missing dialogue for this at the very last moment the evening before uploading to certification we had to patch during the night fingers crossed hoping we were on breaking the entire game so I just want to make up for this to my former colleague Martin if you hear me Sunday sorry man I know I oversimplified things six years after release we had our first user review and Metacritic and it appears to be from a total fan of yours Oh yuck Martin here's another example this time for from the early stages of the development of this one or two so let's take a gown it's called Michelle because we call all our guards Michele that's the way it is don't ask me why Michelle is patrolling when suddenly a

player enters this field of view so at this point Michelle starts fighting for his life and calls for help from time to time and during pre-production we only had a sort of speech so our guards back appeal which is French for let's have a drink the goal of this box is to bring other guards at hearing range to the fight not to the bow so let's say we have another mission in a room nearby that author Michelle here's the Col and joins the fight backing a pillow in return on paper this looks fair and good we agreed on this design and I innocently proceeded to implement it without the shadow of a doubt now guess what happens when you stress the system with more guards and the player vanishes right in the middle of that fast well this is what happens there's no sound but they are shouting apparel apparel apparel forever

because the computer just doesn't care about doing the same thing over and over again so let me send this up so it's crystal clear for everybody here if you take nonsense and put it in a computer well it just blows up plain and simple it stops to respond or are loose forever or crashes our worlds just pretends everything's fine until you find out that your save file is corrupted so the outcome is rarely fun that's the curse of our craft my point is often the germs of failure are already present right at the design phase so we need to be consistent as early as possible also our responsibility as programmers in our buckets is to invest time is improper error checking to prevent a nonsense creation in the first place it's always pays off in the long run so please every day every time we fail at doing so a programmer somewhere has to pay back with a small amount of time and a

considerable amount of sanity and if you don't do it for me at least do it for Charles thank you for listening [Applause] guns shooting cover system sniper AI grenade throwing enemy detection systems damaged systems tactical planners animation syncing to have realistic and gruesome kills blood splatter effect systems I'm done I love working in a AI but I'm really sick of game AI having so much focus on combat and killing and it's the same [ __ ] that we keep redoing in every game enemy AI is where their sole purpose is kill or be killed gameplay mechanics and verbs only focused on combat the thing is the only thing that evolves is us trying to make things look more realistic and it's

actually kind of disturbing because I've started to get jiriz with full-on blood splatter reports showing me how we can make our kills even more accurate so either my testers are equally getting bored or there is a bigger issue at play here now some of my personal favorite games from the past year tend to have minimal amounts of combat and focus a lot more on socialization but on the flip side I also really do love watching horror movies horror movies and action movies which made me start to think well what's different why am i okay with watching movies with combat but I really dislike combat based games well it comes down to the NPCs that we're developing and the gameplay interactions that we give to the player characters on TV have a lot of depth they have conflict between each other they have ups and downs and we really care about them as they progress through their stories they fight they love they grow we care when they get hurt and we cringe when they make bad decisions but we can't actually do anything to help

them because it's TV games on the other hand have this great opportunity because we can actually interact with them but honestly I rarely give a [ __ ] about any of the NPC's in our games they're too simple they're designed to either protect someone kill someone or roam around in a big city like a zombie in short our NPCs are lacking empathy and intelligence and this is what makes us really care about characters in cinema and TV I'm going to call this heart official intelligence as I would like to see some actual real interaction socialization and memorable moments that I can experience with the NPCs in our games from developing love and relationships to hate and jealousy so here are a few examples for inspirations first off if no one actually mentions facade prom week and versu at least once in every AI summit then GDC will totally crash and everything will go wrong so for those of you who are new to the AI summit you are

going to see these games mentioned at least once every year and the reason is because these games these are great examples of social simulation games where the players actions actually changes the relationships between the NPC's now what pisses me off though is that we haven't actually really progressed all that much since these games came out what I wish is for triple-a - please just copy-paste one of your thousand combat systems that you've already developed and actually put some of your AI developers on some interesting socialization models because what was done in these games are [ __ ] brilliant and I want to see more of them next off is generic shooter X a friend of mine once told me that when he was working on one of these games he had work wrote some really interesting barks for their characters and basically they're this rare moment where one of the NPC's is idling with his gun and he looks up into the sky and he goes I hate my job and then he basically starts saying this goes against everything that

I believe in I'm only doing this for my family and I was like oh that's really awesome there's a Great backstory for one of these NPCs but let me guess there's nothing that I can do in this game except kill him right yeah is there an actually family that I can go to can I help out his family no can I convince him maybe to join me and stop doing this horrible job no because he was programmed with just one generic enemy AI behavior and nothing will actually change anything he does which is said next shadows of Mordor was amazing and I'm pretty sure everyone in this room has raved about their Nemesis system for once we had a game that left us with NPC is worth remembering the game celebrated my losses and it made me hate these NPCs which was amazing I had an actual emotional connection to the enemies now what pisses me off here is that when this game came out everyone was like oh we're totally gonna make a better

version of this just wait and see it's been two and a half [ __ ] years and we haven't seen anybody actually build upon this amazing system however yesterday they did announce the sequel so hopefully we will see a little bit more on a happy note finally finally last year The Last Guardian came out and I have to say my interactions with Treecko was one of my favorite AI moments ever what I loved was the was that interacting with with Treecko was actually a pain in the ass compared to most companion a eyes that do everything that I asked them to do it felt like training an actual animal which made the AI feel so much more believable I probably never felt so attached to an AI character in a game because of its realistic flawed and loving behaviors so I hope that we develop more AIS like this and finally everyone in this room if you're involved in a EIU should be watching Westworld it has been one of my favorite television shows in the last year and basically in the second episode there's this

beautiful quote Anthony Hopkins says it's not about giving the gamer what you think they want the killing the garish things the players don't actually want that they come back to the game for the subtleties the details that the imagine no one has noticed before something that they'd fallen in love with they're not looking for a story that tells them who they are they already know who they are they're here because they want a glimpse of who they could become and I love this moment because to me it represents everything that I wish that we were doing with our games actually having meaningful interactions with the NPC is not just through violence so in conclusion what do I want to see more of in our games I want to work on the empathetic side of AI on heart official intelligence I want to actually care about the characters in our games I want to find new ways of interacting with our games instead of new ways of hurting each other combat and violence is easy we've been doing this years and honestly it kind of feels cheap when you throw this into your game because we know it works so please let's

invest more time instead into finding new and interesting mechanics tools systems and AI to make games that feel new innovative and optimistic thank you hi everyone so many interests and I'm going to continue what appears to be this year's trend in terms of complaining about pet practices basically so how many of you have had this experience you're writing some system you're coding something you have to dig into some system maybe you haven't seen that system before and your immediate reaction as soon as you look into the code for this other system is something like this maybe it's this or what I tend to see more often is something like this we've all had this experience right we've all gone in and and you look at some system you're like man the physic system is [ __ ] in this game or the X system is [ __ ] whatever that system is hopefully it's not the AI system and so the

problem here is is that these are basically broken windows and that's that's a term that I like to use for this right there's there's these broken windows that exist in our systems the term broken windows as far as I can to deal with this kind of stuff was a term coined by James Q Wilson and George Kelling sometime in the early 80s 82 I think the idea is that there's a broken window and as soon as you have a broken window it crosses some psychological threshold that now this building is garbage and whatever will just let it fall apart and that happens in our code constantly how many of you have had to deal with that right where it's like oh [ __ ] it we'll just add another three functions in here I challenge you to do this when you get back to wherever it is that you work do a global solution search for the word hack and see how how long you weep I did that once on the sims 4 and it was enlightening so don't live with broken windows why are we doing this we should not live with

broken windows this wisdom comes from the pragmatic programmer by Andrew hunt and David Thomas everyone should read that if you're a programmer of any kind or possibly even a designer you should probably read this book it's a great book so why do we do this why do I live with this I mean I get it right there's deadlines there's bosses so you kind of get into this like mad coder mode right and you just want to get stuff done but I'm here to tell you that the dangers of this are very real I worked on the Sims Medieval and I can tell you that there was one particular file in there and this was only one example of the of the atrocities that were there and there were good parts of the code but there are also symbiote so it was thirteen thousand [ __ ] lines of code no one could find anything in this file and you know we would just keep adding stuff it's fine right the class was actually bigger than this it was split across multiple files what kind of monster does this the answer is all of us we were all guilty of it because it's like I just put it into sim just put it into sim class it's fine on the Sims 4 we I mean we wrote we wrote everything so we're

like oh maybe we should do this again maybe we should it was it was actually bad enough that it was one of our things that we talked about of like let's not have this terrible sim class and we didn't it was much more manageable we use components and all sorts of the fancy CS and and architectural terms but we had to be diligent I and I will admit this because I'm bashing on the code right now I will admit that I absolutely added about six or eight AI specific functions into the sim class and Sims 4 and then I'm like wait I'm being an idiot and then I created an autonomy component added everything in there but we had to go back and sometimes we have to refactor stuff sometimes we'd have to fail code reviews and say no what are you doing but this where it should be had to be super diligent now the complexity issues can rise I'm shocked this is animates correctly uh the complexity issues can arise right and these are these are other other types of broken windows and that's already been touched on by people who came before me and and various people around that if you're AI you have

this super complicated AI that's using you know this Monte Carlo tree search across a neural net of some Bayesian logic and does all this crazy [ __ ] but if you're AI it is indistinguishable from Rand and what are you doing or to quote Steve Rabin why are you employed one of my other favorite things this is actually a pet peeve of mine is the swap is the swap argument and I don't even know why this is an argument people people will argue about this [ __ ] they're basically there are a couple ways you can do swap right there's a one above which is the XOR style of swap which is fancy and sexy and then there's a lower one which is a temp variable like like who uses a temporal oh my god four bytes people love this method they hate this method and they're like why are we creating this temp variable but and since we in the assembly language barrier you're not like this is the actual disassembly of it in in 32-bit mode and and we're not using a temporary rule because we have this thing called an optimizer oh but XOR is faster well no it's faster six six out of earth it's faster four

out of ten times right the very swap is faster six out of ten times in just ten random tests of ten million that I did but really 10 million swaps what are we really comparing here we're comparing like look at this top one how much how much are you really saving why are you optimistic on verse a ssin and I mean at this point I'm just going on a tangent right and I'm basically just yelling at you guys and and all that stuff so probably I should I should just stop and and and have some kind of Zen moment and I know it's all a balance but but for God's sake please don't live with broken windows thank you I gotta ask did you do an architectural diagram for essence did you do an architectural diagram for sense yeah so in the 1970s there's this thing called the AI winter and before that apparently there was huge investment in AI

technologies and I've just learned recently that apparently we're out of REI winter we are apparently inside an AI spring according to Google and of course if it's on Google it must be true according to John and I'm going to screw up his name here I think it's G and Rhea we're in some kind of an AI spring I'm not entirely sure what kind of AI spring he's talking about possibly this one or maybe he meant just a really early spring but regardless everyone's talking about ml now maybe this is a spring for John I mean after all he's a head of machine learning for Google and I maybe I should be excited that we're all talking about machine learning until I get this question I mean really about machine learning is a pretty big field and now of course I would really appreciate that there would be a for dummies book and of course then I had to

Google to see if there was a for dummies book and yes there is machine learning for dummies that's how far we've come I'm gonna get that for my parents but but for me and games that despite a big increase in the number of cores on our consoles I'm not feeling that spring that red line by the way is the increase I've seen in AI budgets now I'm not against machine learning I mean there's some used for that in our games I mean is cool I mean it's not something that I feel particularly enhances my games and I do think you should experiment with it but what I'm saying is I'm not seeing a translation between the increased interest in AI to a commitment of the resources either in terms of people or in terms of real support but there are rising expectations so there is a little bit of hope at least in the AI game dev community I've never seen so many reviews actually start to

mention AI at least in positive terms and our top reviewed game for 2016 apparently was a simulation game and maybe we're getting away from all those combat games after all now the nice thing about those reviews if you go back into them is they're not just talking about how good the game is they don't do talk about the AI and they do talk about gameplay but here's the brutal truth unless you're willing to give me some of those cores and support me by sending me to conferences such as this one and give me time to build the right AI system then you know that awesome machine learning feature you were looking for yeah that's not gonna happen yeah I admit it I'm holding your AI hostage now I'm not asking for the whole pie just like my piece of the pie I mean you can keep the GPU rendering programmers but maybe for us in game AI it's not quite spring maybe it's just

slush season hello everybody my name is Ben sunshine Hill it does say these are Ben sunshine hills slides though I guess that was meant for me who you are so I'd like to start with a small disclaimer in previous years my rants have been fairly silly and light-hearted this year's rant is a little angrier and so I want to apologize to anybody who finds it overly severe and I assure you all they come from a place of abiding friendship and respect I wouldn't bother saying these things if I didn't think you could benefit from them it's also not going to be a very funny ranch and I don't apologize for that the title of the session is tearing Tantrums not let's

giggle at cat pictures oh yeah just one on word so I was reading an article the other day rereading an article by Jeff Orkin the article where he first introduced goal-oriented action planning for a lot of us AI devs it was the first exposure we really had to automated planning at least outside the CS class on stacking boxes with a robot arm it may not have been new but it was new to us and in the article Jeff lamented how the architecture and player of experience of game AI had been basically unchanged for the better part of a decade well go p-- didn't take the world by storm even in the article Jeff said that the results weren't quite as good as he'd hoped and about the same time behavior trees started to gain traction after they were used so successfully in Halo 2 and here we are more than a decade down the road and

guess what we're still using behavior trees and yeah sure I know they've come a long way since then but look suppose that you were starting on the architecture at the AI architecture for a new game and you told your team you know I really liked the AI in Halo 2 we should do and how they did it they'd say sure behavior suits modern beetee's look a little different but yeah it's a approach in contrast suppose that you were starting on the renderer and you told your team you know I really like the graphics in Halo 2 we should do it how they did it what do you think they'd say oh yeah the incredible graphics of Halo 2 we can use those newfangled vertex shaders Oh in the accumulation buffer for all the bloom we're gonna do they would laugh at you because we threw all free throughout all that crap when we found a better way of doing it and then we threw out that approach too because you don't write a renderer like it's 2013 let alone 2005

I am truly just made by the stagnation I see in game AI score technology base I think it is killing game AI as a creative discipline I believe we are killing game AI as a creative discipline I believe our model of what is and isn't possible is aweso fiying and by this we I include you we've all twisted that knife no let me refresh your memory some designer has come to you with some ideas she has some way in which the AI could serve the game and you've chuckled in that avuncular fashion and patiently explained to her that in game AI many things that seem straightforward are actually quite impractical you've said I will do what I can for you but you have to understand that fundamentally this is not something that a I can do and I'll go further I think it gives us a little thrill not just how condescending we get

to be that's old head now I think we get off on gaping at the limits of game AI of AI because it's this grand mystery right intelligence cognition is so innate to our everyday experience and yet we cannot describe it or explain it or reproduce it we try we fail but you know there's joy in the attempt and joy in the retelling of the struggle well no more I deny you that pleasure I rip it from you you are not worthy of that joy for 10 now you have worshipped at the altar of transparency of author ability of convenient elegant mundanity you have mistaken what is known for what is possible you have mistaken what is easy for what is practical and your self-serving pronouncements of what is possible and what is practical have become the walls of your kingdom a very comfortable place where you're never asked to do things that you don't know

how to do you spend your days and your energies on debuggers and user experiences sharpening the pencils of development you are a friend to your co-workers and an enemy to your craft and your and your actions regardless of your words argue that game AI does not matter that it cannot matter that what it is now is all that she'll ever become which means that in 10 years the player experience of game AI will probably be pretty much the same as it is now how do you feel about that does it content you and if not what are you doing about it what are you doing to save game AI because let me tell you nobody is coming to save us we've seen to that peter molyneux is maybe the last game

designed he comes to try to save us you know I've said a lot of [ __ ] over about Peter Molyneux over the last few years so let me also say this his showmanship his fabulus' 'm his assurance his idealistic assurance that if something ought to be possible then it will be possible have done more to put game aai on the map and keep it on the map then all you jerks put together but now it's up to us if game AI is to be anything more than an offshoot of tools programming we need to build it into something worthy of notice and the first step to that is getting over our ideas of what is and isn't possible we must do things we don't know how to do we must create experiences we thought were not creative I didn't say it would be easy but look either you truly believe that game AI has already achieved its apex or you

believe that it has unrealized potential if it's the first I have nothing to say to you but if it's the second well no time like the present time to go home put down the behavior trees and [ __ ] get to work thank you alright so after this I will just ask you if you want to innovate fine but don't leave the players out so is evolving fast and I am biased because I'm working on the watchdog's brand but we see it all around us and yes West world is awesome so now we had self-driving cars and smart cities and big data and robotics

and machine learning and it's fascinating and inspiring and it makes us dream of a future where an artificial intelligence is everywhere but we as game maker we want her AI to follow the same path we want newer smarter more capable agents and the we think AI will help us reaching a new paradigm as well as in any other industry and it's true but it's easy to be lost in all this novelty and great advancement of technology it's easy to get deep in all your AI can learn and create new things and fulfill its needs and completely leave the players out of the equation we are game makers we make games as art as experiences as entertainment our goal is to create an interactive world for the players to experiment in there are no games without the player games that can

play themselves players are not games they are simulation I just said might sound really abuse it is but easily forgettin when we make AI so let's say I try to simulate a town I have all these AI agents that have all these needs like real humans they can eat I have a job go see a movie sometime they have family friends coworkers it sounds great right sounds like I've given my agents roles to live a fulfilling life yeah maybe but is it a great game no it's not even a game yet it's a simulation and the players live outside of it so like no player out there give a crap that your agents can eat an apple and evacuate it later well unless it's part of your gameplay if the players must collect feces do I don't

know create compost and make plants grow I mean it might be relevant but otherwise not that much in other words player don't care how smart your agent is the care all well you're in agent interact with them our smart your agent looks to their eyes oh well your agents cell it's spent easy so we are making virtual worlds robotic engineers self-driving car makers big data analyst they are dealing with real-world problematics they have constraints that we don't have we create the constraints we make the world your AI should care about your game constraints not realities ones how many times have I heard fellow developers say that this is a design problem not a technological con and it's true but then why do I see so

many developers making their own AI system before even knowing what the design of their game is before knowing what it is that their AI is trying to solve in the first place and with what constraints if I put a self-driving car face it wouldn't make no sense to expect it to behave as it was designed to so the next time you try to simulate life or create a smart agent that can learn the alphabet ask yourself what you are trying to solve with it sit down with the designers and see what sort of a I answer best the needs of their game otherwise you won't ship a game at all just a beautiful tech demo thank you hi so for my rant uh I'm gonna talk about how you can't procedurally generally novelty so you know the Mandelbrot set is beautiful and we can zoom into it but after five or ten

minutes you sort of grow bored of it even though you can zoom into it forever but we do have great success stories and video games for a procedural content generation like SpeedTree where we can make beautiful scenes and such but you know if we want to create lots and lots of novelty well the trees kind of all do look alike we have no man's sky witch promise that we could go to millions of planets and actually no man's sky uses SpeedTree but you know if we really would love to go and see all these new creatures we've never seen before and first couple times it's it's amazing a couple like a month or two ago someone looked at the data files for no man's sky and and they use a constructionist approach and this is the Triceratops model and that's so they have eight heads five bodies six tails and you can see that there's a two hundred and forty different combinations you can see there and then when you put different paint jobs on them you put a belly paint job and a top paint job

right you can get to about ten thousand different variations of it now if we look at something like that you try and figure this out well it doesn't feel like we saw ten thousand different things right there's a spectrum of novelty from the the weakest where we are we've seen it oh we've seen something like it mildly interesting and then Wow and then holy [ __ ] I've just seen something amazing so really we don't care that there's ten thousand features we care more like how much novelty there is so rabens novelty number is the the number of Wow and holy [ __ ] items that you actually see so when I look back at this I feeling like I think the number is six right it's it's just it's just not as is what you thought it may have been Borderlands amazing game of course has a lot of weapons right and they use the same constructionist approach where you you can have all the different pieces and such and it has so many gain

so many different guns in this game that the Guinness Book of World Records has certified it and I know what you're thinking like the official Guinness Book of World Records and I'm yes that's that Guinness Book of World Records they've certified as having over seventeen million possible guns and then Borderlands 2 came out and it has even more guns but Randy Pitchford of gearbox actually said there are enough where it doesn't matter right because you just get saying these high numbers is kind of silliness your clever algorithm is not going to make infinite content right it's well yeah because what you really want is infinite novel content right so it's not going to do that your clever algorithm with n novelty will make something like n novel content and this is something like the cold mug order of

complexity I figured out how to pronounce that by checking the internet but that's like descriptive complexity algorithmic compete program size complexity really you're only gonna get the the novelty that's in the original code that you put there it has to come from somewhere maybe you could scour the internet for novelty but it has to you have to put it in there and like SpeedTree has been around for 17 years 17 years of development of refinement adding novelty to it and so it is really good so no clever algorithm is going to give you this this infinite novelty I I think what I just comes down to it you have to make the novelty so that's my rant I'm just gonna leave that guy up there for a little while while I take a drink of water because it's just fascinating to look at it's the face that makes it I'm covered in spoons okay now it's

starting to get creepy we have talked a lot about procedurally generated content the the the craft of it the effect of it not only here in the past two days but actually in these halls and and it's a kind of thing that is being discussed a lot more lately let's face it including I Steve imagine that and you know what Steve's right kinda procedural content isn't a magic bullet it isn't going to be that one thing that just penetrates right through it and you know solves your game and gives you all of that novelty but that's actually not what procedural content a lot of times is used for its use case really it's a little bit more like this where you really need to spray a lot of content around your world because of design you know and asset pipelines and stuff so you just want to fill your game full of lead unfortunately of course a lot of people think that while you're gonna burn the place to the ground or at least drop

very bad bombs in places and so certainly designers and others get kind of defensive about the use of procedural generated content but be that as it may what do we do about this though if it's not a silver bullet and it's it seems to have some use so here's my advice triple-a Studios seriously just stop wringing your hands about you know oh my gosh should we do some procedurally generated content we got a budget for this you know Indies Indies stopped pouring over blogs and articles and message boards and Reddit and say how do I put PCG in my game how can I make a game that does this just seriously just you know stop academics count I love so many of you so many of my great friends and you do amazing work but stop researching PCG just seriously stop tormenting yourself enslaving your grad students and chasing the holy p-value in the name of academic research about PCG all of you just stop doing all of those

and instead just do the damn procedural content already no I'm serious I by way of anecdote I just came off a contract where I was working for a company called pixel mage games on a game called heroes song we made it to Steam early access had to pull it because there's some business concerns and whatnot but it did see the light of day for a while there and the thing was a massive amount of the game was PCG for example big breath we did procedural world gen where we made continents and had many different biomes with rivers and lakes and we did it through the generation of tectonic plates that moved and created elevations we had moisture levels and wind movement that blew things around that gave us those biomes and our worlds were massive 10,000 square kilometers to generate a world maybe they look kinda like this and every single one was different they were really cool you say oh look at this how did this valley here we did procedural terrain across that we had grass and

rocks and dirt and trees and all sorts of things even flowers us lettered around the world and that was all based on whatever biome you were and we could actually even blend between biomes as you walked across the world gradually transitioning from from grassland to Plains to forests to mountains a procedural civilization placement in those worlds that the civilizations had capitals and cities and towns and villages and even small camps littered about the place and we had hundreds of them in those worlds and it was all based on the size of the civilization what kind of coverage were they were a global civilization or are they just regional what preferences did they have four biomes because when dwarves were in the mountains and the elves were in the forests and and the goblins were over here um and land feature preferences did they want to live next to lakes and rivers or along coasts or out in the middle of nowhere and what kind of neighbors did they prefer to live next to that all factored into where we place civilizations in our world and so you can actually look at the map and see all these little this is my debug map by the way all the little colored dots of a Hey

look those are cities and towns and villages and everything and these guys are over here and these are over there we did procedural history generation on these civilizations where we could have wars and assassinations and alliances and trading pacts creation of epic items and loss of a Cappetta items that is important by the way um and this was all based on proximity to the habitations the friendliness of the civilizations to each other the ebb and flow of relationships over time and speaking of time we simulated 10,000 years of history when you generated your world where's Tarn are you here Tarn Adams I love you I love you brother and no we weren't copying your okay we a little we did procedural Road generation in those worlds then between the habitations and even two points of interest in the world like dungeons so people could find them easier we had different sizes of roads some of them were huge between capitols like the Kings Road and some of them were just paths going off to camps and dungeons and whatnot and those were of course

based on the proximity how close were we were to these cities what were the friendliness of the civilizations allies head roads running between them not so much to your enemies because that's dangerous and then of course we use the biome traversal costs you know you don't want to run a road through the mountains unless you have to and then so if you look carefully see the little black lines we had roads all over the place networking across these massive worlds of ours the cities were procedurally generated we had all sorts of different types of buildings I think we had 2530 different types of buildings and we placed them procedurally in a place where we knew that there was going to be some sort of habitation and it was all based on the town size the racial preferences for the types of buildings that would be in this races city and then we had to work around the terrain features what if there's a river running through here we had to place them procedurally around the river the buildings and the dungeons themselves his building shapes and yes we actually had multi-story buildings that conform to the exterior shape building interiors were procedurally generated with the

placement of objects in buildings the dungeon interiors were all procedurally generated no dungeons were ever the same not just modular rooms were gluing things together but they had nooks and crannies and obstacles and everything in the dungeon so that they were all completely unique and you didn't know what was coming next but they looked real the population of the world was procedurally generated the creatures that were in it the humanoid creatures as well as the animals town inhabitants when we created those towns with those buildings we staffed them objects that were littered around the world for crafting items loot all sorts of interesting things you could find and all of that was based on what biome you were in you would find different things in different places the influence of the civilization while I'm in dwarf territory therefore I'm probably gonna see a few more dwarves and what the neighbors were well we're kind of near old goblin territory and they might be invading oh look if there are some and then the rarity are they something that's common or a little bit uncommon rare or are they something unique and then we'd also space them out so that

not all of the difficult stuff was right next to each other all procedurally at run-time names everything in the world had a name not just the characters and the cities but geographic features you could have a forest that had a name sometimes more than one because different races called it different things mountain ranges etc had names and they were based on designer created templates that we would sort out by race sub race gender whatever we there's like one example of that template right there or prefix with the little apostrophe and first and last and suffix so you could have tens of millions of unique names and just using that template for example you could get things like this how do you even pronounce it I had Dave mark master AI Steve Rabin dude and Alisha later wow what are the odds huh cool anyway procedural dialogue in this game now we had text instead of vo so we could get away with a lot more here we had greetings friendly neutral hostile whatever we had taunts we had barks we had information about the world that the characters would say not only to you but

to each other and you could hear worldwide information regional information about the town or even just local dynamic events that were happening they may just tell you about it some of it was generic a lot of it was racially specific they sounded like that particular race they had their own speech patterns etc and we used templates to create these so that they could insert information like race gender name class they might call you by name they might call you by a refer to your race they might refer to each other by their class whatever and we filtered it by all of the above including the disposition they're going to say completely different things do if you're hostile compared to friendly all filled into those templates procedural information about the world that was constantly being generated at runtime using what was going on in the world referenced by the generated names in the procedural dialogue so you could actually hear things like we've heard reports of Knowles to the southwest and if you went Southwest dammit you're gonna run into Knowles they were telling you about this there are goblins attacking the east gate of the city the

orcs are massing in the forests north of the river they would fill in all of this information items in the world informations about the items the world be using the game world information referred to or by the names in the procedural dialog with varying degrees of specificity what do I mean by that the magic tome of amazing AI is rumored to be lost far to the west well that's not terribly helpful is it but as you get people who know more and more about this situation they would actually add more specificity to the point where you would get to something along the lines of the max magic tome of amazing AI has lost far to the West in the dungeon of Moscone near the Californian city of San Francisco where it is guarded by a horde of AI programmers and that was the god-honest truth because they knew it was in the world and they were signposting the player hey there's something interesting over there in Moscone and it's guarded by a lot of AI programmers that was all procedural we even had quest procedural quests that were constantly being generated using

the procedural items placed in the procedural dungeons in the generated world using the names and referenced by the dialogue so we had all of this stuff all of this procedural I did a lot of it just by myself and there was two or three others on the team that worked on various portions of it my point is not necessarily to break okay well it was actually really kind of cool my point is this we did all of this in one year and that's on top of me actually doing the regular game AI that I had to do for combat and for moving around and whatnot and by the way all of the characters knew how to respond to detect and process all of this procedural information how did we do this we didn't write a thesis we didn't do years of research no hand wringing no scouring blog posts or pleading for advice on forums just looked at the problem one at a time we thought about it for a little bit and said well let's figure this out and then

we just did it because it's just programming so again I'm not here to brag about this there is a bigger message here I'm here to pump you up about the fact that you know what you can do this also you don't have there is no such thing as an expert in PCG I'm serious stop all of the hand wringing and researching and just do it because it might make things kind of better spray in some content around your games and if you still don't believe me if you still believe that you can't do it in your Studios it in your games I know a guy might be able to help thank you [Applause] you