Crackpot AI Devs: Unproven Game AI Techniques that Might Possibly Work

[Music] okay so uh and my my mic should be working there we go so welcome back to uh the AI Summit hope you had a good lunch and so so we have uh this great new session coming up but before we do that uh silence your cell phones and all that and and don't forget to uh fill out the uh the the forms for the evaluating the uh sessions as well okay so let's just get uh started with this thing so this is this is a first time we're doing this session at the AI Summit it was uh an idea I had and it sort of embodies what I love about GDC I love coming to GDC to meet a lot of people but I also love to be like inspired and so with this session hopefully uh someone in here at least a couple people in here will be inspired and in order to inspire you I've I've assembled a crackpot team of uh presenters here who will share their crackpot ideas with you and so let's just go ahead and and begin okay

so for my crackpot idea uh I started with this thought experiment how would you get a first-person shooter bot to move exactly like a human player does say in death matches and and such you want to emulate different skill levels and play Styles but what I'm trying to get at with this crackpot idea is you want to emulate the route and the movement strategies and exactly where um people are facing their guns and and and such and when you start talking about you know emulating people in exactly right you're almost talking map at that point and map has some interesting things about it it captures the subtlety of motion because you're recording it and you don't even have to understand why the people are doing what they're doing and yet you capture all of that those subtle motions that's why map is so so well used in say sports games for example but I am not suggesting that we do this in firstperson shooter games that's too crackpot right for I I'm a Call of Duty player and I what would you have to do you have to like make a sound stage and then actually rebuild a Call of Duty level and then have and moat

people through it that is that's pretty crackpot so but that doesn't even solve the right problem because that's not how humans play the game human play humans play the game with a controller on their couch in a reclin position so that's that's that's not what we're trying to capture it but this image here um embodies uh my crackpot idea and it was sort of the inspiration and so if you record a lot of people playing through a game say a first- person shooter and uh their routes and exactly how they're moving and how they pause and aim their gun and everything you have a lot of data like this and uh the different colors might represent different play Styles and if you have enough data Maybe not big data but enough data um you can then find little crossover points between these and the AI can maybe play back some of these and infinitely uh recombine this data in order to give early realistic player uh uh kind of uh Behavior so the recording strategy I'm advocating is that well uh people are moving through a level humans are moving through a level you're going to record the route you're recording the pace the

style of movement any pauses any hesitations and uh the Gaze and weapon direction are critical obviously because as you're going you know in these levels right you're going around corners in just a certain way aiming just in certain places hopefully right where someone's head will be and uh of course the weapon that you're using is very important in this part so the highle strategy once you've recorded all this stuff um you've recorded the humans um you'll need to tag the Styles somehow uh you you could probably write algorithms to this do this or hire interns of course uh you want to find these crossover points Within These Styles uh in order to be able to switch between them and then the AI plays back these recordings they randomly switching between these Snippets and crossovers so let me just uh kind of show you this in a diagram form here's someone's playthrough uh in blue there and here's another person's playthrough and maybe these are all the careful playthroughs we've tagged them as careful and then we can identify places where they cross over and these are the spots where a be playing back play through and then

decide to switch to another one just kind of the threads I showed you before and similarly we could have different play Styles aggressive one with the crossover points and so we have all this data for the AI go through and and perhaps choose and uh could maybe even switch between these different styles and if you want to find the crossover points for that and so um just a week ago I discovered this little thing on the web it's called the infinite jukebox and it uses the same idea basically so for what this is is it's you load up a song here it's Billy Jean by Michael Jackson and it starts at uh the at the 12 o'l position and goes clockwise and this is the whole song and an algorithm analyzed it and found crossover points in the music and so every time you see one of those arcs it randomly decides uh yes or no to take the arc and uh so it's recombining the the recorded music so that you hopefully can't tell it's even changing to the different part of the song and because it's bouncing all around it will play the song Forever infinitely and it's basically the same idea that I I'm I'm advocating in fact I

I think that strategy is so general just like I showed you with the music that you could probably use it for a lot of different things so you want to record humans somehow tag the Styles critical to find those crossover points within the Styles and then you're just playing back the recordings randomly switching between the Snippets and Crossovers and probably the case is the more data you have the better this would actually work and so that's my crackpot idea okay hi I'm Alex um so there are five stages to dealing with a crackpot idea but first I'd like to point out that this is a completely sensible idea uh these guys are crackpots but this is a sensible one um in fact I think there are no crackpot ideas in machine learning uh in general so anything technical that you can think of within 6 12 months 18 months it'll be done so as an example I just want to uh dig into image generation

so in June 2015 researchers at Google figured out you could inverse a neural network and get some interesting visualizations and patterns that came out of that then in August 2015 researchers in Germany figured out they could reverse two neural networks and get the style from one transferred onto the other by mashing them up in the middle it's called style networks then in 2016 just January um researchers also from Germany figured out how to transfer these Patches from one image onto another using a neural network which blends them together in a more visually pleasing way so it's not just a mashup and you can take an image like this and apply it onto an image like this and generate a circuit board City for example so combining them together generates um new variations of of content for example and then just a few weeks ago we showed that you could take the same idea with annotations like take an impressionist painting and then annotate it and doodle things on another image

and you can transfer the style from one to another and generate paintings like this it only takes a few minutes and it's pretty crazy but it works so machine learning has no uh crackpot ideas but the really annoying thing about this is that uh it's not the technical thing that will hold back your ideas it's the actual implementation getting your designs in and uh getting them into production those are where most ideas Fall flat and that's the the most annoying part about having these crazy ideas is that you have to get them by the producer so why not instead apply artificial intelligence to fixing those production problems you know Screw the system Let's uh let's let's figure this out so I have a deal for you um let's take our traditional art pipelines where we take our artists we put them into the pipeline and they generate art that's kind of a simplification let's say of the process let's turn it into a generative pipeline instead where we um create annotations

we call them reference art and then we're using a neural network a patch-based approach to recombine bits of these images and because it's the neural network it has some sense of what fits together visually and it Blends things in a very pleasing way then we have our level designers that are basically creating um artfree but annotated Doodles of levels for example and then generating the final content on the GPU so the good thing about this is that you could potentially use it as an interactive tool to guide level designers in creating levels that are visually pleasing by simula rather quickly or instead you could do a high quality HD render overnight so whatever technique works for you we can we can make this work right there's lots of different options now the sad part about this is that it's going to involve a lot of changes and uh certainly a lot of members of the team and the art teams will have their roles jumbled around a few people may May um no longer be

required their skills May no longer fit if they can't work within this system um but there's new positions that will open up as well so even though the Nostalgia of working with individual pixels may disappear or you people will be even more nostalgic about working with pixels you'll have artists that are working on creating style that transfers well to levels and uh junior artists that may be embedded into level design teams to make it better suited to style transfer or even technical artists supervising the whole process and making sure the quality remains high so I think these types of ideas uh their time has come for AAA games or indie games there's always money to be saved by doing it quicker even if you're taking programmer art and turning in it to 90% Quality Art and there's a lot of value in that um so it's an open source project you can go and download it it actually does work it's a you know the resolution and the performance will be improved over time but it's already a pretty good

proof of concept so I encourage you to go and download it and try it out and uh see if you can get this actually into the real world that's the exciting part thank [Applause] you okay hi um Brian I'm going to go right into it because I have a bit more crack than most so um my first one I called non CPU AI data manipulation this actually isn't too much crack potty cuz I've done a little bit of it like using influenced map techniques on the GPU using shaders you can do character and environmental craziness have a whole slew of them that get all mashed together additively multiplicatively with booleans all sorts of craziness on the on on the GPU and come back out and and now you have a whole slew of better influence map techniques that you can do directly on the Shader or you could even go crazier and start doing computer vision or more

advanced image analysis using compute shaders um there's so many different ways to you know use segmentation to determine territory or resource areas um Edge detection could do front lines between armies or even Financial front lines in particular simulations um you could find a number of different features like high value features like cover points or or resource bottlenecks and then full-blown pattern recognition in in a more transient way so that you could potentially find map SC events like there's an ambush happening to me or or there's a uh an opportunistic moment where uh my my guy has seen an unguarded resource or base that I need to take care of immediately and then again as as Alex is mentioning um one of the things I'm doing in my hobby time is is learning some some deep learning stuff because there's new gpus are coming out that that are really really ideal for running large scale neural Nets like like in real time it's actually pretty exciting

stuff my second idea I call APB this is pretty damn crack potty and this is my this has been my kind of hobby I'd say for about 15 years it's like a mental mental game that I play with myself and this is asymmetric player-based AIS it's sort of like Ender Game On Crack take two or more games and you use the players of each one of them to provide the AI for the other game without their knowing and what the thing is is the reason I say this is a crackpot idea but it's actually kind of fun to think about is that it forces you to abstract the inputs and the outputs of your AI system in a very very hardcore way to try and determine if you have two games where you can cross over like that and so you identify is the the particular sort of temporal flow of the inputs and outputs what types the the the dimensionality of the things that you need and you can actually find some pretty decent matchups like an endless Runner game could based on how you uh um Traverse through an endless Runner

level could be determining which uh uh fighter fighter moves that you're playing against and feed that into a fighting game and meanwhile the fighting game guy could the the the moves and what and when he blocks and that kind of thing could be determining the actual level that the endless Runner is playing sort of feeding it into the procedural system to to drive the actual level that the endless Runner guy is is running on same thing with a match 3 based on which colors of Beed things that I hit or the combos that I do I launch particular waves of enemies of particular types to an AI director system for a firstperson shooter or or a full-blown shoot them up and then in however the human handles those waves of enemies the order in which they killed him how much life he has left you can then feed that back into the random block generator for the match three and get yourself a nice little match up that way likewise you could see something with a tower defense and a and a raid boss sort of situation the spells and behaviors that the party is

using on the raid boss become the creeps for a tower defense game and on the other side the way in which the user handles those creeps become the raid boss's AI That's crackpot and my last I call the AI long con this one sounds kind of weird but I I enjoy it make building AI a rewarding player activity so uh imagine a game maybe it's a a large scale persistent uh um like Asteroids game and you you have a home base and you have a number of small ships and you can fly around you can collect resources you can slowly gather up uh uh technology you can do all this and in the meantime what you can do is like give a a small script a small AI script to any one of your little robots that you have in this in this world and so you can pick one off the shelf or you can make your own little quick little scri quick script for it and we can actually give you premium currency for

for doing so especially if you make your own little script and then you put it on the on the the list of available scripts and other people start using it if it's popular maybe you get more more money from it that kind of thing just to sort of incentivize actually do doing a little bit of AI yourself um obviously the Bots have to exceed some Fitness function that you write or like I said maybe it's some sort of like crowd crowd thumbs up thumbs down sort of a situation but you have to worry about collusion then and I and that's and that's weird obviously this is good with shifting metagame so that it forces the boders to kind of like reevaluate from time to time and then plus you don't have to worry about massive optimization over time well I don't know if that's a bad thing and then eventually the boders might actually start making Bots that make Bots um the thing that I found over time is that if you give premium currency to anything people will do it and so the reason I bring this up is because I honestly think that if you did something

like this in a game people would start creating AI to get that to get their stupid little internet points and this is the thing like like you start to say oh well but you're having human beings do the AI and it's like that's what I want because the the reason I call this the long con is that I think over time if we just have every a larger group of people just try making an AI just try having some control over something that it's going to give them that that little bit of taste that they want that that'll keep them going and keep keep them making AI in our games and eventually maybe that'll be a Direction that's good so that's my [Applause] stuff okay so we tend to focus a lot of time on the sexy front-facing agent intelligence the stuff that the players interact with at runtime and rightly saw however I believe there's a lot of

scope to spend more of our AI Budget on the behind the scenes the development workflow and tools so this is what a typical modern level editor looks like currently or God forbid you're using something like this they're highly complex software and they require a lot of expertise to get the most out of them however there's a problem they're all designed about getting polygons into a virtual world they're not designed about getting game design principles uh iterated on it's like forcing coders to write Assembly Language directly or op code manually instead of high level programming languages perhaps this is why games get better and better looking every year in and our gameplay tends to have plateaued unlike board games which have uh seing a soaring rise in in new potential perhaps we're just not using the right tools so I think game Edis should look something like this oh wait maybe

this you don't need designers spending hours manually tweaking the placement of geometry we want them to instead think about the design intent of the space at a higher level so I'm talking about embracing designer gener designer director generative content here the designer uh could sketch out how he wants the level to play uh and tag certain areas with particular Styles or encounters and then an AI system can interpret that sketch it's a little like Alex's neural Doodles and uh actually generate the the level geometry uh place the decorations place the encounter spawns and so on all the grunt work can be automated something like grammar or L systems are popular example of them uh they would allow artists to develop rules for laying out particular Styles you could come up with rules for a French cafe or an Italian restaurant or an Irish Pub designers then just pick the style they want tweak some generative parameters and off you go

voila instant level you want that level to be slightly bigger 10 m larger no no problem just tweak the size and it automatically lays it out for you no more spending 3 days kind of moving and repositioning every little bottle on the Shelf of the bar now this isn't rocket science it's not groundbreaking yet um I'm not sure why we're not doing this more to be honest we've seen some uptake for foliage placement speed tree and that um there been some work on building forar generation but nothing at the encounter level gameplay and that's where I think there's some room for AI techniques now grammar are effectively a way to semantically break up the space and place objects um so there's no reason why we can't automatically apply these semantic tags to those objects and areas in the game we could feed that into our agent systems our the bo system for example the last of us do this right and they say he's behind the car um we can also potentially feed it into the

behaviors and have richer information to reason about at runtime let's Crank It Up another notch grammar can be tedious to author so what if we could train an AI system by looking at designer made examples artists could lay out specific example areas and then we can get the AI to learn the style of those areas and transfer it onto the sketches for that the designers have made if we can learn paintly styles from uh with deep neural Nets why can't we do the same with layout grammar I think that would be pretty it let's Crank It Up another notch so your QA should be playing through the game constantly during development so why don't we have ai systems monitoring their playthroughs we can try and classify different play Styles uh learning how the players encounter different areas of the game uh we can retrain this constantly during development and uh adjust for design uh design changes and and balance changes it could even help highlight uh degenerative strategies early on and develop Vel mment rather than catching them later

on we could also learn how the movement speed and the engagement ranges affects the play and we could feed those metrics that you've made up your your game design rules about feed that into the generative system and generate layers that are optimized for your specific game mechanics with systems like this we can have designers focusing more on how they want the level to play instead of dealing with the tedious minutia of placing all these individual decorations in the level they can work about the int of the design and the intent of the space and it will allow them to iterate very quickly and be more efficient so the bottom line is we can use AI techniques to make Smarter Tools and with Smarter Tools we can make smarter games thank [Applause] you all right my name is Tobias Caron and I work in Microsoft and my crackpot idea is to explicitly model the play

experience as a story so I got this idea from uh listening and thinking and hearing about the Encounters in Halo particular Halo 2 and Halo 3 in Halo each encounter plays out a story and it's the same story in every single encounter and I was thinking wouldn't it be more interesting if the designers could customize their a story for their encounter and explicitly mod that exp that story as the players experience so the way I'm thinking this would work is the design marks up a timeline of the encounter and on this timeline there would be significant event and this is uh where the designer say this is how the experience would change uh so this is example of uh a mockup I made what might look like on the left here I have what I'm calling the O crap to rocking scale so the Des the uh player starts to encounter reasonably confident after a while something happens perhaps uh reinforcements arrive and he drops down

to three which is definitely in the O craft territory uh the player eventually deals with that and he bumps up to point8 and he's probably thinking he's rocking pretty good and then something really bad happens like an um a boss or some big monster or a tank or something like show up and he drops done all the way to 0.1 and then the player eventually Works him himself through this uh encounter and finishes is all the way at one so the way I imagine this would work is that we have an AI manager that tries to manipulate encounter in order to match the decided uh scale and um uh the a manage will do this by spawning more or fewer uh reinforcements or just weaker or stronger enemies uh changing the AI aggression uh weapons vehicles and tool used by the AI for instance an AI that can revive its uh Fallen comrades or

frow grenades will put more pressure on the player while if the uh AI refrains from doing the same thing it will ease up on the player and as always we have the standard low hanging fruits like changing AI shatter and animations uh music and other emotional manipulators so the idea is that the designer sets the limits and the uh AI manager acts within those limits so the the designer may say that you're allowed to spawn two to four enemies here and manage will have to spawn two three or four enemies and can't spawn the whole Battalion or none at all and this still allows the designers to script up uh say various uh approaches to a particular encounter so if uh the designer wants to make it hard if you attack the encounter head on versus sneaking in the back door we can use very different timelines or just uh different limits for they I manage to work with so the whole point of this is to uh think of an encounter as a story

and explicitly model the encounter as that story so we have something in our game that actually shows the uh story and what the encounter is about as opposed to what most games do today where it's just a collection of spawns uh geometry and perhaps a scripted event or two so at this point you may thinking that the left for days lead for Dead's AI manager is doing something very similar and it's sort of true but the difference here is that uh Li for Dead's air manager Works an entire playing session at the time and it's uh it's non-scripted and an algorithmic approach and more importantly uh it uh it deals with pacing it doesn't really deal with how the story plays out that's just a side effect of the pacing and the setting also Dynamic difficulty systems uh do something similar as it tries to manipulate how the um player experience uh it's en counter or the entire game

but again the goal is something different so the advantages of this is um first of all it allows the designer to explicitly Express his or her intent with the encounter and because we have this down and explicitly uh written down this is something that can be shared with the rest of the team and you can uh now H have this vehicle to actually look at this and discuss what this encounter mean for the entire um for the entire uh game and with uh the rest of the team also uh of course it does allows the encounter to be more interesting for a wider range of players and uh playing STS so we can uh ensure that more players actually experience the encounter the way we intended and finally it makes uh quality insurance better because not only can the QA test now get play for the count say that it works they can also look at the timeline and Ensure sure that it actually matches

the desired result you see when you play the [Applause] game thank you uh hi everyone so I'm going to jump right to it I absolutely love procedural generation in games I've given talks here at GDC about procedural generation in games and how much I think that it's great I also love story in games story is a huge part of why I play games and why I enjoy them so what if we could marry those two things together we kind of already do this a little bit with things like Skyrim is a good example um but really those are tertiary quests and things that are kind of push you out in the world that's not what I'm talking about what if we can generate something like Dragon Age the entire story what if we can analyze the works of JRR

tolken and build a derivative work from that or write have an AI write a lost Stephen King novel I think we can it's all about studying and analy and analyzing stuff right we analyze story we've been analyzing story for centuries this is the hero's journey this was popularized by a guy named Chris Vogler um he took that work from Joseph Campbell who created a hero with a thousand faces who in turn was just looking at myth he was just studying myth we pulled structure from the way that humans tend to tell story we know that somewhere in the beginning of the story we have to actually learn who this protagonist is and care we know that at some point some shit's going to happen and it's going to throw his world upside down and if it's missing we miss it many of you probably recognize this screenshot one of my biggest problems with Dragon Age Inquisition is that the inciting incident was on the main menu

as soon as I pressed that menu button boom right that's the inciting incident I spent the first 15 hours of the game with absolutely no connection to my character from a story point of view I was like I'm going to run around and attack things yay and that was great and it was fun but the story for me didn't kick in until after that compared to Dragon Age Origins where it's like I'm a Mage and I'm going on the heroing and I got some people who maybe I care about or maybe I betray who defines that defines what my character is so I think we can build an AI that looks and says hey right about here something needs to happen something kind of big that forces me into the special world or about halfway through the story something really big needs to happen that spins everything upside down and maybe that should relate to a thing that happened before so can we take an AI and analyze cre some creative work and drive other work from that well yes we already are David cope is a brilliant man who looked at all of the works of uh Bach and

Mozart and others and has created effectively a Bach generator you can actually find it on his previous if I go back here on his site you can go find it and you can listen to like what sounds like the Lost works of Bach Jeremy leech is another one who's done very similar things it's a different technique but it's the same idea so we can totally do this the branching factor is admittedly considerably higher but we can absolutely do this so my final questions on this one are when can I have this cuz I think we all want a game master who can just be this computer and can tell me Dragon Age stories forever and mass effect effect stories forever but my other question is should we actually do this what does it say about our own creative process and our own creativity that we can create a machine that does it for us thank you all right thank thank you guys um one observation please raise your hand

if you actually kind of tapped into this a little bit yourselves is that we were like halfway to Skynet The Matrix and the singularity at some point or another during the past half an hour wait until they come for you mark anyway let's thank them once again and I'll see you in 20 minutes for um what is it it's Eric Eric Martell and uh the n it's a movement thing it's really cool so let's thank him once again [Applause]