Throwing Out the Dopamine Shots: Reward Psychology Without the Neurotrash

[Music] Okay, so thanks for coming to session. Uh, throwing out the dopamine shots, rewards without the neurotrash. Just to get started, we've got the housekeeping stuff. Please turn off your cell phones or put them on silent and please do the evaluation afterwards. Would be really cool. Okay. Uh, so my name is Ben. Uh, I'm a UX researcher at Epic Games. I have a PhD uh, in psychology. I've worked in neuroscience departments and uh human factors departments and I'm a Kiwi. That's the nationality, not the fruit or the bird. Um you can reach me at that email address or on Twitter or I like playing games. So please feel free to add me on any of these services. I kind of hop around between them. Okay, so that out of the way. Who's ever said or heard someone say that a a game mechanic gives a player a shot of dopamine? Hands up. Okay, a lot of people. So what does this mean? I want to talk about that very quickly. Dopamine is a compound. It's

found in various parts of the body. Uh it's a precursor to adrenaline, for example. So anywhere that adrenaline is produced, there's dopamine. But most people when they're saying a shot of dopamine are talking about its use as a neurotransmitter in the brain where you will find it. And I think what a lot of people talk about when they're saying gives you a shot of dopamine, they're talking about pleasure or a good feeling is what's occurring right now. And based on the science, this is an idea that's about uh 30 years or so old. And it was based on an idea that of course if animals are going to work for a reward, it must feel good because none of us work for things that feel bad, right? Um and that was kind of the idea when it came around. And certainly if we look at dopamine and its impact on animals, uh if you genetically engineer mice to not produce dopamine, you get a different type of behavior. So the mouse that's running around, that's normal mousy behavior. The one that's just sitting there and letting the other one walk all over top of it is uh dopamine deficient,

not producing any dopamine. And certainly these mice, they lack motivation and they will eventually die unless you make sure that they eat enough. So it certainly does something. The idea that it's responsible for pleasure and for liking rewards uh has fallen fallen away within the scientific community. And there's various reasons for that. One is that with animals, they still showed that they get pleasure that they like things. If you give mice a choice between water or sugar water that don't produce dopamine, they'll choose the sugary water, suggesting that they still get pleasure from things. If you're not convinced by animal research and want people that can actually talk to you and talk about the pleasure they're getting, you can look at Alzheimer's patients. Alzheimer's is uh defined by a drop of dopamine, but these people still report pleasure. They still report liking things. That's my father, by the way. He doesn't have Alzheimer's, but he looks like he's having fun. Um, so what so the idea that dopamine's to

do with getting pleasure or reward system and all this kind of stuff has kind of fallen away. uh and it fell away quite a while ago. It's just the popular way that it's talked about has not um and even the people that originally put forward the idea of dopamine being pleasure have said okay it's not. And what followed after that was a some thinking that it's to do with learning and this was based on the idea that in the brain just before a reward is delivered there's a increase in dopamine. So you're anticipating something happening. Now this is interesting but there's some problems in that dopamine deficient mice have been able to learn where rewards are in a maze. uh rats that don't produce dopamine have been able to be uh operately conditioned to learn new things. So maybe it's got something to do with learning, but it obviously doesn't look like it's necessary. So where most people talk about dopamine uh in the literature nowadays is around wanting or needing or compulsion. Uh the idea is that dopamine has something to do with uh a real magnetic draw or need for something. Um, often what's talked

about is in Europe as junkies, particularly for drugs like meth, once they take a lot of it, they stop getting pleasure pleasure from the drug, but they feel very compelled to still get the drug even though it's an unpleasant experience. And certainly this is reflected in change in the dopamine system. So if you're saying your game gives people a shot of dopamine, you're not necessarily saying it's fun or anything worthwhile to them, just that it's compelling them to play it, which may not be a nice thing to say. Um, and even if that was the case, it's really not that simple. If our brains were simple enough for us to understand, we would be so simple that we couldn't. This is a nice uh quote. It doesn't necessarily mean don't study the brain. Of course, like that's silly. It's more like it's not that simple and we need to be careful of simple explanations like a shot of dopamine. These simple explanations are very uh appealing to a lot of people and there's been a rise in neuro things in the

recent years. Neurotraining, neuro food, neuro exercise, all these things are real things and there's a lot more of them. You can pay money and have people do these things. And the reaction the neuroscience uh community as a large has been to declare these neurotrash to say the brain is more complicated than this and all this stuff is preying on on uh cognitive biases and praying on people with money. And I want to ask us uh here at GDC is why do we care about this? Why do I hear people talking about neuroscience here? Well, first of all, science. Science is cool. I like science. I have a PhD. I think data is really cool. I think science is amazing. And I think look, you can put a put a piece of broccoli in fMRI machine and it looks awesome. So that's cool. Science is cool. But if we look at a statement like this, rewards delivered on a variable schedule strongly motivated players due to activating a complex web of neurons resulting in various neurotransmitters being released. Okay, but this is what

we care about, right? If we're making games, we compare care that what we can do, rewards delivered on a variable schedule and what the behavioral outcome is, strongly motivated players. So, I really want to say let's throw away the dopamine shots. Let's not really care about neuroscience, not because it's not important, but because what neuroscience is doing is it's explaining already known principles. They want neuroscientists want to know the physiology behind these principles that already exist. They're often not finding new things other than the physiological pathways, which are interesting, but from a practical standpoint, it's not that useful. So, let's get rid of those. Um, if you do want to hear me talk about ways you can measure physiology and stuff like that, uh, look up my Twitter handle. I gave a talk about it a couple of years ago at another GDC. But I'm not going to talk about biometrics anymore. I'm going to talk about rewards, motivation, and what we already know. So I'm going to start off by talking

about intrinsic and in exttrinsic motivation. And this is not because uh I feel that these theories require particularly important u attention paid to them. Rather that I know that they're very popular in in games right now. Who's heard of intrinsic motivation and extrinsic motivation? Okay, nearly everybody. Right? So this idea is a popular theory in games at the moment and it's the idea that intrinsic motivation is personal gratification. intrinsic motivation is to get something from the environment. Now this is just one model, one theory and to give you an idea, this was my wall during my PhD. My PhD was on just one area of human behavior which was driving behavior. And this was a snapshot in time of every single model or theory I could find that had been made to explain driver behavior. And there was ones I didn't have up on this wall. There was a lot of them. And there are a lot of them. And that doesn't mean that they're all worth worthless or we shouldn't think about them. But as applied psych psychologists

which such as myself and working in an applied area like game design, what we can do and the advantage of this is we can look at all these models and we can find the commonalities between them because if lots of models are agreeing on certain things, then that's important. So here's where the models currently agree and it's what I'm going to go over. You don't need to read it all right now. I'm just letting you know I'm going to step through these as we go. And these may sound like the things that fit your model of choice that you may think about. And that's because these models are often uh explaining the same thing, but like the brain not being so simple, human behavior is not so simple. So try to not be dogmatic about what you're talking about. So the first is feedback is important. And I want to start with this because feedback is vitally important for rewards. If you take nothing away from this talk, please have it be that. If players don't know they got a reward, they can't get it again. If players don't know why they got a

reward, they can't do it again. And if players don't know the value of a reward, then they may not care about it. You can give them all the stuff in the world and they're like, "Eh." So rewards are feedback fundamentally and a lot of feedback can be reward and usability and clarity of feedback is a core component of what we do in user experience work which is why I'm here at a UX summit talking about it because UX is fundamentally about reward in a lot of ways. You're a wonderful audience by the way. I really mean it. Paying attention especially you. All right. So, if we're going to go back to kind of basics, we can talk about some learning principles. We can talk about classical conditioning, which is linking old responses to new or responding for a reward. So, as an example of this, seeing this image is probably quite rewarding uh for a lot of people because of the activity that's often associated with it. Hearing this

sound also probably quite rewarding. And that's because they're often linked together. And let's stop that. They're linked together and linked together. in a performative way when playing the game and it gives you progress. So they've been associated, they've become a reward. Uh maybe more practical example of this is in uh a lot of games currencies that are used are gold and gems and dollar amounts and things like that. And that's because we already have an association with these as having value. And by using them in a game, you don't have to reestablish the value of the reward. You can use a completely madeup space thing as your currency, but you're going to have to teach players the value of it. By using an established one, it lets players immediately know this is something valuable. And that's why it's so commonly done. Another example of of classical conditioning is if you look at the uh idol games, often you get those going and then they kind of run by themselves. And because we're so used to uh association numbers going up and

feedback like this with progress, it feels like we're doing something even though that we're not really. We're just watching a game play itself. So another learning principle we can talk about is operant conditioning. It's rather than responding uh to a reward, it's responding for a reward. You're doing something to get a reward. So when we talk about operate conditioning, we can go through schedules. Um, I don't want to spend too much time on this, but it is important. Fixed ratio is when you have to do a certain number of responses to get the reward. So, turns in a unit uh until a unit or building is ready. In a turn-based game would be a classic example of this. You just have to wait and hit that next turn button. Or clicks in a clicker that doesn't have a critical hit system or anything like that. You have to click a certain amount of time, then you get enough gold, and then you can buy your reward and then click some more. And the reason these are important is if you look at these graphs, you you've got the number of responses up this side and then the time across there. And these is

are the responses over time with each tick being a reward. And what you typically see with a fixed ratio, and this is just one single fixed ratio running at a certain length, so maybe you have to press 20 times and then something happens each time, is that what happens is you get pretty fast responding, then they get the reward and they stop for a little bit and then they go up. And it's because once you've got the reward, you've got that whole process ahead of you again before you can get for the next one. So it's kind of like a rest. And the way that these turn-based games work to stop that rest is you always have multiple things going at once, right? So your build building buildings done, but you've got three other buildings on the way. So you just take one more turn. You get a lot of things going. Fixed interval appointment gaming type stuff. This is example from Paragon of our uh jungle minions spawning at set times. So the players know be there at this time, kill the minions, get the reward. it. The response that this kind of interval of reward produces is like

this. You show up at a set time, respond, and then you go away until that set time. It's used a lot in mobile games for timers and daily login bonuses, which is effectively trying to pace out people using the game and have them coming back at certain times. Variable ratio is probably the most famous of these. It's used a lot in gambling. It leads to reward chasing behavior and it is where there's a variable number of responses until the reward shows up. So it's not 10, it might be 10 to 50, something within that range. And loot drops like uh the loot crate system in Paragon are example of this and critical hits are an example of this where you hit a certain amount of times and will do damage. And the response rate that that tends to get is high and relatively steady. The reason it's steady is because it's unsure when the next critical hit, say, will come along. You want to keep on trying to maximize those critical hits. There's no like I've already got it and

I know I have to work harder until the next one comes at a certain set point. The last is variable interval or event-seeking behavior. An example of this uh I found is from Destiny with their public events. These happen at certain times, but around that certain time and may not happen every time. And it generally encourages people to move into an area and hang around because they don't know exactly when it's going to happen. And this kind of random world event is a little bit more if you want to drive people to a zone and then have them wait around for things. It gets a pretty good rate of responding because you're not sure when when the event's going to occur and people will stick around. Now, of course, this this is just a diagram. I've copied off uh data from textbooks. And in a real game, in a lot of cases, you'll be mixing all these schedules together. They'll be running over top of each other. But it's kind of interesting to pull them out one by one like this and look at their kind of fundamental characteristics. So, moving on from those, we want to talk about

reward timing and uncertainty. Ideally, reward should come immediately for success uh mechanisms, progress mechanisms, and choice mechanisms. If I make a choice, if I'm succeeding or I'm progressing in your game, I should know about it straight away and I should be rewarded straight away. Now, this ne doesn't necessarily mean we can't have long-term rewards that we're aiming for, but people are generally not great at long-term rewards. There's the famous marshmallow test where you give a kid a marshmallow and say, "If you wait and I'll come back in a few minutes, you can have two marshmallows and you leave the room and the kid just shoves the marshmallow in their face." And as adults, we like to go, "Haha, children." But actually, we're all pretty bad at that. So, if you want people to work to a long-term goal, they need to know there's a long-term goal. And you give short little feedback bursts on the way there. Progression bars or experience bars are a classic example of this. They're little bursts of feedback, little bursts of reward to let you know you're making progress towards a big thing.

Now, I've been talking a lot about I just talked about things being immediate rewards. Of course, you can also have unexpected rewards, and unexpected rewards can be very strongly motivating. So, an example of that would be in a game like Portal 2. You unexpectedly work out a way to solve the solution. It feels very satisfying. It's not something you necessarily saw coming, and it can feel good, right? Surprise puppy. Um, but the problem with unexpected rewards is if I in six slides time showed this puppy again, it probably would have less impact. And if I kept doing it, it would have even less impact. And even if I mixed up different puppies, you might start to expect the unexpected. So unexpected rewards can be really valuable, but they're hard to pull off and they're hard to pull off repeatedly. The next thing I want to talk with feedback is game feel. The way it feels to play your game. And I want to argue to go back to this idea of intrinsic motivation that this is the ultimate in intrinsic motivation. A lot of uh

companies when they talk about intrinsic motivation will talk about a lot of goal structures and adding a bunch of stuff on top of the game. And that can work. But if you look at the games that are commonly cited, games like Destiny as having great intrinsic motivation, they're games that just feel satisfying to play. That the action of just moving a controller or a mouse feels impactful and immersive. So, there's an example of this from Gears of War, and it's the active reload mechanic where it takes the simple action of just pressing a button to reload and makes it into a miniame. So, Oh, yeah. You got a nice little bit of feedback there, a chance to succeed more or to fail. Oh, yeah. Of course, rhythm games like Thumper heavily rely on having a good game feel in order to succeed, cuz all you're doing in these games is pressing left, right, up, down, and buttons. in rhythm. It's not that complicated a thing to be doing really, but by using the rhythmic design and the sounds and the ramping up

the difficulty, they can create a very compelling game feel. It feels satisfying. It feels like it builds it feels like you're doing an important thing. I already mentioned uh Destiny, but Destiny, the developers at Bungie, work very hard on making the animations and everything like that feel just really satisfying to reach out and every action in the game makes it feel like has impact. Even though say this melee attack, all you're really doing is pressing a bumper on a controller, but it feels like you're really reaching out there and hitting your enemies. And even a simple screen shake, even though it may not be to everyone's tastes, makes this space marine in space marine feel like he has weight. Even though the all all that's happening is pushing forward on a stick, feels like he's physically in the world. This is something we've done in Paragon with several of our heroes. And I want to just give a brief example of that.

[Music] Okay, of course that's exaggerated for the trailer, but Steel's this big android, and you can feel the weight to his moves, the weight to simply moving with him and simply pressing the button makes it feel satisfying to play, even outside of the other structures of the game. I had six slides, by the way. Um, all right, moving on. The next thing is progression. Progression towards clear goals. This is kind of pretty easy stuff. Progression towards clear goals, whether those are internally set or externally set by the game, is rewarding and of course relies on feedback. I already mentioned having to give progression feedback for long-term goals. And typical examples of this would be, this is from Paragon. We have a bunch of things, weekly quests,

uh your progression in game as you level up and get new abilities in our mobile. Uh daily victory chests, living up your account progress. All of this is progression and to show to players that things are happening and that their actions are being rewarded. Progression is also commonly used uh in tutorials or at least tutorials that do things well. Uh this is the tutorial from Unreal Tournament. It's clearly telling the player what to do to jump. It's showing a uh modeling character doing that action and the player cannot progress out of that room without doing the action. They can't leave without jumping over that barrier. And once they do, they have immediate feedback that it was successful because they're out of the area. The interesting thing about the UT tutorial, which I think is a a nice element to it, is it has optional skill tokens in it. Optional skill tokens are as the game teaches you, there'll be skill tokens hidden around the tutorial that kind of build on what you've already learned. So, if you've learned jumping and wall running, maybe in the next room, off in the corner,

it's not required for you to do. There's a token spinning up there that requires you to jump and wall run to get it. You don't have to do it, but it's a way for you to put together what you've previously learned and show your progression. So, this would be an example. Uh, this is probably the hardest one in the tutorial. It's It's pretty difficult to get. You can't jump over there. So, you need to scoop around like slide up a wall, jump around, and then slide out to get it. It's pretty difficult, but it's a nice way to show players progress that's doesn't also make them feel forced. Progression can also be shown via social comparison. Um, this is the postgame screen that was added to Dota 2 a while ago where it shows after the game is over the progression of each player during the match. And it lets the players look and rank rank themselves within this information and know how well they did at at certain times over the match and hopefully learn from this or be rewarded by it. Now, comparison, especially to other players, can open the path to abuse. So, often making it

only available after a game where players cannot interact with each other is a good idea. Hiding MMR is a good idea so people don't uh attack each other for you're on my team and you've got 200 less MMR than me. So I'm instantly going to think that you're worse off at playing the game because your matchmaking ranks lower. And making stat tracking websites opt in. So if you want to see how good I am at Dota by looking on a stack tracking website, you can't because I haven't opted in. And what's actually uh been happening recently, it's interesting in games is the idea of comparison within yourself. So, in uh Overwatch, for example, at the end of the match, they don't compare you against the other players. They compare you against your own progression over the game. So, even if you've lost the game, maybe you've got better at something. Maybe that was your best ever game at healing or your third best ever game at healing or whatever it might be. It's some kind of reward that lets you know that you're progressing yourself, but is not potentially a negative comparison against other people. And

Dota's similarly got this thing that shows you your average for the hero over over certain metrics. Okay, so that's feedback. A lot of this other stuff is also feedback, but I'll talk about it in a different way. Uh control and choice is important. So what we're talking about here is feeling in control. We like to feel like we have choice and control. And generally speaking, we dislike feeling limited, controlled or out of control. And there is a slight difference between feeling controlled and feeling out of control. I I hope that you can get even though they're kind of the same thing. There's an exception to this and that's unless we choose to be controlled and limited. If we choose to cut off paths uh for ourselves that are presented to us making interesting meaningful choices within the game, then that can make us feel in control as well. So the statement like people don't like feel controlled doesn't nec necessarily mean they don't like feeling limited. They just like feeling like they have choice in those limitations.

So, this is often talked about around story. And there's different approaches that can be taken here. You can go crazy like The Witcher 2 where a whole act of content can be skipped based on a decision by a player. They completely miss a massive chunk of hours and hours of content based on a choice. It makes choices feel very meaningful in the game. You can also look at performative stories where there isn't actually much formal story and there's not much choice in a game like Journey except for the way that you play out the game and the way that the story is created around your character. And then you can look at maybe more traditional structures of story games like the original Walking Dead where they very successfully created a feeling of choice. it was uh praised for its feelings of choice. Even though if you go back and play through the game and make different decisions, you'll find out that actually the choice doesn't strongly matter to how the game plays out. There'll be different people that fill in at different times for people you let die or whatever might happen,

but because it felt like you're making a decision at the time, it feels important. And this can be contrasted against Mass Effect 3, which came out the same year and used a similar structure, but because players had expectations built up over two games beforehand, and maybe they were more likely to play through more often, players would know, oh, it doesn't really matter. This play, this character shows up here anyway. And some players were disappointed by that. Now, that's not to say that this was fundamentally wrong. It's just I'm trying to make a point about player expectation about control and how it can be good but can also backfire. It also interacts with activities and progression. So what I'm talking about here is uh examples like uh side activities in open world games. If the site activities feel forced upon a player or they feel like they have to do them in order to progress or they're being bugged to do a side activity, they may resist doing it. It may feel like control. Escort quests can also be very

controlling because they're essentially taking away success from the player and putting at the hands of an NPC they have to look after that can get in trouble or die or break stealth. Modern uh escort quest type games like Last of Us solve this by making the person you're escorting not do these behaviors. So, uh they can't break your cover or they can't die. This, of course, is a trade-off for immersion, but it's a worthwhile one for the player not feeling controlled and frustrated. And then the other one is, of course, grind. If players don't feel like the activities they're doing in a game have meaning, are showing progression, and are meaningful to them, then they're more likely to view like having to make a lot of um daggers to level up in Skyrim as a grinding activity. And in fact, this idea about control being so important has come up in the recent literature as a suggestion to replace intrinsic versus extrinsic and to talk about instead autonomous or controlled behavior. And internally

motivated behavior, if you feel obligated to yourself, can feel controlling and be demotivating, just like external behavior can feel autonomous if there's a lot of choices set up for you. So this idea of control seems to be a core point where a lot of theories are agreeing and I've been saying it's perception of control that matters and that is the important point. You can give players an illusion of control and get just as much uh impact as real control. So games like Pegle where you fire into a random zone but because you pick where you fire and you have a have a special skill can feel like you're more skilled than the game is actually uh working on. And of course match three games can be masters of this. You have no control over what's about to fall on the screen at all as a player, but because of the way that the feedback is presented to you, it feels like you made a really weighty important decision even though you just like matched one line. The last well, one of the last things

about control is this interesting idea of counter control that uh came out of behaviorism. It's been around for a while and it's typically the idea that when people feel controlled, we often counter to this and this is often talked about in a negative way. If people feel controlled in your game, they feel like it's grindy, they'll stop playing it. That's how they break the control of your game. They're like, "Nope, stop. I'm out." However, it can actually be used as a game mechanic because it feels very rewarding to break control. So, a classic example of this would be The Stanley Parable, which is a game all about a narrator trying to control you and your attempts to break that control, and it feels very satisfying to do so, even though you ultimately fail. Um, so the last thing is control and tutorials. Now, when a lot of people hear this stuff about control and feeling uh controlled is negative, um, they'll say, "Okay, we're not going to put tutorials in or we're going to do tutorials that have no controlling elements, no guidance in them because it's better for people to learn things." Now, that can work out, but the problem

is it's feeling in control and not feeling out of control. And one of the really big ways to feel out of control is to feel confused and unclear and not know what the hell is going on. And so having a little bit of handholding, a little bit of guidance in tutorials, if it lets your players get to the point where they can feel in control, like you're telling them what to do, and then suddenly they have control and they can show that control, it can be worth it. So this idea of control doesn't necessarily mean don't make tutorials. And of course, that's one of the nice things about the optional skill tokens in Unreal Tournament is that they're optional things players can do to show that they've mastered uh the mechanics that you're teaching them. Okay, last thing in control is non-contingent rewards. These might be known as participation awards. Who thinks participation awards are a dumb idea? Well, not as many people as usual. um participation awards kind of came out of SDT and the intrinsic model theory

and there's a reason for that and it's that uh participation rewards which are rewards you get for just showing up are less likely to be be seen as controlling which we've said is not good and more likely to be unexpected right because you don't have to do anything to get them they're not based on your performance of course because they're not based on your performance they're not really giving you feedback on anything however there are multiple occasions that they're used in video games. So, probably the one that the people have the most negative association with is rubber banding. This was where mechanics in the game like in Mario Kart will help losers equalize with winners. And it's good for casual games cuz nobody wants to just be beaten over and over again if they're supposed to be just having fun with their kids or family. Another example from more hardcore games is gold drips in MOBAs. As long as you're playing a game of a MOA, there's just some constant resource coming to you, no matter how well you're doing that might let you buy that one item you needed to win the team fight and come back in a situation you are otherwise

losing. A similar example in Titanfall 2 is the Titan drop uh progression which like a gold drop in a MOA was just constantly coming in in Titanfall 2. So even if you're getting beaten and not performing very well, you probably still got to use your Titan and experience that part of the game and that get that reward during a match. So that's what these non-contingent rewards can be quite useful for is this kind of balance and making sure people experience more of your game. You're a really wonderful audience, by the way. Okay, so I'm not going to do a bunch of slides about this one. In fact, this is the OD slide, but not getting a reward you expect can make a reward a penalty. So, this is the idea that if you build something up as really big and important, like players go around and they pick pick up every single consumable and collectible in your entire game and they get nothing for it. This can feel like a a bad thing if they were expecting something to come out at the end.

The next is that task inherent rewards motivate more than uh task irrelevant. Or to put that more simply and basically rewards that are meaningful to a player are more motivating. A classic example of this would be uh the agility orbs from the Xbox game Crackdown. These orbs you had to show agility in order to get them. You had to climb up buildings, jump, and generally show that you had agility and power in your character. But by collecting them, you also increased your agility. So they had meaning in two ways to players. you showed competence in this to get them and then you're rewarded by a be able to do more of this. You got the progression of being able to do that. And one way you can do this in games is to personalize if you can. If you have data on what people are already doing in your game, you can reward them in ways that you know uh will be rewarding to them because it's the type of thing they're already interested in. But you don't have to go that far. All you really have to do is show value. And

when I was working as a consultant, this was often a problem we would see in uh mobile games that would come to us is in the tutorials, they would throw all their rewards at the players. They would say, "Here's premium currency. Here's this, skip this timer, all the things." And it was like, "Here's all the things." But we're not telling you why or what the meaning or what the value is. We're going to get you to skip a 30-second tutorial. What like why is that important to somebody? And it doesn't it hurts people's learning and it hurts their valuation of the rewards if the reward is not meaningful. Okay, interacting with others positively is motivating. And when I say positively, it's positive for at least one person in the equation. So if I beat someone at a game, it may not be positive from them, but it's positive for me. So when you're interacting with others, you can do performance comparison. You can compare your performance against other people. And this is potentially rewarding if it's in our favor. It's also potentially u motivating if it shows progress or changes expectations

positively. So even if it's not in our favor, but it shows we made progress or it changes our expectations like we did better than we thought we would. That can be good. But of course it can be demotivating if it's against us or if it changes our expectations negatively. Now I'm going to example of this. It's not necessarily a reward example, but in Dodo they have these uh conduct summaries. They'll show you about your behavior in terms of how you've been reported by other players. Uh this is a report of an of a negative player. They've got six reports and they're being told that 94% of the population their report rate is higher than 94% of the population. So the idea is you're supposed to go, "Oh, oh no. I'm a I must be a really bad person and it's not normal to do this and I'm supposed to change my behavior." Now, this is a conduct summary of a nice person. It's me. um who's got less than three reports when this popped up and they're being told that only 77% of players have

uh three reports. Well, 77% of players have less than three reports. But you might view that as okay, that's good, but it could also be telling you that like 20 over 20% of people have more than three reports. Maybe I don't need to be so nice. Um, and certainly in research where they've shown people in neighborhoods energy use where you get to see, hey, your neighbors using more energy than you are, but congratulations, you don't use much energy. The people that are being told they don't use much start using more cuz they're like, well, everyone else is doing it. So, you got to be careful when presenting this information. Now, competition is a social interaction that comes naturally to game designers. I think quite often we we think about it uh nearly straight away and it's a strongly motivating force especially if you think you can win. Okay. And it does have a potentially slight demotivational event immediately after winning. And that's often because uh competitions are often in a fixed ratio or fixed interval. They only run at set times or only for a set period. And so after

you've won often people kind of take a breath and then start competing again. It's just something to be aware of um to keep people motivated between events of winning. Of course, the downside of competition is it can demotivate those who think they can't win or know they can't win. So, in a mobile game, if you load in and it tells you you're 31st thousandth in the world at this game when you first started playing it, that might be really demotivating to you, unless of course you make a lot of progression quickly. So, one way to get around that is to put people in local leagues where they can actually see uh progression. It's also I'm being hard on competition here maybe and it's because I think as game developers we often don't think about the losers in our games and how we can make losing a pleasant experience. An experience that we can let people learn from and come back from and feel like they can stay engaged with our game and don't just go well this is over. I'm going to stop playing. Cooperation on the other hand compared to uh competition is a very powerfully

motivating uh factor for a lot of people and this because it allows everyone to contribute to it. Nearly every top game you can think of nowadays has some kind of cooperation element in it. Cooperation uh according to psychological research motivates people a lot more than competition which has been shown to competition has been shown to do things like suppress creativity, cause conflict, all these kind of things. And it does allow for cooperative competition where guilds can compete against each other. And this gets you kind of the best of both worlds. You've got the competition stuff, but you've also got the cooperation between people. Allows for replicity, which I can never say, but it's basically the idea that people feel like, oh, someone's being nice to me, so I'll be nice to them. A monetization example of this is in uh Game of War Fire Rage, which is anyone in your guild spends in the game, it benefits everyone in the guild. So this is a monetization application of it in that by uh spending within this game it benefits everybody

and that's one of the things I want to talk about here with that's kind of the idea of positive externalities which is a uh economics term that is by spending in an economy I don't just benefit myself I benefit everyone else around me and this can be a very powerful motivating force because while it's not a selfish competitive force it says by me contributing to this game, I'm part of something bigger. I'm helping other people. I'm helping friends. And also, if I've already spent as much as I want to help myself, I can continue to spend uh to help other people and feel like a good person. There's an example of this in uh Paragon. And this is our master challenge system where by buying into these master challenges, you get an experience boost that goes up the more you play. And the more people that have these experience boosts on your side when you're playing, the faster everyone's experience goes outside of the game. So the idea is by purchasing this, you're not just helping yourself, you're helping others.

All right, so last one on this uh big long lists, and that's that rewards can in certain situations uh be demotivating. I've already tried to call this out where it comes up in terms of if they felt to be too controlling or they're against expectations, things like that. Now before I finish up I do want to just briefly talk about some cognitive biases and only a couple of them um that I feel can be important when it comes to reward. One of them is the representativeness horistic and this is the idea that the more time cost effort you put into an activity there's a basic expectation that you get more reward from it. So, imagine you're a relatively tough fighter in an RPG game and you encounter an unconscious goblin and you knock it out and kill it, you probably don't expect to get much reward from that. You haven't put much effort in. You haven't done very much. But if you fight something like this,

you probably expect to get a lot more from beating it. And this is somewhat common sense, but it can be a problem for games that have a lot of randomness or procedural generation in them. I've played quite a few rogue likes where I'll go through a room that was really, really difficult because of the configuration of monsters in there or the way that it was laid out. And then the room on the other side is just an empty hallway with nothing in it. And that feels against expectations, against this horistic that if I've put effort in, I expect that effort to be rewarded. So this can be taken uh into account with the reward systems also being tied to the uh randomization or the proceduralization that comes out at the end. If it's uh over a certain amount of difficulty, then there should be more reward for doing it. This can also apply to monetiz monetization. If people spend more in your game, they of course to get expect to get more from it. So, you need to make sure your economy is balanced to support the people who spend more in your game or at least they feel like

they're getting more value. And in fact, some games uh so it's important to go back to this idea of feedback and making sure that everything's very clear and the value of buying things is very clear and the value of the things you can purchase with money is really clear even before you make the purchase. But some games will also once you make a certain level of purchase, they'll without telling you modify the odds of certain good things happening for a little while afterwards. So the game suddenly is more in your favor after you've spent perhaps a dark pattern. Another thing with uh this is after a string of bad luck, we expect a payout. This is what keeps people gambling. but it can be something that we can use in games to reinforce this expectation and make people feel like your game is fairer and more in line with what they want. So, an example of this from Destiny would be the vanguard bonus. In Destiny, when you run strikes, you get loot, uh, random loot based on how you play through the strike. It just pops

up. And the more strikes you run in a row, the higher your Vanguard bonus gets, which increases your chance to get better stuff, which meets this expectation of the more effort I put in, the more I get out. Another example of this is non-replacing randomness and variable loot rewards, like a loot crate. Once you've got something out, once it's out, and that increases the odds of you getting things that you actually want out of it, and that feels better to players. It's more in line with what they expect for their increased effort and increased spend. You also need to make sure you set expectations. This is another Destiny example. When Destiny launched, they had the Ingram system. Uh the Ingram system shows you is kind of random loot you pick up in the world. And when it launched, the color of the Ingram was the highest possible reward you could get out of the Ingram, which set the expectation. If I see this, it's got to be an exotic, which is really cool. But you could hand it into the Krypark and you get a common item out of it. And people didn't like that very much because it was against what they felt

was being represented to them. So in my understanding, what Bungie did is they didn't change the drop tables. They didn't change the loot or the underlying math of this. All they did is color the Ingrams at the lowest possible thing you can get out. So now you saw a blue Ingram. You knew you were going to get at least a blue item out of it, but maybe you got a surprise and got an exotic. So, it's important to be aware of what expectations you're setting up for your players with the reward systems. The final bias I want to talk about is the idea of loss aversion. This is simply state that players will often work harder to keep something they have than to get something they could gain. So maybe a complicated example of this could be if you're designing a mobile and you want people to attack towers, you want to encourage them to stop playing defensive and get out there and attack. One thing you could consider doing is lowering the hit points of of enemy towers, making them easier to take down and attack. You can achieve your attack more easily. And that might work.

Uh or what might happen is defend people might work that out and they might start playing even more defensive because people view that tower as something that they own and they control. And even though now it's easier to go out and gain, it's also easier to lose. And people will often work harder to keep something they have. Other examples from games will be continuous login bonuses where if you don't log in uh every day you lose progression. Of course, that can be a very negative thing if you actually hit that not logging in on the day. So sometimes you can buy that progression back or sometimes games don't use a continuous login bonus. You just get a increasing bonus every time you open it rather than it being continuous. Consumables and durability. People often hoard things like potions and consumables because of this reason. We don't like spending something that's going to go away. So you need to make sure that the base rate and the drop rate of these things is good enough that maybe people will use it or you really demonstrate the value of doing so. And pay to continue is a classic example of

this. You're paying to keep the progression that you have. All right. So I just want to summarize the points I've gone over today. It's not everything about reward, but it's what I feel uh there's a lot of agreement in the literature. First of all, what I talked about with dopamine at the start, I really hope that you can be skeptical of any simple expectations, whether they be neuro or not. Shots of dopamine are not needed. There's practical answers out there if you look for them already. They don't have to be the newest and hottest things. There's a whole load of research and there's more that you can find to go over what I've gone over today. Rewarders feedback. Please do provide feedback on rewards as close to the behaviors you want to reward as possible. Uh this for long-term rewards this can be progression towards them. Mix certain guaranteed rewards with rewards that feel more random and unexpected to get different types of behaviors. They get different kind of response rates.

Control and value. Help players feel in control. This does not mean not guiding them or teaching them, but help them be able to access your game and get at the rewards and not feel controlled by those rewards and show the value of the rewards and behavior, which can often be a good way to reduce control. When we talk about social rewards, comparison, competition, and and uh cooperation can motivate, but they do come with risks. really try and think about the players in your games that are losing out on your competitions or not coming in first place or consistently not showing any performance uh increases. By giving these players some way to stay motivated, some way to still feel in control, to feel like they can progress, you can keep them in your game and keep them retained. Social pressure in cooperation to do things like give gifts to other players or or work together. It can be good, but it if it's

presented wrong, it can also feel like control, which may cause people to abandon it. And you should set and meet player expectations. Try and understand what these expectations that you're setting up, maybe unintentionally, and at least meet them, if not exceed them. And preventing loss can motivate more than working for gain. Okay, thank you very much. And any questions? If you have questions, please come up to the mic and you can contact me at these uh addresses down here. Hi, thanks so much for the talk. Um, so in a lot of especially casual and mobile games, there seems to be an order of how these things are unrolled. like early on there's more simple rewards and like a variable schedule and then only later do you get things like social rewards. Is that is there actually like a hierarchy or is that just kind of convention? Do you think these kind of roll into each

other? I think there's a there's a hierarchy in terms of the cost to the developers in some ways. You don't want people getting into social too early in some cases because that might be more load for you. You want to make sure that they're engaged with your game first. And a good way to that is to show the value and roll out these more simple rewards early on and show them kind of the fundamental nature of your game and why they might want to have this value of working with others, right? Rather than just forcing working with others on them before they understand what what the potential of doing that is. I think that could be what some developers are thinking. There's not really a formal um hierarchy to it, but I think it could be around this, you know, get people into the game, show them the value, and make sure that they uh engage with these later systems that might be a bit more uh ask a bit more more of them in terms of how much they time they're giving up or how much effort they have to put in. Yeah. Thank you. Thank you. That was very interesting talk. Thank you. So many of the free-to-play games uh rely on boosters and powerups

to win levels. I wonder if you have any insights on when using these boosters and powerups to have less or more reward. Mhm. Yeah. So, I think um one interesting trend I've seen in in freetoplay mobile games that use boosters and rewards, which are consumables, which people can be kind of adverse to using, is to instead of having them as things you collect and then they go away, they have them the last one of them goes on a timer. So, it's not gone. You've you've used it, but it's going to come back in a certain amount of time. And by having it on a timer, which means it's not a permanent loss, that can help people be more motivated to try them out, see the value of them, and keep using them a little bit more than perhaps they would. But I've also I've experienced doing UX tests on mobile games that people don't like consumables very much. And this is one way to overcome it. Yeah. Thank you. Hi. Hi. Um, as somebody else who plays Dota 2, I love Dota, but um, for people my

age, it's kind of this learning curve is so steep. So like I was wondering um, there aren't the rewards for Dota consistent. So do you think for like new players if they offered I don't know more rewards or like different rewards, do you think that would help bring more players to Dota too? So, I think the thing with um Dota and other mobers is the thing about while they have this really complicated learning curve to become good at them, they're actually not that complicated to throw yourself into. And and compared to an RTS where you had to control a lot of units and all this economy, all you're doing is you're controlling this one unit and clicking on things. And you can kind of do that pretty fast. And each game of a moa is this nice progression loop where it's a even playing ground and you're cooperating with a bunch of people in a competition. So it has a lot of these other elements. And I think those are what's the really rewarding things about mobs and made them so uh popular even without the need for a bunch of external rewards or extra things on top of it.

Thanks. You had a question. And if you have a collectible game where uh people are opening booster packs to to get rewarded a lot of the times, uh I've noticed that sometimes opening the 50th booster pack, it's not quite as fun as the first one. So a question for you would be, do you uh have any strategies or recommendations for exploring rewarding players uh whether in booster packs or related to booster packs uh that could continue to be motivating even during demonetization? Right. So, I would say the way to address that is to to maybe talk to players and work out why it's maybe not so motivating. And it could be just there's no value in it anymore. Like, I've got all the things I need or I've realized that the things I actually want have so little value uh or so little chance of coming out that it's not worth me trying. And if those cases, you maybe you can play with the underlying rates. Once people open a certain amount, then you make certain things more likely. There's adding new content. Um, but ways to demonstrate that value to the players is important to keep them engaged, I

feel. Awesome. Thank you. Hi. Um, so using Overwatch as a case example, when it was in beta, one of the complaints I heard from people I was playing with was that there was no metagame progression, which is something they did add when the game launched. Do you think that's something that is necessary to keep people interested or is that like an easy way out? Uh, I don't know. Certainly progression is is important and um, if people were feeling like there was no metag game progression and that's the feedback that was coming in and that's what they're getting, then I would understand doing it. It's not something you necessarily need. If your core game is compelling enough, but uh, it can certainly add on, but I'm not familiar enough with that case, unfortunately. Thank you. Hi. Um, yeah, when you talk about setting up uh player expectations and making sure that you're meeting them,

what do you think is the right balance between having a cadence that's so expected that your players sort of maybe like save all their money to only purchase on sale days and now they're so trained that you can't really break them of it. So, what do you think is the right balance of having those expectations but still wanting them to have that intuitiveness about purchasing and using and if you see them coming down this path that you don't really want them to be on where they're so trained, how do you break that without making them feel cheated? Right? So, that's going to be different on a on a case- by case basis. Um, but generally speaking, you can try and cut off that by mixing different kind of schedules. So it's not always like what's important is that players expectations are met and they feel like there are certain rewards coming but you can still use randomness or still use variable schedule so they don't know exactly when it comes so it's not completely predictable. It's just be aware that if you have built a

expectation and don't deliver on it, that can be a problem. But certainly what you're saying where um because players have built such a strong habit, maybe all the funds gone out of it can certainly be a problem. And I would say just mix up mix it up to to get around that. Okay. Do you know any examples of games that you see that's done this successfully? Like in some of our games, we have like a daily login reward and we might have changed that to be like a collect wheel. So the expected value is lower, but maybe we think like they won't notice that or something. Um, but then the players who are the core players like they do notice based on the feel. So do you have any stories of So I think that's an interesting trade-off because it's hard when you're designing a game because you're designing for most players who will be satisficers. They don't really understand the core base levels of of all the probabilities. they don't really care. They just want to play a game and that kind of thing will work for them. But your really hardcore players are optimizers. They look up the tables and they work out the odds and and everything like that. And certainly the

idea of adding a random uh reward as a daily is a on a a base level kind of a good one because it's moving to this more variable schedule. Um but I don't know off the top of my head an example of a game that's got over that. I would just say uh you need to think about when you are designing your rewards that somebody's going to work out the base rates for it and how upset are you going to be when they do that. Do you feel it will demotivate them if they know the base rates and if the answer to that is yes then maybe you need to reconsider your base rates or you need to say they're going to be such a small amount of the population we don't care if they get demotivated it'll be a trade-off there. Okay. All right. Okay. Hi. Hi. Do you have any particular framework you kind of use when you're designing rewards or do you kind of just because you're a PhD you know everything like what used to help you when you're coming up with I mean

basically the the list I had is where I feel like the research uh comes together. It's like try and think about all these things when you're designing your reward system and and think about how you're hitting the different parts of them and is there a good clarity and all that kind of thing. Fair enough. Thank you. So, you gave some cool examples and some counter examples. I'm wondering in your opinion, what's the biggest sin? Like where has the dopamine naive theory or naive like incorrect model been misapplied uh widely in gaming? Like if you could wave your wand and fix For me personally, and I don't know if this is the biggest problem, I think it's that point of Okay, thank you. of um a lot of game developers don't think of losers. And if you're uh competition is fair, usually that means people are losing 50% of the time in your game. And there's often we like to think about winners because we like to think of ourselves as winners. And winning is already rewarding in itself, but we also

like to throw a bunch of rewards and celebrations and everything on top of it. And often our losing screens are kind of like heart you lost and then that's it. And we don't allow for our players to still feel engaged with our games at that point. So I I think that that is something at least on my personal level I think that could be improved. Thank you. Okay. All right. I think that's it. Thank you very much.