Propaganda Games: Sesame Credit - The True Danger of Gamification - Extra Credits

Okay, so one of you recently brought this to our attention, and when we first started looking into it, we couldn't believe it. Even now, looking at this, I keep thinking that there's got to be something about this that I just don't understand. Like, I must be reading this wrong. It's just so clearly something out of dystopian science fiction, and yet, here it is in our world, and people are actually embracing it. Apparently, China has gamified being an obedient citizen. Going under the innocuous name of Sesame Credit, China has created a score for how good a citizen you are, and that's one of the scariest things I've heard in quite a while. It's jointly run by Tencent, yes, that Tencent, the one that owns Riot Games and has a significant share in Epic and Activision Blizzard, and also the ascendant Chinese competitor to Amazon, Alibaba, hence the name Sesame. So, the owners of China's largest social networks have partnered with the government to create something akin to the US credit score, but instead of measuring how regularly you pay your bills, it measures how obediently you

follow the party line. They dredge data from your social network, so if you post pictures of Tiananmen Square or share a link about the recent stock market collapse, your Sesame Credit score goes down. Share a link from the state-sponsored news agency about how good the economy's doing, and your score goes up. But, Alibaba and Tencent are also the largest online retailers in China, so Sesame Credit is also able to pull data from your purchases. If you're making purchases the state deems valuable, like work shoes or local agricultural products, your score goes up. If you import anime from Japan, though, down the score goes. And this score has real-world consequences. Like many games, Sesame Credit has tiers and levels, and having a high score gives you special benefits, like making it easier to get the paperwork you need to travel, or making it easier to get a loan. Now, currently, there are no consequences for having a low score, but there's been talk about implementing penalties once the system becomes mandatory in 2020. Penalties like slower internet speeds for low-scoring citizens or even restricting the jobs that a low-scoring person is allowed to hold.

But, there's one more layer to Sesame Credit, and here's where this goes from being repulsive to downright insidious. Because this is all part of a social network, it also scans your friends. So, you will lose points for having friends with low obedience scores, and it tells you this. At any point, anybody can check anyone else's score, and when you check your own score, Sesame Credit provides a handy map of your friends to show you who's dragging your score down. Have you ever had that thing where you play a game with somebody who wasn't doing very well, and you've tried to change their behavior to make them do better? Or maybe after a while you just sort of stopped playing with the people who were holding you back. That's at the heart of how this system works, and it's also what makes this one of the most terrifying tools of authoritarian oppression I've ever read about. Because like mass censorship, jail time, assassinations, those are all big messy implements for keeping a population in line. That messiness and severity foster resentment, eventually rebellion. They're expensive, they're unwieldy. In the end, those tools are impossible to maintain. But, social pressure,

ostracization, those things are free. They happen on their own, and as a government tool, they don't have nearly the same potential for going embarrassingly disastrously wrong. With a system like this in place, a government doesn't even have to tell neighbor to spy on neighbor, to rat each other out, because that's all built into a seemingly innocuous game system. The government need not step in. Re-education will be handled for them by friends, classmates, and relatives who want to maintain a high score. And if that doesn't work, then potentially dangerous ideas still end up quarantined by the social isolation this game system causes. Express or help to spread too many radical ideas, and people will stop associating with you. And not because some jackbooted thug showed up at the door with threats, but simply because associating with somebody with those ideas will lose them all the privileges they've worked so hard to obtain. It recontextualizes obedience to an authoritarian regime. In the past, you obeyed such powers because you were afraid. Fear kept you motivated, but fear is negative. It fosters resentment.

The world we're stepping into instead uses positive reinforcement to promote being subservient to the will of the regime. It's Big Brother's kinder, gentler hand. And the things that make this scary is that we've seen the efficacy of this only too well in games. You may not actually know this, but when World of Warcraft was in its early stages of development, it had an unrested penalty mechanic that started limiting experience gains for players who had played too much. And players hated it. They resented it and they complained about it every day. So, after brainstorming on this for a while, Blizzard had the idea to simply change how the mechanic was presented. Without changing any of the existing numbers or systems, they started referring to the unrested experience penalty state as normal and made it the default. And they started calling the original normal experience gain state as rested. That's all they changed, and everybody loved it. People would log on every day just to get that bonus. Positive reinforcement works wonders. But unlike World of Warcraft, which built a system just to get people to embrace

meaningless grinding, Sesame Credit has built a system to get people to enjoy falling into line. Now, the system's not mandatory yet. It's opt-in right now, but it's going to be mandatory in 2020. And there's a terrible brilliance to phasing that in. The early adopters are going to be people excited about this system, people who are already patriotic and are eager for anything that'll help display that patriotism to the world. And as early adopters, they're going to talk it up. They're going to give it an air of being positive and fun. Then it'll be foisted on the society as a whole. More than that though, the early adopters are going to compete. Already, you can see hundreds of thousands of tweets displaying people's high scores or showing off the new milestones they've hit, giving a hard numbers to their patriotism and giving them bragging rights for being the most patriotic, most right-thinking person they know. And that's going to set the tone for how Sesame Credit is intended to be used as a competition to see who can agree with the government the most. We've talked about propaganda games on this channel before, but for all the time we've spent examining and deconstructing terrible games that

espouse hate, and for all the studies on propaganda games James has done, this is the use of game systems that frightens me the most. Because to most people Sesame Credit will seem benign, perhaps even fun. It's a conversation starter, something to share with your friends. But it's making heavy use of all the psychological motivators that we game makers deploy in scoring systems and ladders and levels. Systems that we build to shape play habits and to keep people coming back. Like I said, I'm still kind of in a state of disbelief looking at this. If any of you are watching this from China, please tell me if I'm misunderstanding this thing, because I would love to be wrong about it. If not, well, I hope this episode can do some small amount to help the fight to keep such a system from becoming mandatory. For everyone in the rest of the world, I hope this helps remind us all how important it is to be aware and to be vigilant. All of these gamification techniques we've learned through making games offer incredible opportunities for making this world a better, more engaging place. But every great tool carries with it the potential for misuse. And it's on us as a community who understands this amazing

new medium to do what we can to stop that. We'll see you next week. Hey everybody, just so you know, the Extra Credits team is going to go on hiatus for the next couple weeks so that we can all get some rest and spend some time with our families and such, but we will be back at the start of the new year with more Extra Credits, more Extra History, more stuff on Extra Play, everything we make. So, I hope you guys have a fantastic holiday. Thank you for watching and yeah, we'll see you in January. So long.