Why Do $200,000,000 Games Keep Getting Canceled? | EA’s Black Panther Greenlight Failure

[Music] All right, Adam, we've saved 20 minutes for you for our philosophy corner. Here we go. Philosophy Corner. Okay, so the topic here is around the problem with AAA greenlights. So, there is an excellent article by Owen Mahoney, ex-president of Nexon called Sympathy for the Devil, which he went through basically the problem with green lights. Also last week we discussed there was the recent news that EA has canled and shut down the studio working on Black Panther. Jason Shrier came out afterwards and came out and with additional context some quotes here from the article. The news came as a shock specifically to the team. Black Panther had recently passed what EA called a development gate. The team had momentum. The project was coalescing and the rumor behind this was that EA executives were frustrated that the game had not left pre-production phase after nearly four years in development. And for context

around this four years, this might be a surprise for people coming from mobile. It's not unheard of in AAA to have a 4-year pre-production cycle. Six to eight-year productions are unfortunately becoming more and more the norm for some of these epic scale AAA openw world games. We need to try to find a way to to solve this. 6 to 8 year cycles lead to a lot of bad things, but especially when you've got a new studio, a new engine working through COVID, you've got a recipe for these kind of six to eight year productions. So this blog post specifically talks through the challenge of making a decision at a green light and Owen Mahoney as I said president of Nexon from 2014 to 2024 CFO from 2010 to 2014 and then previously was corp dev at EA. The challenge here of like a green light meeting is sitting and putting your head in the the executives's mind here. Do you trust this plan right this

plan as presented? Do you believe this game will be fun? Will it sell? Will it cost as much as they're saying it will cost? Or how many delays do you bake into your mind that this thing will actually cost? Really? Do you plus the plan, the team? And unfortunately, can you even say no? And I think this is where we get into a lot of the some cost fallacy, but there is a lot more pressure to stay with the default path of saying yes than there is to say no and to cancel. There's tremendous amount of talent risk. There's cost risk. But sunk cost fallacy is real. And of course, what this translates to is most of these green lights end up never being a full no unless there's a smoking gun. It ends up being more like yes, but with caveats or yes with additional concerns. So digging into things like is this actually fun, right? The team is presenting the world, the art being pretty, bugs are getting squashed, momentum is good, but unfortunately most of these green lights don't focus on whether it's actually working, if the

game is actually fun, if the game will actually sell. It typically focuses a lot more on production momentum than it does on commercial viability. Typically research backs up some of these decisions. So they're bringing in saying, "Hey, we got great fun scores. Players are loving it." But the reality is I think much of this research, especially presented by the game teams, can be bent, can be adjusted, can have the finger on the scale. Additionally, sort of plans are then built. You know, maybe it's not fun now. Maybe the executives don't have full trust yet, but hey, trust us, it will be in x amount of months and then eventually the big decision around a delay can happen. And this goes into the article around like do you kill the game? Do you force the date or do you delay? And of course, if you delay, then you're killing credibility with your board. So answering all of this in a 90-minute meeting is just not realistic. The big flaws with this is that most executives

assume linear progression of games. When games are jungles, even in production, there's always going to be es and flows in conviction and momentum. Big decisions are happening in these meetings with actually pretty limited information. You know, no one in these meetings ever says that there's not enough information. Teams put together decks and decks and decks, tons of spreadsheets of checklists, but ultimately most of these green lights turned into a dog and pony show. This then starts to emphasize more around the schedule than around fun and commercial viability. And this is especially true because burn rates are intense in AAA reaching 30, 40, 100 million yearly burn rates. So the problem then becomes these green lights frontload the wrong risks. Dex becomes more important than software. A good sounding strategy becomes better than execution. Having amazing pretty artwork becomes more important than real fun. Having those pretty pictures for your decks, tutorials become more important than

deep systems to make sure execs have fun in the 15 minutes that they actually play in these green lights. And USPS is more important than the game to get your marketing team on your side in these meetings. So this all points towards like clear underlying issues that like the process in which we build out these green lights. The way that we kind of coales this decision puts a lot of emphasis on the wrong things as well. He points at things like tools development process and our overfocus on graphics as our selling point makes this problem worse and worse. So my take on the article is that I think this is absolutely a discussion that we need to be having. That's why I'd like to bring it up here and I'd like to have even more discussion after this podcast. What I'm missing from the article is actually a discussion on real solutions since I think in general, yeah, 90minute meetings with 30 plus executive stakeholders are just never going to be the forum where decisions are really made. Like these will always be dog and pony shows because unfortunately there's too high of a

stakes. There's too many stakeholders in the room. The reality is that there's just going to have to be more sort of closed-door meetings afterwards where the teams objectively evaluate the momentum. And that's why like most of the stuff will end up being more dog and pony shows as long as everybody's careers and stakes rely on it. It's really hard for any game team to present an objective truth about their own game. And as games become more complex, it's no longer just about the box and about USPS for executives to have conviction in these investments, it takes a lot more granular checklists and breakouts of objective experts bubbling up a clearer look at the overall game health. Like especially as we're talking about not just box premium products, we're also talking about live services. It's not just like, hey, the USPS look good. It's the fun, it's the retention, it's everything else. So on top of this, research and forecasts are flawed. Research can be bent. Forecasts can be

goal seeking rather than bottoms up reality. Making P&L's typically a pretty useless exercise. So what's better is having a conversation mapping execution against key assumptions in those forecast, key assumptions in that research to make sure that we're always saying like, hey, we said this game relies on XYZ to exceed. Do we believe we can get there? and constantly having a very objective analysis of that to drive the decision here. That's not simple, but that has to come top down mandate. That that's the way that you make decisions. And in general, you have to embrace the fact that game companies are terrible at killing games in general. Like I did I literally did a GDC talk about this I think 10 years ago. Even though I talked about killing games, like Wuga was [ __ ] at it to be completely clear. Like we talked about, oh my god, we kill games all the time. It's like no, we were pretty [ __ ] at killing games. We're even talking about Supercell. Like Supercell, hey, we kill games all the time. Super, you're also [ __ ] at it. I'm sorry. AAA devs never can make a decision quickly. So, this is

the really hard part. It's like, how do you break people out of their human nature, saying, "Yes, but please fix some things is easy versus fighting against some cost fallacy, objectively looking at the evaluation, and understanding when you're really off track." That's a really difficult problem. What I really appreciate about the blog post was that him just calling this out for everything that we all feel every time we go into one of these meetings, no matter what side of the table you're on. If you're on the game side, you're like, "Uh, it's a dog and pony show. I just have to figure out how to tell them what they want so they greenlight me." And if you're on the executive side, you're like, "I know they're bullshitting me and they just want a green light and what I'm really looking at might not actually be true." And so I think there is a lot of human nature of how do you actually have a real conversation about the real state of the game. And one of the things that you pulled out that I thought was really interesting and I wanted to talk about was research and how difficult research can be and how much of a crutch it can

be either way depending on how objective your research team is. In some cases, your research team goes native and then they just support the game team and other times they're too harsh. And other times it is garbage in, garbage out. Adam, and you and I were talking about this before the podcast is you can get false positives or false negatives if the assets you're putting into research don't do a good job of reflecting what the product is. Because players and consumers can never ever tell you what they want. they can only react to the stimulus that you're showing them. And if your stimulus sucks and you're doing a bad job, then you're going to not get great data coming out at that. And I would say even to that point, like the executive team can never be worried about that, right? Like I have never met an executive that's going to go like, "Oh, that research, what was the exact asset that you tested with? Is that an unbiased?" Like nobody's going that deep into the research, right? Because they don't have time. It's a 90-minute

meeting. The bedrock of our decision is like, hey, this number is green, right? Yep. But the reality is that that number was created in an biased way. Then it becomes, how are we really making good decisions at this point? Yeah. Every research deck, you know, I either read or work on has 50 pages. And you're right. So, it's like all of these different insights. I mean, the time and energy I put into creating the assets. I was on a project last year where I spent so much time going into and creating what is the paragraph that we show people first. How do we talk about the USPS and different features? How do we talk about what we want to understand? And then what is like a visual asset that we're creating that showcases the gameplay when the game team only has like gray box stuff to show. What do I make up and put in there? How do I tell the story? What's the voice over? like those assets are crucial and then how do I make sure the game team isn't like gonna [ __ ] all over the assets that we

know we jointly created together so that they then [ __ ] on the results and say oh well that's not actually true about the game and so to me one of the places I think we should all put more attention and focus on is in that early stage play testing consumer research so I wanted to ask you actually you know a lot of what we you know we talk about mobile all the time should AAA Hey, start taking learnings from, this is going to sound crazy, from hyper casual studios who prototype things more quickly, who get to things more quickly just in terms of like finding the fun and the play without worrying about, you know, the beautiful crazy graphics and the beautiful art in a sense. Do we care as much about that in finding the fun? I know it's not a perfect one for one and you're not going to like take a shitty hyper casual game and be like this is the next W which are four with a tech demo that's the most beautiful thing you've ever seen but are there learnings? Am I crazy? The only point I was going to make is that these decisions get so complicated because

they're mired in a bunch of other decisions that have been made whether you know Laura Mielli is on the hot seat right now because she got the Star Wars licenses, she got these Marvel licenses and they have to make use of them, right? So there are financial considerations about how much money they spent for these licenses and and what their contract is required to, right? With the Star Wars license, for instance, they never hit the milestones that they were required to. So they canceled projects and whatever else. So, this is actually Laura is like really in in a tough spot here because any she was in charge of those two licenses as far as I'm concerned. Yeah, the cancellation fee is like tens of millions of do. I mean, I'm guessing. I actually don't know, but I would imagine and they have to have a certain amount of projects in development or whatever. So, they still have these these things going, right? They evidently still have Iron Man and maybe one other Marvel game, but they definitely have another Star Wars game. So, those are part of the considerations that have nothing to do with the actual assets and the and the development process or whatever. The other thing that like a little bit of context for this is that EA has been trying to build

an action game since the 2000s, dude. I think since the '9s, right? They've been trying to be competitive. They had Action Girl for a while. I think in Montreal they built that Superman game in Tibbron that was [ __ ] terrible, right? Like they made those Lord of the Rings movie games. Remember those? Those were good. Those were good. I thought those were good. But they really wanted like a single character action game and they finally made one with the Star Wars game, right? The Star Wars game was like kind of their first success in that genre material success. So they've been trying to build this stuff for a long time. So there's that as well that they're like trying to be competitive in that space and they're using these licenses to do so. So there's political pressure there. So what I'm trying to say is that these green light processes are far more complex because there are other business considerations that are not part of what the game is. there's other things that are going on that needs to be, you know, Laura needs to prove that these licenses were worth getting, right? And so, like, you can't cancel a project like that, you know, after signing these big licenses. You

know what I'm saying? So, like there's political pressure there, you know, like I don't know, there's all kinds of considerations. I'm not close to this one at all, so I don't really know exactly what's going on, but it's just something to keep in mind. I'm sure that like slowed down any decision-m process, right? Like the weight of you signed up for this license, you're telling me 4 years after pre-production that we're now going to cancel it. Yeah. Yeah. Yeah. Like that. Yeah. But I guess to your point on like hyper casual, that is insanely tough. Let's think of this as like AAA single player action adventure games, I don't think there's a lot of learnings you can apply from hyper casual given like how much time it would take to get to a testable state for these games that would apply in any sort of hyper casual style test. the way that they think about it, right? You've got small teams prototyping different kinds of play patterns rapidly getting to a playable thing. It feels like, yeah, I listen, I get it. Like, you know, can you mock that up to be the next Jedi Survivor or the next whatever? I get

that's tough. But are AAA teams so stuck to the point of this article in the old ways, in the old best practices that people aren't taking a step back and saying, "Hey, could we prototype differently? Could we mock something up in a different way?" I I guess that's just my challenge. And maybe I I was in going extreme and saying like hey the voodoo method of prototyping 2,000 things like I'm just going to extreme just to say that AAA is stuck in the other extreme which nothing has changed in the way that you make a AAA game in 20 years and I find that concerning. Yep. Absolutely. So there is a part of this around getting faster to a faster to fun faster to a testable state better benchmarks for really really early in production so it can gives you the signal but like the issue with hyper casual is hyper casual lives or dies based on CPI and D1 retention and they actually don't care so much about D30. They don't care so much about some of those longerterm metrics. Yeah. So

that's why like even hyper casual doesn't always apply directly to normal mobile games, right? Like you still do CPI tests, you still do as many of these things, but because hyper casual could frontload so much of their risk in terms of those early signal metrics, that meant that those iteration cycles could be that fast, which is great for them. for AAA. You're we're still talking about like can we get 8 million, 10 million, 15 million people to pay 70 bucks, 80 bucks for this game? That is a lot harder problem. Absolutely. No, I I I hear you. I just in the world of solutions, how do you teach, you know, 300 person AAA team how to get unstuck from the ways that they've been doing things? I think we need better objective tools to break teams out of that earlier in production. I just don't know what those are yet. All right. Well, well done. We're not solving the world's problems here on today's podcast. We are only surfacing

them. I like the fact that Owen Mahoney after being CEO for a while and CFO of Nexon's out there just philosophizing, you know, like he's out there writing LinkedIn article like league article like just throwing grenades like getting the discussion going. I think that's good. I love it. Yeah. All right, everyone. That wraps it up for this week. We will see you next week. [Music]