How Supremacy: Warhammer 40,000 developer Twin Harbour wins back lapsed players

If you're running UA in the US or in Europe, the Christmas times would be an ideal scenario. People at home, they have more time. So, if they played the game in the past, that's a good time to remind them. In the Mena, it could be around Ramadan or Eid. It fits also the same thing. People have more time. They see an ad, they are reminded for a game they played in the past, and it's a good time to remind them. So, those are the best two to start like investing in re-engagement in my
[music]
Hello and welcome back to pgc.biz in the hot seat. Now, most mobile marketing budgets, as we know, go towards winning new players. Labs players, who may know the game, get a lot less attention. Now, my next guest, a speaker in fact at next month's PGC Jordan, is all about re-engagement, and he thinks studios are actually leaving money on the table when they focus too much on winning new

players. Faisal Nimri, welcome to the hot seat. How are you doing today?
Yeah, doing well. Happy to join you today.
Yeah, I'm so excited to talk to you. We just off camera, I was So, Faisal's based in Jordan, and I was shocked to learn that it actually rains and snows in Jordan. I had no idea.
Yeah, very rainy today.
So, um Faisal, for those who don't know Twin Harbor, I haven't given you an introduction yet, but what is the Supremacy franchise, and what does your role actually look like? Do you mind just giving us a little bit of an intro to us to who you are, and then we can kick off?
Cool. Yeah, so I'm a senior growth marketing manager at Twin Harbor. Joined the team around 4 years ago. I am part of the UA team, but I'm leading two growth initiatives, retargeting and CTV. So, more about Twin Harbor, Twin Harbor is part of the Stillfront Group, the developer of the Supremacy franchise. We have multiple games, Supremacy 1914, Call of War, World War 3, and we

launched a new title recently. It's still in soft launch. It's called Supremacy Warhammer 40,000 um in partnership with Games Workshop based on the Warhammer IP. Very [snorts] excited for that title and when it's fully launched. Yes, so our games are long-term strategy games where a match can take from a couple of weeks to more than a month to end. They The game started on the browser and moved to mobile in 2019. The company is based in Hamburg and Malta. We have offices there.
Nice. So, Faisal, diving straight into things. I gave a little bit of an intro about what your thoughts are on reengagement, but why did reengagement become a priority to you rather than putting more into new player acquisition?
So, we were mainly focused on UA, uh but as CPIs are getting higher and higher, UA is becoming more challenging, we

started looking into other growth opportunities. Uh retargeting was a natural fit for us. We've been live for years now. We have a large user base. Our games are mid-core strategy games. Uh so, we started looking into retargeting as a growth lever. Um we only started like one and one and a half years ago uh to invest in that area. Uh so, we're we're still new. Uh we are still focusing on UA, uh but retargeting is the next thing that we want to focus on because we see the value in uh this growth opportunity.
So, Faisal, you're somewhat of an expert in the reengagement strategy. So, when is the best time do you think to launch a reengagement campaign?
Yes, so reengagement work best when it's paired with a big update. You're launching a new hero, a new map, a new game mode. Uh you tell the user, "Hey, uh we have this new uh update." Give them an the opportunity to come back and experience a new thing. So, this is maybe one of

the best times to invest in re-engagement. And the second time is based on seasonality. So, if you're if you're running UA in the US or in Europe, the Christmas times would be an ideal scenario. People at home, they have more time. So, if they played the game in the past, that's a good time to remind them. In the MENA, it could be around Ramadan or Eid. It fits also the same thing. People will have more time. They see an ad, they are reminded for a game they played in the past, and it's a good time to remind them. So, those are the best two to start like investing in re-engagement in my opinion.
So, Faisal, in plain terms, what is incrementality testing and how is it different to what standard attribution tells you?
Yeah. So, incrementality testing, it's like an AB test. You have the test group, which are players who will see the ads, the re-engagement ads. And you have a

control group, which is players who will not see the ad. Usually, it's the ratio is around 80% test, 20% control. Then, what you are looking for, you are looking for differences in revenue on and on other metrics. Then, you measure the uplift between the two groups. By doing it this way, you will know what is the real impact of the ads that you ran at that time. We need to make sure that the result is statistically significant to make sure it's not like a random chance. And the incrementality methodology, I think it started in pharmacy industry. When when they want to test a new drug, they always have like the control group, people who will not take the medicine, and the exposed group who will take, and then they measure the effect. So, it's the same methodology. And to answer your questions, like what's difference between regular

attribution, Uh, attribution tells you who clicked, where this click is coming from. Incrementality tests uh, the uplift between two groups. First group of you saw the ad, the other that don't.
Okay, and how did you actually set up your first incrementality test? And what surprised you about the results?
Yeah. Uh, so when we started, when we want to run incrementality for the first time, uh, we had to make a choice. We had three options on how we run it. So, the first option was to do it internally. Um, the pros is that we have the data, we have everything. Uh, the con of this is that this requires a lot of team resources. So, we didn't take this option. The second option was to run it with a retargeting partner where where we run ads with them and they can test incrementality on their side. Uh, the good thing about this option is

that it's free. Um, but the con, it had some limitations when you want to compare two different channels. Uh, so ultimately, we went with the third option, which is using a third-party tool. In our case, it was our MMP, uh, AppsFlyer. They have uh, an incrementality for retargeting tool. It was an easy setup because they it's already our MMP. They already They already have the audience, the uh, the integrations, so it was overall set uh, easy to set up. So, this is how we started to run our first test. And to answer your question, what surprised us the most, uh, the first test was a failure. It didn't generate any uplift, and the reason was that we had like a broad targeting. Uh, the audience was very broad. So, we learned the hard way that you need to do segmentation, segment the player and the audience into

uh, non-paying users. Even the paying users, you can segment them into top, mid, or low. After that, uh we It's It's started to become irregular ex- tests that we do.
Mhm. Learning from your mistakes, I love it. There's no better way. But, so that leads me to my next question, Faisal. How do you actually split your audience when you're testing more than one channel? How do you actually go about doing that?
Yeah, so uh I We do like head-to-head tests. So, we have like channel A, channel B. Uh so, what we do is we take uh the audience we segment it in our preferred way. Let's say we're targeting top-paying users. Uh we give to 40% to channel A, 40% to channel B, and then 20% control group. Then we run for like 3 to 4 weeks. Uh then we will know the winner between the two channels. After this test, we do it all over again. We take the

winner, we pick another channel, we do them head-to-head, which is like 40% 40% 20% control group. Then we uh have the next uh winner. So, this is how we split the channels. If you have like a really big audience, you can uh add like a channel C, so it's will be A, B, C, and a control group. Uh but that depends on like how big is the audience.
[snorts]
Mhm. And you talked a little bit about the mistake that you guys made. Um but what mistakes do you see that teams commonly make with reengagement campaigns or with measuring them?
Yeah. So, one of the first mistake I will touch on the creative side. Um so, recycling UA creative for retargeting is a big mistake. Uh you need to invest in the creatives, the messaging. You need to give the players that play the game, that know it, uh a reason to come back and to experience new thing, maybe new update. Uh so, recycling UA creative is one of

the first mistake. The second is I touched point on it using broad uh targeting. So, mixing uh players and non-players and all the other things that I already spoke about. Another mistake is with retargeting and with incrementality. You need to reach statistical significant and that takes time. Uh so, being patient is very important. Sometimes you have to keep the campaign running for an extra week just to reach that significance.
Okay, and if a smaller studio Faisal is looking at this and they're thinking I have a limited budget. What can a small studio with a limited budget actually take from this including studios that are in Amman, you know, it's a region that's known for indie indie games and smaller studios. What can they what can they do?
Yeah. Uh so, you can always with re-engagement you can start small. You don't have to be to go big. If you are running any UA, if you have any UA budget, you can take

5% to 10% of that budget and invest it in retargeting. Um this is the first thing. The second thing maybe it's like a tip. Instead of doing testing incrementality with the first group is 80% and the control is 20% you can have it 50/50. This way you reach statistical significant faster. Instead of like 4 weeks you have it in 2 weeks so you invest less budget to know which audience bring um more uh growth opportunities. Um the third is something I already touched on which is seasonality. So, if a studio in Amman maybe uh test retargeting for the first time during Ramadan. If they see good result, they can invest more but these are the tips that I can give as of
I love it. Now, Faisal, final question for you here and that is um I want to give a bit of a as we as we said in the beginning you're joining us in PGC Jordan very very shortly which we're so excited for. So, if attendees can

remember one thing from your PGC Jordan session which is titled a guide to re-engagement and incrementality testing, what should it be? What do you hope that they walk away from your session thinking or maybe a new piece of knowledge that you've imparted onto them.
Yeah. Um maybe the the message that I want to give is that your lapsed players are one of the most valuable audiences that you already have. You've already paid to acquire them once. Um and with CPI rising bringing them back could be uh an efficient way to grow uh your game.
Well, we can't wait Faisal and thank you so much for your time today. Thank you so much for joining me.
Yeah, very happy. Thank you for the opportunity. Looking forward for PGC Amman. I attended it last year and looking forward for this year edition.
Amazing. Now, for all the listeners, uh you can head to pocketgamer.com for tickets to PGC Jordan where you can catch Faisal's talk. Thank you so much to everyone and until next time.