Session: From Reviews to Ratings: Scaling Player Experience with AI Automation - J. Rodríguez.

[music]
I'd like to welcome to the stage Jen Rey Rodriguez Indias from AppFollow to talk about from reviews to ratings scaling player experience with AI automation. So, please come on up Jen Rey and round of applause.
Yeah, today we are going to talk about reviews, ratings. Um, probably that's more on the retention part of the gaming experience, but um, I think we've got

really interesting insights to share with you. It's good. It happens. So, while we are waiting, uh, is there any game uh, here in the audience that is responding to reviews? Good. Good. Okay. Good. Happy to see that. Um,
So, you're using
That's not mine. No. Not Igor.

We're getting there. Will I get three additional minutes?
[laughter]
I can see it here. Okay, good. So, just briefly, um what do we do at AppFollow? We help uh games, apps turn user feedback into in in into actionable insights. And in trying to read all the feedback from customers everywhere in different stores and give you the insights you need to improve your games and your apps. And also, we help reply to these reviews at scale. Probably, we don't have the teams, we don't have the capacity, and this is why um at AppFollow, we try to help um you doing that. Today, as I shared, we've built a really

cool industry benchmark report for gaming. You will have the link to to the report. Uh so, I really um suggest you to download that after the session, but I'm going to share some insights, the ones that probably stood more to me. Um and just to start, in 12 months, um we have analyzed over almost 23K gaming apps in Google Play and App Store. And this is the volume of reviews that gaming apps have received in 12 months, which is almost 52 million of reviews. So, it's a huge volume. Obviously, it's difficult to analyze and understand what's coming from those reviews. I think the most important question here is what are we doing with those reviews? Are we reading? Are we replying? what's happening with them? So the first thing I'm customer success, so probably that's why it surprised me

the most is that almost a third of gaming apps never reply to a single review. Like nothing. Not blaming anyone. I know that there are different priorities. Teams are tight. We don't have the time, too many priorities. The only thing I care about is the user experience. And in this case, we have users, players leaving reviews. And they are not feeling hurt. They don't They don't get anything from us. So I think that's the worst part of not replying to reviews. I am sure that these reviews are taken into account, that you are analyzing, that you're taking action, but they are not getting anything from you. Obviously in this report, we have analyzed the global market and our customers. And obviously we put a strong effort in showing to our customers that replying to reviews matters. So that's why we can see here that our customers on average

are responding over a 50% to both Google Play and App Store. And the global average is roughly 23-35. Then what's the data What's the connection between those two? Because I get it from a lot of customers, why should I even respond to reviews? That's not my focus. And I completely get it. But this is what we got in the report. So those apps and games that reply consistently to reviews get a higher rating. So I think here you can see that in general, our customers are on a 4.30 stars rating and the global average is 3.48. Obviously it's 23,000 games, so it's a lot. But this is what the data is saying. Here we can see a a bigger drop down, like correlation between reply rate and stars. Obviously, what we can see here is the more you reply, the better ratings you will have.

Then, um we've seen that the industry has gotten over 50 million reviews. I don't know if in the audience from the ones that are responding to reviews, how many of you receive more than 1,000 reviews per month? No? More than 10,000? No? Think it's probably guys were here before. They are receiving that amount of of reviews. The thing is that we cannot manage that manually. It's impossible. You cannot keep adding head count to reply to reviews. It's not scalable. So, we need to use automation AI for that. And probably most of you are already doing that. But, how this translates into the this benchmark report. So, with AI we can reply 12 times faster. Obviously, manually we're not working 24/7. We don't have people speaking every language of our customers. So, with AI,

we can respond in all languages 24/7, and this is why we can on average respond in 25 hours. The thing here is that we do have store moderation time. So, in Google Play usually, you send a reply and in 5 minutes it's published. Apple Store with the store moderation takes a bit longer, so that's why this is the average. But, there's a huge difference and again, the main difference is that the players are feeling heard, and they feel that they what they say matters, and this helps with retention, growth, etc. etc. So, not only we are responding faster with AI, we are also covering more reviews because we don't need more people. AI can do that for us. Automation can do that for us. So, we can increase response time and also reply rate. And as we've seen, reply rate translates into higher ratings. Then, um

we can reply to reviews, but obviously replying to reviews does not improve our product, our games, right? So, if you have such big volume of reviews, you cannot read every single review in every different store, in every different language. This is why AI overviews, AI summaries can help you with that. Can help you understand what you need to do. What are your strengths? What you can promote? What are your pain points? What the customers are saying? What you need to escalate to product engineering? What needs to be changed in your product? So, AI overviews, AI summaries need to be done to understand if you want to understand what's being said in specific country, specific market. This can help you with that. And then obviously you can get more granular with AI. You can, for example, if one of our biggest issue is monetization. So, AI will allow you to go a bit deeper. So, in this case, 35% of your reviews

talk about monetization. What are our customers saying? It's about ads. User complains about too many ads, not a surprise. Subscription issues, paywalls, etc. So, this is how AI can be a good tool to highlight the issues. And then, obviously, by replying to reviews you can tell your customers, "Hey, we are on this. This is what we are going to do, and this is what we've changed, for example." Then, again, proof from the data and from this report.
[clears throat]
In this case, it's the outcomes from our customers from responding to reviews consistently. They can improve the rating. We call it reply effect. You are responding to customers, they are feeling heard, you are sharing what you've done, and because of that they are updating their review and giving you a higher rating. Eventually, that's what we want, updated reviews with higher rating. You can also see here that 77% of the top-rated games in the industry reply to

more than 30% of reviews. It's not a lot, but it's uh 30%. And then you can manage higher volumes of reviews without adding additional head count. This is a bit controversial, I know, but it doesn't mean that you don't need that people. I think is you can focus that people in what matters most. So, I would use automation for easy reviews, like five stars, no issues, and then focus your people on those reviews that are really showing something that is not working and giving a personalized response to them. So, having your team working on what matters most. And then the secrets the cycle of review management, we have covered that. We can reply faster, reply faster, and reply more. That will take us to higher ratings. And then higher ratings will give us more visibility. Yesterday in in different stations uh with all the games that are published at the moment, the issue is visibility. So, having higher

ratings will help you be more visible in the stores. And then if you are ranked higher, you will get more downloads, more revenue. And eventually it's what we all want. So, I want I brought some case studies. This is what usually our customers request. It's like, "Okay, but how how are they other customers using your tool and what they've done with they've achieved?" I think it's really three cases that I that I really liked. Toca Boca is a gaming studio for kids games. Their challenge is that they were receiving over 75,000 reviews per month, which is impossible to manage manually. They were trying, but the thing is that they were having a negative reply effect, which means that the users were not valuing the responses. They were They were not They They were not valuing there. Maybe it was too repetitive. It was not working. What they did, they were using

templates. There are some companies that have strict deadlines. Use template, that's okay. What they did is use templates with AI rephrasing. So, you can have one template and AI keeps rephrasing that so that you don't sound robotic because if you go to a app profile and they are all sending the same reply, that's not it's not personal. So, AI is rephrasing this so it looks personalized, same content but in a different way. They are also tagging all the reviews and this helps them spot issues fast and also have an accurate reporting. So, if if they want to have report on bugs, so this tagging these reviews is helping them analyze things and and sending it to the relevant department. And how this translates, obviously they are covering they have a higher reply rate, they are covering more reviews, but the most important thing for me is the reply effect. So, the reply effect changed from minus 0.3 stars to 0.45.

So, customers were seeing the value. Reviews were being updated to the better rate rating and this helped them increase the rating overall. Probably my favorite, Hyper Hack. I have two cases from them. I think they did did really a an amazing job. They were having issues with negative rating below four from a legacy game that they improved. So, they wanted to increase the rating, obviously. So, what they did is they sent to those users that had sent a negative reply, they sent a campaign to more than 50K users saying, "Hey, we have an improved game. Why don't you test it and share your feedback with us?" With that campaign, 1,000 players updated their rating. It's a 2% conversion rate, not a lot, maybe, but they managed to improve their rating by almost 1.5 stars with this campaign.

Users tested the new game, loved it, changed their rating. And as their main concern was increasing the rating, another thing that they did is report all those reviews that were spam, offensive, nonsense. Probably you have a lot of that in our profiles. So, they did it retroactively, too. They reported concerns from 2021. And I think the good thing about reporting concerns is is you get two things. One, your public profile is cleaner. You don't have that offensive those offensive comments in your profile. No one wants that. So, you can report that. Obviously, there is this the store moderation, too. But, on the one hand, you remove that those comments from your from your games, from your profiles. So, I think it's also good because new potential users don't see that. And probably if they see that would think about downloading download your game.

And then, you are also removing those negative reviews. Usually, spam and offensive reviews are one-star, two-star reviews. So, by reporting them, you can remove all of that. So, I think it's you you kill two birds with one stone. And this is what they got. From 7.5K reviews reported, they almost deleted 45%. So, we helped them report to the stores. This is something that you can do manually. And they removed almost 50% of these reviews, which is which is huge. So, just to wrap it up and give you some insights after, again, working with a lot of customers. Um my first advice would be start replying to reviews. Um again, it's okay if you haven't done it before. Today is the best day to start. Maybe Monday.
[laughter]
Uh the second, target a 30% reply rate. But, very important to have a clear

strategy. Don't start responding to everything if you don't have the time or the capacity. Focus on negative replies reviews first. Feature reviews first. Feature reviews are the ones that your customers will see in the stores. These have to be 100% responded to. So, negative reviews and and and feature reviews. Use AI. Obviously, with AI you can cut response time, cover more reviews, and your customers will feel more heard. And then, I think that the most important one, how would you know if your strategy is good? You have to measure the rating effect the reply effect. After you've replied to X amount of reviews, what's happened? Have your users updated the review for the better? Have they uh updated for the worst? If it's for the worst, you need to check your strategy. If it's for the better, keep doing that. And I think that's all for me. You can get the slides there. You have the link to the to the full report there. It's a

long report. There's a lot of interesting data. And yeah, that's it. Thank you very much. If you have any questions, happy to take them.
Thank you very much. Really insightful uh really insightful talk there. Does anyone have any questions? Yes, we have a question there from the audience. Um, can we get you a mic? Excuse me, I'll tell you what, I'll grab one of these.
Oh, yeah.
And then you can then you'll be able to hear it now. So you can hear.
Yeah, yeah. Thank you.
Could you tell us a bit more about reporting the bad reviews you talked about? Is it automated? Do you detect them? Do you report them automatically or does someone go through them manually and then report them on a map?
You can do both. I mean, in the store you can report them, but it's a manual effort. And I follow what we do. We have semantic analysis. So we we tag these these reviews are offensive, spam, or nonsense. And then you can automatically report them in bulk.

Or as soon as they get it, they are reported.
Okay.
Yeah. It can be deleted or not, but at least it's worth trying.
How long do the stores usually take to act on them?
It varies. We don't have a clear timing. It could be from 3 days to 2 weeks. Yeah.
That's great. Great question. Thank you very much. Uh, do we have any other questions while we're here for Janira? Uh, how if people want to uh follow up with you afterwards, what's the best way? Are you in the meet and match system? Are you here all day?
Yes.
Great. So people can catch up with you as well. Yeah, that's great. And uh and your slides are available uh they're on the
Yeah, you have the QR code and again, if you want to get the report, you can send us a message on LinkedIn and we will happy to to share it. We also have some cool stickers that we prepared for the event. So you we can we can uh Olivia has them.
That's perfect. Well, thank you very much for that. Really good to see some practical uh uses of of AI. Thank you.

And a round of applause for the our speaker for our talk. Thank you. Thank you very much. Really good.
[music]
Mhm.