Session: AI Powered Community Relations In App Management - Anatoly Sharifulin, AppFollow.

Heat. Heat. [Music] So, next up, uh, we're going to continue on with practical AI track. We've got AI powered community relations in app management. And for that we've got uh Anatoli Sheriffulan uh from from Appfollow. Sorry, I'm British. I'm I'm the worst at this. That's okay. Yeah, please take it away. Uh hello Barcelona.

I'm Anatoli uh from Helseni Finland and I'm happy to share some my insights about EI and automation how to can how you can use it now uh for uh communic for reputation management uh in gaming. Okay, first of all, uh where uh community lives some essential channels as a social media or like modern one, Discord, Reddit, X, uh some one-on-one communication as a help shift support when you could talk directly to your customers or players and uh store reviews, uh App Store, Google Play as a major platforms, a secondary app store and Android stores. Um but volume depends on sources could be different and depends on portfolio or game and the thing that uh in some feedback you could find really good insights for your product some like buck reports or feature requests some issues about monetization and something like that and this example of Facebook comment under

like uh event announcement and nice to see that some issue with account could be exist there uh and my focus will be about apps on Google play when we in a follow have more expertise. The thing that if we understand power of ratings and reviews uh for example look just like numbers how many ratings at the moment for really a good title subway surface like several several uh millions of ratings and if you compare to iOS it is completely different numbers but the thing that at the moment reviews has like more complicated sometimes uh information about player experience and source of deep insights. Uh and the thing if you compare average rating by stars to average rating by reviews you will see difference and this is visible for everyone in store for users for you as a managers but this is visible only

for you and you know oh our game has some different more complaints and lower average rating than like benchmark because benchmark at the moment 4.0 zero but bestin-class 4.5 and higher. What about volumes and this is like three uh games what we are prioritized for this slide. Uh and when Activision launched Call of Duty War Zone, it was in first month really a lot of reviews 240k and this is really huge and need to analyze. Uh but do you know for what non-gaming application uh you could see like really a lot of uh reviews what kind of application have it is just WhatsApp like who even my mom or grandma has access to WhatsApp and potentially uh write some uh reviews but they have 300 uh K reviews and the thing that Chpt at the moment after launch a lot of new cool stuff has the same level

and the thing and the problem how to detect really uh problems or feature request and improve uh customer or player experience. If we compare some like Finnish uh portfolio uh games, you could see like really different volumes. Some titles has like really a lot as a Brawl Stars by Supercell, but in Roio you could see like really less reviews, but the thing the value exists in every uh game. And this is our first stop for AI and automation that first use case what you could manage easily because you have now summarization. You could understand in like 10 sentences what uh customers or players uh want from this play. uh you could have after translation from different language to your native one English or Spanish uh and you could chat with this reviews or with this feedback easily this is now available and this is really really cool okay let's go next uh

what uh users are saying and if you go like by uh stars and subway surface legacy one and new title you could see some interesting thing that uh legacy uh game has more than 50 80% of five stars and it's it it means really good game at the moment. No issues at all. But the thing that new title could have different distribution by stars and more problem exists and number of five stars really less. And after our some research we found that five stars some sometimes have negative sentiment when stars five but some negative topics could be there. And this is why you need to care about all uh reviews and topics really interesting to understand that it could be thank you or feature request or UI monetization issues. And the thing that featured reviews something unique for app store and Google play. This is like top five reviews for every country. What

visible for everyone and the thing if like no response here from you as a developer or from your company there's something not so good. We know when we try to choose something on Amazon or demo we really uh read reviews featured reviews and then send better is it like issue complete like fix it or not and if no response this is the problem and the thing after our analyzing we understood that uh gaming company as a average have 20% uh reply rate but who started to reply they like 40 50 could have but some of them on this chart you could find some names have like really good uh results but the thing that like size of cycle it is size of volume of reviews and doesn't matter what is it because if you optimate uh this will help to achieve better better results and uh if you check uh categories like puzzle board kazino now has really a lot

of reviews what need to manage right and this is some examples of current best practices how to reply to reviews. First of all, if you don't know how to redirect your users from store, for example, you could redirect to social media. Something interesting could be in discord or uh other communities or you can like easily go to uh help desk by uh support email address and it will be uh useful when you discuss it uh directly. Okay. Okay. And this is a second stop for AI and automation. Uh because at the moment as a legacy approach like manual work on reviews when support agents uh have like Zenesk or other system and it means low reply rate and high cost to manage it. But in the other hand with EI and automation you could automate it if you have some categoriz categorization

sorry uh you could uh use templates rephrasing after translation or EIP replies based on some custom instructions just to make these replies more human not just robotic. use some tone of voice, some uh instructions what really your company or your game uh have and improve quality and it looks like as a human way and the thing at the moment what we see on the market like hybrid approach really work when you automate as as much as you can uh if you're familiar with support uh word like line one it could be EI and automation but line two complicated cases featured reviews or reviews when rating down it could be on human on support team or like community team and I'd like to share some uh best practices at the moment for example uh advanced level best practices uh when you fix the bug for example one to many

approach will be really useful because if you faster answer to reply after fixing the bug or after improving something you would need to update and now we understood that up to 20 30% it depends on game could update the original reply original star and it will increase average rating. So one to many approach just to have bulk reply still work and anomaly detection something interesting because you could understand this issue what you find now it is smoke or fire for example my favorite anomaly detection is the moment plus 20% for example in Brazil uh market dissatisfied uh users or players because you change or your marketing team change pricing or monetization or like increase ads and this is what you could know not just uh one quarter of the some like changes in marketing but after one week for example. And last thing about aftermation

reporter concern now consoles has uh feature when you could spam offensive and no sense reviews uh sent to reporting and a store will remove it. So you could easily automate it and have as a hygien thing just uh like only good and valuable reviews available on your uh store story reviews list and last thing and this is will be short point for EI and automation about measurement about reporting and we prioritize and we see how customers current customers prioritize what could be useful first of all average rating as a essential metric marketing product and support team understand it. It is easy reply time average reply time it is really good and on this chart you could see that if you start to use automation more and more potentially your result will will be really great and something unique thing uh how you use automation it is

automation coverage or tech coverage uh if you want uh to cover all reviews automated it will be 100%. This is really good stuff but in hybrid approach 40 50% plus it is okayish and some uh other stuff especially for leadership or higher level overview when you could understand better okay if you have several uh games or uh countries and you want to track average rating why rating down or why it is increased and AI could help to understand better okay it is because crashes it is because uh new ads or uh new levels what you designed and now every player satisfied. So this high uh overview high level overview will help to understand on different level what happening because you know uh customer voice and what about uh benefits? Yeah, as we

know now from current customers who use it, automation and AI, it could uh optimize your money like costs, time and make potentially you and your team happy. And as a summary uh and this summary EI helped me helps me to create first of all collect player feedback across different sources really important and now every uh voice matter and need to focus on actionable insights and react as much as possible quickly because if you from support and player community team share this product producers okay we have some issue or we need to fix it and improve next version or next release there something what will work in your case. Hybrid approach something what at the moment work and we recommend to use and once again aftermate as much as you can and everything else could be on uh support agents and we believe that stronger community uh relations uh will drive

users engagement and revenue in the end. And last thing, last point, this is uh feedback from our customers that at the moment I see shifting uh shift gaming to live services and understand player sentiment uh at the moment uh really super valuable and if it is easy just one click one report this is something what will help uh everyone. So that's all. Thank you for your attention. These slides you can get from this QR code. Thank you. testing. There we go. Uh, thank you very much for that. That was really great. Does anybody have any questions they would like to ask? No burning questions. I have I I have a question. So, uh, yeah, please. I guess when you're when you're kind of checking checking community feedback at scale, how do you account for and check for any

Well, with AI, people always I know it's more generative AI. We're talking about kind of hallucinations or things it just gets wrong. So how do you kind of account for anything you might find that sorry how do you check if anything's incorrect in what it finds incorrect in reviews you mean or so if it's it's probably I imagine it's like tracking for keywords and things people say but is is is there a way to check if it's actually giving you incorrect information? Interesting. Uh or maybe it's just all correct because it's all based on actual Yeah. Yeah. It it could be some issues uh especially if a model start like add something new for for for the model. But at the moment if you have some adjustment on your end this is like at the moment secret source because like pure models uh train it by general information. But if you try to train on your for example feedback and reviews and you as a owner of this data know

what is the best what is right what is not right it will help to get really good quality for example in our case now when we focused on quality for replies this really important and at the moment current quality is good but if we for example add your knowledge base or uh some help center articles this will help to achieve better results in replying right answers in right time as a human can do. So like hybrid approach once again something what should help in all all these cases when you think about quality or incorrect data. Yeah. Thank you. Yeah. Thank you. Uh any any more questions? All right. Uh I think thank you very much. So yeah, another big round of applause. That was fantastic. So thank you so much. [Applause] [Music]

Heat. Heat. [Music]