Fireside Chat: Rovio's AI Transformation for Improving Player Experience - Rovio & Keywords Studio.

[Music] [Music] [Music] uh my name is Samantha Pang I'm from help shift which is now part of keyword Studios we are an AI and driven conversational support platform that provides uh the support experience for mobile of course but also PC and console um with me today is Pascal let him

introduce uh his uh experience in gaming yes so hi my name is Pascal I work at roio as the player support lead and yeah I'm all about sharing my experiences with keywords and help shift thank you Pascal for joining us today um I want to start by asking you how does roio um think about the player support experience in relation to your game success and what is your current vision for the ideal player experience and this will segue nicely into the rest of our chat all right so as you may know roio entertainment has this small IP which is called Angry Birds uh you might have heard about it um so our mission is to create games that people will enjoy for years to come and in order to do that we have also come up with the idea that it it's not enough to actually just make great games but we need to operate our games in such a way way that they can

engage and retain players for a longer time than just first 30 days or something like that and de well or tied with that that means that we need to be able to provide experiences that are suitable for our users but especially as well suitable whenever they get into trouble because our motto is that we want to make sure that players don't have to worry about what's going on in the game we're trying to help them to get back as fast as possible within the game so that they can just enjoy the game that's it I've heard uh that a lot um from the mobile gaming companies we work with especially with all of the um changes from Apple side and kind of all the data privacy the retention has definitely become a lot more important um next question I want to uh ask you is uh about your change in your support Tex stack you've made some major changes in 2023 moving off a primarily email-based support support platform to

help shift so starting your journey on AI and automation at roio can you tell us a little bit about uh why you decided to do that and what were some of the sort of the objectives in doing that yeah so like Sam said indeed we were using an email based system and that was one of the main things that we wanted to get rid of because it makes no sense when people are playing mobile games that they actually would need to open up another browser or an email client and actually start typing their message in there while they could be just in the game send a support message through the game the SDK from helpi for instance would get all the data that we would need in order to assist a player so that was one of the main things um there are a couple other points um I'm just going to list them they're not in any order of importance but they might uh shape the whole picture so one of the other things that we were actually noticing was that by having an email based service provider we actually could not Double Down on our e efforts in order to

increase deflection and deflection is actually one of the main things that we wanted to look at because it made no sense to us that we would have human Workforce answering what we would call dumb contacts so questions that a person would have to which the answer should be quite obvious uh does it make any sense if you have um particular kind of question that's being asked 100 times it like what's the point of having human Workforce actually like work on that you should either automate or you should provide an FAQ that a user can read so that they can get the answer there by doing that you also are able to push them back into the game faster because they don't need to wait for that particular response that has been in your C response folder or something like that um so our deflection was before we migrated to helpshift was 20% after we migrated to help shift just by implementing help shift we actually bumped it up by 40% so we have actually

a bit more even it's 63 that we are at right now so from 20 we went to 63 just by implementing help shift um another uh reason was that um our service provider was not focused on games which to us that made it a big hurdle because they did not understand the pain points that we were talking about um there was also the billing model the billing model was technically based on seats instead of volume or how you actually deal with volume um which all means that you're more dependent on human Workforce which inflates your cost especially if you have inflation so that was also one thing and while they actually offered Bots and automation they were very clunky and they did not allow for a lot of customization which meant that while we were paying premium price for Automation and Bots we never saw actually the return of investment on that so all that link together that made

us want to want to switch thank you Pascal um I remember when uh you were at next games um that was our first interaction and uh we hadn't launched Bots yet at that time and you had left next games and then you went to hatch uh and then it was roio um and I think when we first had the conversation about switching to help shift and I I shared some of the deflection rates in the automation kpis you didn't really believe me I had to take it with a grain of salt come on I mean they're promising Like Heaven and Earth so obviously I was going to be like okay I had worked with help shift in the past so I had the idea that yes what they have is valuable I can see the uh the benefits towards the player experience but I was cautious about it but I have to say like damn it actually does work so this is not I'm sorry for saying it but this is not definitely thank you I appreciate that and and just to kind of clarify we didn't really build help

shift um intended for the mobile gaming industry I think they found us because they're all trying to keep players inside the game um when they need help and not take them out to a browser or get an email um because retention is just becoming so important and um just the game behavior of a mobile it's like you know few minutes at a time rather than PC console which there's a bit more patience to wait for support um and our journey is about starting with FAQs and just by having good quality content you can really uh Drive self-service and you're not trying to deflect the player you're just trying to provide content for most commonly asked uh questions so that does really work and what we see typically is uh clients that move off an email-based platform and come into help shift immediately they see a 20 to 40% volume drop because of the faq's um as

kind of the first uh stage for a player to really try and self-service so I'm really glad you saw those results and we're just starting the journey I mean our Benchmark is 90% for faq's deflection and you're at 63 so I think uh we can work together to uh bring that up more I I truly believe so because when we had it implemented at next games I did get to 96 even yeah they're still main yeah it's amazing so I mean at the end of the day we're just here to make sure that we can provide a good support experience for a mobile gaming community so really appreciate uh that you were able to achieved that and followed uh some of our best practices to get theirs which is what we really want to do is have the SDK out of the box and then all of the uh customizations can be done without heavy developer work and you can chune the experience as uh your games or player Behavior changes as well so moving on to the next question question

um this scale of change that you did at roio was across 23 games right correct multiple Studios I can't even remember the number you told me before five Studios studios3 games there's new ones and there's uh older games as well so we definitely had to create a very solid plan uh I only saw the plan that we work together but I can't imagine what it's like internally to uh kind of develop that strategy and the human resources behind kind of uh planning out that migration can you tell us a little bit about that and how you kind of navigate a change management because the larger the gaming studio the larger it is uh the the effort and also the key stakeholders that you need to get buying from okay let's book this room for 30 more minutes if I am going to have to do the whole story but I'll keep it like quite top level um we already knew for a long time that our provider was not a provider that we would partner up with

for a very long time um we were already planning one year before we signed with helpshift that we were going to go into a particular kind of Direction Where We would want to utilize Automation and Bots a lot more that actually meant that well one year before we actually made a shift we already started to uh to review the way how we had set up FAQs how our C responses were set up so where we were working with emails before we could use long text or long form text that would not work with Bots you cannot on a mobile device send a user like 120 lines on this is how you do this in the game or how you utilize this feature so we already started dividing up our can responses into particular kind of steps that we thought this is where a bot would have this interaction then the end user or the player would actually respond with this question uh with this answer or this answer let's already create um a wireframe of what kind of forks we would

have into our interactions with the bots so we already started doing that way ahead and had we not done that I am quite sure that we would have run into serious problems especially because we had a huge amount of games in our portfolio in uh to deal with um so what we also had to think about was how do we consolidate technically everything on our from all our games but also all our Studios because we had as as proia we had acquired several Studios started up new Studios and we had these different kind of cultures within each of these Studios while still being part of the bigger Roo kind of culture but it also meant that the operations work completely differently they had different ways of working so we try to consolidate everything like that we try to to put on paper or digital paper um what the ideal ways of working would be with support how we would actually transition from an email based service provider to an inapp service provider um what we also did was we did our homework

with our with our Central Tech Team because what we knew was that we can just like integrate help shift and and that would solve it no obviously not because that would just allow us to like just send responses or put automations out there but it would be working on no data so what we had to do was use our centralized data platform and try to integrate that with or at least link with helpshift so that when an agent actually gets uh a conversation from a player they would have a a link that would immediately bring them to the dashboard of that particular player profile so that they could interact on that but also on the other end we also could pull data from our centralized data platform and put it into help shift so that we can run automations and bots on top of that data which makes it super powerful because that would allow us to like create ideal workflows make sure that people who have been stuck with a particular kind of problem uh or have a particular kind of spend tier that they would follow a different kind of flow than someone else would same with

languages if you have like one market where you know that there's a particular kind of event going on uh and it's not in other language based markets then you can just use that data and create a workflow to actually just only get those people in a particular kind of queue have that visible to particular kind of Agents um yeah I'm I'm nearing those 30 minutes already have the feeling um there is a lot more that I can tell but it it it was it was not a small feed it definitely it isn't and and I think well when we're touching on pitfalls I think I can elaborate a bit more about this but it's not to be underestimated what you're doing it is not just yanking out one system and replacing it with another you need to be fully prepared with that you need to assess the risks not to mention that you went from 20% automation which is kind of the um threshold you hit with the previous platform and then coming into help shift where we started you with a 50%

Automation and then adding AI on top of it so how did you orchestrate all of that and what were your kind of thinking behind testing Ai and automation with us I'm actually still not sure that I I fully realize how we actually managed to do it to be fairly honest um no what we what we did is that we actually we did it piece by piece right like instead of just like going full on with this and trying to implement it at the same time in 23 games that's like crazy that you should not be doing that uh also we did not have the resources to do that because someone still has to make games so or answer players or assist your vendors um so what we did was that we we kind of like identified a couple of key games in our portfolio where we thought like okay if we would change out our current provider with help shift we would potentially see the most benefit out of this so those are the ones that we immediately we went for we did a couple of those and and it seemed to

work more than we actually anticipated like like quite honestly like we did almost not to anything like other than the stuff that we had already been preparing for for a year ahead so I'm sorry for saying we did nothing because technically we did a lot um but it was in the end it was relatively effortless the last bits of that um so we weren't setting up like a lot of automation we weren't utilizing the Bots a lot in the beginning um but once we started doing that we did it piece by piece and we again have been like needing to redo Bots we have been needing to redo automations every time when we started adding a game in the beginning we felt like okay no we need to review this this doesn't work anymore like this um we kind of like had to sit down at one point and say like okay it's time that we determine what is the ideal setup for us in order to actually build up our automations how is it applicable to all of our titles within the same IP because that would make it the easiest and since the major it of our games actually rely on Angry Birds well that was rather easy

um but I would definitely have to say we would not have been able to do this if we didn't allocate one resource with within our team that actually was 120% dedicated to figuring out how help shift actually would work how it would work best within our portfolio and that they would actually lead the efforts and and with 120% that did not mean that they actually had to do overtime that that's not the point I think really worked well was we were able to take your data history without the pii in your old system run it through our AI system that helped you with um understanding your most common problems that players were contacting you about and then being able to design the AI and automation road map from that and I think that's why you felt it was a little effortless because that's basically our model we start the journey with AI to understand the player's problem one once you identify the problem then you can send them down

a defined path that a bot handles to collect some information verifies their identity um and then decides whether to hand over to an agent to finish off the process or talk to your backend system in which case Pascal did integrate his um backend uh game data system to help shift so we're able to push and pull data based on that defined workflow and I think that's how we kind of have a very structured uh plan that allows us to iterate and continue testing with data behind it um so talk about um then tell us tell us some of the pitfalls or learnings um that you learned along deploying AI in automation with us all right so early on we we had figured out that this is me touching back on this whole point of like 23 games uh game teams having autonomy about how they're implementing things um you just can't you have to like put your

foot down you have to just explain that okay this is the one type of integration that you can do it has to work similarly because otherwise if you have 23 games you have 23 different setups uh of help shift then that means that you need 23 different types of procedures in order to deal with one particular kind of issue so that makes no sense so try to get all your game stakeholders together and try to make them aware of like okay if we unify these kind of things if we consolidate and we do it in a particular kind of way it's going to be so much more efficient and so much easier and and well you you can use success stories from the previous kind of games and utilize that to convince your stakeholders that's what we did um another thing that you need to be extremely extremely mindful of is that implementing something like this and especially like working with AI it's not a one-time investment it's not like hey I'm going to have this one person dedicate this whole week into setting this up and it's going to be done no you

have to constantly be meddling with it and I don't mean this in a bad way but it's like every every every game works kind of differently there your audience Works differently so you're going to have to fine tune constantly um so it is a lot of work actually and and especially when you are changing something it might have an impact on other aspects of the experience so then you have to find unit there um but still having that kind of investment someone who can constantly like keep an eye on things it's wor it it's it's definitely worth it because that is how you can control the experience towards your players I'm quite sure that we yesterday we were having drinks that that we were talking about some other things but now I can't come up with I did not drink that much that's not the reason though no that's good um I think in summary it's really you know you you they it's garbage and garbage out for AI so you do have to invest a little a bit of time understanding the data and uh what we

want to do initially for the training and then having an iterative process um with data right and then you know working closely with your team we also have our own data science team um so I think that would that makes the process a little bit more manageable and we'll get there and we did um in nine months we we were able to get everything on boarded and working correctly and you hit your kpis that I promised um yeah and and you're right 9 months 23 games in 9 months time like wow uh so I want to move on to another AI topic which is machine translation uh in 2023 this is definitely kind of one of the top things uh we heard from gaming companies that they were starting to think about machine translation um I know you've been testing it um so can you tell us a little bit about that experience and and sort of the potential outcomes you think about when you test machine translation yeah so what actually

matters a lot is well this is tying in to a little bit of a different topic but I already touched on that as well it's like speed of response like it's it's more important the faster you you actually like reply to a user um using machine translation it also means that users technically can get service in their native language faster uh you using humans in order to actually provide that native language experience it's quite costly so I would say that's the main one there yeah what we've also seen is now um players that uh have languages where you can't provide that human touch is now able to get support 247 and what we are seeing is that for gaming companies who never provided that uh support in the native language and just started with machine translation they actually see cat go up in that language so it kind of helped us uh understand that players just want to get support fastest at the end of the day and they're accepting the quality of

machine translation which is also getting better um so in the interest of time this is our last question um what kpis do you look at um when you're kind of testing Ai and automation obviously seat I mean I know it's a little bit of a vanity metric but it it at least like it gives you the right kind of direction of like how how accepting are our players of uh machine answers uh turns out not always as much as with humans but the tradeoff is actually better in my opinion and and it changes completely when you're starting to use like language translate well well uh language AI or or yeah machine translations actually then it works actually really well um another one is uh reopen rate like that's also one that we're actually looking at trying to see if if a bot or AI has been responding to a player is the user satisfied with it or will they want to actually reopen the

ticket so I would say those are the main important ones actually no the third one speed of response obviously because we've been talking about that yeah speed of response like we have games where it takes less than a second to get a response great well uh thank you so much for sharing your experience with us you're welcome and and sorry to chase you off it's like so much nothing there I mean it's thank thank you round of applause please thank [Applause] you