Curated Panel: A Practical Approach to AI Transformation Across Player Support

[Music] [Applause] [Music] so now we are going to talk kind of combine a bit of live Ops players a b and also AI so now we're going to have a conversation about a practical approach to AI transformation across player support and we have some amazing people who I get to spend time with at Gamescom which was fun so we're going to hear from them uh we got s uh from keywords we've got Samantha from keywords uh help shift we've got Pascal uh and we got marrick so I'm going to get you guys to come on up and tell us all your marvelous uh stuff thank you very much hello hello this work everyone

Perfect all right hello everyone uh welcome and thank you for joining us we realize it's been a long first day at uh here at PGC Helsinki and uh we're uh really glad that you could join us um we hope to provide you with a couple of valuable takeaways today as well uh my name is s I'm a strategic account manager with keyword Studios uh and it's my pleasure to host this uh panel for you today um so as you most certainly know there's been a lot of Buzz around AI uh this year uh and with the appearance of chat GPT and um Google bard and other uh competitors it's been made available to the to the public at large but uh Ai and and machine learning have been around for for longer than that um so um applying Ai and automation to uh your player support process processes

in operation may seem intimidating at first but it's really quite practical and it's um relatively easy to um to show the ROI in a in a relatively short time frame so uh in the next uh 40 minutes or so we'll uh we'll try to break down uh some uh best practices some common pitfalls and some real life examples uh that can hopefully help you uh build a road map towards uh AI driven player support if that's what you're what you're looking to do um so with that said um it's time to introduce our our panelists uh like to give a chance to each of them to introduce uh themselves uh you maybe say something about yourself about the company you work for and tell us what your favorite game of all time is all right so hi my name is Pascal I'm the player support lead at uh roio entertainment so I guess I don't need to introduce the company technically we make Angry Birds games and and some

other stuff as well but mainly Angry Birds um I've been working in player support for more than 10 years mainly mobile um my favorite game oh my God careful now yes it's not going to be mobile it's going to be civilization because just one more turn that's unfortunately the thing for me so yeah that'll be it all right thank you Pascal want to go next up uh my name is Mea I'm the uh VP of customer success at helpshift which is a technology on by keywords um help shift is a conversational Ai and um automation support platform we have a SDK out of the box that gives you everything you need from The Help Center to the agent platform um through the SDK integrated directly into the game which we'll talk a little bit more about later um my favorite game and I don't really mean to say that because roio is next to me but I really like Angry Birds I've always liked it and now my son loves it so it's kind of a thing we do together so I love

it awesome and hello everyone I'm Mark I'm with keyword Studios we're a technical provider uh we also provide the player support services to the industry uh help is now part of the family I've been doing uh player support community and uh moderation for the past 13 years across a very large number of uh of games and working as a vendor uh we we are able to get that exposure uh to the industry as a whole so that's very very cool to be in such a position and grateful for the opportunities uh for a favorite game I I I would go with with what Pascal said I've lost too many nights and too many hours on civilization I probably the the fifth one uh although right now on my iPad I have the the sixth one so I I guess that's a mobile game now so that that counts there has to be all right thank you all and uh thank you for joining us um all right let's

start our discussion Pascal with you if you don't mind um and and roio how how important would you say player support is for for the success of roio and its games and a a secondary question like what is your vision of the ideal player experience so they actually tie in both both very well uh with each other so our mission as a company is to be able to provide the most fun the most enjoyable kind of puzzle games that our players can play for Generations um for us in player support that as well it means that we want to make sure that those players can actually focus more on playing the games and not getting stuck with either a bug or a question that they might have on how they could progress through the game so for us it is important to be very Hands-On being able to provide players as fast as possible an answer so that they can get back to playing and enjoying the game that makes a lot of sense now um it's our understanding that you've recently switched to help shift from a from primarily email based solution what are

what are some of the the reasons that made you think about switching tools so initially we have been working with a with a provider that is mainly email-based or or uses more email based conversations and we noticed that that was not necessarily the best kind of way for players on mobile to contact us because the least thing that you would want to do is actually be in your mobile game you encounter something and you would have toess set the app go into your email client send an email send that off go back into the game to see if it is fixed or constantly referred back to your email in order to check if you get a response right so that was one of the reasons um we also felt that we weren't doing enough with automation itself like in the industry we call a particular type of question that should be straightforward where you have a clear answer uh to the to the player we call it the dumb contact sorry for that

but that is what we call it um and it technically means it's an answer that could or it's a question that could have been avoided because you could proactively already have provided the answer um so those kind of things are things that we taught that why are we not automating that why are we having agents actually spend time answering individually to each of these questions so those kind of things combined they made us think like okay maybe it is time that we look at a better solution for this because email plus a lot of dumb contacts and lack of automation it doesn't allow us to scale with the success of the games and the growth of the player base right now question we get a lot from from some of our clients and Prospects how do you get buy in for for such a for such a a big move internally yeah it's a it's a it's a very common question and I talked with a lot of peers actually and they also asked me that question how can I get Buy in in order to switch a tool

or to get extra budget for this um the most common kind of answer I guess that I would get when I'm asking others about this is that yeah prove the ROI the return of investment that you would get from it um it's not necessarily only that it's also what is better for for the player experience can you provide self-help to players to get them back faster into the game it's a very important part um it's also making people aware of well okay there is a cost savings obviously that it's it's a big thing and especially in the past year uh a lot of companies have been going into this cost savings kind of mode um so that is technically it's still an issue or it's still a topic not an issue but a topic rather uh it is an issue for support however it is an issue for support um but obviously you need to be able to prove to your stakeholders that that you can actually make

operations more efficient and with more efficient it can be that you have a cost cutting aspect uh for us for instance it was clear that by automating more of these again sorry dumb contacts you could technically save a whole bunch of money that you otherwise would just be throwing through the window um what we actually or what I actually did was that um together with the people from from helpshift we actually we came up with a plan on okay what would this look like for us if we would be implement ing this what would our projected volume be what would our projected cost savings per conversation be and how would that affect our total budget and with that kind of information I called out an all handson deck kind of producers meeting so all the producers from from all the games we all sat together and I just explained um look this is currently what our operational cost is for each of these games how we are supporting them by switching over to help shift we would actually be able to to save this and

that alone might have done the trick for some but then I try to phrase it in the sense of like the money that I save is technically gain in net revenue for you and that actually made the majority of them say like okay this is something that we want to be on board with now you're talk um making a switch like that can be a huge undertaking right how how did you sort of manage that what were some of the things that you that you and your team thought about when when planning out this this migration was there I'm sure there was plenty of change management involved how did you how did you approach that yeah so that's a pretty loaded question a pretty big question as well so we have half an hour right yeah yeah we have half an hour left so we're good so this is the rest of the panel by the way yeah uh um so I would have to say that we were already early on very interested in moving away from one tool to another um

and we actually when we the moment that we started to have a little bit more concrete conversations with help shift we actually started in player support started to make a plan what are the things that we need to do in order to make a successful transition mhm and we realized it was actually a whole lot of work indeed I'm not going to lie it is scary um but by identifying each of the points that you actually need to deliver on you can make good progress on this so what we did was we understood that we had more than 1,500 FAQs in total for our whole portfolio uh caveat we have a portfolio of 23 live games right now so that is quite many uh 1,500 or more FAQs uh we wanted to make sure that those would be moved over from one system to another and there was no readymade solution for that because one vendor doesn't want you to move over to another vendor so they're not going to make it easy for you except hell technically but but I'm

not I'm not thinking yeah I'm not thinking of leaving don't worry um but so we we already made early on we made a plan of like exporting all our FAQs at the same time making sure that we actually revise all our FAQs because we understood in email you can have long form responses within help shift it feels more like a chat within the mobile game and that means that you need to have short form responses m so that meant that our FAQs and our K responses that we had they would likely not translate well from web platform to uh to a mobile platform so we revised everything we revised the languages the tonality within the messages we cut everything down to the essentials because people don't care about fluff when they need a support request they need to get the exact information that they require so that is one thing that we did we also planned out with our Tech Team what the actual user flow would be at what point or in which scenario would

a user want to contact us what would be the exact touch point that they would have with our client or with the game client and what would they have with the support SDK within that that client so we started mapping all of that out our Central Tech team did an amazing job like testing out all these use case scenarios they actually create a documentation uh for each of the game teams on how to actually integrate create a centrally controlled SDK through which we can actually update all the games at the same time if needed um and that has taken a couple months I would say so the prep was actually quite long um but once we actually started to implement it within the game teams it took us less than two days for each game so it actually it is extremely fast to implement it once you have the correct information documented awesome so that's kind of the journey that we have

and we are currently uh I do have to point out this is technically change management because you're yanking out a support tool from 23 games and putting a new one in 23 games uh I calculated this at one point it would have taken normally it would have taken us 20 no sorry no I'm saying for 23 games it would have taken us around two years in order to complete the whole task and we're going to do it in less than eight months that's ambitious it is it is ambitious but also the benefits are there and if I have to buy in then why shouldn't we do it absolutely well good luck thank you uh Sam let's hear from you could you uh expand a little bit on on what Pascal shared from from his experience and tell us how how help shift supports customers with the with the migration process what change management resources uh they make available yeah absolutely and and thanks Pascal for kind of uh sharing all the

great work that we did but you know I think it was a very um you know hand inand partnership to kind of create that plan what we try to do is make sure we focus on onboarding um I think if we onboard our customers correctly to the product and making sure that they're using um the core features out of the box that achieve their goals against the ROI plan that we put together um with Pascal then you know the the the schedule of migrating 23 games on eight months is going to be on track and that's kind of how we work together so we provide onboarding Specialists um that kind of stick with our customers through the implementation of the SDK and then making sure that we provide the data that can help them set up their um classification of their top issue categories um and then from there they can plan out their automation Road mapap using those top categories that they've identified um based on the contacts from their um actual data from the previous tool um then we make sure that uh all

the features out of the box from the SDK from The Help Center with the AI the Bots set up in the in the dashboard um is done correctly mapping to their processes that they've given us um so we work hand inand to make sure that it's very seamless and then we also provide training to the admins and the Agents because they're coming from an email-based platform switching to a chat model with automation it's definitely change management that we have to be mindful of to not hinder the agent productivity or they're kind of buy into using um automation alongside their work and understanding that work is to support them to enable them to provide responses faster to the players awesome thanks Sam Merck what about uh from your side like you have a lot of experience managing these big teams of people that actually provide the support is there anything that you've seen go well or not necessarily so well in in during a transition like that or or more specifically when

switching from a from an email Bas based system to an asynchronous so you can imagine that after we acquired help shift a lot of our partners want to capitalize on that and we since what January we have been transitioning a lot of of our partners over to to help shift now the people side of learn agents learning help shift or migrating from one CRM to another that's that's fairly straightforward uh they get it it's not it's not a complex task however to be to get the must out of help shift and AI that it enables you have to be mindful of what Pascal was talking about like it takes two days right to implement but Pascal had to get those resources right he had to have those game teams available for that time to integrate dsdk right that's the first step once you have that you're able to have the the platform in in in the in the game now ai is not out of the box it's not

your chat GPT right it has to be set up correctly to capitalize on what it can offer so that's one of the important messages that I I want to give everyone like yes the technology is there but it has to be also set up correctly to your needs it's easy but you need to invest the time into it uh the other thing on The Human Side as we're migrating from other crms over to help shift I as a vendor right we we we right now provide technology and people before we're providing just the people which meant like hey if there's automation I'm kind of losing business here so H what was the interest but we knew that was coming that's why uh we partner up and and now working as one by Design with proper AI we know that 20 to 30% of our staff that was working on the game will no longer be required to support tickets they might be moved to like VIP stuff you

might enable like you because you have your budget right you want to spend it towards the player right engage with them so just be mindful that 20 to 40% of your workfor wors you no longer need them for this particular task you can use them for something else that brings value but the the level of AI just enables you to to have less the other thing is if you have kpis right now when your player support operations you have specific kpi for time per ticket right be ready that they will go up like people will spend more time now once you have proper automation per issue simply because all you're easy that you I don't know if you're calling it dum dum contact dumb contact goes away you no longer have that right so but what that means that people that are answering tickets are now dealing mainly with those complex stuff right and here the pitfall if you want to say that way

is if you don't have proper process or tooling I think the previous uh speaker was talking about Tooling in in in in in in liveops and how important that is so you have to be able to enable the agent or the whoever is replying the game master to act on that player's account so you can get a lot of out of that transformation of AI it takes away a lot of the manual work to get that to the upper level you have to make sure that other tools are also in line and if you want to get the most out of AI invest the time and make sure that your game team is great thanks Merck that's great so let's try to make this a little more more tangible and we promised some some takeaway so uh Sam could you speak to to some of the most common like pitfalls and best practices that uh people can can take away from this yeah I think when you're kind of trying to test out AI or even automation for the first time

make sure you have um clear success criteria what are you trying to measure what is your goals are you trying to reduce your response time improve agent productivity and improve the player experience all of the above making sure that you have clearly defined goals so that you can measure along the way and then pivot or change your approach once you see some results otherwise you're you're not going to have a clear outcome or clear conclusion second is change management implementing AI is not just about adding Tech adding AI to your technology right you have to think about um organizational mindset shift how do you get buyin from moving from um email 100% to kind of AI and Automation and um telling everybody about the value of that so that they understand why are we doing this and what are we trying to um achieve at the end of the day third is that um AI is not a set and forget it process right AI requires training and it's what I call garbage in garbage out

if you are not training the AI with data that is good quality and when I say good quality you also have to think about the balance if you clean the data too much then it's not really going to mirror how your players talk to agents or how they contact you um how they talk about their problem but if you um kind of don't clean the data then you're going to get a lot of like um gibberish sometimes right or spam so you have to clean it in a way that actually is reflective of enough for the AI to learn with a good foundation and then that way they can improve over time and you can then understand how to retrain in production once it's live um so those those are the kind of things that I would suggest when you're kind of looking at Ai and automation initially and be ready to have an open mind um that it might impact your kpis like cat in the short term because because you're introducing something into the play experience it's

going to take time for players to understand that at the end of the day it helps them get a response faster so initially you might see CET decrease but then look into why improve those setup of the AI and the journey and then eventually you will see the sead go up because we've we've done this over and over again especially like now we're working with Pascal so I would just say keep an open mind but have clear goals Buy in you know top to bottom from leadership down to kind of operational level and then most importantly um making sure the agents understand how the AI and automation is going to make their jobs easier and where they're going to now be able to spend more time to Mer's point on VIPs on complex problems that you don't want um automation to touch yet so then you can start to really strategically shift the humans to do what Ai and automation can't do yet or um you're not comfortable with them handling sensitive processes so those are the type of

pitfalls but also best practices that I think can be thought about super valuable thank you Sam um Pascal you're you're going live with uh help shift what is it later this month we're already live oh well well with you're going live with more games what some what are some of the the concrete improvements that you're you're expecting to see yeah so as I mentioned oh my God this seems like this is the topic of this whole conversation AI and dumb contact but no like we're we're really like trying to cut we should chosen a different title different title for this panel definitely um so no this is definitely it's an outof thebox solution right like putting Automation in place identifying which are the Hot Topics that people keep or not hot topics even but recurring topics that people keep uh keep writing to you about automating that it seems like very low hanging fruit with a huge impact on our operational efficiency so that's the first thing that we actually did um the

second thing that actually would be a really interesting case is something that I haven't seen anyone else do this but correct me if I'm wrong with this Sam um but help shift uses these what they call Smart intents which is technically it's you let the end user select a classification of this is the type of issue that I want you to know about and we created an extra one there which is called bad at reporting and this is something that we had been thinking about out with our Tech Team and our ad uh ad and monetization Department um because every mobile gaming studio that is serving ads to their to their player base they will know that there is a lot of fraudulent ads out there or that there is a lot of broken ads around there and as company you're paying for that for for that volume so what we actually did with that is that we actually wanted to set up this whole workflow through which it would be a lot easier for players to immediately report an ad so they would just open up help shift have the smart intent that

says report bad ad they would tap on it and we have a whole automated process through which the player uh is asked what type of what type of problem did you have with the ad which ad is it can you add a screenshot to this can you tell us the date or the exact time when this actually happened or otherwise was this the last ad you saw like a bug report yeah like a bug report but we made it like a whole automated process and then we're not fully there yet but we have that flow set up ultimately we would want it well okay no let me tell you what we currently are doing is whenever we get a ticket in like that or a conversation in like that we actually mention this towards our at team we are telling them hey report came in about this one this is already the fifth report within one hour that we've gotten about this ad this technically means that there is a problem with the with the offering that we're currently having and they're very responsive to that but the ultimate scenario here would actually be that Sam can you get something like this maybe integration or something API call that actually goes

directly to slack and says we have received this many issues reporting this particular ad this is the data that we have been getting from this and the ad team is immediately aware of this because they just see the slack message pop into the channel they know something's up wow very creative use of the technology there there you go is that a promise do we have that on yeah no problem we're all about automation so it's right up our alley sorry Sam you just uh talked yourself into some trouble there so speaking of which uh Sam what um what else is on the road map for help shift what can we expect to see in the future yeah so uh what Pascal mentioned was um our AI classification feature called smart intense um what we do is we we take um your data from the previous CRM and then we can build a model that shows your most common problem types um and then we would use that as a foundation to create the Bots that tied to those specific us cases so um we've already had that out for about

two years um so we've got kind of a lot of gaming companies using that right now um what we AI continues to be a really core Focus area because um of you know things like chat GPT and um other uh AI tools out there that gaming companies are now testing so um it continues to be a heavy investment area so what you're going to start seeing is chat GPT in some of our core features um we have analyzed the strength and the weaknesses of chat GPT so we want to couple them with features that make sense for us to continue uh using it to uh support self-help for example um you're also going to start seeing machine translation with AI um it's another tool that keywords own so we've just integrated that natively into helpshift so you can now use U machine translation coupled with AI training to talk to your players in any languages um and provide 247 support um the third area is uh you're going to start seeing um AI

implemented across what we call a modern support Journey so which it starts with The Help Center then the smart intents which is the AI then the Bots um then the Asian dashboard so we will be um allowing kind of using chat gbt to be able to summarize the um entire ticket so that you know agents don't have to scroll to the top and you can kind of get that context and continue to improve those resolution times so and uh lastly is sentiment so sentiment analysis with AI so those are the areas that we'll be investing into in 2024 sounds like you'll have your word cut out for you Sam um I I want to zoom in quickly on one of the things that you mentioned and that's machine translation uh Pascal you mentioned that you um were going to be starting to use that to to enhance the player experience uh can you can you tell us a little bit more about that yeah so I I think this is an inevitable kind of topic uh for for support

um I'm just going to throw out the question here how much as a percentage do you think of the total gaming population would feel comfortable writing to support in English throw out number speak up more or less I'm going to surprise you it's it's less than 26% so it's actually it's quite low amount of the whole player base a global player base that we would have that would be fluent enough or feel comfortable enough in English to reach out to support in English so that means that there is three4 of the whole player population that we would have that actually would want to reach out to support in a completely different language than most companies actually support um from experience we all know uh agents with other language native language skills other than English they can be horribly expensive so so it can be quite expensive for your operational budget or your operational cost machine

translation actually offsets that kind of cost one thing that I have noticed working at several companies in the past is that players generally don't expect that when they are talked to in their native language that the answer is 100% accurate MH like one thing that you need to consider is that um the more effort a player needs to do in order to be a able to conversate with support the more of a friction point you have the more effort a company puts in trying to communicate with a player the more brownie points you get which then translates into loyalty so even if your your answer would be machine translated and it would be 98% correct it would miss a little bit of the nuances that a native speaker would actually have the player is most likely going going to score their satisfaction higher when they get helped

with machine translation right interesting American Sam I know you have uh some some uh some case studies already with clients that are using uh machine translation what what have the results been like what have you seen so there's a very sign Pascal mentioned that it's it's terribly expensive to stuff in some some languages uh versus the amount of volume or tickets that are coming in in a specific language Finnish might be one right how many of you have interacted with the customer support in Finnish I'm assuming that's the portion of the group here is finished right raise his hand yeah unfortunately so it's not it's not common right so you're missing out on that on on that group now with with language AI embedded directly it's a seamless experience the key element that we've seen when it was implemented is being honest with the player that it's not a human but you're speaking to a human but

everything they say and what you say to to to them is being translated to an engine the the perk of language AI so the solution that keywords own and one of the reasons why help shift with keywords made so much sense for us is it's a language technology developed specifically for games and what we can do is we can build language engines that are specific to your games Linguistics use right so if you have Sword of Fire translated into German or polish it could be translated as sort of fire sort of heat or something else right because of the pairing of of languages and how like if you Google translated it get gets random but in the game you have it specifically so what helps is that we can get that level of detail and make sure that what is being said or translated by the technology within support is ex

translated exactly the same as the localization team did it for the game itself so there's there's 100% accuracy those are custom engines but it just works so much better that way uh the biggest break is you reduce your waiting time for for players so you get to them quicker and again kind of repeating what Pascal said you you're able to keep them longer right they don't go to another game because they weren't able to to look log in and they had to wait 16 hours because your German uh agent just finished their shift and they will not be on until 9:00 a.m. next day right uh you can now stuff 24/7 in any language you don't care about language the other thing is as a as a vendor right we we're stuffing a lot of people and uh often have to look for specific language skills right and that used to be a a one of the core component of who we wanted to hire like we want to

hire a finish speaker right because roia wanted to have support in I'm just making things up right but but imagine such a situation right that no longer is the case now we can look for people who are best for support because that language element language component is delivered by AI so you can get people who really care about players not just people who speaks specific language and need a job right so that that helps a lot I'll add to that um for customers who have tried the machine translations the language AI feature in help shift what they've seen initially is um we can expect cat to be maintained and then cat will improve over time um but as kind of Pascal mentioned you just have to be honest that it's not native but at the end of the day players just want their answers as quickly as possible even if it's not perfect they're a little bit more forgiving um that you're trying and so

CET will go up over time so we see that but obviously the cost savings you can almost immediately see an impact if just by testing one language and then the second and then the third um so there is a lot of value in being able to provide that in 247 as well and making players happier awesome well I'm afraid that's uh all we have time for today uh I do want to do a quick wrap up uh to to um repeat the pitfalls and best practices that you mentioned uh Sam so uh pitfalls lack of clear objectives not having a change management strategy and treating AI as a one-time project bad ideas stay away best practices measurable success criteria training and using the right data being open to trial and error and clear consistent and transparent communication to all of your stakeholders um Merck Sam Pascal it's been a pleasure thank you so much much for uh for taking the time thank you um I think we have three almost four more

minutes left so if anybody has any questions I didn't want to leave space for that cool cool yes sir Sor we got a question over here I'll just go grab the mic for you hi so I was wondering about the scale when do you want to start uh thinking about this AI in the customer support like what's the amount of tickets per day that is uh reasonable to do with an AI instead of using a human to do this customer support thing thank you Sam do you want to take it I'll take it first and then maybe you can add to it since your volume is bigger um we can start AI on any any any size of tickets because um for like customers like roio that have 23 grams obviously the volume is significant but like even if you look at kind of like the the supercells that get a million tickets a day right and we just use their data to make sure the training is Fast and the training you can do without having your own AI team we have the training mechanism inside the agent dashboard admin dashboard so

it's really easy so for clients that have data and want to train it that's an option so we can do it right away and deploy it immediately um no engineering work required um for smaller clients because we have so we work with over 300 gaming companies globally any genre so we have um the the structure by those genres out of the box when you look at top gaming issues for any games I mean like 20 30% is account related issues right then it's 20% like gameplay right Technical and bug and then there's maybe a feedback or something really specific to your game so four or five out of the box and then we give you kind of the subcategories you can fine tune you can launch with that without onboarding team out of the box once you go into production and start collecting enough data then you can train it and we don't need a ton of data to train like 30 or so messages at a time you can get the quality and you can start to see the data and iterate from there so you can start any any any

volume Pascal yeah me before you run out of time yeah I was going to add quickly that I mean at which point does it become worth investing in AI um I would I would turn the question around and I would actually bounce it back at you and say at which word do do you think it's not worth your time answering the same question 30 or 40 times uh so so that is that is as well one thing to take away I mean like Sam said like they have actually a very scalable kind of solution um help shift is more than only the AI part like like that that is like let's say that is the most impactful kind of solution that they can provide right now but they also have the kind of stuff that other uh tools have like Automation and my God I have to say like they have the best automations that I've actually been working with because they are so easy to use they don't require understanding of code or anything like that like others have um so the basic

concepts are there and they already are providing like a better experience by actually being integrated within your game client doesn't even matter if it's mobile anymore because they actually just open it up to be available on console code so so yeah automate as fast as you can use AI it's a matter of scalability at that point um but you can test it out like like Sam said you don't need that much volume in order to train it already so all right hope that answers your question but that's really all we have time for today uh if you have more questions feel free to approach us uh we'll also be at the booth tomorrow just when you come in to the right side all right thank you everybody thank you thank you you thank you very much guys that was great um I mean just the whole idea of this using AI in this space is like really enlightened to me because particularly when we were talking to these guys in uh Gamescom for the community clubhouse that we did it's just like real realizing that we have these tools now and they I think particularly that machine learning

process now being good enough to get positive customer satisfaction from it the the speed at getting somebody therefore to be able to then support people live as well again massively interesting n