How to Encourage Social and Emotional Learning In-Game in a Data and Research Driven Manner

[Music] thank you hello and welcome to the talk about how to encourage social and emotional learning in-game inner data in research driven manner so let's start a meme always says more than thousand words and today we will prove at at least one of these two exist if the right circumstances are there my name is James Cullman and I'm executive producer for Star Stable and also the platform for the pilot that we're going to talk about Star Stable is one of three products within Star Stable entertainment and the other two being wild Curry and wild song and Curry besides being the executive producer for

Star Stable or at the time taken the producer I also for this pilot also the driver and initiative taker on stage I have two friends with me and without them this pilot wouldn't exist hi my name's Alex I'm the founder of ultra Lu we build AI technology to create and maintain positive communities I previously worked at Google within their trust and safety teams for a number of years and hi my name is Paulina and I'm the co-founder and CEO of Pepe agency we're a swedish-based agency specialized in developing social and emotional learning experiences through play and today I'll share a little bit about how we went about when we crafted the in-game message that could encourage a more positive and kind Behavior online so before we go into the pilot let's talk a bit about the platform that we use the Star Stable game we are an MMO

and our target audience is girls you come for the horses and the equestrian dream but you stay for the friends and the community on screens we see four of the main characters the soul Riders the soul Riders are a symbol for friendship kindness and a bit of magic they all have their own horses just as the player and the players can choose between 100 different horse breeds and still counting some other facts about the games we have been around for 10 years and we have 650 000 monthly active users we are translating to 14 different languages and are played in 180 countries we do weekly releases so we promise our players something new every week and we have never failed actually and we also are cross-platform today so if you are on a mobile iOS and I'm on desktop PC or Mac we can play in the same PlayStation

and it's also a very proud to say that we are 54 Women Within the studio and uh let's visit our world the magical jovik [Music] [Music] [Music]

[Applause] [Music] [Music] foreign [Applause] [Music] so let's go to the pilot we had a challenge and we asked ourselves are we the police or are we the paths to our players being an online game and Community for girls between eight and above there are some jerusal decorations to follow when it comes to the social elements of the game social elements like the chat and other in-game Communications and of course we also want to keep our players safe and be able to play in an environment without being bullied or

treated badly and at the same time not compromising the fun of connecting to new friends so we asked ourselves if there was some other way to create a good and caring environment than Banning the players that behave badly and acting like the police almost two years ago we stopped using the more traditional method for in-game shots that filter outwards and ban players automatically when abusing these words we first had a blacklist approach which means the list of words the players can't use they get hashed out in the chat that was not enough for creating a good environment so we switched to White listening which means a list with allowed words and those are the only words that the player can't use which of course creates some kind of limitation so this let us both approaches create this fact this satisfied players due to the limitation and also not understanding why you got banned

as you can see on the screen here so this led us to take a more modern attitude and we started to explore what AI moderation could do for us today all our in-game communication channels are moderated based on AI with support for our human moderation team but it was still not 100 satisfied with the player environment and we started to ask what more could be done at the same time I got in contact with Alex through Linkedin and we had a first conversation about AI in relation to online communities and games and we soon realized we had the same Future Vision for in-game communication the vision we discussed was something that I believe was the solution to our challenge our players seek to exchange ideas about issues like mental health identity family school and life speaks questions policing created a lot of frustration for a player community because it didn't let them to talk about things they

wanted to talk about we of course comply with all regulations there are but beyond that who are we to stop prayers from talking about what matters to them and just like in the schoolyard they will talk about these topics our job is to protect them not to cut them off from important conversations with friends so with the joint Vision we started to become more concrete and I even invited some other key people from Star Stable our director of a customer experience Cecilia Monta and director of community Jane Billet they got on board with a vision and the three of us created space in our current road maps in order to enable this pilot the pilot also included three Tech teams it was the game server team back end and also the client we also was supported by marketing and our communication and Community team of course

each team brought their knowledge and together we started to create a foundation for the pilot so with the joint vision and a great pilot team we were ready to go and the aim of the pilot was to effectively address and decrease negative social behavior in-game without sacrificing our players social engagement we decided to run the pilot in UK using Tree Service during sex weeks period in order to verify the pilot we would use two groups of players a treatment group and a control group so when a player in the treatment group was detected to send harmful messages they got a pop-up with a message to stop think and act before sending the next message no one was blocked from sending more harmful messages but instead encouraged to take a better decision and in order to get the right message on the pop-up we added Pepe agency to the pilot

yes so some of you might be thinking why social and emotional learning what is that and how can that be used to reduce toxicity well to start off when we're using traditional methods like Banning players we're risking to leave them feeling perhaps even more frustrated or unaware of their hurtful Behavior to give you a comparison I'm sure many of us in here have found ourselves in moments when we've lashed out either at our partner our kids or maybe a friend and in the moment after we thought oh gosh why did I do that and so what if you would had a little person on your shoulder that told you hey Paulina why don't you stop and think for a moment I'm sure that our Behavior would have been completely different and that was the exact feeling that we wanted to provide players on Star Stable

to give them guidance to change their behavior so that they could form positive relationships and we saw that moment as a skill building opportunity but it was important that we did not judge their current behavior and the skills that we wanted to install are not only about being able to label your emotions and the feelings that you're feeling in that heated moment it's also about understanding other people's emotions and how you can use that information to form positive and healthy relationships now when we design we have a principle to design with empathy and in this case we needed to go to the root of why people bully in many cases it's because you feel unhappy scared or unheard it might be because of something that happened at home or it could be something that happened in the game it might also be that I lack the skills

to control my impulses or to label my emotions and in the case of Star Stable which I think is true for many online games we could see that players were acting out of Rage when they lost a race or perhaps they spotted bullying in a chat but they were too afraid to act and they might even continue and tag along with the friends in the behavior that they could see we could also see that players felt disappointed about a recent release and they acted out in that frustration so with this knowledge and the awareness of what situation a player might find him or herself in when they're they're acting in their anger we developed a three-step approach that is based on a research-based method called non-violent communication and we divided it into three steps where the first one is to help the player identify his or her feelings and the

situation that we would like that person to change so we talk about self-awareness and self-management the second step is to help the player understand how the hurtful behavior is causing others to feel so social awareness and then we wanted to encourage a more positive behavior by giving giving an example of what that player could do so talking about relationship building and in the case of Star Stable this is the message that we ended up using so first you see hey it seems you've sent an unkind message we're hoping the player identify the behavior that we would like to change but without judging secondly stop and think how would you feel if someone said that to you so giving the player the opportunity to experience empathy and put themselves in the shoes of another player's feet

and then third remember being a good friend can also get you lots of new friends so why would I change my behavior well if I can get new friends um that sounds pretty good now the message would not be effective if we couldn't identify the right moments to deliver this and in order to do so we needed the powerful technology that other Luke could contribute with so as Paulina mentioned we now have the message right so we needed to know when to trigger it and also how do we actually measure this how do we measure that it's effective at all so what we used was something called Transformer models and to give a very simple explanation of this imagine an AI model that ingests the entire Wikipedia and then through that learns language and for example cultural contexts and things like this um and that's something we used as a base

but I'll also show you how this can go wrong with this Cutting Edge AI technology so I've put this sentence here um which you can see windfall today you suck can see you're only 10 right and all of you read this here and probably made up your mind in terms of what this actually means now if we used an out of the box Transformer model that would be coming from say it read the entire Wikipedia it might interpret this as being windfall being a person who is 10 years old now in the context of Star Stable this would be very wrong windfall is actually a horse breed and tan is most likely referring to the level of the horse in this case now the way we kind of made sure that our AI model was adapted to Star Stable was that we let it ingest data from Star Stable from the community so just like say a community manager builds up this

intuition of how people communicate in the game the AI model was learning those different associations and connotations so when we get to the last layer where we're actually saying what is as hurtful Behavior towards another user the AI model that was fine-tuned to Star Stable would know that okay here you're actually writing to your own horse in this case not another user so maybe we shouldn't be triggering the message however if we used a kind of out of the box AI model it probably would have triggered the message in this case so you can see there even with this Cutting Edge AI technology we have to be wary of what can go wrong and this is still very effective compared to say keyword filters but it has its caveats as well now we had this so we could we knew when to trigger the message and as Jane was mentioning we needed a way to kind of measure this and I'm sure with covid uh being around we're all somewhat familiar

with kind of clinical trials and how they work and you could say we used a similar approach here so we had a treatment and a control group now there were some considerations we had to make here so the treatment group would get the message triggered when they were writing something hurtful to another user the control group would not get the message however Star Stable already had some built-in safety features so that if you're writing something hurtful to another user it would kind of get blocked out and we didn't remove that feature for the control group so both groups would actually still have their messages removed and the reason why we did that was to make sure that we're not exposing other users to more kind of a in a negative experience that they wouldn't have had otherwise um and those are what some of the considerations you need to kind of make when you're trying out these type of things another thing that we did was also making sure that we could measure this in real time as the data was flowing in because our hypothesis was

that this would have a positive impact however once out in the wild things can change pretty dramatically and we might experience a really adverse reaction to this so we wanted to know that we could kind of hit a kill switch if something went really wrong or if things look really positive we could actually roll this out even quicker and stop the actual test now we did this with in conjunction with the community team so we were looking at the quantitative data that was coming in to us in a live manner but we're also listening to the community team that was getting the qualitative data coming in from the actual community so seeing if there was any feedback coming in there and I think it's very important to look at both those aspects as you're trying to do something like this and you also have to be very wary of you know what potentially could go wrong when you're running this type of test um but through doing this we could actually measure how effective this message was and I'm going to hand it

back over to Jane now to kind of talk about the the results that we saw so did we succeed and what was the conclusion after six weeks the first week we did not see any significant differences between the two user groups but as the pilot proceed that would change already after four weeks we could see a significant difference between the two groups which made us 99 confident that the message would we displayed for the players was reducing the number of harmful messages in the control group interesting as well no players contacted our players support during the course of the pilot with other questions or complaints so they accepted the pilot but we did see some interesting behavior in the chat with our players first discovered the pop-up not all but some would try to trigger it again most likely as a challenge to see where the boundary goes before getting a ban or

some other punishment so it could look like this as we see on the screen in the first session same player user a first say something harmful to another player hope you get hit by the bus and then oh I get some warnings and then uh self-reflecting how many warnings do you need before you get a chat ban and session two I want another one ah Noob and then got it so uh quite interesting behavior from from our players thank you so with the two different script treatment and control the length of the pilot and the amount of chat messages sent during this period we came with 98 confidence say that we did reduce five percent on harmful messages among

our players in the UK maybe doesn't sound that much but five percent translated into actual messages are more than 800 negative interactions not being sent and that done without policing no Banning and instead encourage our players to stop think and do what a nice friend does so I would say yes we did succeed so what's next in the game we will see the dark Riders take entrance but with the pilot well are where do we go want to go from here after the pilot was done we're quite sure that this is one of the missing pieces in our strategy to create a friendly environment in our game in our overall Vision there are three parts that we call the moderation puzzle there are the rules and regulation element that ensure things that absolutely should not be in the chat are

not part of the chat there's the self-moderation piece which plays alert us when there's something going on in the shot and then there's the third part about encouraged positive behavior in-game which the pilots showed was possible so right now we are preparing to set up a long-term structure for this to be a permanent part of our in-game communication we have the vision and we do have the knowledge now so coming to the end some key takeaways and I would say have a strong vision makes sort of the key and also a strong team and then I include both the internal team and of course good partners that can enable where you don't have the resources yes and think about how you can support and build skills on the players terms so

you created the game but your players created their community so how can you in your game create guidance and help players behave in a more positive way without judging and last but not least if you have the right circumstances the right environment nice people in online game do exist so that was the end so I think we are in the Q a [Applause] barely I don't know if the it's on hello hello um I love the three point research kind of framework that you had um I noticed the phrasing the first sentence I noticed you sent an a harmful message I I thought of the first point was more

of like self-awareness of an emotion so I know it's like kind of hard to identify what they're going through any kind of thought about what that first sentence should be um kind of like to fully get at that self-awareness framework um yes so we had a couple of ideas on how we can use that research-based method um and there are a b testings that we hope we can do and one way could have been to say um hey it seems that you've sent an unkind message take a moment to stop and think how are you feeling in this moment and then go on and say okay well build the empathy skills how would you feel if someone said this to you what can be a little bit risky when talking about very strong emotions or big emotions is that we couldn't in that moment offer also a skill building to how to deal with that emotion so we

decided to focus on the change in behavior and focus a little bit more on the social awareness and the relationship skill building although I'm sure there are moments where we could have had a stronger focus on the self-awareness instead cool that makes sense thank you foreign can you talk a little bit about whether or not that five percent reduction was specific to certain players and you saw an actual reform and some of those player behaviors over time or was this just generically across the board regardless of specific player Behavior so we actually looked at this on aggregate across the two different groups and what's interesting what we saw was like the most so you'd have a small majority of players you kind of have like this one-humped camel so majority of players might send a you know a couple of harmful messages one harmful message and then you'd have kind of like uh a few players that were

sending a lot more and what we saw was that this group actually didn't uh reform uh didn't change Behavior overall so you kind of had the same kind of camel shape afterwards and there those kind of insights kind of tell you that okay maybe we need another approach with this group you know okay we're over on aggregate we're actually able to change some of the behavior but for this group we might need some other type of intervention there as well which I think is important since not every player is going to be the same and respond the same to these uh like this type of messaging and then as a follow-up were there any long-term changes that you observed in player behavior that could be linked to this type of model like for example where players who are consistently seeing this warning less likely to engage in the future or was there no change in Behavior um so we ran this over a six-week period and what we saw during that period was that they they you would have are you

referring now to specific like harmful messaging or other more in terms of like retention of players oh we didn't look at the actual retention of players in this case we were more specifically looking at the the behavior the harmful Behavior towards other players thank you hello first I wanted to say that I really like the methodology you guys use and this is the perfect population to use it on like a young audience that they're already learning social skills how did you guys measure success of of this methodology foreign that's where we wanted to have this more quantitative measure you know to actually see that there is a reduction and uh you know I'm speaking a bit from my personal experience coming from Google and trust and safety and it's a lot of the time you're trying to mitigate risk and kind of reduce harmful behavior and it's been difficult to

measure I think one area here which we can all agree on is like how how do we see that where we have some success and that's why we kind of did this more had this more clinical approach in terms of a b testing to actually be able to quantitatively measure that okay messages from the specific group we're going down in terms of the harmful harmful messages um yeah and I can add to that as a um I have a very strong Vision that as long as one child or teen or player is not being presented with this hurtful message that's a success for me personally and then obviously there are things that could be tested in terms of optimizing things and maybe as we talked about having another message for certain players Etc but I think for our pilot the success was to actually see a decrease in harmful messages thank you I I can continue on that as well uh so yes that that was the goal in

itself we could have done a lot of other sort of metrics to look up on but but uh we'd like to keep it simple uh we started to talk about a lot of messaging trees and it started to be complex but it said let's try it rather than to to get it perfect I I think one last thing to add to this is that the positive impact I think from this is that what we present here today we hope that many of you in here if you know you are a developer or you're working with Community Management you can go back to your teams and you can show hey this actually works you know there's hard numbers here that can show that this actually does have can have a positive impact on your players thank you I think that's the last question yeah okay uh thank you very interesting pilot um so a lot of the Neuroscience studies show that it's extremely difficult to change Behavior without awareness first

and also it's much more difficult to get awareness when you are in the heat of the emotion and the action so have you been thinking of any like preventive or like reinforcing positive experiences even before like the deteriorate or anywhere to enhance like the trust and you know relationships strong trusting relationship also helps to reduce this kind of behavior so it's like you almost try to tackle like the most difficult aspect but like what could you do to build up to kind of you know reduce in advance yeah what we do have in the game as well is ambassadors that are in the game and that do talk to our players and they also try to be nice friends and Role Models so I think that is some type of awareness and we do have guidelines and the the game in itself stands for

kindness uh we have the one of our values is kindness is the superpower so we talk a lot a lot about sort of friendship kindness and what that can enable it's not just being sort of kind it's it is a superpower and what does entail takes so I think that is spring to some awareness if that was what you meant uh yeah apparently it's it's also like um you don't want to suppress negative emotions neither you know so it's more like learning to how to deal with them also when they arise but learning to do that in the heat of the emotion is super hard so it's yeah that's kind of what I meant like how do they learn to deal with um you know negative emotions yeah before they lash out so I think your question is around emotional regulation and what can we do

to promote that in a game and um of course there are many ways such as if it's if it's losing a race for instance you could have encouraging friends that say oh you next time we'll do it or like um you still did a great race and just as we would in a a real life setting with a lack of a better word um and I think in terms of lashing out and behaving perhaps in a hurtful way as I said in this pilot we decided to focus on sort of changing a behavior and how to build the relationships but I'm very curious to see what we can do also with emotional regulation and I think that question will be more around not minimizing the freedom of a player and not becoming too educational or um you know being almost their psychologist in that sense so um it's a tough question and I I hope

that it's something that we can work with in the future great thank you yeah and thank you so much for listening thank you [Music]