Designing with Community Safety in Mind | Develop:Brighton 2024

Good afternoon everybody.
Well, how lucky are you that you guys are the ones who really care about mental health and safety and civility. So you guys are going to get the ultimate secrets here at this presentation. I'm Tammy Bomik. I'm the vice president of civility at Roblox. I um full transparency, I am a recovering global head of marketing. And six years ago at the company, I um I started this new thing called the civility initiative at Roblox. And it was all about empowerment. It was about educating our community. It was about educating parents and it really became truly my heart and my passion and our

CEO Dave Bizooki who's the founder of Roblox embraced it um tapped me on the shoulder three years ago and said listen I know how much this thing means to you how would you feel about leaving marketing alto together and heading this thing full-time so it was a dream come true so I've been leading the charge my focus was on civility I will share a lot more about what that is as we as we continue on. But I wanted to give you a little bit about a background on myself. Um there have been a lot of things that we've been doing over the course of the last three years that I'm very very proud of. And what I'm hoping to do is share some of those insights with you today as you develop and create your experiences, you build your companies, some things to think about in order to make your experiences more positive. And hopefully that positivity will turn into more engagement and more success for the games that you're actually creating.

I'm going to start out with a little video for those of you who are not so familiar with Roblox. Perfect. If you're down for the cause, cuz I keep it real. Yeah, I keep it fresh, baby. I'm be What's next? Cuz I'm breaking news. Shake an exclusive. I'm making a boo when I talk to the press. Give them a show. There ain't no stopping when I'm on a roll. Yeah, didn't you know? I'm taking control. I make appearance and I got to go cuz I get a dancer with a handsome. Yeah, I walk in and they go bananas. I'm a bad and the Miami [Music]

So, that's just a snippet of some of the really cool experiences that are made by our developers. If you don't know, Roblox is a plat we don't make any games at all. We are a technology platform that enables creators and developers to be able to tell their stories and publish it out on our platform. And really what we have become is a platform for immersive experiences. Um, we bring people together from all over the world. We have imp we we really really look to empower creators and allow people to connect and find community with one another. We are big. We are really big. Um, over the last 20 years, and I would have to say 20 years, we've been around for a really, really long time. At one point, we were um, a startup and we were indie

and we were just create trying to figure it out. 20 years later, we are pretty massive. And so with 77 million daily active users on the platform, we are larger than many countries in the world. And with that comes an incredible responsibility. A lot of things that we have learned um just by being and doing. A lot of things that we have learned from our developer community in terms of feedback. Um and we continue to to um focus on that journey of empowering developers. It is the number one thing that Dave Bizooki who was our founder focuses on and that's empowerment of the developer community. Um you may be wondering why Roblox has become so popular with so many creators and there are a lot of different reasons why. Um, but a lot of it has to do with the storytelling element and the area of

enablement. It is the community that we embrace, the community of developers that truly love one another. I've been in the technology space for 30 years. And I have never, you know, I was pretty jaded honestly. When I came to Roblox, people kept telling me, "Wait, you're just come to our developer conference." I'm like, I've done a million developer conferences. I know this. I went to the first um RDC, Roblox developer conference, and it blew my mind away because these developers were truly a family. And I know that a lot of people say they're a family. These developers were a family because they started playing Roblox when they were 10 and they lived life experiences with one another, supporting one another. They're now 17, 18 years old. The average developer age now is 24, 25, but there's still a lot of developers out there um

who are full grown adults. They're adulting and they have their own studios now, but their base they formed when they were 10. So when we when I talk about family and developers who this community being so special, that's what I'm really referring to. The other area is we're continuing to always innovate. We have immersive 3D communication. Our vision is to remove the friction um on in the digital world that people have. Um we want people to be able to communicate as easily as I'm communicating with all of you today with emotion, with eye movement, um with hand movements. I tend to move my hands a lot, which is the reason why I had the glass of water taken away because I was afraid I was going to splash it all over you. But then in addition to that, we're we're continuing to always innovate. And it is truly one of um our missions

within the company of bringing a billion people together with optimism and civility, which is our vision. And the continued innovation has to do with continuing to improve tools for developers, making it easier for them to develop and tell their story. One of the things that I'm really excited about is Genai and the ability for um democratizing creation. There are so many people in terms of making sure that um this creative world can have different storytelling, can be diverse. And with Gen AI, you don't have to be a hardcore coder anymore. A lot of people who have that story to tell, but didn't like math, weren't into coding, can now be part of the ecosystem. And that's exciting for me because that's going to bring more women. It's going to bring more people of color. It's going to bring marginalized areas into developing

and being able to create and and storytell. So, that's something that I'm very excited about. The other reason why um our developer community is growing so dramatically is because we're continuing to innovate with brands. So um you know as a small developer out on their own it would be impo next to impossible to attract the likes of a Gucci or a Vans or a Nike to be able to collaborate and create. But at Roblox we are dipping our toes in the water and we're figuring out and we're learning a lot. We're making mistakes to be completely honest with you. We're like, "Oh, that didn't work. Let's let's let's try something else." But we're working with these brands who are who see the future, who understand like how are these what are the innovative ways for us to reach Gen Z, right? Because they're all trying to reach the same audience. How do we innovate around this? So, we're they're

coming they're they're seeing roadblocks be their future. So, um, with great success, we've we've done some really pretty amazing things with Elton John. We just launched, um, uh, an experience with E.L.F. Cosmetics. Um, we did something with Elton John. It was a lot of fun things and the developers are really getting noticed for it. When I talk about Gen Z, we did um we we launched some research last year and it was even surprising to us. We knew because we'd see the numbers, right, how um digital expression is so important to Gen Z, but what we didn't realize was just how important it was. And now we have this incredible sense of even more responsibility. So 88% of Jenzers say that um it helped them to

find their self-confidence. A lot of Jenzers are trying to figure out who they are. You know, who they who they think they might be. Um but it's a safe space when they're trying on different hairstyles, makeup, outfits, pairing things together. They do it in the digital space and then they take it to real life and they're getting a lot of response back in real life and a lot of thumbs up and they're feeling really really good about it. So, you know, it's no surprise that over half say that their digital appearance is more important than their real life appearance. Um, and it's also indicated by the numbers that we see in our marketplace. over 30% of our users go into marketplace every day to change something or to take a look at what the different outfits are. And so our creator community who are creating these digital items, there are more and more

and more creators because now they're creating little fashions hotspots. They're they're creating popups and they're making names for themselves. So this is not just a game, right? This is a complete ecosystem that we are creating that allows developers and creators to participate and and contribute. So, you know, one of the things that um I was mentioning about Genai, the creator community that we have is pretty special. And one of the things that I that I saw way back when when when I went to my first Roblox developer conference, there were like like 10 10 women. 10 10. Now there are hundreds when I go to RDC. And it's something that I'm I'm really I find really special. And part of that is and again we're going to continue to innovate on bringing more diversity from every age group. We need to hear

those voices. We need to hear those stories because if we want to create a virtual world that we want to see that's positive, responsible, and inclusive, we've got to have those voices. So, we are unique and we continue to grow. We um now getting to the heart of this talk, safety and civility at Roblox and I'm also going to venture and beyond. A little bit about our background. So back in 1989, I know many of you weren't born yet, but back in '89, our founder Dave started a company called Knowledge Revolution. And it was quite simple. It was, you know, 2D physics. They he wanted to really um showcase how um physics would, you know, they had like different cars and houses and

things like that with um you know, different balls that would fly through the air and like knock down a building. What were what were the physic models that were actually created due to that? He went into classroom and he hung out with these kids and he had a lesson plan and they were bored to death. They were like and so after the official lesson was over, he opened up the software and he let these kids play and what he saw was amazing. It was these kids were so creative and they were creating their own physics models and it was unbelievable. So he sold knowledge revolution and he realized wait what am I doing as a middle-aged man I have no I have like what am I doing why am I doing this let's let these creators these kids create so Roblox was born um so that's a little bit about the background and you know 20 years later we are we are still focused on helping those creators but

because we focused on kids in our very early days. Safety has always been fundamental to everything that we do. We have all we have all of the features. We have both human and AI moderation. We have image review. We um are really focused on AI. Again, another thing that I'm very excited about is AI in terms of moderation, in terms of language translation. Um, we have chat filters that are automatically translated. We're continuing to refine those models. They continue to get smarter and smarter. Um, and eventually, you know, we have um multimodal models that will allow us to review full experiences. So, um, it is it's exciting for us to have that be the number one priority at Roblox.

In fact, so much so we have something called trust by design. And trust by design are multiple teams within roadblocks. Anytime a product manager has an idea for a feature, they have to go through this rigorous assessment. This rigorous assessment in terms of is it leg are these things legal? Do we p will we pass you know regulatory and legislative um rigor? Well, is this morally and ethically the right thing to do? There are a lot of different teams. You've got to get through these hurdles and there's purposeful friction put in place in order to keep us safe. Civility, which is near and dear to my heart, is what I started out with. It's built on the foundation of safety. Safety has always got to be there. It's it just it's it's table stakes. But civility, it's it's a three-legged stool in how we in terms of how we look at

safety and civility. Safety is all about continued technology innovation and policy innovation and civility is really about education and empowerment and awareness. So at Roblox civility, our north star is about thriving. It's about helping people to thrive not only on roadblocks but in their life. And we feel that goodness is going to come back to us and hopefully make the platform safer and more civil. We do this by empowering people with education materials etc. We have a huge network of global partnerships around the world where we're learning constantly learning from them. My team's job is to go out and work with researchers from some of the top universities from around the world that are doing really innovative work around mental health, mental well-being,

how gaming impacts adolescent brains. Um, is it true, you know, screen time addiction, is that a real thing or is it not a real thing? What's the evidence that showcases that it might there there might be risk, right? And then in as as we take a look at what we need to do in order to move forward as as as a society, we need to do it with evidence-based research, not sensationalized headlines that are written to scare and to provoke, right? But have these honest conversations mired in data and evidence. So that's one of the things that are it's really important to us. We're working with the likes of um Oxford. We're working with the likes of the digital wellness lab at Boston Children's Hospital, Harvard Medical. Um we take all of that information that we are getting fill our

brains and we take it back internally. And I always say internally and and this might be controversial inside of Roblox, but I look at my job and my team's job to be a pain in the behind. And that what that means is that taking all that information and constantly hitting up our product and engineering teams to make sure they're thinking about things in the right way. to make sure that they're thinking about things ethically, morally, and what's good for the soul. So, I have constant um weekly meetings with our chief safety officer. He actually I'm sure he regrets saying yes to that, but I have weekly meetings with him, right? I take all that great information back and and I'm constantly advocating for different features, pro-social behaviors, um, innovative things that we could be doing to to make

to to get it onto our road map. Um, we call it civility by design. And then ultimately, we're taking all of that information and speaking at events like this. We're creating podcasts with Sesame Workshop. Have I in the UK? Have you heard of Sesame Sesame Street? Yeah.
Yeah. Sesame Workshop um is the organization that produces Sesame Street. So we work very closely with them to um to publish uh out podcasts. So again trying to open up to the world to make sure that all the work that we're putting into these resources are getting somewhere. So this is just a small snapshot of some of the organizations that we work with. Um, the National Association for Media Literacy Education is really important. We will not be successful if we don't build awareness and empower this next generation. I honestly think we are our systems and

I'm I'm talking our global systems. We're we're broken from an education standpoint. I'm probably not telling you anything you don't already know, right? But media literacy is so important and at least in the US um you know we teach the core curriculum math and science and history etc. But we need to get media literacy and digital literacy into classrooms in the very beginning. The moment a two-year-old has a device in their hand there there should be education that helps to empower them to be really strong resilient people as they age. Eventually, when you're on the internet, you're going to see bad things. As as the many safeguards as we have, we're it's still the internet and it's a whack a game of whack-a-ole. You are going to see bad things. There is going to be toxic behavior. There are going to be scams. So instead of reeling back, these parents and kids reeling back and not

knowing what to do about it, how about we empower them and we show them how to use the tools, where to report it, and most importantly, how to mentally process that information where it doesn't cause toxic harm. So these are some of the organizations that are helping us to think through really innovative ways of thinking about that. some of the things we've learned. So, oh my gosh, a year and a half ago now, I guess. I I guess it's yeah, about a year and a half ago, um we formed a hypothesis. And the hypothesis is in 5 to 10 years from now, the online world can be safer and more civil in than in real life. If we were to make that a reality, what would need to happen in technology, innovation, policy, and education and awareness? We partnered with the digital wellness lab at Boston Children's Hospital, Harvard. We convened over a 100 thought

leaders from around the world. I'm talking about leaders of industry, um, regulators. Um we had somebody from Ofcom um join us. We had the e-safety commissioner join us from um the e safety commission from Australia. Um we had physicians, pediatricians, psychiatrists with you know all sorts of people with so many letters behind their names. They all thought they were incredibly brilliant. Brought them all together to workshop this. There were seven virtual sessions and one in-person convening. We put out a white paper of the results and I'll cut to the chase if anybody's interested. It's on our civility. site as well as the digital wellnesslab.org site which is um part of Boston Children's Hospital. But I'll cut to the chase. The number one thing that came out was include youth meaningfully

through everything. And honestly, I come from industry. I've spent 30 years in industry and I can honestly tell you we don't we have great we have focus groups with with teens and with kids but then we say okay thanks and we we they we usher them away but what's missing here we cannot build an internet for youth without youth and so the one thing that came out resounding was that we need to do that and when I say we I mean industry, I mean governments because everybody thinks they're so smart. Everyone thinks that, you know, they're smartest people in the room. When we were at this physical convening, you know, we made some sort of sort of assertion. I don't remember what it was. But I look across the table and there was this young woman. She's 17 and she was shaking her head. And I I said, "Yeah, wait, wait.

What are you thinking? What what's going through your head?" She goes, "That's not how we use it. that's just not how we use it. So, it was this dawning moment where it was like, wow, these kids, it's the rights of the child, right? It's these kids have a lot to say in terms of how to form this next generation of the internet, how to form roadblocks, etc. So my appeal to you is you guys are developing your own games. You're developing your own studios. Integrate youth meaningfully. You may think that you know what they want. I can guarantee you you don't. And the way that they think about things is very creative and innovative and will probably bring up things that you never even thought of coming from those workshops. We are putting more money where our mouth is and we coming off of that we developed a

youth engagement council and what this youth engagement c it's a teen council and you know we'll be launching it um we'll be announcing it you're getting like a little pre my my comm's person is sitting there she's going like this but um we'll be announcing it um at towards the end of this year but we are right now working with other youth councils to get feedback and you know again we're putting our money where our mouth right, is we want to include people meaningfully into the development of these next generation of how do we make our Roblox platform more civil? And so um their input will be on product and policy design um innovative ways how they how they view civility, what the definition is for them and how the how developers can actually work to develop more positivity into the experiences that they're creating. So that's one of the

big roles that they'll they'll be playing. Um, the other lesson that I want to share is who is your audience? Who are you designing for? Right, is something that's so incredibly important because I know that a lot of our developers get an idea for a game and they they go off and they do it and it's super fun, but then they realize I'm like, wait, how can I apply business practices to this to in to increase engagement to in increase um word of mouth, right? How do I how do I do that? And so, um, one one example that I use is over my last eight years at Roblox after I do these talks, I've gotten off stage, and I'm not talking one time or two times or three times. I'm talking like 50 times, 60 times. I've gotten off stage and a parent will come to me and say, "I can't tell you how much Roblox has done for my child."

and more oftentimes they are on the spectrum, right? And so I knew that there was something um so I wanted to do a research project about neurode divergent teens and so I partnered with internet matters. I don't know have you guys heard of internet matters? They're in the UK. They're quite prominent, well respected. They helped launch a research project for us. We partnered, it was a it was a research project in the US as well as the UK. We partnered with autism charities. Um, we learned a lot. We learned the good, the bad, and the ugly. So, the good news is is that, you know what we didn't realize? 93% play Roblox. Wow. That's that that's the reason why I had so many parents come to me. Um, and more than half shared that it allowed them to find a community to to make

friends and to practice skill sets. Um, they it helped them to feel part of something even though some could not necessarily articulate it, but they felt like there was some community that was there and it helped to improve their overall confidence. there are areas where I know we can improve upon. Right? So, one of the things that like would never occur to some of our developers as they're developing their experiences is that sound and sensory and light can be a triggering effect. So, as you guys are developing your experiences, doesn't have to be on Roblox, but like if when you as you guys are developing your experiences, keep that in mind in terms of your audience and who you're developing for. You may even want to go and put some sort of a labeling or warning or something in the description that says that if you're if you're triggered or bothered by light or sound,

this experience has it. Right? in order to be respectful for the the audience that you have. Um, and then again, one of the things that we found was and this was coming from these teens and and just the age that we we uh researched was 13 to 17. They said that they actually wanted more help from their parents and their caregivers about what was happening to them online when they were playing games. And this is not just on Roblox, but it was it was posed to any game, right? And they wanted information um on how to play games so that they would be safer, right? Because they're not picking up on the signals. They don't they have a hard very difficult time picking up on nuance, sarcasm, and it it's easier uh for them to be tricked. So as again as you're developing your experiences, think about creative ways

where you can be the protective factor in terms of making sure that you know um people are following the rules that you are setting in in your games and that can be a myriad of of different ways which I'll I'll get into in a moment. Um, as I mentioned, we have a lot of resources, a lot a lot a lot resources. You can go to civility.roll.com and there's a a wealth of information there that you can that we can share. Um, Internet Matters helped us to develop all of these materials along with um the autism charities in the US and the UK. Um, and then there is we just released guides to Gen AI for parents who weren't who are not aware of what Gen AI is. It's a very very practical guide that's easy to read. Um, and then teens as well.

The other areas because our fastest growing demographic on Roblox is Roblox is 17 to 24. I'm probably not telling you something that you already don't know. Mental health is really, really, really challenging right now for everybody, but especially for teens. And it's important for us to meet them when they need help. if they are um looking for care, if they're if they're looking um potentially self harm in some way, we need to make sure that we have resources available to them at the time of need or their friends or the community can report. So, um, we actually, uh, partnered with a company called Throughine, um, down in New Zealand to offer this widget that is on our site that it it covers over a hundred different countries. It it's not just suicide and self harm, but it's eating disorders, it is um, depression,

it's loneliness, etc. if you need help and you can you can filter by country and it will identify your IP and you will be connected to the resources that you need. So again responsibility right as you're creating your experiences and your games remember that the people who are playing it are often time teens and in a in that risk area. So making sure that they have the ability to reach out and get the resources they need when they need it is is super important. Um, these are just some of the organizations that we work with to be able to provide the education for those services to be able not just at the very end of the spectrum where people are looking for help or self harm, but really focusing on how do we build resilience for young people the very before they and hopefully avoid the downstream. Right? So the Jed Foundation

is an organization that focuses on empower empowerment and teaching young people the skills they need so that they can be healthy. So it's not just the back end of the spectrum. Born This Way, which is Lady Gaga's Foundation, we just um partnered with them. They're doing some really amazing things in terms of inclusivity and acceptance for community. So, we're doing some really innovative things that I can't announce right now, but um really exciting things that will hopefully help our community to thrive. Um and then the role of AI in moderation. I already mentioned we have um active real time intervention modes that focus on if you're calling out for help, if you're being bullied, if you are um expressing self harm in such a way, we can actually model it and we can actually put up a little a little note that says, "Hey, do you need help?" Or,

"Hey, be cool. That's not nice. that's, you know, we don't we we never want to be finger waggy, but we also want to flag people when they're stepping out of line. Um, and then the last one of the last things that I'll share is um inspiring experiences on Roblox. And again, these are just some of the sample things, you know, practicing mindfulness. Um, Aloe Yoga is a um brand of yoga wear. Great stuff. Um they created an experience called Aloe Sanctuary and in five days they got 8 million plays on this experience and it was all about meditation and it was beautiful experience with like really calming music. you walked in into a room and you got to do yoga poses and one of the things that one of the people their comments that really stuck in sticks in

my brain. He says that his his parents are getting a divorce and they're fighting all the time and when that happens he puts on his headphones and he goes into the aloe experience and he finds calm. He'd never known anything about meditation before, but it was really really it soothed his soul. And I'm like, wow. Like, if we could do that at scale, how do we do that at scale for like the whole world? Um, so as you guys are creating your experiences, think about ways that you can you can help with that. Um we have other experiences that teach um kids how to be kind, how to be inclusive, how to stand up for one another. Um so the the range is is limitless. And then um again just some words just to leave you with is how do you set the stage for creating positivity within your own experiences? Right? One, think about the age and

developmental readiness of the audience that you're trying to reach. And as an exarketer, one of the things that I hate hearing is, well, like who's your audience? You know, everyone. Well, no, it's not. No, it's not. It's never everyone, right? Think about who the and the narrower your target audience, narrower the age limit, the more successful your game is going to be. And so think about the developmental readiness and design for that. Um the other thing is again clear onboarding setting the stage setting the tonality of the game and what is accepted what you will accept in your in your game from a cultural standpoint. And again civility does not necessarily mean kindness right? It's contextual and it really has to do with um you know if you are creating exper an experience where

you're in a library and you're putting books away and you're filing books that's that could be like a different tone. If you're doing esports, then some smack talking is should be allowed, right? But again, think about the contextual nature of the experience that you're creating and set those boundaries. Um, when you're creating your experiences, make sure that your your onboarding, your help areas are really really clear. One of the biggest complaints um we would get from a lot of in our research is that people couldn't find where to get help or couldn't find where to report. So as you're de I'm not even talking about Roblox, but as you're developing your games, make sure that that that button is very very apparent. Um, and then you know like the the neurodiversity examples that I gave you, make sure um you're you're really focusing on different audiences and

their needs. Um, there are so many research pieces that are out there. I'm going to go back to include youth voice into the development of your experiences to make sure that your experiences are hitting the mark, right? And you can get teens from your community. you can reach out and get um you know tap into a lot of NOS's have teen councils already at the ready who who would love to support the work that you're doing but again get that feedback don't don't develop in a vacuum that's like one of the biggest learnings and then again don't be afraid to try out pro-social um features AB test things for example that if you have a guardian or an ambassador when a noob comes into your game, maybe there's like an ambassador who takes them around, shows them how, you know, what the what what the culture is, what to do, what not to do, what's allowed, what's not allowed. So, just

think about ways where you can incorporate everybody into into the development. I think it'll make um for a much more successful game and it will make your studio just that that much um more impactful. So, um, my last slide is a call to action. Please go create that positive world that you want to see. And thank you so much for listening to me for all of this time. I appreciate your time. Thank you. [Applause]