The Mechanics of Explosive Growth

[Music] hello all right all right let's get this talk started thank you okay please silence your devices so hello and welcome to this talk thanks for coming really appreciate it my name is adam miller and the talk today is titled the mechanics of explosive growth so i am a vp of engineering at roblox and for those of you who are not aware of roblox we're the largest entertainment platform for kids under 18. and i've been at roblox for five and a half years and while i've been there i've been fortunate enough to witness really interesting trajectory of growth

so i'm going to tell you a little bit of that story today oh i got to turn the clicker on that would be a good step all right anyway so uh this one slide kind of captures the essence of this whole talk so this is the story of many many years of moderate and steady growth and then all of a sudden in the beginning of 2016 boom hockey stick we hit the inflection point and things really take off so today i'm going to give you a little bit of background on how we got to this point how we experienced it while it was

happening how we analyzed it and some of the conclusions that we came up with so i am not an expert on growth theory on company growth in general this is one data point uh that i think is interesting and hopefully has some takeaways so a little bit of background on how we started roblox first launched in 2006. and we began with a fundamental premise about people which is that humans want a place for self-directed imaginative play and increasingly in this world it's harder and harder to find that especially for teenagers and and kids so when roblox started we wanted to try to give a place that allows that

online and it started out as a physics educational tool or toy but quickly we added some basic functionality that defined our platform so giving every character every person who plays on roblox in avatar that's the 3d representation of yourself in the game uh we added lewis scripting so from being just physics adding scripting we let we enabled more compelling experiences and games to be created on the platform and finally we added the ability for our community to create content as well as just creating in-house we thought first we could create the content but by bringing these tools and releasing them to an existing passionate user base we found that they could create

content better than us not just in terms of quantity but even in terms of the quality and in the end we have this system where there's all these games these rich experiences they stream from the cloud people play them with their friends and kids can jump from one to the next and have a good time so this video is a little bit of an overview of what roblox looks like many years later you can see it's 3d it's avatar based although the avatars can be kind of funky and fresh you still see a lot of social activity people are talking to each other running around doing things like working in a pizza place exploring worlds chatting driving cool vehicles uh playing in platformer type situations or even being a bird flying through a forest scavenging for grubs things like that

so there's a phase that i'm calling the formative years of roblox and this is the long period of time where we focused on building our core vision this is the 10 years from that launch in 2006 until that really big growth thing we're going to talk about in 2016. so in this time we focused on trying to solve these really hard problems and to do that you build an amazing team you build a team that focuses and thinks about these really hard long-term problems and some of the things we've built in this time we're enabling easy play with friends really doubling down on that social experience so kids can be together they can be on different devices they're in one same virtual world they're jumping around talking to each other having another experience in another

game making that seamless so it goes across pc mac ios android xbox vr and you can all be playing together in one world that's not so easy not a lot of people do that but we focused on that as one of our big things and enabling these tools at work in a way that creators can create a piece of content and quickly push it to the cloud have lots of people playing and it just works automatically and we scaled lots of people so in this time we're experiencing a growth that's in the 30 to 50 percent year-over-year range and it's steady and there's challenges that came along challenges for our own toughness so competitors would launch potential competitors sometimes these games would launch and they would just explode they would take off they're getting ten thousand percent week over week and the challenge for us the big thing we thought about was

do we try and copy them how they're having all the success over here should we do some of the things they're doing or do we stay to our true vision and i would say this is the period of time when we really focused on our vision and kept that same course so 2016 and to step back a little bit the beginning of 2015 we said accompany gold we said we're going to grow at 85 percent this year and it was crazy it was outlandish we we'd grown at like 35 the year before we're like no we're going to grow 85 we're going to set this really high bar so everyone's like okay i don't know if we can do it but we're going to try so the 2015 holiday comes around and it's it's the best holiday season ever we're taking off uh there's tons of traffic but we always do

well during the holidays so it's not that big a surprise it's like okay things are good but then january comes around and things are even better instead of dipping down again we're just continuing to grow so this is crazy like what's going on people are looking around maybe it's a stat collection problem now things seem okay maybe it's a botting problem got all these bots attacking us uh but no they're paying us money bots don't normally like to pay you money so uh it's probably not that so all of a sudden we realize that this is real this is uh we're hitting 300 percent you know all of a sudden it's just it's going crazy so very exciting um and we want to figure out why because if it's one thing we just started doing well we better make sure we don't undo that and we weren't quite sure so um we started to think about it but at the time so much of our effort was just

keeping things running right we got this amazing growth and we want to make sure the servers stay up and the site doesn't come down the database doesn't collapse under the load and we just like ride this train so people talked about theories but um it wasn't even time to figure it out at the time so we said okay cool we we know something's going well let's let's write it for now and see what happens so um we had a bunch of theories at the time when i researched for this talk i got a bunch more theories there's there's there's more there's as many theories as there are people um that you can talk to about it and uh a lot of them are very interesting so i'd go to the marketing team and i'd be like hey guys why do we grow like this two years ago like oh it's very simple uh yeah we we did these awesome co-brandings we really got our brand out there and we got our name out there

it just like took off i was like ah cool good starting then i go to the web engineers and be like hey why do you think we started growing oh yeah it's because we fixed the polish on the site like site used to be so ugly now it looks professional like we took comic sans out of the website take comic sans out i mean there's just no limit to how much you can grow that makes sense right so uh you think i'm joking i'm not joking so uh and i'd go to like the product team and be like why do you think we're growing oh it's very it's a great reason for it we finally enabled excellent party play like the ability for people to jump in game together we fixed all those problems we really unlocked that social dynamic that sounds interesting so there's all these theories going on everyone's got a different one um let's let's dive into a couple of what i think are some of them the most compelling ones and and and

take a look at how that plays out so theory number one youtube influencers these are the youtubers who are either new to roblox or started streaming roblox around the beginning of 2016. right around the time when we're growing all of a sudden there's tons of youtube videos going up and some of them are getting millions of views tens of millions of views clearly they're driving traffic to our site so this could be it like maybe we hit some kind of big thing on on youtube and and that just took over and drove all the traffic so looking back at the charting of it i'm plotting two things here so i'm plotting the roblox activity on youtube versus the roblox signups and when we see this the

youtube activity turns out is a trailing indicator so it's actually growing a little bit after the roblox sign up indicator and so what that means is it's actually the new roblox users who are saying oh i want to also go to youtube and watch videos about roblox so that makes sense um right and it's still a great thing for the company we appreciate it uh but it can't be the one factor that explains our growth trajectory so theory number two we've always had some international presence back around this time 2015 2016 we made a big effort to put servers in different countries around the world where we had a lot of players and the theory was our games are all networked and if you're connecting to a server you have a lower latency you're going to

have a better experience you're going to want to play roblox more and this graph shows what happened when we launched our servers in germany all of a sudden in turkey which is pretty nearby you can just see this jump in activity it seems very clearly correlated i think the internet isn't so good in turkey so this seems like a pretty compelling explanation when you see a jump like this but then you start to look at what happened in germany itself so germany itself even though they got the actual servers there's no change um and it's not 100 clear why but we think it's because their inner internet was already pretty good so they had some kind of connection to somewhere else that was already good enough whereas the turks really had that jump in experience and when we looked across the world at all the different places where we added

servers and all these countries there were some pockets where there was some change but it wasn't enough to really explain the whole change in our growth so another good thing but not the answer very compelling theory our xbox launch so we first launched an xbox at the beginning of 2016 right in january and it's that little white bump that starts appearing at the top of the graph so it's a lot of new users coming in and there's a theory that eve beyond just these new users you're getting there's additional benefit for the existing users so if you have one house and two siblings want to play and all they have is an ipad and xbox all of a sudden if the xbox can play too you got more benefit because they can play together so you take these two things together it's a very strong theory

so we have to dive a little bit into how growth works to analyze this theory so for growth there's really only two things you can move you can either get more new users at the beginning of the funnel or you can lose fewer users at the end of the funnel and looking at this growth in this particular time we realized that it was new users at the beginning of the funnel that was changing the retention metrics didn't particularly move so okay that sounds good that seems to match what we see with xbox we see new users with xbox but here's the problem so this is a graph of overall traffic but we're seeing a change in news user acquisition now new user acquisition is a leading indicator for traffic meaning the users that were acquired and the change at rate at which they're

required happened earlier than this growth so when we look back at when it really changed it was actually august and september the year before where we started to see this uptick in new user acquisition so that unfortunately invalidates xbox as a theory so this is where things get a little bit more murky but super important there was a really fundamental change we made on our platform in 2014 and we call it devx or developer exchange this is the ability for the creators on our platform to take money out so up until this time for the previous eight years they'd been working in a way where they'd make content and share it and they'd get some virtual currency back but it would just accumulate in their account they could spend it on things on

roadblocks but that was it in 2014 we said okay you guys can take that out for cold hard cash and this was actually a really hard decision when we made it because we already have this huge community they're producing tons of content clearly people like the content why would we give them money when they'll do it for free but what we've seen is this is probably the best decision we ever made because it changed the whole nature of how people make content on the platform they started to take it much more professionally these people who had built up all these skills and connections and community began to create in a much more serious manner and that produced a level of game quality that far outstripped what we had before so this is actually very difficult to chart because it happened over a period of years

but what we see is a combined network effect we have two network effects here that are feeding off of each other there's a community of creators and we see the more creators they are there are the better it is for each creator because they can form these teams and groups of people and work collaboratively to create awesome content and then the better content they make we have the player community and they have their own network effect where the more players there are the better an experience it is for each player and these two things feed it off each other so i do believe this was a factor in our growth change although it's something that happened over a long period of time and probably can't be a sharp uptick in itself so back to our theories i can disprove all of them i can prove that every single one of

these theories i believe i could be wrong uh is not the one thing that made roblox grow uh they're all great and they all helped and the tough thing is people want to believe them we all want to believe them because they're things that made our product better and they happened around the time of this growth so they make such a convincing narrative so what did change the growth so my theory is that it's a lot of these things added together and this sounds like a little bit of a cop-out but i'm going to explain so if it's a combination of all these things there's a natural question which is why would there be such a sudden uptick why wouldn't it just be slowly there's one thing and then

another thing and then it just layers in over time and that that's an important question so i believe i have an answer to that question and it has to do with the nature of how people want to play together and i'm going to illustrate that by showing a video of kids playing this is kids playing roblox these are two of my kids and one of their friends from down the street and i'm going to show the video and then i'm going to explain why i think it's relevant [Music] you got an assist kill for leo we both killed leo together now i'm just gonna take out his choice [Music]

is it a good spot yeah i'm surprised we could deal high five daddy we just killed leo when polly does you can see my daughter at the end she realizes i'm recording she's like oh dad um but the the key thing about this watching kids play like this is it's maybe a little bit different of a model than how we adults play it's very social it's completely engaging and they're constantly talking this isn't like an isolated incident when they're sitting there and playing they're always talking to each other sharing screens like messing around with each other it's a very different way they engage with the product and i think this ties into how roblox became viral so when you think about virality with our

demographic it's not on facebook and twitter they aren't sharing links on social media they don't have social media this is old school virality this is kids in a classroom on a playground talking to each other and when they have a great experience that they want to share they'll tell people about it and they'll bring them in and so my theory is at a certain point we'd unlocked enough of these things we'd we found we'd had a product that people wanted we unlocked the blockers we fixed the things that were kind of broken we made the experience work for our audience and that opened up their desire to bring in more people when you have some kind of critical threshold three or four kids in one class and they're all playing on one system they want to bring in that extra kid

because that adds so much to their experience so this is our ultimate answer in our story we took a long big vision we worked really hard on it for a long period of time we worked and invested in the people and in the product and in the end we created a system where it was viral and it fed this word of mouth experience so i'm going to conclude that there is no one trick that just let growth magically happen and is that a good thing or a bad thing i think there's pros and there's cons so the con is well it would be great if there was one magic trick that everyone could use and then you just grow like crazy but

i am glad we live in a world that's different than that i like living in a world where we try to build really difficult products really challenging big things and when we succeed that's where the real interest happens it's better than a world where it's just armies of growth hackers fighting against each other with tricks so i can't promise this approach will work for everybody in every situation but i think i encourage everyone to consider it as a potential strategy thank you guys [Music] [Applause] [Music] are there any questions

while other people are thinking of questions it's not totally on topic but i'm curious in this growth spike how your creator like what percentage of your population is creators how that's grown with your player base like is it have you seen like a linear relationship or is it lagging now like how does that look yeah that's a good question so the question is is there a linear relationship between the growth in our players and our creators and mostly we find it's a little bit sublinear so they both grow and they track each other but there are a little bit faster growth in players than creators and i think this makes sense that there's a little bit fewer people especially as you get out of the early adopters that are as excited on the platform but for us in particular we find that our community there's a lot of blending between the two types of people and especially with children who want to

be creative they'll play one day and then they'll create another day and so there's always going to be some crossover thank you thanks hi hi did you ask any of your users i guess before the growth spurt and after the growth spurt to see if there was any delta and reasons why they might have signed up for and started playing so that's a good question uh did we gather good user data on what was what was changing before or after the answer is not great data unfortunately we have started doing more of that user gathering data since then but we didn't have it in place at the time so we unfortunately can't go back and get that information with a bunch of different creators in your ecosystem and a bunch of different people spending money in those games do

you take any responsibility or how do you manage the longevity of something that somebody else has created or is still creating for your users while knowing that your spenders in that economy have an expectation that their money is going to be valuable in that user generated content today but also months from now great yeah so uh i'll reap the question which is if users are spending in a game how do we ensure that that game what they've bought is still valid down the road so the first thing we do is we have a commitment to make sure we don't break old content we go to extraordinarily lengths to make sure that old content is still relevant and we work with the creators to make sure that we don't cause them problems um we find that most games still operate if they're making money and they have a significant community involvement in their game they

will continue to operate that game and not break it there are times when when things like that do happen somebody bought something in a game and the game creator broke it and doesn't care it's relatively rare it does happen i'm not exactly sure on our policy at that point in time but we do sometimes offer refunds in certain situations hockey stick growth is kind of a double-edged sword uh with that growth comes management problems scaling problems not of the systems necessarily but the humans additionally it increases speculation on your brand and i'm not sure if roblox is publicly traded or not but that can tend to be quite volatile how are you addressing uh the the non-engineering risks of explosive growth good question so how do we how do we handle the other parts of explosive growth the downsides

uh so first roblox is a privately held company and we um so we don't have to worry about external pressures which are tough i'm glad um at the time when we were going through this phase i think a lot of it came down to our culture and there were challenges um but what we did was we grew slowly within our means we didn't say okay now we're a different company now we're going to be like that company over there we're going to rethink ourselves so we stayed within our existing plan and didn't try and overshoot we didn't take i think where people can get into trouble is saying i've got to capture this thing now i better hire a thousand people tomorrow uh because we're this other company now and we didn't do that so that seemed to work for us thank you

hi um since observing the growth in the turkish market when you opened your german servers for the game have you developed any sort of scheme to track potential player bases and see if there's interest in having servers set up in that region of the world good question so are we tracking interest in different parts of the world and how do we determine how we're going to approach it we do so it turns out that currently about half of our usage is international maybe just a little less than half so we have a ton of early signal on where users want us to go and it's interesting that some countries tend to be much more popular than others so for example south korea loves roblox japan less so today and even though we haven't done anything in particular for either country so we get a lot of strong signals off of the existing traffic and then we get to target where we want

to deploy more services translations things like that are there any identifiers that are specifically outside of the game that you have noticed in particular so uh one that i've noticed is youtube streamers so that seems to be a strong indicator that there's a cultural affinity for the product so with south korea thailand some other places unexpectedly you get these streamers who have millions of views of roblox that's maybe out of proportion to the traffic from that country itself and so that potentially is one of our indicators for hey we should take advantage of this okay thank you thanks hi uh were there any factors uh you identified as critical to the very steady 30 year-over-year growth prior to the hockey stick that we're not talked about already yeah so what were the how did we identify the steady 30

so what's funny about that i thought about this uh i guess this is natural to humans you have some kind of status quo and then you just accept that oh this is the thing right and then but we don't actually have any way to prove that that was the thing we should have expected versus the change um so the answer is not really um the nature of the growth at the time i think was word of mouth viral growth we didn't do a lot of advertising we never did but um so i think there's a coefficient of how fast people come onto the platform and that 30 to 50 percent represented that but like i can't prove it i can't there's no like mathematical basis for either phase i guess hi i have two young girls uh that absolutely love roblox i spent hundreds of dollars on great

games and nintendo switch and they prefer to play roblox which uh sometimes infuriates me um my question is my question is uh as a parent um since the uh players are so young um and anybody can create and anybody can create whatever they want uh or do you have uh are you thinking or do you already have something in place that cure monitors the content and curates it in some way yeah good question so how do we monitor moderate and curate the content so because we are targeting young children this is built into our dna from from every phase of our product so every chat message every piece of text goes through scrupulous text filtering we have teams of moderators that stand by and react in very quick time to anything that's behaving badly any kind of anti-social behavior any content

that is not appropriate every image sound mesh and so forth goes through a moderation queue before it can view it on our site and so forth we can't be perfect we can't absolutely catch everything and guarantee it but it's it's very good and we're always working to make it better uh thank you for your presentation um so you mentioned you have quite a few players in south korea and japan so i'm wondering are you thinking about creating some content that caters to the asian audience and also just how do you think about the timing of international expansion in general yeah yeah so the question is what do we think about the timing of international expansion and and do we want to create content for the asian audience so what we want to do we don't create any of the content ourselves our goal is to make the best tools the best platform that enables content creation from the

community and we do believe there are cultural differences in different parts of the world and people do want to experience different content there's going to be a lot of overlap but we want to support all of the variety that's that's one of the great powers of having a really long tail and a huge catalog so the way we achieve that is we build really robust communities of creators around the world in all these different cultures and countries and we make sure that we provide the tools that let them create and then we make sure that the discovery part works as well so in fact today on roblox you can go to our games page and see games that are popular near you and that list will be different in korea than it will be in america and as we really expand internationally over the next couple of years hopefully we're going to see a lot more really cool differentiation in that area

thank you thank you hi um you mentioned that this basically a double network effect did you realize that you in order to unlock one you needed another one was there like a chicken egg problem how did you solve it yeah so uh how did we solve the double network effect in unlocking creators versus players i think we started with a single network effect and we really started very small we started with just the simple vision and built up from having two players online to wow we've got 10 players online at once that's amazing now it's 1.5 million but it was a very slow growth to get to that phase yeah did that mean you were creating content initially so only at the very beginning like literally only in the first few weeks of creation because in the beginning we had to create a tool to create the content anyway and as soon as we had that tool we could give it to the people who wanted to play

got it yeah cool thanks hey did you find that the distribution of play within the games changed as the growth grew ah so did the distribution of play change within the games as the growth grew so uh we didn't see any major change i would say we always have a pretty long tale of content there's a few games that are super popular um there's maybe a top thousand that have like dozens to hundreds of people in them at a time and then there's a super long tail of pretty obscure stuff that fundamental distribution did not change when we when we had our growth story that's time okay thank you everyone so much really appreciate it