Product Management at Riot Games | Paul Belezza

everybody I'm here today with Paul bellezza who is product manager at Riot Games and I don't know if you know this Paul but League of Legends is one of my favorite games of all time thank you Paul works in the R&D department I thought we could first start by asking you about you know what is it like working at Riot in terms of you know the work and then into the culture I've been away a long time to been able to see a lot of evolutions of how we work but I would say the thing that's remained constant is a dedication to the mission of who want to be truly the most employer focused game from in the world and what that means is all of our decisions are heavily weighted with that in mind everything we do whether you're putting on a new character we're putting out you know a new piece of content we want to sell we go alright if we release this thing what does it mean for the rest of the game what does it mean for the audience is it too much of a Forli terms are we doing too many atcs andro are we not balancing out the support rule so that level of thinking comes in

to everything we do and it's been there since the start so as a developer that's pretty awesome because you're starting from a place you're like we're trying to make the best experience for the player above all and everything else business value monetization value even like balance comes into play with that versus versus not and so that's that's cool the second thing from a cultural point of view is a where company of like-minded people who love games most of the company the majority of grown up loving games playing games of all types of all genres so you bring that kind of audience perspective into development both as artists engineers game designers producers but also people in finance and legal in eSports so we all have this kind of common vernacular so when we're talking about making decisions a lot you don't have to give people a lot of context ability why would we why would we rent out X venue to do this why would we you know spend all this money at PAX to give away this type of stuff a lot of

people get it because they go oh yeah that's what would be fun for the audience a very fun for players etc etc and that's that's part of the culture in itself and that's really cool as you can just be at lunch with someone in the legal department and you can be talking about what favorite games you guys grow up playing and what you're playing at and you can have a deep conversation about it so that's a little bit about like both the development and just how we are as group humans working together I think that's pretty fun yeah and I know you've been at Riot since the really early days and we can go into that later but I thought maybe the other thing that it'd be great to hear about is a little bit more about the history of ride could you talk about like how did start you know just kind of that the history yeah okay so bright games was founded in 2006 by Marc Merrill and Brennan Beck they were both roommates at the time they were both classmates from USC and they were both and I actually have met each other in high school I believe on some type of like like leadership camps at each other high school camp your high

school camp and they met each other and there they connected because they're both they were both really into video games and Mark is funny as a human he was very much like athletic captain the football team type of guy but when you talked to him he plays D&D and he plays video games hardcore really into EverQuest and that's the type of guy who'd be like you don't play D&D let me teach you how let me get you in there and Brandon who was always just like just a cool down-to-earth guy who loved video games and particular he geek they're all fellow Los Angelenos like me so we have this kind of connection of growing up in the various gaming pockets of LA and she came up living in the city and yeah I went to k-town and playing games and PCs she was very much let me go communally play games with the groups of people so that was the genesis of their relationship and so after after they graduated USC after they went on to work in various business places they they decided hey you know be rad would

be really fun if we made a video game company and we've talked about what types of games they would want to make and I like what we both want something competitive something we can all play with our friends together dota this team dota is amazing it's an amazing mod kind of reminds us of early csgo back in the college days there's something special and as they did research they realized how big how many people across the world were playing them on I think over a million to two million at the time right this is something really really special there's something cool here what if we could bring this to the public in a way that nobody else is taking advantage of so there is that was kind of the genesis of what game League legend was like why don't we take the stand alone mom let's take this mod and see if we can make it stand alone and second they had both been studying a lot of free to play games and East and in particular they knew you know that was the way you play games in Korea for example very differently played games in North America it's a culture of

you go after school to a PC bong you pay to play with your friends by the hour you play as much as you want but you you come together communally in there right there's something special about that the majority of the games there were free to play and so they're like let's study that market and they're like this is actually a very viable way to make games the the rub of it is some of the monetization mechanics are a pay2win like hey what if we took a skill based game and applied a free-to-play model to it without having pay2win mechanics ooh try to find a way to just focus on serving the core the core player base in terms of making a very compelling engaging competitive game and then we'll find personalization content to monetize and see if we can make that business model work so this is 2006 at a time western publishers were like what get the hell out of here that's doing that you should be in the web you should be on on like what Facebook was even that big a deal is my species be on MySpace yeah it should be this type of product and that was the genesis of their pitch

and I think they went around for a year trying to get money to do it and eventually they did angel investment from themselves and their family and then a small investment for the very small PC and that was the genesis of the first you know phase of riot 2006 and 2007 and eventually grew out to what it is today so humble humble nitty gritty start back then right and then since then obviously the company has grown quiet quite a lot could you talk about so in terms of riot today how was the company structured in terms of you know the types of people the types of teams and in terms of you know maintaining running live operations on League of Legends versus kind of new titles sure okay Wow so we learned as we went yeah and I was fortunate to be able to see a lot of that evolution so in the early days was very much get it out there in fact one of the earliest stories is when we launched League in 2009 we had a store but it wasn't working and we had already

communicated to the eventual player base that we have started to build their community alpha testing that the game was gonna be released on next day and our store wasn't working so instead of delaying the release we just said let's release the game free and without any thing you can spend on just will release all the content for free so you can play with it and we worked for a month to get the store up and running and eventually we turned it on and started growing so big that that lesson still holds true to this day when we onboard new people because we're like instead of delaying instead of doing what was you know potentially better for us business-wise we preserved the audience and we in a month we figured out how to get a store and later on I ran the store team and I'm still dealing with some of the decisions so anyways like so the early days was very much like just get stuff out there good stuff and I will figure it out over time as the scale of the game grew you know we learned how to deal with server architecture and scaling that up and how to communicate

with your player base when your things are falling over so over time we invested a lot in live operations and we built a network operations center that we should have investing in those types of things and over time we smoothed out the way we service a game before three years it was very much trial by fire as we were releasing new content things will fall over get it back up paste another server do this shard our servers to make sure we can support players around the world and so we learned the hard way of how to scale up live operations so now when we approach our development of I would say our bigger bets these are gonna be our I'd say our it's similar in ambition type of games we're thinking about that stuff way up front so in addition to having you know game developers who can ID eight and think through what the product is going to be and what the gameplay problems going to be we also partner early with live operations producers to make sure technology is going to potentially al appropriately to what the demands of that game could be if everything goes

well so we do a lot of scenario planning now to figure this stuff out and to me I just smiled I'm like I'm glad we're thinking about that I remember in 2011 or 2012 when we just assumed X and then Y happened and we were freaking out for three months so you know we're doing our best to approach that but it's very much a holistic development point of view for some of our bigger ambitious projects at the moment and then can you talk a little bit about your role as a product manager at riot and certainly like product manager at different companies can mean a lot of different things product manager vs. producer versus design or whatever but for you at riot what does that mean to be a product man yeah yeah so for me at riot it's it's it's more of a hybrid role and I'd say a traditional producer and some aspects of product management that you would see in other companies like other companies might see revenue strategy and progression design and monetization mechanics being analytics being before skillsets pm's do have that but you don't necessarily have

to you could be partnered with people who do so for me I'm a little bit about where my products I'm accountable for defining what we're doing and why we're doing it which meets you know the business objectives and audience objectives and I'm also accountable accountable for getting it out the door so I'll lead the development team on the ground floor to build whatever product we're making and make sure it's hitting or at least set up in a way that we can hit those goals and then once it rico's live I'm accountable for measuring the progress of the product and iterating on it or you know making tweaks on it or sunsetting if it's not achieving its objectives so in that way it's similar to other types of product management but it's not all it's not all it's not all the same oftentimes we are coupled with various other disciplines to help them so for example I was on the skins team for League of Legends for a year and a half so I is the product manager was in charge of figuring out what skins were going to release during this quarter of the year so that would mean picking out

what champions we wanted to develop content for and which thematics we would do based on past performance history and I would also be the creative team to figure out you know should it be a cowboy or should it be a samurai or whatever but and I would be coupled with a revenue strategist you'd be like here's an analysis here's what we think the potential is here's what we think the audience reaches but I advise you to do X or Y and we would do it together so that's very much a partnership with so could you talk about those other specific roles that Riots you-you'd meant you mentioned like a revenue strategist so what are like the other roles in terms of you know do you also have a producer and a designer and this or that or what look like if you can describe those other roles that'd be great absolutely so depending on the product you might have a whole slate or you might have a few but in general some of the types of roles we have our product managers like I just described revenue strategist which used to be called ecommerce but they're basically our monetization and business mechanics developers and say go crunch all will do

that but do the math the spreadsheets say okay but but I appreciate that closely within my insights which are going to be your your analytics folks so they can either do qualitative research yeah connotative research and there's various flavors of insights pokeman mostly you get a researcher and you work together to prove out your thesis hypothesis you work together so those are some of the roles there there's also development managers which is another flavor producer these are your more typical project management for software development roles yeah be kind of what we say it riot is product managers should be able to do all the skill set of a development manager but a dev manager doesn't even necessarily do all the skill sets right okay they're pretty pretty closely intertwined and I've been partnered with a lot of great dev managers and we see it as like a 50/50 partnership in all the way we go here's what I want to do and product manager here's what I think it's valuable here's what we here's the right decision here's what I believe the right decision to make in a dead manager those okay based off our resources our timing here is

what we can support or not support and we negotiate together got it so the dev manager would be more responsible for interfacing with the engineers more directly and like managing the sort of backlog in terms of engineering yeah exactly so it's it's still a partnership so the product manager is expected to be in those backlog grooming session site for Ortiz but the day-to-day operations of the team and the team he'll for the team culture that is jointly held by the democratic manager should live more than right I've done both roles so you know it's it's they both they're both synergistic when you have the right partnership at it and what is your typical sort of day to day look like for you and your current role yeah so I'm on an R&D team so I'm working on a Content team and I earn and a more exploratory team and Arne so kind of where to kind of do two different roles within that so I my conduct team is very much working with my development managers and my discipline leadership to look at the roadmap for the next six seven months of

development and go hey we're trying to make X amount of content this is the flavor of the content here's why I believe it's the right content to make for what game we're trying to make disciplines are you staffed appropriately is your team in a healthy mindset do they have everything needs you to succeed except etc do you need any clarity and the decision-making for your team for example like if we're making this you're making a character like is it more serious character isn't a more gregarious character and so how does an animation look from one of those it would be my job to make sure my animator understands you know what we're going for and they could leave a team to it so there's that layer and then there is like the day-to-day running operations and so I might attend a stand-up everyday to make sure the team has what they need for me and they have what they need development I do a lot of the high level vision work and then they work on executing it and we review week to week together so that's typical on a typical day and then what skills do you

think are required to be successful as a product manager in your role and then you kind of worked in a lot of different roles at riot do you think there is a difference in terms of the skills required to be successful as a product manager in more of a live ops capacity working on League of Legends kind of you know like feature update skins characters versus kind of your your new role in terms of R&D and like trying to come up with new game concepts is that is there a differentiation I think there is I think I think there can be people who do both I think they're they're different mindsets yeah so an R&D mindset is hey we have a hypothesis we want to prove out we want to build this feature let's go build it let's test it locally within our team and a small group testing of human beings outside the company even validators and it's very much a you know experiment test experiment test experiment test framework when you're live ops you do have some of that to agree your lie so you have to look at what's out there about what is the community saying what are the players saying what are the

behaviors of people saying based on for analytics what decisions do we need to make should be tune of game design feature this way should we retune pricing for a set of products this way I think when you're live you have to have an iron resolve of information gathering and not be swayed by the whims of just your data or just put the community saying where we are for example our reddit presence is very large it's very very loud yeah yeah and so as a product manager I've seen some more junior are more new to live ops development people look at reddit like we need to make a change what I have learned in my experiences reddit is a point of view right mostly represented by this group of humanists and the less here they what they may be saying may be directionally accurate let us take a look please engage but we don't need to we don't need all right quick quick pivot just because some people so I

think you need to have this mindset of like let's wait and see let's let's give it a week if it two weeks ago or three weeks before you decide to make a change and sometimes you may find that the voice of the community is loud and right and you can see enough across all the different sorts of informations to make a change and other times you may see that it will pass with the right context so that's another part of right that's actually I think is special is everyone is encouraged to speak to the community directly regardless of your position you don't have to go to a publishing team right tell them this you can as a product manager get on reddit or get on boards going hey everyone here's what we're going for we hear your feedback here's what we're thinking but let us know and that's part of the job too so you have to have a proactiveness to jump in and discuss things with your audience directly so those are some of the things I think differentiate that the nine cents right and you know League has been around for a really long time super successful you know one of the most successful games of all time but one of the big questions I think with respect to riot just given that you know how

much love there is for League is when the next big game going to be coming out in terms of can talk about in terms of like new game development like it's what's the current thinking at riot and what is their sort of a plan and you know we can talk about TFC in a second but but just generally speaking yeah so I think it's not a secret that we've been lined with the s and right and if you've been following the community for years there's been starts we've canceled we've canceled some things some of which the community knows about because there's been leaks like a long time ago we were exploring a type of card game right but we we the point is we what we try to do is look at the opportunities that we think will serve an audience and we as we as you know people who love games - we have our point of view of what we think we should make or not make and what we now - I'd say much more rigorously is we try to validate our gut and so we'll start up an R&D project we

have an R&D pipeline now we're a team of humans can pitch pitch a game based off like these parameters that we're looking for and they it's on them to do the due diligence to say hey here's what we think the opportunity is here what we think the twist and gameplay is here's what we think our capability to do it is that we would like to find an X amount prototype they go through this process now and using that word work will be much more deliberate about what we find there are products that have been in development for a long time some may see the light of day some may not but those have been very much guided by okay what do we think serves our type of you know the type of games we make with what is a type of ranking for us I think and deeply skill based potentially competitive things that have great social hooks to play with your friends cuz it's it's a lot of what B does and so we've had some good good results on that yeah I look forward to what we do whether they come out like the project I want to come out see and some of them don't make it cuz it just doesn't it doesn't it doesn't make the light of day

and I think if you look at League you can see us experimenting we've done a lot of stuff with them game modes in particular they credit test the waters and like what we think the appetite is for certain mechanics and we use that information to inform other R&D products so net net of what you asked is yes there are other game I will they come out I hope so I really want one I do for one for you I'm impatient and I've been really a long time but yeah I trust in the people doing what they're doing I say stay tuned right we've been speaking of new games I wanted to talk to you more about teamfight tactics and so you know I believe the original mod on Duda 2 came out and like January February it's like it was already out it's like so fast can you talk about so how did like if you can tell us a little bit about the inside story of you know what was the thinking behind team fight tactics and how did it come out so fast so I can I could do them as much inside baseball's I'm safe to say okay in January this year the dota 2 Auto chess game came out

yeah a lot of people from insights into game design in particular omni's and legends were playing in this game including and they were at the same time paying attention I believe to just some of patterns you can see from Steam dated like wow so the two numbers are really good right now well I wonder if it's out of just waited and the hypothesis was it probably was and as it can you the play they're like well makes sense it's really really deep as a lot of breath there's a lot of you know takes advantage of like a lot of them the thematics and the the combi new aspects keeping games playing don't have items special and so I think a small group of rioters were like pain why don't why don't we try to prototype with this enemy let's see we can do so at this point it's some history it was been hard for us to prototype games in league because virtually all of our tools proprietary someone who's worked on the game for a long time he wanted to get like a piece of art on the map changed

really difficult or at least it was a few years ago cuz everything was kind of custom and manually put together so over the years we've evolved our tools to better facilitate creating new maps and changing up gameplay we've invested a lot of map over the course of our in-game events with past years we've done things related to the star Guardian skin line we did an odyssey event last year we literally make game mode so building those tools with game modes gave the game designers enough like options to be like hey I think we can pack something together quickly that's inspired by auto chest and they did and they they showed it to a small group of leaders on League of Legends and the leaders are like from what I understand this is pretty awesome why don't we why don't we get a small group you guys continue to working and if it becomes something that we really think will meet audience needs we can try as a game mode with in League of Legends because we do game modes all the time we had Nexus bliss lab like last year which was like a wild interpretation of summoners rift that had all sorts of weird mechanics so

this is not something we we don't not do we like right we definitely experiment and so there was enough momentum behind it and enough expertise that built over the few years that they were able to put something together quickly and then show hey this might actually be really fun so they put a bigger team behind it and that's how it kind of manifested over the course of the few months Connor yeah I think it was three months they went from idea to at the door which for us we we made events it would take six or seven months for us to get the game modes and scripting of the content all in the right place but we've gained some capability and were able to utilize that right idea right time right group two so bring it to some players yeah that's awesome and when I look at some of the twitch numbers it seems like you know TFT has just been skyrocketing right it's been doing really well work where people that riot kind of surprised by just how well it's been embraced by the community we've been really humbled I don't think I might guess because I don't know all the details I'm not on

the awareness campaign team was we knew was something fun it was something fun that people they liked it they would they would enjoy it and I know that they partnered early with some of our community team members to get their feedback and show them the game before a couple months before launch and I think a lot of their sentiment was positive and that allowed them to when it was time to show it on PvE partnered together to put some guides together and I think that helped propel some of the awareness for it and then steamrolled into what it naturally is and what I think is interesting is for me playing it the past week and a half intensely with my wife and my family it's it's it's familiar but different enough like yeah you play summoners rift or you have to be very intensely concentrating on the game like every click every position could cost you a strategic moment and TFT is much more

I wouldn't say purely relaxed but you don't need to be on your computer as intensely you can lean back and play and so what I've noticed is that it's just seemed some very uniquely put to benefit streaming oh yeah gene you're clicking you're standing bill you can talk through it and I think that natural fit in that different type of gameplay has passed has helped with that indeed so the net Menem is I don't think we expected it but I think we had an inclination based on how good the internal feedback and the small group of people saw it and it seems like when it when they saw the numbers like people on the side channel were like oh my god so shifting gears a little bit you've been at riot for a long time just kind of moving forward if you're to look into the future what do you think are kind of emerging trends or changes that will be coming to riot maybe from an organizational perspective or from the

perspective of a product manager and what that role means at riot or even from a product perspective yeah I think what what we will see in terms of trends is we're going to continue to pay attention most importantly to where the players are yeah and we're gonna be where they are so for example a lot of people have asked teamfight tactics is it gonna come out on mobile and the answer is maybe you know Mike for me I want it online because I would much rather be able to sit on my couch and do that and players are asking for it and I think in the past we may have been more hesitant about even entertaining that because we think it's old-school idea is like no games the games are on a PC or games on the console but you're seeing groups of humans all around the world going no I play where I can I'm want and given that I think we're opening our eyes to feel like we need to meet the players when they are so that could mean the world work you know we'll invest in it I'd say we are but we would look at TM or other games for example and then if you look at the long term you're like

our peoples going to be changing the way they play games it's streaming technology should we be thinking about like what types of games work for that I think we're we're open to all those done like for example when PR was hot but still less hot but you know when it was really hot we had people come jumping on that going let's try to prototype experiences we always we always keep that stuff in mind yeah and I think it taught us is product and game developers to be like look what serves the audience best where are they and what type of experience we do and we should be moving fast and T of T I think is an example of let's capitalize on something we know is fun we know we can pull off we know we can get there and create a little spin on it we quickly inspired by the original game but we find our own twist on it and we're able to move fast so I think we're gonna continue to do that because we're in the best team and capability to move much faster and hire more people with expertise in like managing and managing how to tune your audience needs accordingly using much more data

informed processes like see how we can exponentially make games better versus just relying on so I just have one last question for you just in terms of your history at riot I thought it was super interesting and you've been there since like almost the very beginning so I was wondering if you could just talk to us about your story how you got connected to riot and adjust your history that okay so many many years ago I was a high school teacher and I flipped out and I decided I love teaching I love education but my true love is video games I need to enter the game industry long story short started off at retail I actually got a QA job and decided I wanted to go back to school for game design and game development this was 2005 so at the time it wasn't a lot of video game schools there's usually are those commercials for like these video game schools were like oh polish the graphics on level three you know like all this kind of like whimsy about what game developer iron and I knew those not to be reality so I went to USC because I had a new graduate program in game design and I've

had some funding from EDA to do some experimental game stuff so I went there certain media like-minded developers and eventually we would realize you know what I can with this like-minded group of student developers I can meet industry vets because they would come in and talk to us and see if I can get an internship in a studio so that was like man go get an internship at a studio using the connections at the school and through my school friends eventually my friend and I and I were both fighting for an internship at pandemic back in the West LA back in 2007 and my friend got the job and I was happy for him but I was sad for me because I wanted to work mercenary says what they work on and in in a in a day a day of like pure determination of sadness and this one on Craigslist in any way with the mindset I will apply for any job that I can see at a game company in LA like I don't care how small it is I'll do it so there was an office manager position open for a startup video game company in West LA

and I sent in an email expressing interest and I got an email back by my guy named Mark Merrill it was a USC alumni would you see a You Missy that was nice and he asked me to come in for an interview and he told me the company was Riot Games and they were starting out and they describing in the interview on site a very small office very humble there's only five or six people definitely four people at that time he was like they were describing the game of the opportunity and I was like hey that came sounds a lot like Duda and that model sounds a lot like gunbound which was the game I was playing at the time they had like a free to play worms calm I'm like you're gonna take some of the you know free-to-play opportunity you have a game like gunbound or some of the next on games with a game lakota and find your own version yeah I'm like that sounds amazing and so they're like awesome they were just surprised it like I you know that another person saw the opportunity right there without a lot of context

yeah we're all like-minded video game players so they're like would you like an internship I'm like hell yeah hell yeah I'll take an internship so I became Wright's first intern in September of 2006 and so my job is you know help out with whatever need helped off but they were like hey we need help kind of concepting some of our champions for this game would you be willing to pair with our game designer and do that I'm like of this is my dream so I got to help out on everything in the earliest days and it was it was wow wow man so this is Craig's it's also just an interesting story of how you know the initial disappointment actually led to a much better outcome can you say today if I wasn't motivated to keep looking based off the rejection yeah I would have never found the opportunity so I'm I'm thankful all right well great well Paul thank you very much for your time you've got a great story and obviously you know everyone everyone knows and loves rioter

you know most most of the world does so thank you very much for your time [Applause] [Music] [Music]