The Future of UA+PM | N3TWORK's Nebo Radovic talks Product Management and User Acquisition Strategy

everyone welcome I'm here today at branch headquarters but it's talking to Nebo from Network and I thought I'd start with first questions just asking you about this issue about the integration of user acquisition teams and product management and I think we've seen a trend over the past one to two years that there's been an increasing need for both of these two teams to kind of work more together and I wanted to ask you about you know I don't know of another company that that has actually stated that they've done that more than the network and so maybe you could talk about what are the advantages of having user acquisition and product management teams work more closely together Yeah right so so to just provide some context Network is a company behind legendary game on heroes yeah it's the game that that's monetizing using live events and these events are running every week and they've been running for like over like hundred fifty weeks I

believe so far so basically it's a very dynamic dynamic environment there's always something going on there's a bunch of a/b tests there's a bunch of events there's a bunch of different cards you can buy so it's not it's not one of those games where you just download the game and there's like a certain it's like a saga saga games where there's like 100 levels and you reach like a certain level a certain time and there's a table so the game is very the game monetize is the monetization begins quite complex and as a result you know the you a theme and live ups teams basic work together on understanding what's so different and what's the impact on early user monetization if we're honest I maybe test for example and what's the impact on payback windows because you know if the game is has different content every week then that has a pretty significant impact on on payback and you know cash flow of the company okay and so you know in previous companies I've worked at kind of you know we started this trend towards UA teams and management teams working together and we used to have

like a weekly meeting and in that meeting we would talk about like on the user acquisition side the UA folks would talk about like like a weekly review what worked from a user acquisition perspective what didn't work sort of targeting new channels to experiment with experiment with a try and from the product management perspective we would talk they would talk about things like you know upcoming features events you know a be tests and things that would require more more users and you mentioned you talked about some discussion about monetization pay back windows but could you maybe go into a little bit more depth about so what what are these so two questions first what should these two teams be talking about in terms of like agenda and then secondly like what is a good cadence is it a weekly cadence I know some come at some company if they talk about having both of these two teams me daily and talk daily III just don't know what they're talking about on a daily basis so so maybe you can go into

those there's two different aspects yeah that's that's a great question I'll try to answer it in a fairly short way so so again like legendary it has like live events pretty much at any point right at any point whatever you enter the game yeah so those events monetize differently so one of the reason why we're meeting on daily basis is because like different packs are sold in different days on top of that we're like constantly running different a/b tests that in fact early user experience so like early user experience is not this like box solution like black box solution where you know there's not like things are same every every time yeah there's actually they're actually live events that come after you like complete like certain number of levels or you know what's you done with the tutorial so those things can have a pretty significant impact on on monetization at the same time you know like you have we have you have the Y of gaming then you have all these like machine learning algorithms there's dynamic content there is dynamic pricing

there's their different push notification campaigns and all these things are changing all the time and as a result of that like you know user acquisition team works together be data science teams to tweak the machine learning algorithms to improve early monetization of late monetization is that if that's goal and the same time we need to be aware of what's going on in the game what are the events they're you know what are the events they were running at the moment so like we can isolate the effect of those so one of give you some KPIs that we look at unlike pretty much daily basis or maybe like three times a week you know we're trying to understand okay what's the early retention or what's the early early pare-pare conversion and how like different things were different like types of traffic can have an impact on that for example if you're running an event with like russian themed characters and we're doing a push in Russia you know and suddenly we're having like love you know driving a lot of installs and driving a lot of revenue from Russia then the game team knows you

know that that's because we're like we're being the heaviest like combined effort to improve the the the revenue coming from that particular the world so I know it sounds like like you know it doesn't make sense to meet on daily basis but there's actually a lot going on I see and and thinks thinks if you don't pay close attention things break all the time you know you can do like a simple thing as you know moving like you know moving like certain packs from like rank 5 to rank 10 yeah you know some not showing that before and then and it has like a huge impact on the early monetization again that has like after that it has a huge impact on for example how Facebook you know performs because they're early the early revenue is not there then faizon debt you know there's no there's no enough signals for Facebook algorithms to optimize the performance Thanks so we need to know all these things in order to do our job successfully and as a result you know we're like pretty much we're you know there's always one person of the one

person for our team sitting together and would like with users and trying to understand what's going on so just to help me better understand it sounds like because like network aren't much legendary game of Heroes monetize is so strong against events and then these events can have an impact on monetization which may be impacting things like LTV and Rose yield if as a you a buyer if you're buying it you don't understand the changes that are being that are being done that you may not be you know it's not like your your targets are gonna be static and so you need to understand what changes are occurring is that fair to say yeah that's that's exactly what's happening okay so uh you know to give you like an example so when Facebook introduced value-based optimization yeah you know the basic of the signals they're receiving from are adding like you know are subtly different you know I don't care more about revenue or the signals they care about to be more precise they care more about revenue

than about that about retention for example or the the sole amount of transactions and and you know we had to like understand at a time you know okay whether we should change the early user experience to kind of send more revenue data and says they like trying to front love the purchase events in the first seven days so Facebook could optimize better and you know do better in terms of performance right so those are the discussions we have at the front of team and the data science team and then we work together to optimize the early user experience with late user experience you can optimize for those for those like events or actions okay and just just speaking from my own personal experience I know there has been more discussion about you a teams and PM teams working together and there's some level of that happening but you know practically I think in the industry we're not actually seeing teams work as closely together as see it Network so how much of a competitive advantage like if you were to try and characterize this how much of

a character how much of an advantage does having this tight level and integration provide to two teams I think quite a bit and one of the one of the things that are being sold with this set up is that you can spend more aggressively because you know the game theme is accountable for us spend as much as you a theme is and vice versa we're like as accountable or like responsible for the game game fremming you yes the game team so you know if we want to spend let's say you know millions of dollars a month and you're just like okay my d7 target is like 15 20 % you know that's it I don't care about anymore and and you know then like you have these two siloed teams right and suddenly something changes the game you have like a serious of really poorly monetizing events that don't do well suddenly the payback windows goes from what will go from and 150 days or like 200 days to like 400 bags that can be they can be quite devastating for the company right so when these two themes

work together they define target's together which is another important point I forgot to mention so we work together on defining monthly targets both on the UI side of the revenue side yeah you know then they're both like responsible for the results and you know they work together and towards the same goal which is really really important what I heard like is the case in other companies and you probably have more experience than I do and from some of my previous experience usually those two teams would be actually silo Danya and as a result there's always like this dislike set up where people are complaining go or not spending more money on my game and you're spending more money as the miner game or you know beginning things always transparency isn't there in the discussion yeah basically you know teams are just trying to play in the other theme on why the why something's not performing well in this case we work together we're pretty much the same theme and we work towards the same goal and that helps us to be aligned and together you guys establish the KPI as a map a back window that could adjust fairly dynamic yeah exactly

okay exactly so as a result we can spend more aggressively yeah on because we can spend more aggressively the king crow much faster than some other companies right and you know people are just curious and they like the game team knows more about about you a then probably what an average game team does right and the USA team knows more about the game so in every little while you can come up with something hey you know we there's this opportunity to scale instant traffic but we need you know a little harder for like first twenty levels need to be a little harder can we change that the game team understands these like different levers and they can optimize the experience to help us do better as well great and I wanted to ask you next about genre specialization and you work with Eric super and you know we did a podcast a couple weeks back and you know one of the points that he made very strongly was that he was really pushing for genre specialization or hdu or he was at least saying that if both like the you a team

and I think he was he also suggested p.m. team is not as specialized as possible that you can't possibly be competitive I wanted to see if you know if we could get like your sort of you know you know your opinion in term in terms of that concept and then to provide a little more color about what you know like specific examples of what gives you that competitive advantage if you do specialize yes so so I work in the past I worked on like top eleven which is a commitment like it's not really a casual game it's like a soccer management game and then I worked on like a more mobile strike and now a legendary and and you know most of these most of these games are gonna make two hardcore yeah so they're they definitely monetize differently than the casual games for exam and on the other hand you know user acquisition is much harder because your funnel is much more their error so so you don't necessarily need to optimize

for like the Optima is creative to an extent where it performs at the same level as the casual game creative performs so you know like the thing but if you don't know that if you're just applying the same kind of one-size-fits-all approach then it's likely that you'll optimize for the wrong metrics and you know you you won't understand when the money's coming back and how to scale the game successfully so so that's why that's why I think first like for me right now if I wanted to do you a free candy crush for example I would probably need to spend a few weeks at least or maybe a few months to understand you know what are the levers that that I in fact the the payback windows and all right and before I could do it like at a large scale so in that time like genre specialization is is really important because you know you'd think creative is important for example it is reported by the way but it's important only to a certain extent or you know how L do your curve curve looks like it's quite a friend that between

like casual games and supports games and and hardcore games card collection games so so to be able to get the most out of a certain genre you need to really know you know how to use a ship for that genre how the monetized and genre and India I like to be able to do use our position properly you need to understand the audience so those three things that are really important and if you don't know them then you probably need to get to like a certain level of expertise in that base time right so we've talked about kind of two concepts today the first is like this increasing collaboration communication teamwork between you and product management and the second concept we talked about was sort of genre specialization really understanding the game that you're working on or marketing extremely well and so I want I was hoping you can help character I like how much of a competitive advantage both of these concepts provide because if I'm being

honest I think Jen speaking in the industry I don't think it's that well adopted and so like what percentage chance do you think and sort of the medium to long term would a team have of being successful if they don't kind of move towards these two concepts I mean you can be successful to a certain extent so you know if you have like a lot of money as a company and you can afford those like yeah sure just like you know spend as much money as possible I know it's gonna pay back eventually maybe not it's fine yeah but if you're like oh well we're still a fairly small company we don't have like hundreds of millions of dollars you know in our bank account and you know that's why we need to like be as efficient as possible and work together in towards the same goal and try to be you know even just try to be more successful so again you know there you can probably be successful to a certain extent but in the end you know there there's there's like the same ten or twenty games and

like top twenty grossing games out there when you look at at the top like where the top spenders in the industry I mean there's still like five to ten different company that have been around for a while now like you know peak games and and which is still there it's not a gaming company but they're spending a fair amount of money you know we mentioned combat common fewer games so there's like a handful of companies that do it at at scale and there's probably a reason why they're there not all of them have a lot of cash just that they have a better understanding oh ha like you know user acquisition and product work together and what are the things they can do together to become more efficient so again it's hard to provide like a like a binary answer you know like yes or no but I think I think there's definitely there's definitely like that last 20% that you need to become great that you gain by understanding the product better and working together towards the same goal so you know kind of a kind of changing changing gears a little bit I wanted to ask you about so

just just recently I've been kind of hearing a lot of comments from kind of two opposing schools of thought with respect to user acquisition more specifically in the first is user acquisition is easy like anyone can do it and they're kind of pointing towards how like with the machine learning based bidding algorithm things are becoming more of a black box right so you AC in particular you kind of push the button and then it's like the focus shifts towards more creatives optimization or something else and then there's another school of thought where it's like where it's complete opposites like well that's what those guys think but actually the guys who can do user acquisition really well and know it extremely well to get the most performance are extremely rare yeah and so I was wondering if you could kind of weigh in on your thoughts in terms of you know is user acquisition easy or is the this really rare complex skill yeah so I mean it's it's a rare complex skill

so so to priety like that what I think is it's getting easier it's getting easier because of those like machine learning algorithms you mentioned and there's simply fewer levers that you can use to even to find like unique ways of buying or like spending money or user acquisition to give an example Google used to be really a fairly complicated channel right three years ago different networks like YouTube Google Display Network search all those were different and then you could find like different pockets of traffic you could like control the cost much better could do a lot of different things to become successful and when you spend a lot of money those specifics are very important so you know right now that's not really the case because you AC is this like you know easy to use solution where you're just like I'll put creative you say this is how much I can spend these are my

bids that's it but you know it's not all that you know like the reality is that you can do that only to a certain extent if you spend $10,000 a month sure you know you're gonna do well but if you try to spend more aggressively things get significantly significantly harder right and that's why it's it's it's you know very there's a handful of people who can do it successfully so if you want to spend 10 20 million dollars a month I mean their hope to dispense significantly more than that but even if you want spent five million dollars a month it gets really really hard to to control the cost right so if you want to spend a few million dollars in Facebook and you want to drive all the payers you probably rely on one of their popular machine learning algorithms and the results is gonna be you're gonna pay like tens or like hundreds of dollars for an installer for like a fine action and you know at some point you know you're like guilty be to like CPI formulas gonna break right so so

basically what's really hard is scaling scaling to like a dual level where you know only the good guys are so I heard this analogy long time ago about like poker - poker tournaments I'm not sure what it's true okay like where people say you know pokers are about luck and some of the else says yeah but why do you have the same twenty thirty people and the final tournament every year so I think you it's the same for you I it's not really all about like you know thrown it on its going to do everything on its own it's actually you know trying to kind of fight fight this trend of like automating everything and moving like everything from machine learning and you know there's maybe fewer levers but there's still like a lot of other options you can you can use to optimize your companies it's certainly on the creative side there's a lot I know that's some of the stuff that the network platform this time yes yeah exactly so so what's happening with Google again we started that Google is like you know you have all these different levers like like listing white listing the publisher is different

networks or different properties so that's not available anymore but you still have creative and depending on the type of creative like you know the you're gonna receive more of this or some other traffic and then you can just control creative or like improve creative to perform better in a certain segment of traffic it's the same with Facebook right now you can buy every weren't video Facebook you can buy like interstitial is it better traffic as well as Instagram and Facebook and you know if you know how different creatives perform in different kind of sub properties or like different segments then then you can improve the performance but to be able to do that you need to spend a lot of money in to be able because spending all the money allows you to receive a lot of data on this tool side when you kind of data then you can make it educated decisions and like this loop is not really it's not really available when you spend ten thousand dollars you're just like okay set it and forget it okay so last question Nebo you know think thanks for every response or for all of your answers so far but for a you

a team to be as competitive as possible what are some of the other concepts or skills that they can be or that they should be developing besides the two concepts that we talked about today to you know be as good as possible I think either there there are obvious things he needs to be like everybody needs to be data-driven itself fully people like on the team knows equal not to blow it so everybody can't do their own reports understand what's going on marketing creative is becoming this like you know one of the most important important segments that you should pay attention to yeah simply because it's it's such an important such an important tactic right now is there no other leave is available and and they're they're something they're you know they're more of a common sense but people don't really talk about them is you know if you want to be a successful user acquisition manager directly or whatever

a curiosity really helps I mean I play a lot of games probably more more and more than like most other people in the industry it's it's a pretty bold claim but I just play a lot of games i watch ads I tried to understand what other people what other companies are doing it's sometimes you can catch fraud and all these other interesting interesting you know events and industry just by doing that but on the other hand you just get inspired and trying to you know understand what is working for others and then you can copy those or just kind of cross pollinate some of your ideas with their ideas and create something successful so those are kind of two or three things that I would I would emphasize okay great well I'm one there you have it here with Nebo from network thank you very much for your time and yeah that's it thanks yeah thanks very much fun I'm a little I hope you don't mind all right

[Music] [Music]