User acquisition – Can robots do it better?

[Music] [Music] thanks everyone for sticking around so the panel is called user acquisition can robots do it better why doesn't everybody get up on stage and we can kind of introduce ourselves cool so when we just start with you all okay hi my name is Aang and I'm a senior managers in wargaming I'm based in Berlin I'm sorry in Helsinki at the moment Cheers Robert lagers I'm at next games I'm

doing user acquisitions there hey my name is Mika I lead the user acquisition team at use ition are we missing Camille oh yeah so we're supposed to start in a few minutes right well you know well we just get started and if he shows up we'll make him sit in the front row cool so I think a good question to start with because it's the only one I remember and I can't find the questions on my phone right now yeah but I think it's also an appropriate question to start with is what what do you think is so we're talking about like user acquisition automation that kind of means setting up like autumn just for maybe people that don't fully understand what that is or unfamiliar kind of setting up user acquisition systems to run by themselves right without human oversight or without humans having to do any sort of like

data entry or reporting or or targeting even so here yes do you want to grab a chair and then just stay there No cool yeah cool so we got everybody you want to introduce yourself oh hey I'm cam at future play games apologies for being a bit late I'm handling all things analytical including user acquisition and bi and add ups etc at future play cool so yeah anyway so automating the the sort of like the end to end process of doing user acquisition so acquiring users analyzing them changing bids what do you think is a good place to start right so I mean if you think about that task right of doing all that like from you know from from zero to getting a completely autonomous user a user

acquisition system that could be like pretty intimidating for someone that's you know that's just starting what do you think a good place to start is like what's a first you know kind of good initiative just to get them just to just to make some forward progress on automating your UA so like from our perspective at least I'd almost turn that on its head rather than picking a particular area to do or having a plan for attack for automating systems is just to hire the right person in maybe and having someone on the you a team who's technically able to build out these systems then you know in our case my day-to-day job I was doing things and then I realized hey perhaps a machine could do this better or more efficiently or eliminating the need for me to do this task and so I built it out so I'd almost argue that rather than for taking a particular area to attack getting someone on the team who can actually automate systems is the right approach would they start with though so I mean it really depends on what your needs are first we certainly started with automating a lot of the reporting and

then automating figuring out LTV and ROI per campaign and then these days I have a dashboard which are come in in the morning and per site ID or per campaign we it specifies whether we need to move particular site ideas up and down out of them or change bids and then even automates emailing account managers for example to tell them what they should do to change the campaigns or linking into API is for networks to change bids etc I think agree with cams that you need to be able to be a scalable data system first to automate your workflow because I think most of the time for the you a manager is doing the planning and optimizing and also like doing the reporting for the results and it can cyclize 60 to 70 percent of your daily time so that's something that you need to automate first so once you have like I mean because of the you a guy doesn't know how to code then you need to have like a bi team or actually building live data ops and then like

analyzing and model for them to actually work with so that's something you should start first with the bi team yeah I don't know what more to add these knowledgeable people on my right side it's is just get your data straight and from there you can take the next steps if you don't have the data I think we've come back to this data topic I think from a if we were to consider starting you a from scratch and picking up automation or machine learning maybe not the best approach from the get-go maybe start with just running user acquisition and figuring out what's the grunt work that you're running on a day to day basis and starting to automate this and also keeping an eye out for tools that could potentially ease the process of optimization on the fly that's all I got for this the only one else is giving

good pointers already yeah I think if if you're starting as like the first like from the basis just you can also start with some alerting if you're spending too much if you have that capabilities the technical capabilities of building that out the alerting when saying when you're spending too much it's a little that could be very useful I can add just from personal experience I think as a starting point if you can get your cost data into like a Google Doc and have that being done automatically and then and tie that to some kind of an LTV prediction which which is married to that the source of like some campaign you're doing pretty well like you're probably ahead of 75 percent of advertisers right so like if you can just if you can get those two datasets merge together and like a Google Doc it doesn't have to be like a really sophisticated proprietary dashboard that you interact with but even if you can

just get those two datasets married together like connected like joined in a Google Doc you're that's a good starting place I would I would aim to get to that because I mean once you're there and you've built that foundation you can make it as sophisticated as you want but even that would save you a ton of time cool so that's my answer that question so I think a kind of a good corollary to that question would be so like we talked about starting points where along the lifetime of the game or the app should a company start doing that stuff right so it if you think about what you all just said about here's the starting point here's what you should aspire to do as like a first step well at what point does that occur is in I think just to just to add a little bit context would that be like pre soft launch during soft launch global launch first title second title ten person company 100% company when does this process begin you answer

why I guess the question depends a lot what your needs are for the company and where the games are if they are doing a lot of user acquisition or not but I can speak from our example at least which is like we started doing it from day one and actually it's a good point Eric that sort of first thing to do is maybe get in the costs per campaign and then figure out what your LTV per campaign or per channel is and that's what we did pretty much on day one energy sheet which helped a great deal so yeah I'd say like everything you can do right from the start to save yourself time and improve the efficiency of what you're doing is a smart move here's my thing lights on from day one or is even like you can wait until the stop launch when you actually have some data because this I mean for the one who not running you wait and it's a bit hard to actually building something like a template so when you have some days that you can actually merge it together so at first life hours is the most annoying part it's like merging the cost and the and the revenue per network Biagio per pups

and per site ID so it's always I it you cannot do it at once and just building like at the revenue and the retention first and building LTV model on the even on the spreadsheet and then after that by merging the span in there and the robust yeah the order you can start the more you can iterate over your processes over your GE sheets whatever you have so the earlier you can start the better I think all all of these or the basis of automating anything is just having access to everything we're having having access to the right data and beginning to identify patterns that you can then automate on-the-fly so definitely from day one putting putting the groundwork in place to then later on being able to build on top of your learnings when is that then the time that's actual machine learning or automation comes into play then it's that's a bit trickier these obviously depend on the maturity of your

product and metrics if there's need to focus on automating bid adjustments if you're in soft launch and you're not hitting your targets then this is probably something you shouldn't be spending an immense amount of time on words yeah I think it's also like it'd be problematic to be trying to tune your you a spend in a soft launch because that's not really the point the point though the point of soft launch is to validate the game and the validate sort of like mark like like product market fit it's not necessarily to be optimizing campaigns and getting the cheap as possible I like to sort of like you know whatever the the cheapest possible installs at like some LTV target right it's to really actually be you know I think figuring out what the LTV targets are so I would think that even if you were somehow able to build some of this infrastructure in a soft launch you want necessarily want to use

it because you might be optimizing to the wrong thing you might be optimizing so like sort of like a local maximum instead of really understanding if the game works yeah so I think one one kind of important consideration here too is so if in building all this stuff what is your goal is it to actually increase the amount of money you spend like profitably so like it basically increase your capacity to do you a to do more UA or is it to optimize the current budget so is it is it to basically they've become so efficient that you actually unlock more budget and you're able to acquire you know more users that with more money or is it to acquire more users with the same amount of money just by making that process more efficient I mean I think it's just from my point of view it's firstly things to be optimized the current money that you spend first and then after that when you know that the money that you spend is actually you get the return and you can really is measurable then after that is you start

to open more budget further for the new campaigns and so on so like for us at least we don't have a concept of budgets really we try to avoid that as part of a company and we just spend when it makes sense to spend so if we are optimizing the campaign it means we'll be spending less on the ones we are at the moment or making more of a profit margin off them and it means we'll be able to enter new campaigns or expand the reach because we can do them more efficiently so it's for us it's a case of yeah optimizing what we're doing and being able to expand rather than dealing with a specific budget that we have a budget no budget so you could you spend ten ten million tomorrow okay we got two mil in the bank but if we wanted to spend ten mil and it made sense to do it and we're 100% sure of it like I'm sure we could come up with another eight mil somehow if anybody knows how that would be great but yeah like two million dollar budget two million budget to spend on one day but as long as we're doing it in a smart way if we spend two

mil we should get some of that back and be able to spend it again the next day but one now in 30-some days right okay 30 some days yeah sounds like you've a budget day well well regarding the the question so is it to improve or increase volumes or is it to optimize performance and and I see that it's both definitely it really depends on your product metrics what you can do if your metrics don't allow you to go to broader targeting or higher CPI's or or or so forth then you'll be looking looking to make the most out of the money you have which then leaves but more time to allocate that budgets into running new experiments so at next we're more along Kam's line so we we do optimize the things that we buy that we want to make sure that we're buying the right users

at the right time for the right price and that in later stages open up the budget or the the volume that we can spend on new user cool so this next question all all kind of split into two because I think it's more interesting that way so the first part is what areas you a technology do you see being fundamentally innovated on in the next few years and what areas would you like to see innovated on in the next few years so for me I think the attribution would be one of the area that could be innovative so at the moment is for the direct direct you a direct advertising attribution is working really well for tracking the direct download but in the future we will do more some sort of like online offline campaigns YouTube campaigns or even TV campaigns or that kind of thing that at the moment we also tackle some of the challenge who with the attribution partner to actually measure in the proper way and also we

also have some also in the which we attributions for the user who will actually download the app and also we in some and in so and we install the apps that also another things that we see that some of the attribution partner will also have some challenge on that that's something that I would love to see innovative in the future we kind of got the same answer to both questions and maybe it's wishful thinking there probably is but both the bit I'd like to see in our hope see will change going forwards is a lot of the top networks at the moment still deal with email and talking you know changing you're setting up new campaigns or changing bids through an Account Manager and yeah like I say we've managed to automate a lot of the emails that go to them and sort of try and use them as a part of machine and treat them like an API in a way but we could iterate a lot quicker and make it a lot more effective if we could you know we're the top networks use an API to set up new campaigns change bids change website IDs we target etc so maybe it's wishful thinking but that's

kind of what I'd like to see over the next year so just a side point whenever an Account Manager like takes you out to dinner or takes you out to drinks do you ever wonder like I wonder how many sense of like every install I got yesterday this represents cuz it was like a nice place and you didn't spend that much yesterday it might be like that was like 25% of might spend yesterday went to this dinner Thanks it's a good dinner good dinner thank you yeah actually there's regarding this there was actually recently launched a company called acquired Isle it's founded by former app Scotch founders that does exactly that with the goal of automating campaign reporting and optimization on one dashboard and their focus now initially is in mobile games it may be worthwhile checking out but not to pitch anyone else back to the questions are definitely here definitely working with smaller mobile ad networks

is currently very tedious in the sense that critical decisions are sent over email over spreadsheets or shared spreadsheets rather than any sort of technical integration and you essentially don't decide what happens on the end it's sent through an email thread and an additional area to this something that's sort of I guess sort of everyone's talking about it I don't know how many are actually proactively working in that space I know I know Roberts and next games are are interested in the in the sphere of a fraud prevention an area where as these tools become more sophisticated or that will definitely be a big deal in the near future I think what what I would wish in the pain point that I see right now is very much expressed by cam

because a lot of these optimizations that we do it if everything goes over email and we know all of us know how we implement it post install event into our attribution partners but it's a crazy amount of the networks that we're buying from still can't optimize on these post install events and I mean if I could wish for something from these ad networks just start optimize on these ones and we tell you do this look at this but they're still not able to automatically increase look at these returns or these post install event metrics that's what I would wish cool I think a good question to follow up on that one with is what what do you think Automation can't solve so like what pain point do you have that you think just probably will never be able to be automated away that you're gonna always kind of have to deal with as a human being so I guess another way of rephrase this is why are you worth

your paycheck and why can a robot not replace you I'll start with this I your people person exactly I mean it is it is the personal relationships because what what we've seen as well we've done you way for quite some time but actually the personal relationships with the ad networks the video networks with the social networks as well that do pay off you do get a better relationship you do get this contact that you couldn't get you couldn't be on top of their minds without you actually speaking to them so the fact is that you're a person that you have that social contact you know their face and they know yours that that's something that you can't replace so those drinks actually reduced so I was just about to say that you can't automate the dinner but but continuing

straight off that something that what creative at least yet you can have a robot automatically create new concepts for creative to run on these networks definitely the as more and more the optimization work becomes automated the focus of user acquisition will then be more in in building out the infrastructure to support machine learning based media by and creative so that's probably their missus paces that that can't be automated yet you I was gonna say the same thing actually like creative optimization I've seen one startup to come back at you on the startup front who saying they'll do creative generation they'll pull off random images from the net and come up with text and the creatives I've seen

them come up with I don't think would work at all so I think that is something that's gonna be you know hard to replace and the other side of it as well is you can you know build a machine but every company is different it has different needs you know for example we're more ad based we also take into account cross promotion network value bonuses from networks which we can't pull from an API etc into our LTV equation so you can sort of build a machine to automate things but you're always gonna need someone to understand that machine and tweak it and make sure it's running and sort of oil it so to speak so a caretaker maybe and I think for even for the optimization part because at some highlight people may think that the machine if you set up the rules if it hits a certain span it is not hitting this return on adspend and you just opt-out the campaign but it's actually it's an open market out there so even if you set up everybody set up the same rules for your campaign but when we're pitching through the marketplace is everything will be running like differently so even you still need to

have somebody who actually understand the number looking at that and seeing like making like a constant changes - because it's - based on your skill and your expertise instead of just letting my machine because otherwise everyone which is bidding if the machine would do your machine could do to the other company machine then everyone is just bidding higher higher that's what the machine could do so I think apropos of news today which was that uber is suing their agency fetch for 40 million or so they're seeking 40 million damages for fraud so fetch is like a mobile ad agency owned by Dentsu I'll ask what do you think is the future of mobile adding disease so this one was ok my bad this wasn't on the approved questions list so you can if you want to answer please feel free if you don't I I can answer on this so I think that for the mobile agency it could be a good

addition to the company hotel other smaller developer other tiser who doesn't have the in-house capabilities you actually running and building the UI or the BI system to actually running it and even for the big companies could be something that they can outsourcing or certain projects that they think that it's not worth the time for the UAE internal team and then there's ourselves and father or other agency to do it but if the future is not very bright in my opinion because all the big company is actually the one who spend the most so they're already building like a sophisticated uit into actually doing it so the need for the big clients who actually need an agency would not be that huge might not be the best person to answer this because we've looked into using agencies a few times and never saw much value in it over only a pitch so hard I've seen but I don't know like so from that perspective a bit amis like yeah yeah like they'll all go away we'll

all get more capable but at the same time like Mobile's spent spend on mobile advertising increasing more and more brands coming into the market and they're not familiar with the space and needs somebody to go in and you know talk to these networks and place bids for them so I see them sticking around for a while and I'd like to think they'd improve some processes and get more automation but like I haven't seen much trend in that direction now I'm gonna get some uber pitches yeah yeah I think what what also PNG cm all revealed that the de Mexico here this year was the transparency piece that if if agencies can't deliver this transparency they will kind of not have any clients anymore and I think that's something that will need to change also on the well if we consider the media or advertising industry past what

what 10 20 years time going from a brand advertiser agency relationship where a brand has out sourced essentially the the day-to-day operations of running marketing but with the ability to access data on your marketing performance which is something that we're all very familiar with this the infrastructure to measuring that performance is built in-house and the more of that technology as more of that technology's built in-house that leaves less and less justification for an agency to spend on on a certain media and what this so if we that in mind if we consider the different types of advertisers currently and how or different types of agencies currently and who are who are specifically at a competitive advantage or those that have a technological piece

where they're offering also souls a problem that a a company might not be able to or might not have the resources to solve in-house so rather than having an advertiser go to an agency and say hey I want to run this campaign go it'll be hey I need this technology can you help me implement this in-house and there will always be advertisers who need more help in setting or building out the infrastructure so in the sense of an agents or the future of agencies will definitely change not that it's ever been stable but that it will need to be more focused on technology and probably also more focused on consulting or helping out building tools for the client cool so why don't we do we've got three and a half minutes left we could probably take one or two questions if there are any does anybody have any questions if not we can do one last question from the panel

there's one in the back okay this is Pepe as good as why did you deserve your paycheck that's my favorite question either he'll be popular from Rovio so I have a three-part question about retargeting are you guys doing retargeting and then are you guys doing retargeting continuously and then the third part is that do you have any specific things to take into account to automate those operations compared to more regular UA campaigns I could say the answer will 3-1 which is no we're not doing retargeting I haven't heard great things like there are some success stories but it from what I've heard it's not something that we should invest into in this point what was your second question second question are you doing retargeting campaigns continuously and if yes then anything like to take into

account when automating that stuff compared to normal campaigns because this was basically three yes and no questions so this was yes yes and yes okay now the last one was what are you doing what are you doing retargeting are you automating retargeting and then the last one is what are you doing to what what we're doing is basically keeping the audience's the retargeted audience is up-to-date that that is a key if you're not having that up-to-date you're basically exhausting that audience over and over again if they haven't converted they won't convert so we we do run a retargeting for one of our legacy game and the problem for that game is that even is heavily huge amount of download is so very hard to have enough cohorts

or the amount user ID to actually do the root proper retargeting because if it's too small and we didn't and then with the result we see that it didn't really bring like much of the live into the actual campaign that we're seeing that okay we see some people return we see some like good return on investment but the cost of running it is just so much and then the time for the UI manager who actually investing to running a proper UI to acquire a new user it's even like less than spending more time on their doing the retargeting campaign so for me I think it's not very worth it unless you have like very big user base that you want to really do retargeting properly yeah I can add something here so I don't think we're targeting is very big in games I mean I think Robert it's pretty clear where there would be a use case at next because the show is cyclical and it comes on it goes off and so there'd be you could you could perceive a value in retargeting at some point a user who had laps because the show and off and it's probably easier to get them to convert when the shows on but retargeting actually is pretty prevalent in

verticals outside of gaming so social casino they do a lot of retargeting travel they do a lot of retargeting mobile commerce they do a lot of retargeting I would love to walk you through some of the projects I took on in that space with clients but we only have three seconds left [Music] [Applause] [Music] [Music]