Mobile Marketing for Card Games | Growing WildCard Games Ultimate Cribbage (UA Coffee)

you everybody welcome to another edition of you a coffee talk today we have a special edition which is kind of like ask the experts and by experts I mean John Lao and Nebo radevich but joining us today is actually Josh from wild card and Josh maybe you could tell us a little bit about your company in your situation yeah me first though thank you guys so much for the same time talk to me today I love the way coffee talk segment it provides a lot of insightful content that frankly I haven't gotten anywhere else and just watching it my number one of my first big insights was you guys trashing d7r poo in d7 row s that was like a big breakthrough in my head so yeah it's like actually changing the way that I've been approaching this that's really cool so I'm hoping it's going to

become really valuable today so in terms of like what wildcard is the business we are all about reimagining kind of classic card games removal our audience is older and heavily concentrated in North America and there is this massive nostalgia factor so as families grow these games that they played together as a kid whether it's cribbage gin rummy spades things like that they become harder to play together because the kids are now in different states or different countries or even just at school so they're they're looking to play those games and we're really competing in the card game vertical but also against board in word our current flagship is ultimate cribbage and what's cool about that taking a step back is in that in the multiplayer card kind of sub vertical within card there's melding games like gin rummy and then there's trick-taking games like spades and cribbage is amazing because

it's neither of those so it requires a lot of thinking and it's just Leslie and fun so the reason for this conversation is because we've had some level of success with user acquisition and predictably there's a lot of growing things we started off just running a couple hundred dollars a day on Facebook because that's super easy it's really easy for us because we make classic games so we can do an interest look-alike where it's like people who like cribbage and that's super effective audience size is like 50k so that gets used up pretty quick so we had to scale and scale and scale and add new platforms and overtime it you really need to be a you a expert so that's kind of the context behind this conversation great so in terms of like specific questions can you kind of help help us understand to the extent that you're able to share some of the data like what does your kind of if you can describe

what your marketing campaigns kind of looks like today to the extent that you can share and you know we can dive into like any feedback from Nebo and and John about specific issues yeah that makes more sense so we're still primarily running on Facebook we also have a little bit and growing really really fast on Google and then just some low-hanging fruit on the SDK networks like Longueuil unity adds map 11 and stuff like that really the focus though is right now it's storing Facebook so the way our campaigns are structured is we don't really use any kind of big caps we're mostly using just daily spend thresholds really heavily targeting 1% look-alikes 1% BB look-alikes and targeting US Canada and a little bit of the UK but that gets used up pretty quick for us too surprisingly so even though we're about

85 to 90 percent ad revenue AAO and vo is working fantastic for us even just straight-up purchased be a purchase AO and nvo it works really well so the way we're kind of structuring guys we're kind of creating new lists every week and every month and trying to find new things that work on the audience side then on the creative side we're working with a fantastic agency consumer acquisition and really in terms of the top performers to me it sticks out then and dollar are the two that are most interesting that we can consider it the most heavily and CA produces about ten to twelve videos for us a month and they have a really strong testing process where we start in India with a small budget and just try to screen out the creatives and we promote it to Canada and then if it works we promote it to the US

that's our high-level strategy there are definitely some oh go ahead oh and how you how are you determining your your kind of daily spend know your budget is that is that off of a DX Raz target or at yeah yeah so that's one of the questions I actually have for you guys that's gonna be fun conversation but the way we're doing it right now is we know that we want to be breaking even between D 90 and D 120 on our ad spend night total cap and just ad spend and what we've done is we are responsible internally a wild card for synthesizing a d7 target by OS and media source for that and then CIA is responsible for creating the d1 d3 kind of targets to get to that we've seen that there are even within a media source there can be some changes for us so like vo matures differently than ma I so they're responsible for getting to that d7 but we're respond at the d7 is wrong that's

on us okay so just to be clear with consumer acquisition you're not just choosing them for creatives or else they're also yeah they're running the UA oh yeah so we're that we're driving at each portion of that as well okay then even John what do you think channels approach any thoughts initial thoughts yeah yeah I have a few things that I wanted to share initially it's like first of all I want to cover like card games in general what's good about card games there is actually organic demand for them so you know there are people who are potentially searching for cribbage and that's not really the case with a lot of like especially made to court to court titles unless they're kind of big IP or big brands so I think what for games that have like a strong organic demand it does make sense to like pay special attention to yeah so it does make sense to potentially try preloads and just like you know try super kind of low cost low quality channels that could

work for games that are well-known so think about like that risk that Network publishes or or I don't know like you mentioned Jim Ramius Dade's and solitaire etcetera yeah I think it's really smart to focus these like old car games because of that because that ultimately lowers your CAC and you can you can be more aggressive on the CPI front I mean there are studios like triple Studios whose almost solely focused on these kind of like evergreen games that that monetize the monetize well enough and you know compiled with kind of the organic installs you can you can turn them into very successful businesses and the one thing that you mentioned one more thing is that you know you mentioned just like look-alikes and and you know like refreshing those your audiences on a weekly basis I think for games that do have organic demand it does make sense to work with interest and the keyword targeting just because there is an actual interest car games that you can target so these are kind of like top of

the mind things for me and I wanted to like us to maybe cover like the holic vo and AO and Mei impact games that that have good monetize with ads because I think that's interesting and in line with and it's in line with lady Sandra and why it sucks so you know maybe I could ask you a question real quick for these other networks how should Josh be thinking about the amount of budget that he should be spending against those in order for him to to definitively determine whether it makes sense or it doesn't make sense yeah it's uh it's a little hard to say because when you like a on VR for lazy people they do work really well but it's like you know it's not really hard core you eight looks not like really going and like and I like trolling for whales it's more like just Hale bringing payers and that's that's it so the thing when your game is at monetized it's like the LTV curve of a oh and video players is

significantly different than what's the case for M AI players and rewarded video network players and then you need to like spend enough money to understand what that LTV curve looks like and and then based on that you should like invest as much money as you can as long as it hits your day nighty day back right so so what's likely happening is that these seminars is significantly higher in a young vo right and then by like day 30 and I and rewarded video takeover because the cost is the cost plays a very important role for ad Montes games and you know if you're like if your CPI is like a dollar on the reward video in my eye and it's like four dollars on a do you kind of have to be very kind of cautious about the CPI because after 30 days it's likely that monetization takes over and it grows pretty slowly it grows steadily but it takes like 300 days to pay back like a dollar right so that's what's happening and paying back like a dollar or three dollars after 30

days makes a huge difference so so that's kind of like what's happening here what's likely happening here that's a great insight thank you and that's pretty good pretty good guesses on your part too and Josh how long has this game been operating and since you've been operating for for a very long time so we first launched it all the way back in 2016 it was the first Unity game he ever did and actually always heard scaling in the last six to twelve months yeah because some of the backend technology that we built that allowed us to really run AV tests kind of non-stop and be measuring our our qu curve for those AP tests at an impression level granularity what am i doing that I'm sorry to hitting these just massive our food growth curves so really it's been the last six months we've been spending on you weigh heavily for it right but at

least I would say you have a very long historical record right in terms of data you can see where you can you'll you'll have a lot of data against a long term art boot curve and so when you have been coming up with your payback windows today how have you been determining is is it based upon where you're looking at where it flattens out and you're trying to build in a margin for yourself of you know a certain percentage or how have you been thinking about that that's a fantastic question and it's more the most frustrating things actually about growing our food really rapidly is so we put a lot of effort and to measuring out our curve by country and mediasource with a lot of variables for about a year out to D 180 so that we can measure that and I started to really set the buy against that aggressively and it was really helpful and we had these really different targets for like iOS us Google

versus Canada Android Facebook and then we started growing and we talked about in these ad tests and I remember the test it was it was be tested dramatically cranking up how many initials we showed and it was bonkers it what it did was it boosted our my day one ARPU by over a hundred percent and had boosted our D 30 when we printed it out by like forty percent and then just couple months ago we've measured it out to D 90 and it's like twenty percent and then that effect was that the whole time that we were buying on that are historical data we didn't realize that oh my gosh our our cou curve is fluctuating like crazy because frankly we're resolving a test almost every two weeks that has a net five to ten percent rg3 and actually it introduces a lot of uncertainty to the modeling process so you know one of our

questions is in real time no say we're going on our pretest and we've measured a 30 cent measuring boost to d3 is that you know a 10% boost with the whole curve or is it just a 30% boost of the whole curve or is it a 30 cent boost to do 30 that degrades to penny or even negative PI D 9 C and maybe that's worth it because it promotes the payback period for maybe it doesn't but it means that it's really hard for us to use historical data when we're small and growing rapidly if you've been at scale and mature for six months or a year you're our poop data is fantastically valuable and for us we can do things like generate lists but using it to actually create fantastic targets has been more difficult for us so what we've done instead is take a blended approach where we are trying to look at those old models but

also be constantly projecting forward our most recent d30 data out to our target breakeven which is d 120 and try to understand where we think the curve is going as opposed to where it is definitely easier said than done yeah you kind of lost me a bit okay I'm not as smart as Nebo and Josh so you're seeing a lot of variability in terms of your RP curves is that based upon like the same targeting same gos based upon different changing your site so changing the game so an example would be like if you go from showing interstitials every eight minutes to every two minutes I think even if it ends up being net LTV like that theoretical number positive yeah he's gonna change the shape of your curve because you're gonna make more money up front and less long term yeah though when you're resolving tests like like like nuts your curve doesn't stay accurate long enough for you to really buying on it you know if you're netting

a three five percent ARPU gain a month maybe maybe not our targets for monthly growth or fifteen twenty percent a month still one one quick comment I had regarding your earlier question is that looking at some of your user data your demographic overlaps you know heavily with Melanie card games but like the match-three puzzle games and a lot of slots as well right so there that audience traditionally can be relatively expensive and when you're running a vo and a EO and to Naples point the front cost is incredibly expensive because typically it's more meant for games where there are large purchasers you know whales that can come in and really float the entire audience eyes now my concern with those is that they're and they're never going to be consistent

right so you could be running vo it's working great for you and then for a quarter in the next quarter Ayoo is doing better despite the fact that vo is supposed to be the premium with the Edmonds ties game and potentially even a lower our Pugh kind of like payer game I'm always worried about costs I'm always worried about that CPI and and you can look at it two ways like do you if you're your base basing your performance on paint only it's going to be very very very difficult for you to achieve that payback if you're if a majority of your campaigns are in the vo or a bucket because the CPI is is so high but then you also in if you're the other way where you're okay with a blended outlook to to look at your recoup it's also very dangerous because you could all of a sudden like have a heavy emphasis of oh

I need X number of organic in order to achieve a break-even within that time so I do agree a lot with Nebo strategy of looking closer at how you could utilize interest targeting to help lower down your initial cost that's one ASO strategies is definitely a must given that organic traffic I agree with that because that's the commonality of looking for card games but when you're testing outside of that where where would you say have been working well for you outside of the Facebook what what has worked really really well in terms of both scalability and performance outside of Facebook that's a great question so really what's worked great for us scalability is actually right what you guys are saying it's those keywords it's that low cost Google text and Apple

search those are both being fantastic we've been able to get good performance and other places but nowhere near at that scale and it one question for you so you said something about a blended outlook can you explain John what that means yeah so some companies are okay taking organic revenue and blended that into your overall growth strategy in order for you to grow a little bit more aggressively but it is it can be a slippery slope when you become too heavily dependent on that organic growth in order to achieve whatever like LTV over cap ratio that you need to maintain and and I'm not big fan of it because you it makes it seem like you're you a or your overall growth strategy is doing better than it is but I think even if you're gonna go down

that path is important for you to really focus on paid only performance and see how that is actually evolving over time John we lost your video by the way yeah went to the kitchen to get coffee okay okay I got my coffee as well so it's a prop for you a coffee so so one question for you two is the one thing that we've seen that was really really interesting is you can take a step back first so we use mop up remediation and they have an impression level Avenue reporting tool it's fantastic and then in our mediation kind of waterfall and we run programmatic - but in our mediation Waterfall the see the line-item floors for dangerous digital might range from $2 ups like 120 and one thing that we saw when we switched from average CPM reporting to

actually report in the impression level and seeing when there's like 120 CPM AdView versus a $2 the veal in the EEO campaigns exploded especially on Facebook because what Facebook is saying with those AE onvo bids is hey we know that user and we know they're worth a lot so when they're in our game and we're requesting fan for an impression they're willing to pay up be high for it so I'm kind of curious to know if you're aware of that affecting if that still you still feel the a on vo is kind of scary for ya merrily ad based games yeah actually wanted to piggyback on like John's point and talk about it I mean like that your like previous question probably from ten minutes ago I mean I would model out at revenue LTV curve and like IPL ticker differently because what what happened is when you when you starting a you and vo you're probably you're like ATS you like ad CPM skyrocketed because of a yawn vo so what

you really need to do is basically you need to have user level a drive and you to understand what's the value of each user and you know what's the overall impact on on unlike returns really not necessarily not necessarily just campaigns because a onvo can help you monetize the existing users better and by like improving the ad CPMs however I'd like that strategy is only kind of it's a little tricky because you know they tend to cherry-pick so they don't really pay $120 for like ten percent of your impressions but probably for like one to two percent so can you kind of need to and that's why I need different LT beakers to understand what's going on and and then like based on those two you can make decision on work and less money so maybe like a o and vo bring you like $120 impressions but the overall impact or the incremental revenue you get from those is not like high enough to kind of set the cost so that's something you

need to be aware of but definitely having user level ad revenue is crucial and having like a header bidding partner like like mo pub or applauding or iron source is a is important because then you'll be able to understand that better and the last thing one thing you might like learn over time is that local o CPI low-cost campaign am I still do better because the average CPM is K like higher than what it was before and the risk profile is much better because like your CPI is lower and then you can you can pay the money back sooner so the reason why why John mentioned would he mention about like that the high cost of a ovo is that you know if you end up feeling like very comfortable because your ad CPMs look good and then suddenly you're like CPI moves from $4 to $6 and you're pretty much because you're like long term retention is not that good and the last two dollars take like

2000 days to pay back so no that's that's exactly right yeah it's a set ban the CEO of networks like he has this saying like you know whenever someone's if someone gave me a dollar every time someone said like hey we started spending more in video and the cost is going up he would probably be a millionaire because like you know video can be very dangerous if you don't manage the cost properly no that's exactly right especially when you have a admins a monetized game like you're you're LT view like it is probably a long time significant longer than let's say if you if you're looking at like someone who does make or for example right so you really want that person to stay in your game as long as possible to generate those like to the 15 cents arc tau on the daily basis and unfortunately that's that can also fluctuate right like depending on your country depending on your user segments or where you're

putting your ads your your ARP Dow on on ad revenue is if it can be inconsistent you have competition you have seasonality you have a lot of different things that can impact that so all that will bacon - I'm assuming why you're seeing a lot of fluctuation and and if you throw a wrench like vo costs skyrocketing or dropping like on top of what is already very relatively volatile I think is gonna be very very difficult to manage I'm not saying that ad revenue or at based games are always very very volatile but I think there's a lot of things that can impact like I really like what you said earlier about changing the an interstitial ad Instituto every 2 minutes to 8 minutes and how are from 8 minutes to every 2 minutes how that impacts not only you our front revenue but potentially the lifetime of that user so I'm actually very very impressed that you're you guys were thinking about these things awesome

well thanks and actually I feel like I'm learning a lot right now that my big takeaway if I'm gonna get in this correctly is there is definitely a hidden risk it's the AO in the vo campaigns and we should be looking at that how they age out compared to ma I very carefully that is frankly something we're not doing right now so that's honestly you've given me might asking for today yeah yes basically the the graph I shared in that b7 the Ross article is explaining that I mean I can share it real quick if you guys want to just so you can see what it looks like but it's you know with a on vo because they know players much better like the LTV curve looks significantly different and and then the early data looks great well with you know ma I and potentially lower cost CPI channels you might be able to achieve better results just because the cost is lower and then you

know over like 60 to 90 days that things will look significantly different let me just load up the the graphs I can brag about it guess why Lonnie Bo's pulling that up I think that really big emphasis we have is really diversify when when you're scaling up you know to the level you're scaling up today and the reliance on a EO and vo to drive your business becomes a very very risky proposition and I have no good answer as like what are the good channels for you to diversify into you have to test in this very very game specific but I can give you an example right at play studios at one point are spend on our reliance on a EO this before vo was roughly about six seventy percent of our budget plus right and at one point we looked at the entire portfolio and we're like oh wow we can't if if anything were to happen and what things were doing really really well

but if anything bad were to happen it could completely tank our entire more marketing portfolio and from that point we made a mark effort to reduce our reliance on not only the the aeo campaigns but on Facebook overall so that we can leverage other channels because we know fundamentally you can get that user in in the ecosystem from a different medium it is much more difficult to me both point it takes a lot more work but they are out there and you can find them and yes initially when you're growing you can rely super heavily on Facebook because it is the requirement for entry is relatively easy right but as you mature not only as a company but as a growth marketer you have to move out of that space if you want to go kind of move forward into the next level it's look at other channels where it will require a lot more manual work and

thinking around how you want to strategize your campaign optimization and when we're talking here about a env o we're mostly talking about a vo for purchase like you could like basically your focus should be on controlling the CPI and making sure it doesn't like skyrocket so you could easily do like a do for an ad view for example well for like a certain engagement the event that could work really well I'm gonna share my screen real quick just so you can you can see the graphs like basically it's these two graphs so with vo and AAA what happens is like your early revenue looks really good and then things flooding out I mean it could be that also your game is not optimized for IP and then when you're like like solely relying on IP you know first 10-15 days look really good but then when there is like less content and that would people just stop spending what happens is this Ltd curve flattens but like early 7-day like first seven days if first fourteen days look pretty good and like the LTV is growing pretty aggressively but at the same time you're kind of paying a pretty high cost for that well we've kind of like lower

CPI strategy like for traffic that's solely add monetize the LTV growth is much slower but because of the lower cost as well you might actually pay the money back sooner then what's the case with like vo nao because the cost is lower and then the CPI is fully paid back by ads so that's something just do like have in mind and that's why I think it's important to have LTV curve and IP LTV curves separate to understand what's the effect of each one of those on on payback and then buy campaign by like optimization model like mobile app install a lvl radio for view it or a reward video etc etc everybody video network so I'm talking too much I'm gonna stop shaming Josh is fascinating sorry just kind of thinking out loud but because you have a lot of historical data and if your operators are jumping around one thing you might want to consider and you know again you know Johnny will check me if I'm wrong on this is like maybe what you should be also doing is checking your scale to

spend against your row as target so like for example if you were to evaluate what your d7 row as against the your daily spend is sometimes what we see is we'll see like a like a like a knee bend at a certain point of spend sometimes for some games if you know if you're unsure about spend you might want to just you know sit but that's a good I hadn't even considered that okay yeah awesome maybe in terms of like more at the executional level because I do think we've got you know actual unlike me we've got practitioners actually executing stuff John and Evo do you guys have any thoughts in terms of like the you know like the ad format mix or creative types or things like that or maybe Josh you can tell us how you're approaching that and then you can get some actual practitioner level advice yeah that'd be

awesome so one of the big questions that I have is creative type statics are just crushing it for us right now they I think we're about 75 percent static there was a tennis band but we're investing almost all of our effort into video production under the hypothesis that when we find a video winner it will outperform the best static we've invested a couple months in that strategy now and we're getting close but you don't really pass the performance yet so I'm kind of curious to understand from from you all especially on Facebook is that a valid path or is there a point when you say like this screw it we're leaning on statics even though it's not what's supposed to work the best Wow how come you think and and and this is not critical in something this is very curious so how come you think that video will would outperform static that's a

good question frankly I don't think I have a great answer for that the what we've seen over long periods of time is the videos tend to go stay a little bit slower I don't have any solid evidence the videos are gonna outperform universally I guess that was just my assumption to be honest so the reason why I asked that is because I'm not I don't know and this could be very game specific but I'm not convinced that like one format will drive better ro ass for example than another format necessarily I'll bring you a better user than another user what I think whoa change is the lifetime of the creative so a good video might last longer to your point then then statics but at the same time you could probably knock out statics significantly faster

than videos so I think if statics are working for you and they're working really really well I wouldn't worry too much about looking for that one video winner that's going to somehow be a game changer but more focus on diversify in your creative mix so that you can elongate the lifetime of your overall creatives right so kneeble wrote it incredible medium article about maintaining healthy I PMS and and stuff like that so I think if it were me my focus would be about how can I maintain a higher average ipm of my overall creative mix rather than like one or two creative that's gonna be one because like if you're chasing hits it's it's tough to know what's gonna be a hit what's not and and then I I've had experience where things were a hit actually consumer acquisition was the one who made the video that made the hit

that did really really well for us at play studios and when we first looked at it we thought this is not going to work at all it became a hit for a year our best performing creative for a year right so so my my point is you just don't know so it's it's instead of focusing on let me just crank out a lot of the videos and and and hope one of them is a winner focus on just whatever your your your your your average IPM is across your entire portfolio mix making sure that you're maintaining that more consistently I think that consistency is more important than one or two hits that's going to somehow change the game for like a week and then everything kind of comes back down all right that's and it's awesome feedback because you totally changed my perspective on it Thanks yeah I mean my stance of this is that I think statics will only take you so far I think there are some limit because like most of the big advertisers

are not really running aesthetics or not at scale and because you're spending now like you're not spending like millions of dollars a month you can do that but once you start spending more money you kind of have to fire from all weapons and then video needs to be part of your your creative mix and you'll have to invest more in that particular unit I think good thing about statics at this scale is that they're easy to kind of produce and like it's pretty quick but you know over time like as you start spending more you'll have to create video and like video ads and multiple dimensions of video ads to test like the reward video specific sizes you know newsfeed specific size etc etc so I think you're now kind of because of the scale you can afford to just focus on statics and things that worked well also one thing that's interesting we didn't talk much about it but I did read the presentation you share it like the audience that's playing these games is slightly older and I think

because of that they're kind of more responsive to ads than was the case and when I say older it's like 45 to 50 plus and they're more responsive to ads ads in general then you know like 15 to 25 obvious that done doesn't really look at any any content anymore just girls impulsively so so I think that's that's one one reason why statics are working it's because like the audience that you're going after is slightly more responsive to statics than what's the case with potential younger players and maybe clean yeah I was gonna add like from a video perspective one thing that has been helpful for for me in the past and it's something even today and I'm so asking my creative team to do is focus more on potentially creating what I call micro videos all right so so view and and I actually stole this idea from from Nebo in his article a while back and so what I do now is instead of like I

telling my creative or motion graphic artists to say hey give me a 15-second video we actually shoot or create like multiple three to five second videos and then from that I create like 15 different iterations of it and of course they have their their different concepts they focus on different things and and and then they focus on different either focus on the product feature some of them might focus on like the game itself why it's fun you might focus on that there's a good play you know so from that aspect you could actually do a very little like actual video production on onset but have a lot more value in what you create that can help you create a lot of different combinations to run forward these are the amount of the time interesting so when you make those it sounds like those are those are unique creatives you're making they're not just like cut up longer videos they're they're brand new content that's can you

clarify what do you mean by kind of kind of both their new content but the same concepts yeah yeah so it's so first I'll give you an example like right now I work for or e-commerce grocery business right so one thing that we we're doing is we might have some shots on like a certain merchandise or certain product right and I couldn't have one video that is really just focus on that single product for 15 seconds but then I can because there are a bunch of four to five second shots of multiple different products I can then mix and match and create a lot of different iterations whether it is I want to talk I want a 15 second video that focuses on dessert or I want a video that focus on both entree and dessert or a bunch of different videos that focus on snacks and dessert right so that combination allows me gives me the ability to create a lot of different

video not this concept the concept could be the focus of the what merchandise category we're targeting but at the same time it gives my creative team a lot less pressure of consi having to come up with a fresh video of fresh concept and that way when we're looking at our creative Sprint's it's a lot easier for them to knock out assets for me on a monthly or weekly basis so that I can then continue to refresh them on my campaigns Josh do you have any sort of in terms of like outstanding questions or have we if we address some of the and when I say we I need more Jana Nemo but did we address some of the bigger questions is there's more should we go deeper into a specific area to try and address something or if there are any other outstanding issues should we tackle those yeah there are some other questions I'd love to cover I think we absolutely crushed the creative type question I'm kind of quiet so I'm just thinking all these different things I

could be doing now it's hard we got to start like writing up a plan but can it be already sort of touched on this but one big question is as you scale through that let's say hundred quarter million monthly spend range what are the strategies that tend to stop working or things that need to start working so some of the things I've already heard from you guys is like statics tend to stop working and stop allowing you to scale and the aeo and vio performance may fall off a cliff or just become dangerous at that scale if it's a high percentage of your spend are there any other things that stoppers start working yeah I think we scale like the number one things to be very cautious about the cost I mean you know a you and vo might continue to work just like your are I will probably go down because of the costs so make sure that you're always you know checking the the cost and making sure that it's not growing like exponentially

you know the good thing about scaling is that you'll have more information so like you understand better which placements are working for you what it's like audience network or newsfeed or or messenger probably not you know like you just have more data that you'll that will help you make kind of more better decisions and and potentially try can a more granular targeting so that's something that's a good thing of scaling when you're scaling especially because your game is kind of very it's it's not like a global phenomenon it it seems like it's known in like a few Gio's you should identify those years and potentially group things together to kind of maintain the cost group countries together to maintain the cost and make it easier for Facebook to learn I think what's what's what's happening a lot of cases when companies scale their campaigns they just create a ton of campaigns that end up stuck in the learning phase and then as a result of that your are goes down and the cost goes up so it's potentially better to

kind of group countries together and make it easier for Facebook to learn and hit your goals sooner and yeah like the last thing is that your creative production you need to scale your creative production as well so you need to make sure that creative fatigue is not like a big problem and that they have like could help the influx of new creative every every week or month at the minimum it might even be worth while to just group countries outside the US at least outside these group countries outside US except for like Canada for example in to worldwide campaigns just so that you have that kind of minimum criteria of scale to go through the learning process as you're scaling up your budget one thing that I've actually run into the situation before but stand firm on your payback stamp firm on because it's it gets easy for you to tell yourselves like well I'm making this much revenue maybe we shift from like a hundred days

120 days maybe might shift from that 120 days to beyond Sam firm on that because you have to measure apples to apples when you're spending either 100 K to 200 K or maybe 500 K a month so that's one thing I did take a look at one of your questions about user lists and of course like idea for a deprecation I don't actually I shouldn't say idea of IGF a deprecation it's more user opt-out it's more the more a good statement but I would be interesting to see how that impacts your programmatic campaigns because a lot of programmatic algorithm is based on optimizing on the device ID levels or that ID level so I'm not sure how that would impact granted please understand that no one has a good answer of how this is going to impact the work I think like me when I are both kind of sitting and waiting to see how things

will be impacted in the next couple of months but we know several things we know that Facebook will likely be less impacted because Facebook has a lot of other means of identifying users across platforms across their properties and so as long as you're collecting things like email or user information you could continue to feed that into Facebook to manage your Facebook campaigns programmatic they don't really have that and I don't know the match rate of emails and programmatic I I don't doubt I don't think it's gonna be as high as Facebook for example so it'd be interesting to see how those campaign change over time yeah I also think they might have a negative impact on analyzation because Facebook won't be able to cherry-pick impressions anymore assuming that they don't know who the users are that's something I'm researching right now I don't still fully understand the impact but just be aware of that a potential good thing the CPI's might go down as a result of that because CPM will go down but you know it's good to have gonna help the ad

monetization as well just to make sure that your knows not super dependent on on Facebook and and Google was the idea phase gone yeah I've got a question actually just related to being stuck in learning phase so if there are a lot of product level changes you know on in the product which seems like that's the case for for for this game does that I mean certainly on the campaign side if you make changes that can you know get you back into into into learning phase but on the product level if you make big you know potentially like thirty percent change in our pew curve or things like that does that does that get you back into learning phase as well changes on the product side I mean they so not not really I mean did because what what really gets you out of the or get you back into learning phases if you change something significantly on the campaign side like change the budget like for

more than like thirty percent and for like cpa whatever you're gonna target target cost is it could help campaigns learn faster if you for example front loaded more purchases like you know if most of the purchase happened in the first day or in the first seven day days that will that would have helped facebook and google learn faster so it's something we tested and you know it it worked in some cases it didn't work in some case and it could potentially hurt you down the road if you're too aggressive with monetization or at least my recommendation just a/b test and see what's going to happen because no one really has the right answer to that and facebook keeps changing things all the time i would say the opposite interestingly can be true though so i have had examples where especially running pricing tests on really fit on early IPS and stuff like that if you're skewing really hard towards a EO and vo it can actually skew your results

towards the more expensive and things like that win when perhaps when you even out your ad spend it actually wouldn't have they take another vote for multi-armed bandit stuff I suppose but go ahead sorry I I wonder if there's a difference in terms of like how Facebook targets Al and vo audiences right so there's a fundamental difference because then if you think about it ecommerce is probably buying a EO + vo along with gaming and with e-commerce you're looking for buyers who pay within the first three days and they pay a lot and then so the purchases are if I'm a gaming kind of like perspective you're essentially buying a bunch of wheels that's consistent buying on a weekly or like by monthly basis whereas if you flip it on the other side and you're looking for people who let's say the purchases is heavier on the tail end of

that seven-day look-back window you might really be looking for a different user so you might not be competing directly even though it's Silvio and AEO and so competing within the purchaser pool but the purchasing behavior within that pool you might be able to carve out a different audience that you can target of course you can't manually and you can't quantitatively know that but I am just curious like is there any difference between even within the payer pool of the vo or al audience there is going to be different kind of opportunities for you to play around with where you go hard on your pricing or way you want to really get that user to monetize because a lot of people want to front-load purchases and if you went the other way around well being a Facebook algorithm look for fun of money a different user for you yeah that's such an interesting question it's one of the things that as we're

looking to integrate game soon sometime soon which is unity ads kind of map feature it's something that we're kind of thinking about what we have seen and so far we have we don't have a great kind of definitive answer yet but it definitely seems like the aeo Amiens okay okay cheese tends to have a lot of people that make one purchase and it's just like remove ads where it's always like three bucks it's great but it's whatever whereas vo are more likely to actually buy the currency's which which bodes well obviously so that's one thing that we're starting to look into a little bit but frankly we aren't very sophisticated there yet okay well I think in the interest of time is there any last question or comments from you John Andy

Bell I think for me is that just looking at your presentation and your write-up in the way you're thinking about this you're already incredibly sophisticated in terms of how you're approaching this so there's that I know a lot of companies might in my peer group that aren't even thinking about some of the things that you're thinking about so I'm incredibly impressed by how you're even approaching this and I think this is really going to be a springboard to how you can think about growth for a very much longer period of time so we're talking about the next five years and keep at it and I think from a creative perspective niebo and I are in love with creative because we really feel like that's the fundamentally the biggest lever you can pull so play around with that and don't fall in love with your ideas I think that's for the biggest advice is it with the creatives never fall in love with your own ideas of what you think or what your own hypothesis really just tests different things even things that you

don't think will work tested and let the data speak and and that's probably my biggest advice I can give today yes in here I mean also it's worth mentioning they're just a two-person company right so great great job guys I mean everything that John said you know when it comes to creative just trust the process believe in the process not in a specific tactic and yeah I my number one recommendation is you know just look at that revenue and the IP revenue to separately understand what's the effect of those on your campaigns and that will probably unlock a lot of new growth opportunities for you guys so I get a run there's great touch with you guys and you know it's good being an another you a coffee talk yeah I thought thank you guys so much for the kind words [Applause] [Music]

[Music]