Which mobile fraud solution is right for you?

all right so welcome to the talk which mobile ad fraud solution is right for you I'm hoping by the end of today's session you'll have a great idea of what key questions to ask when you're choosing your mobile ad fraud solution and I have my clicker also we have a short Q&A session at the variance if you have any questions feel free to ask me at the very end so to quickly introduce myself my name is Reena Matsumoto and I'm the performance optimization lead at bungle and I lead efforts to make sure advertisers are successful on our advertising platform I've been in the gaming and mobile advertising space for about six years I've got a chance to see the industry from different angles from the game development side I agree the mobile attributions I had adjust and now on the mobile ad network side and fungal on the

topic of fraud specifically I've had the opportunity to really understand fraud schemes and fraud detection the techniques that adjust and now I've been working internally to help develop fraud detection processes a little bit of angle where we are a global ad tech company with regional offices across the world we our main goal is to serve as the trusted guide for growth and engagement for developers on a monthly basis we serve over five billion ads to over 1 million unique devices and we work with over 60,000 apps amazing from indie developers to powerhouse brands so now for today's agenda I'm going to start off with laying out the mobile ad fraud landscape and how things have changed over the years and then I'll go over what exactly exact solutions it can actually actually purchase into and then finally the meet at the topic will be around the five key questions you'll want to ask to figure out which one will add fault fraud solution is right for you so first in regards to the mobile ad fraud

landscape obviously this has been a hot topic for the last several years but it's been continuing to grow over the years so this is a chart provided by apps for mobile attribution companies showing how fraud rates have grown in the of course of 2018 you can see both in the US and worldwide the rates are growing for me what's particularly alarming is how worldwide about 25 to 30 percent of paid app installs are being attributed to fraudulent installs and so this could mean that you're you a but it's 25 percent of it could be going to fraudulent publishers not only that fraud schemes are getting more and more sophisticated when I first entered the industry about six years ago fake installs were kind of the talk of the industry particularly around Bop funds however there's new types of fake installs now is through SDK spoofing but now the majority of fake fraud comes to fake engagement through clique spamming and clique injections and so it's clear that fraud rates are growing and this

landscape is getting more sophisticated and it's clear that it's not a fight that an advertiser can find on their own so next I want to talk about what solutions are out there and what can you actually purchase into to help protect yourself from fraud in terms of the main players that are involved in the advertising and attribution space it'll be the ad network MMPs or mobile measurement measurement partners and the advertisers themselves I can definitely speak for ad networks at bungle it's definitely our duty to make sure that any fraudulent publishers don't sign up with us from the get-go but also we work really hard to make sure that we're shutting down publishers proactively if you detect any fraudulent behaviors on the MMP side over the last couple of years they've all been starting to build out their fraud detection tools using their installed data and engagement data and actually block attributions from being sent to us the networks and of course the advertisers we're dealing with the this is the advertisers have

the most skin in the game given that these are there you may budgets that we're dealing with here and then finally third-party solutions the last couple of years I've seen many many third-party solutions are rising that are specializing in fraud detection and they receive data from MMPs and works in order to do this fraud detection so in terms of solutions that an advertiser can actually purchase in - it'll be why MMPs and third-party solutions are providing and I've also seen some advertisers start to build their own fraud detection tools internally as well so now that we know what solutions are out there let's talk about what questions you should be asking to figure out what solution is the right one for you the first question I would ask is how is the data being aggregated when you're looking at different fraud KPIs you can aggregate the data in many different ways it can be on the network level publisher level a creative level campaign level or even on the device level publisher level is definitely going to be the most effective given that publishers are the main perpetrators of fraud I also wanted to

highlight how network campaign and creative level can also be helpful ways to look at your fraud detection KPIs for example if you find that one particular campaign has been flagged as click spamming you might find that one that one campaign has the wrong set up and has clicks being fired on a rung engagement like a view and so it's being flagged with expanding or you might find that one particular creative as being applied does click spamming that might just be a particularly creative that's serving as a click beat and has a really high CTR and serving a lot of clicks and so being able to look at the data in different aggregates will help you uncover different types of issues not just publishers that are committing fraud I also wanted to show what device level reports might look like and how they're not always helpful this is an example of a device level report that we received from an advertiser showing fraud on the device level not a third party solution actually provided and you can see it's a lot of data on the device level however I wanted to give an

example of how this isn't always helpful so this particular install on the first row was flagged with click injection because of a short click to install time of for a second this is extremely short but knowing that this one install is being detected with click injection doesn't tell you a full story of how that publisher is behaving doesn't really tell us whether or not we should shut down this publisher this one ends thought maybe had an extremely fast Wi-Fi connection or maybe this install had happened previous to look like and had a really fast clip to app open timespan so unless we know that the majority of install is coming from this particular publisher is majority within the first five seconds or it's showing a pattern within that publisher this install reports really difficult to action upon from a network and advertiser perspective the main thing to take away here is that you really need to grasp the statistical significance of our data within a larger context the second question I would ask is

whether your fraud solution is preventive a relative or not there's varying different levels of prevention and detection I'll go through each one and how preventive they are the first one is rejected installs and attributions and this is one of MMP will actually block the attribution and prevent the network prevent the postback from being sent to the network this is definitely prevented it because the post buck is never being sent to the network and so the end publisher will not be getting paid out and inherently will be DNS on device from committing fraud the big thing I would consider here is that if this is an accurate this is immediately a worthless solution so since you're blocking attributions and they're not accurate your this is a pretty drastic measure to take so I would definitely make sure that your solution that you're using is very accurate the second level of prevention and detection is fraud detection reports this is a report that identifies a group of installs or publishers as fraudulent this is not necessarily preventative given that publishers are still getting

paid out and networks are being notified through post backs similar to what I mentioned earlier granularity matters and so those device level reports may not be actionable for networks so I would make sure that those aggregates are on the publisher levels so that networks can act and act appropriately finally the last level prevention detection are going to be suspicious install reports this is what a report highlights a group of installs and publishers as suspicious but doesn't make a definitive column whether or not it's fraudulent again this is not preventative given that postbox are still being sent to networks and the end publishers are still committing fraud the biggest problem I have at this particular report is that it leaves the responsibility to the advertisers to make the call on whether the installs were fraudulent or not it becomes a matter of opinion on whether it's fraud and it's like definitely strain the relationship between you as an advertiser and the network so this is definitely something that I would keep in mind if you're a

fraud solution is using this next the next question I would ask is whether or not your fraud solution uses dynamic or static thresholds so clearly fraud fraudsters are getting more and more sophisticated and we need to make sure that we're providing dynamic specials so that we're providing a moving target that is harder to gain for fraudsters so I'll give a quick example of what a static threshold might look like by using an example of CTR CTR sometimes as a proxy for clicks panting if there's a high click-through rate so let's say the static threshold is set at 30% anything above 30% a publisher is going to be flagged as fraud anything underneath is going to be considered legitimate if a publisher finds out that your threshold is at 30% a static amount then they'll work to trying to slither right underneath that threshold maybe at 25% 28% range to make sure that all their installs are getting attributed to so the more dynamic and the more that it relies on statistical deviation it'll be

harder to gain for fraudsters it'll be easier to identify anomalies and fraud trends next I would try to understand how likely you're a fraud solution will have false positives or false negatives so first to define what they are a false negative is when a fraudulent install or attribution is not caught and are included in your campaign analysis a false positive on the other hand is when a real install or the correct attribution are falsely described as fraud and are excluded from your campaign analysis the reason why I wanted to highlight false negatives and false positive is because I tend to find that you a managers and flawed solutions focused primarily on false negatives and trying to capture every single fraudulent install out there but false positives come at the cost of false negatives because the more overzealous policing you have the more likely you'll Holocaust positives but they're just as dangerous as well some of the consequences from having false positives

you might be preemptively blacklist in collisions that are actually good quality sources for you as a for your campaigns you might be effectively lowering your waterfall positions and your network partners you might also be well you'll also just have marketing analysis that just inaccurate they're not having the correct data in your campaign analysis so false positives are definitely dangerous and also highlight two questions you can ask to evaluate whether or not your fraud solution tend to have higher false positives so the first question that would ask is what is the normal expected KPI and how far is a threshold from the actual so let's use the example of CTR just go that's easier to understand let's say the normal expected KPI for a campaign is 10% if you find that a particular campaign or publisher or creative has a CTR of 20% is that far enough from the expected KPI to be considered as fraud or does it have to be as drastic as 90% for it to

be considered as fraud so those are the questions you would ask to understand how much doesn't deviate from your expected KPI the second question is are there instances in which a real install or real attribution could still fall into that fraudulent threshold so let's see that example of CTR again if you have a creative for example that is at 20% CTR it's above that time prasun expected KPI but perhaps that creative again is serving as a cook bait and you actually want those installs attributed to your campaign and not be caught us fraud so these are some instances to consider when you want to look at false positives for a fraud solution and finally perhaps most importantly the key question to ask is how transparent is our methodology I'm a true believer that any fraud solution that you use is only as good as your understanding of it this is going to help you implement the recommendations that they provide and also have transparent conversations with

your advertising partners only then can you really trust your marketing numbers and really evaluate your campaign performance thank you that's all from me I'd like to open up for any questions that you might have [Applause] here with microphone should I kick off I mean I think this whole idea about the false positive and false negative is actually really critical it's one of those areas which I don't hear enough people talking about is you've kind of failed at Parramatta by looking at the deviate standard deviation as a sort of methodology is that the best option or are there other alternative ways you can look at how you balance between those two kind of measures because it seems like this is a system thinking problem rather than a link yeah definitely I mean once the deviation is the first one and then the second one is kind of more fundamental of understanding in what cases where there's real situations that might

actually be falling into that threshold so understanding really how your campaigns are running from a creative level publisher level perspective is gonna really help you understand that so maybe there is a user experience on the publisher side where you have a really high CTR or the creative that is causing that so really going deep into understanding the user level experience can really help to understand how it might be caught in the fraud solution and it sounds to me that sounds like the classic thing we need to do with any kind of analysis that we do is that temper that data with actually understanding what we're all the customers the player is experiencing at that point yeah whether it's a game design question or a or a use of lad or even targeting as I had understanding the kind of user experience of that moment I think is always critical so that was me asking a question anyone else want to grab on I'll just come up with what mobile ad fraud solution would you

recommend a great question definitely not here to promote a specific one in particular but if I knew all the questions all the answer to these question to every solution out there I would probably be able to provide an answer to that but would definitely recommend everyone to evaluate these questions when considering their fraud solution obviously the big ones are going to be come from mobile measurement partners as you're probably using them for mobile measurement already and it's a simple add-on on from there fraud detection side and they also block attributions which I find really helpful from a network side so that we don't have to deal with refunds and things like that though just block them from the get-go so great any more questions yeah I think one thing I will do wonder about is obviously when we're planning our kind of campaign strategy how much allowance should we put in our budget for handling fraud I mean it's something

we should be separating out and kind of anti-fraud kind of strategy momen or is it just about the way we think and we is it a separation of budget or is a separation of mindset that's a really good question I wouldn't say it's something that you should be allocating fraud within your you a budgets it's a little counterintuitive in my mind but I think as you're evaluating our campaign and how effective it is that's exactly the time to evaluate whether this fraud because that might stifle your marketing analysis so I wouldn't necessarily allocate a certain amount but definitely as you're evaluating your budgets and effective this liberal campaign that's when you want to start thinking about and I'm also thinking about forecasting as well because it's one of those things I think is quite challenging too because you can you can know how much you're spending you know how many players you get it you can understand the attribution of them but if you're not incorporating kind of an understanding that there is a risk of fraud are you able to properly model the

forecast for the game with that make sense that's kind of where I was coming from with but it's difficult because I mean maybe I'm in a slightly different position because as a consultant I'm coming in and trying to give advice to people but I think it's an interesting kind of thing in general which is you know if we're not paying attention to this stuff we're not going to do well but actually we can incorporate in our planning and choosing the right partners and making sure we work closely with them and I think maybe I'm biased because of my time at unity ads when I was the Evangelist for them you know transparency is absolutely key and I think you know that's why it's worth working with big players like yourselves to make sure that you're getting somebody is giving you the right information for casting I think as you try to renew budgets for different partners that's also a good time to evaluate whether there's fraud coming from these partners actually so you can maneuver your budgets accordingly so thank you very much got a quick one because we just ran out of time but I guess you go for that

so it seems like there's no magic bullet right for preventing fraud so over time do you see people just constantly rotating strategies and just having to keep up with it I don't know if there's new technologies that come out that require new strategies or what are you seeing right now yeah I see fraud ever going away unfortunately I think as fraud detection tools continue to be developed fraud will move elsewhere whether it's to advertisers who don't have fun detection tools or bigger targets networks that don't shut down publishers quickly enough it'll just keep being a moving target so wherever is more susceptible there's gonna be more schemes coming out to gain the system as well so I don't necessarily see this ever going away unfortunately but it's important to be on top of it so that you can be at the forefront instead of being kind of susceptible and being a target for those publishers I'm always amazed how much effort people go to for you know things like fraud because it's

often more effort than it would have been to actually just make your game yeah thank you very much [Applause]