Gamified panel: Which ad won?

[Music] [Music] so as it is appropriate for Pocket Gamer connects here in London we're gonna this is unlike your traditional panel so there's a bit of a gamey fide panel so it's gonna be a bit of fun and we're gonna have a little bit of audience interaction so I will keep going in with that is do I need I need a clicker or should all right well a for thank you so so yeah this is this is actually very much based on so lift-off as a company we are very data-driven so we do a peck position campaigns app install campaigns

and then also re-engage in retargeting and pretty much everything that we do is based on data we release a lot of reports if you care to go to our website you'll see there's a resources section with tons of reports which is all database so this game is actually based on real data real ads creative shown to real potential gamers and gamers and we're just gonna be running through this and have a little bit of interaction with the crowd to understand how important creative is in terms of driving installs and then down in downstream engagement with with the app the game so just as a in the form of a quick intro liftoff is actually based out in Palo Alto and we have offices here in London and then also in Singapore and Tokyo so we work when she started off traditionally as non-gaming and over the last probably 18 months a lot of PA Murs have started working with

us some of which are mentioned there including my toner who's up on stage with us so grateful to have a customer up on stage as well which is great so we very much focus on not just driving install but what actually happens after the install we track you know do you go on to complete the game tutorial do you reach level 25 which may be indicative of somebody using the complete an in-app purchase has somebody purchased but then hasn't engaged in the game or played the game for a while to try and build segments for reengagement etc so everything that we do is very much looking at data and of course one of the really important components when you're putting ads out there into in front of people is to understand you know what you actually put the visual of you put in front of them is going to be very critical in terms of getting them to convert for an install or actually going to engage with your game so so I'm going to get a get going shortly but before I do that I'd love to obviously

just allow the panelists to quickly introduce themselves so maybe we'll start with with yourself I am a freelancer and I help indie developers market their games across all platforms of mobile in HD hi everyone I'm Elena from might onna is a free-to-play mobile game developer and publisher I'm a marketing manager at my Tona and we are running pretty successful campaigns we have lift-off thanks for the plug ok my name is Jana I'm a head of marketing for Aviva Games one of the biggest polish mobile publishers and developers excellent ok so here's you see we've got real games experts right so this game should be a lot of fun because you would

expect that they're gonna get every single question right so so let's so let's find out so so these as I said this is a real gay sort of ads that we've been running this particular one actually I'll just just double check this was actually run across a number of gos so so this was actually run in Canada Germany France USA and Austria now on iOS and you can see its version a is the interactive ad versus animated ad okay so quick let's get our gaming experts which one would you think had actually warned in terms of outperforming and this is of course all

based on real a be testing with statistical relevance I'm kind of I'm gonna I'm gonna go for the 20,000 Linden by the stat because I think this consumer is kind of I think they're after a lot of kind of puzzles are kind of into like very open about playing against for mind you know to kind of betterment themselves basically so I think that would appeal to them rather than a simple kind of piece of interactive okay so I'm gonna get there's some game expertise there what about Elena I think I'm one of the target audience for this game because I'm a female and for me version a looks much better because it's a show me a real game play so I can I already know okay this is puzzle I need to drag and drop the pieces of puzzle and make some nice pictures so I think like as a gamer

and user I think version a is like more interactive and show me more how game can be look like inside the game right yeah I absolutely agree I think that the version name would would create much more engagement not only because of the of the fact of the interact interaction in it but as well it shows in a better way what the game is about but I think when we see those we need as well to consider the costs usually those kinds of ads costs more so it is I think that this could create much more engagement but at the end of the day it would need to calculate their organs as well okay great so we have to going for a and one going for B just a show of hands from the audience which who would go for a the interactive ad overwhelming anybody in the audience brave enough to go for four B now okay a

few okay so let's see the winning ad was [Music] [Laughter] you got that one so again yeah you know this is interaction right so you would think that if you're this is sort of mimicking the gameplay of the app this is obviously an app from za mad magic jigsaw puzzles so yeah fairly interactive as you both said and in terms of cost which is your question actually I don't know I don't know what other marketing platforms do but what we do at liftoff is that we have our own creative team and that is actually that is part of our full-service offering so the cost of that versus that is zero as far as we're concerned because what we need to do is we need to make sure that we can drive installs for our customers

and then of course we're you know they pay us on on a CPI or a CPA basis etcetera so there is no additional cost as at least unless you obviously working with a separate agency to do your creative that will be a different scenario so and they're just here like ads so this is kind of like maybe like a guide for the panelists and a guide for the audience as well so really if you're able to mimic you know the the app play the game play and and sort of demonstrate the core benefit of the game those those tends those ten those tend to win out those kinds of ads tend to win so hopefully this will be a guidance to make sure we'd get everything right from now on right here we've got woo Gers pearls peril another hidden objects game so in this particular one here you've got obviously

again there is a interstitial and there's also a whoops there's an interstitial and then there's also the video a sort of video and interactive so there's a video and interactive here for an interstitial ad unit this was actually run on iOS in the US only and it's actually running as we speak at the moment so we've already reached statistical significance - to judge which one is performing best so maybe start with you this time yeah yeah I would say the version a would win in the similar manner as the previous one I think that it has a movement in it much more well it's not interactive per se as far as I understand it's just video right it's just video yeah but nevertheless it's there is a movement inside the frame it's much more reach in a way so it

would engage better at least that's what I would guess and at the same time I like the call to action at the beginning of it it is much more I don't know direct and explanatory rather than help me find nine wines I like this one help parallel find something whatever it's like it describes better what the game is about this one for a Elena just question this is an interactive right so it's not interactive so these are both these are this is it's just it's just as it is so there's no interaction in that one it's just video versus interstitial as for me I think version a is much better but maybe if version B was interactive so means when I click on it I actually can find these nine bottles yeah so probably version B will win P okay so that's again so one for age I'm not I'm gonna go with the popular vote

on on a I think it just it works better and I I agree I think about the idea of interactivity of Babel if it isn't the video will really work it is actually so my mistake it is instructive you've got to find the nine bottles it is inter it is interactive the version B isn't around yeah it is interactive appalling and Dan is shrieking it is because who does not want to find 9y if it's if it's playable it definitely has much better KPI so it's just this so it's tricky that the version a it's much more appealing when you look at it like that but if you can it's the finger as well I wanted to talk about there is a face there's a big face of it so it's very opening well that's tricky I don't know I would still go for version I mean I would change the version B then okay if it's interactive then I would go for a version two bees and yeah okay I'm gonna go fray this no

face of a human okay and then a quick one for the audience again show of hands for a show hands for interactive be slight win slight win for the orders for interactive beat and the answer is interactive wins again interactive wins again so even the human face does not sway it even for even for this one so again lesson learned here as well so here's one just actually for the audience so as I said at the beginning we released a lot of sort of reports we released a mobile how to market mobile games report last year and wanted to understand like wow which according to the report that we released about four months ago I believe which subcategory enjoys the lowest cost to install and purchase rates so is it a casino be

casual see strategy or D action let's go for a show of hands that's probably ease this in the audience what about a who would go for a just three or four people who would go for B for casual a lot more anybody for strategy no one it's right EU lowers cost right okay and then action/adventure that's about fifty fifty I think between casual and action-adventure based on the audience so on this one actually it is casual so yeah it is hugely competitive but it's certainly it's still cheaper so on average based on all of the data or form of the game reports it's approximately two dollars to acquire a casual gamer CPI across the board right

so this is data across the world across all sort of Kurdish casual gaming apps that we've run and and then just to give you an idea the most expensive is actually adventure adventures double the price at full dollars CPI so significantly more expensive the reason why is obviously LTV with LTV with casual games is obviously lower we know that LTV for strategy and action adventure is much much higher so the value to acquire those users as a result is is is a lot is a lot more competitive so although it's not mass-market for those users that do play strategy and an adventure their high-value users so to get them is is a lot more expensive so back to the game we're ready for another round it's getting more difficult as it's hopefully getting a hang of this so

this is localized versus non localized but it's both are being directed for Sweden both are just in the Swedish market okay one is doing it in english in sweden and one is doing it in Swedish in Sweden which one which one would win B I guess it's all those sweet speak better they do but are nevertheless it's always better to localize right okay so Swedish for localize really yeah I think version B will win but maybe the up leaf is not that big because yeah Nordic and euro I think most European countries speak English really well and they understand English ads so I think in this case version B is still winner but maybe it only gives an

answer she qualifies and gives her some data behind it nice ok and I'm gonna let go meeting this is that I go me it's got a microphone and he's telling me so I used to work at Chillingo and we used to do a lot of localization and experience was very similar in a lot of territories it would still be positive but the margin wasn't wasn't as big as you would expect right and thought I do still think the localized one would work better ok so pretty much everybody's going for Swedish winning in Sweden who goes for Swedish winning in Sweden surprising it's only about half of you ok and the yes a 56 percent uplift so significant so this is this is actually important because yeah I mean it's a little bit surprising as well because obviously in Sweden they speak very very good English but generally localized

always works better but very much to your point so of course if you're going in a country where English is not spoken don't use English so so there's going to be huge differences in certain Geo's you know you know there's no point showing I don't know English in in China for example right or Japan for that matter and that's that's just not gonna work so generally even in Sweden which is obviously a country where they speak English perfectly for the most part there is still an uplift even even in that country and there just in terms of another from one of our other reports so there's there's a lot of data and you've probably seen some of this is that people install apps and then they quickly uninstall them or they just

don't engage with with those apps so only one in four users who install a game or an app are using it one day later so it's just one day later you've spent all that time developing your game you've got it in front of App Store and Google Play Store you've done your marketing and all your campaigns etc somebody downloads it today Wow amazing fantastic so you know 25 percent had gone tomorrow and then actually the sad reality is one month later 30 days later ninety percent are gone so yeah so those are the sad statistics coming obviously we you know gaming is incredibly competitive so so reengagement is clearly like a big and important part I think more and more games developers are doing very engaging so once you've built a decent sized user base and you can

monetize fairly well then I think reengagement becomes more and more for an important piece any show of like here on the panelists I mean do you do you do you guys invest in reengagement whether it's paid reengagement or obviously you know whether it's email or you know the you know the standard push notifications and so on and so forth well this is something we start to explore we have experience like two or three campaigns like that which are fairly successful I say I think there's a big matter as well of the KPS on your game and how it performs and how easy it is to retarget the players to that as well so it has an impact on that but generally that's a big trend I would say now so it's definitely something that should be enough interest to a lot of game development and from you from for us I think from their beginning just using simple push notifications and adapt like different push notifications

to users bring your like retain your users much better than maybe if you cannot do paid acquisition right we we same as vivid games start to exploring their retargeting for now I think our games like mature at the current level so it's actually necessary kind of kind of acquisition as well just retarget you're churning losers what about when you're consulting out there with with games developers yeah so not had any re-engagement campaigns in the past year I've just been consulting for a year before that I was a TA Chillingo when he had a tank battle gank or iron force and with I enforce we did see a lot of reengagement when we Canada coordinated

campaigns around in-game events so we do kind of big kind of global world war events and and we used reengagement for that very kind of strategically and that seemed to work so right in game and then I know with EA for example they would do actively engagement around their licenses so you know if there's a new Star Wars movie and you've got Star Wars Galaxies heroes well actually you may be tempted to go so they're just like use that as the hook - yep it's so easy to like - you know that's great for EA who are those big licenses but it was using yeah and then just a show of hands like you know in here in the audience who are games developers have you as anybody invested or thinking of investing in INRI engaging users rather than pure user acquisition so not that many okay interesting cool

alright here we go here's a good one leprechauns or coins which which one wins okay I actually visually I can't even see much of a difference here but there is a basically you can see the difference is here is coins falling down or leprechauns falling down other than that they're identical so this is from play ticker or one of their casinos slice titles okay maybe I start since it's a casino or slots game I think coins should work better since I should win coins right and like just visually coins looks much sharper I think there should be but yeah but

actually not much difference yeah I think I agree coins because I think the kind of gamer here is after that reward you know they'll see the endgame for the casino type gaming is that reward if it's real or virtual and I think coins would take that box for them oK we've got two coins anybody for a leprechaun well not me I'll go for reversion B as well it's just as their previous guest sets so it's obviously something more symbolic maybe it's a bit confusing on the left side so people may not exactly know what's falling from the sky the prop the problem I have with that is actually I I'm not sure why even bother with tests like that it's so similar that I would say that it probably would get similar results maybe I don't know but uh I don't know it's like variable so I would go for much more difference when testing creatives at least me version anyway so unanimous B coins

anybody in the audience that would go against the panel and go for leprechauns nobody so this is a unanimous scenario right we're all going for coins are we sure about this so again not a huge uplift so again what we do is say and you say we go you go for big changes actually what we tend to do when we're doing a/b testing is we go for very subtle changes because you know people I guess you know if you think about it people get bombarded with ads all the time when they're on their phones and I guess what we've found is that when there are drastic like huge differences between a B it's very difficult to get

like a like a an understanding of what check what was different made the difference what was different what was it that was different actually made the difference because with a/b testing what we're trying to do is you know essentially have two things that are relatively similar and change one component to see if it has an uplift or not so that's what which will constantly do variations now of course this is again it is very in my opinion very random as well right I guess it's just the luck of the Irish with the with the leprechaun but but yeah it's it's pretty random but I think I guess this what this means is that you know literally test everything the most that you know the things that you would not think that could have an impact sometimes they do sometimes they don't but until you a B test and have sort of statistical relevance then you don't know what the potential uplift could could be so

another one for again this is another we have a guide to marketing games which obviously pretty relevant for for the crowd according to this report that was released how much revenue was expected to be generated by games mobile games in two 17 is it on the low-end a 2.2 billion in the whole of 2017 or on the extreme end D one hundred and eight point nine billion or something in between so what do you guys think on the papacy well yeah actually just to get your opinion before I go to the audience as far as I remember it see see yes I go for biggest D D I'm gonna go D D as well okay audience would you go for let's go

for anybody for eight C and then D okay so most people went to see it seems whoops you still hear me and the answer is in fact C so 46 point 1 billion and I think there was actually a report just a last week confirming the numbers for that from having data from a Pawnee all right I think this may be the last one so this is from out play and obviously a match-three type game which I think a lot of people are very familiar with casual gaming mystery match so here we have as you can see it's got a institutional it's a video compare that the two different ones so both a video

no interactive no interactivity in this one that's just which one wins visually which one is visually more appealing and which one would get users to actually click and then install I think the version is appealing though I have big concerns about one thing as you can see only for some specific amount of time you can see the CTA and the button and this is something that may be a little bit confusing for players and version B and that matter is much more clean and it's with probably more efficient because the install button is being visible from the very beginning of the ads and I think it's quite crucial the call to action essentially yes because on the left side for some specific time you don't know what to do right you just see the video and right now you see the battle and version B as has a has a button from the very beginning and I think it's an importance it might be tricky I might be wrong but I would go for a version B anyway like that I'm working in

marketing so which one you're going for B right okay so yeah again I think personally I like that creative you know but I think B would probably perform better I think it looks like more of a challenge as well to to the gamer I'd like to say a but I think it's I think it's I think he'd be we're probably outperform it okay cuz but a is the only one because I remember I'll never mention that the face again that's that's got four faces in there or at least you know so it's different scenario but so you still go for B I still go to pay yeah Elena for me acting version a will win because those match free animation and how the makeup these matches and bombs I think like from

players perspective is looks more attractive than just three matches in a row yeah I would click on version a ok so we've got 2 B's in 1a from Elena show of hands who would go for a so this is the last one in fact also overwhelmingly a okay anybody for B okay so I'd say that's that's ninety ten percentage wise for much repair this is the last one let's see the experts versus the audience ah so a bee wins out and this one's a 99% SETI uplift so pretty pretty significant so again I think you know really this is that was the last one that you know the whole idea behind this was really want to put some visuals up there we're all in this space of getting

people to click on our ads install our games and and ultimately engage in pay in our games and of course the visual element is critical so that's something that we're constantly testing so I think you know for those people that got their questions right we've got to stand up on the on the second floor I think we've got some some schwag up there so feel free to to visit lift-off up on the second floor and collect some swag but I just like to thank very much the panelists and and of course the audience for participating as well thank you guys thank you [Applause] [Music] [Music]