Superstar Session: Art Definition: The Queen's Gambit of CRO

we are going to move right along uh and we have our track sponsors up next um with a superstar session with a fantastic title um which i'm really uh i'm a fan of uh queen's gambit so i really like the idea that the the the title so uh we have uh uh jose jose garcia i'm gonna get us wrong either way where are you joe so you in spain somewhere yeah based in madrid right now fantastic okay fantastic i won't ask you about the european super league that's another thing we'll talk about at the time but a different conversation but um no great so um fantastic welcome uh welcome along thanks for being our sponsors and you you've got a really fascinating talk lined up here um uh i like i love the title uh i'm gonna not

gonna take any more of your time you'll have your full time don't worry we're running a little behind so we started a bit late but you can uh you can launch in are you ready to go i am i am of course ready to go i just have to share my screen with my presentation and at the end i'll come back i'll gather some questions and come back and do a bit of a q a at the end and then but yeah go for it cool here we go it's all yours okay so thank you very much for your time today it's amazing to be here with you um we are going to talk a little bit about chess we are going to take to talk a little bit about cro our definition mobile how this is all related right so if you know what queen's gambit apart of course from the next league so um just stay here it's going to be very interesting and we can see how tactic and strategy a proper thought about that is going to boost your organic [Music] i'm going to download to to do a very brief introduction of how who i am who we are at labcape what do we do um what is conversion rate optimization

uh what is our definition inside cro and how we apply the scientific methods uh how we hypothesize and how we a b test to confirm or to eliminate hypothesis but first very quickly my name is jose garcia i am aso and cro specialist at batcave and i have more than five years of experience now and within the mobile industry with app and apps and games i can safely say that i am very lucky to work as the same my working hours and my free time is basically more or less the same because i love to play with games and with apps so it is just want to speak with me you think i can help you to unlock your your growth your number of installs just drop me an email to jose.garcia at latkeygames.com i'm very happy to help you i'm i'm part of the strife

that we are at lab cave we were founded almost 10 years ago we are now part of some media group [Music] and since then we have worked with more than 65 clients and we can safely say that we have put our skin in the game first as we have launched more than 700 apps ourselves we have reached more than 300 million installs only with with games and also of course with clients we have a lot of expertise in mobile growth with aso plus cro but we also can help you to monetize uh or to help you to publish your game and we also do user acquisition so if you think we can help you with our 360 degrees marketing strategy we are happy to help okay let's go into the subjects into the matter right and what is conversion rate optimization what is this cro that we are going to talk about is the same as eso well it's not the same

aso is the art of improving the visibility of your app or your game okay we have keywords we have optimization we have analysis etc but art creative assets are very important i think now more than ever before to increase the number of installs to to increase and to boost the number of installs very much uh but is it's it's not something static it's nothing dynamic it's a process it's something it's sleeve all the time it's live [Music] and we we work with them um we optimize it's just okay we hypothesis we optimize but what do we do with that okay so when you are going to to download a game or you are going to just for search for an helicopter rescue game uh you see some some things okay so you see here the icon you see the the the keyword but the thing that appeals the user the most is the icon okay so once you have got the user to your store listing the second step that's half of the battle okay you have

a lot of people in your store listing but if they are not downloading your app it's like nothing it's like okay yeah i have a lot of traffic but i don't have any money um so this is the second part the second step of the process we have to increase to increase the number of installs we have to optimize all these things we see here we have to optimize the screenshots we have to optimize the feature graphic and we have to convince the user with the with the eye with something very eye-catching and of course also with keywords but first the first step is to establish a strategy okay uh we have if we have to if we want to have good answers we have to ask good questions uh and how are we going to achieve our goals uh what do we have to do how are we going to do it we have capabilities do we have a artist team a team of artists dedicatedly specifically to this to my game are they going to study the market are

they going to to do a proper research who is our audience are we targeting the right people are we maybe our game is fantastic maybe our assets are fantastic our keywords are fantastic but the audience is not the proper one so maybe we have to think about that right when will we know if we are succeeding okay we are testing things but this we can see some improvements we don't see an improvement how do we know if we are succeeding and if we are succeeding what do we do next we are going to stay like this the market changes very quickly um what are we gonna do right so we are going to play chess right um if you have played just a little bit you know the the openings are the first tactic movements and queens gambit the the current one the the one who netflix take the name is the one of the oldest so as you can see here in the image this is the queen's gambit um [Music] and as squish gambit the openings are the first movement in chest our

definition is the first movement in cro and our definition spans three steps um first one is competitor research okay um we want to do this uh but our game is small our game is big but it's not big enough what are the big ones doing uh what is the market doing how can we research uh what order others are doing right uh second one is differentiation why and where our game is different okay there are a lot of games in the market all of them are not the same why our game is different answer one game analysis um what are the main features of our game okay our game is doing something very cool that other games are not doing we have to show that we have to convince the user that our game is different our game is better than the other ones so for that we follow the scientific method it's not you know um marketing is every day is more science it's also art it's like chess art and science is chess but it's also

marketing so we have to follow some very thoughtful steps um intuition is night okay i think this is going to work very well i love this icon i love this concept i love this color i love everything but do you know how do you know it's going to work you have to trust data right so to trust data you have to to test things um so the first step is just a strategy strategy definition is just the ones we saw the one we saw in the previous slide but second one you have to formulate an hypothesis okay i think the competitors are doing this and we are going to do this this this and this and i think it's going to work because this and this and this reason okay that's so very cool but we have to test so we go to a step third we go to a b testing and a b testing is something very scientific uh we have to to think very well what are we going to test how long are we going to test um and and how are we going to know if it's working or not except fourth

data gathering okay we have to we have this hypothesis we tested what the data is showing what the data is telling us what is working what is not working i interact to the first to the fifth step conclusion okay this work this didn't work if this work we are going to implement it or we want to optimize it more okay so we go to another step to go to iteration and we have to second step again we have to formulate a new hypothesis based on the data we gather and then we have to go to the whole process again right um so now we are going to go through a real case study how do we apply all these principles all these fundamentals in real life in in real life so in november 2010 we started working with a great game development students a studio in bigtooth games and since then we have increased their downloads globally more than 800 and only in the u.s more than 500

so the icon a as you can see here this is the icon we receive when they contact us and the icon b as you can see okay we see we see two helicopters we see we see shoulders here but it's not the same it's not diff it's completely different the colors the intensity is not the same the order is not the same in the in the final icon b we can see the soldier here and not here that's that's testing it's not just the mavic um so how and why did we go from a to b okay and as you can see here these are the steps of the process uh we follow all the time this is just like a religion for us we have to follow this all the time to know that we are working properly um [Music] the first icon a b testing we did uh it was through the through these three concepts the number a and a was a a soldier trying to be deployed for an helicopter you can see very khalid color very in the background the the icon b is just a different one

completely different one we saw just we see just two canyons just shooting to the helicopter and other helicopter behind and then the icon c is completely different one it's just the blue black background just a helicopter trying to rescue soldier um in this test the results were not conclusive you know it's like okay this is working a little bit better we'll know exactly why but the other ones are not working very well varian v there is no soldier this is not working okay we tested this okay it's time to to formulate an hypothesis because it's good it's time to go back to the step sixth to the second one again and let's test the soldier with helicopter because it it was the one who worked the best so second iv testing we go through the same steps again hypothesis formulation a b testing data gathering conclusion and iteration in this case the variance c in in the previous slide the variancy was the word the one who worked the best

and here we can see um uh the same idea okay it's just an helicopter trying to save a shoulder trying to rescue a soldier um icon b is just uh the helicopter is landed there are three soldiers trying to get to the helicopter trying to to to be rescued um and the icon c is just a completely different idea again it's just a a soldier trying to help to helicopter to land uh it's not any it's not doing any rescue mission it's just trying to land um okay so we confirm our previous hypothesis if we put an um a soldier trying to be rescued by a helicopter trying to go through the steps and variant a was the winner with seven uh with more than seven percent of the improvement uh okay so in this in this case we confirm our previous hypothesis the conclusion is positive but we have to improve even more so we are going to test a third time um

we are going to optimize the the previous icon the one that worked very well but we are going to compare it with two very different icons and here we can see the rescue of a shoulder with help it's the same idea but it's not the same we can see here rope instead of some steps the icon b is just some silhouettes of the helicopters just completely different background there is no sky there is no sea there is no forest just a wooden background and some helicopters and then i can see is a very cool one we love this one but the the result is the a work much better i can see it's just you can see a first plane helicopter trying to fly just shooting with some warships behind we can we see that she we see the clouds everything okay this is very good but variant a in this case was the winner as well so with this result we confirm our previous hypothesis and prove that any uh it's true we added

any soldier here you can i didn't sell it and here in the helicopter trying to aid to the soldier to be rescued okay so we finalized the the experiment the scientific experiment we we confirm our hypothesis conclusion was positive and then okay we implement this icon but we don't do that just with with icon we do also with screenshots in this case first two steps okay what's going how are we going to find a strategy for this for these screenshots and what's going to be our queen's gambit what's going to be our first movement uh in this game match right in this chess match um we can see here that the uh the screenshots of the uh that we that we have uh they are showing some very um basic features of the game we can see here you can see very well the soldier here because it's green the grass is green

the helicopter is green you can see here another gameplay but you don't know what's really doing uh you can see here another it's a little bit cloudy okay where are the ctas where are the call to actions where are the ribbons what do you know what the game is doing here so we do the market research and we see okay other people are doing this how can we improve that um okay so we formulate a hypothesis i think this is going to work better with some some ctas which is this choose your favorite chopper don't let them shoot you down and the ribbon is this green blocky um part that you can see here so that's very important because okay if you are a user and you go to the store listing and you want to download the game but you don't know what really what the game is about it's not okay maybe it's not for me but if you see here okay i can choose choppers i can get medals i can i have some campaigns i have to complete that that's very cool um so

we have to maybe test our our thoughts our hypotheses we are going to choose some blocky uh ribbons because they use because the game is very blocky we can see here the solder is blocky the helicopter is blocky um so if our gain is blocky it's very nice to have blocky ribbons right okay let's test so we test we gathered the data and we saw that with this test we managed to increase the conversion rate by more than 16 16 so the conclusion was very positive but we could iterate a little bit more so we did a second screenshot av testing um just for for you for united states so here as you can see this very this person squeeze so it's very eye guide catching you can see here the helicopters flying through to the sky with the sunset with some pink sky very nice and here we tested a second thing it was like okay we don't know what other features we can

add that we can explain what the game is about so how can we do it okay if we go through the reviews of the game we can see what the characteristics of the game the users highlighted the most what are the key words that are repeat are being repeated the most what are the messages that people are living there what are they loving what are they hating uh so based on all that research we build a different concept here like this one join the most epic campaign and prepare you to prefer battle um deploy your best troops all that stuff uh to see if it worked better than the previous one so once again we hypothesize all this based on on the on the research we did we a b test we gather the data and then we go to the conclusion that this worked very well because we managed to increase the compression rate in the us by more than 29 which is huge um and last but not least or only for google play of course we also can test

the feature graphic uh and this is one this was the feature graphic that we received uh just a fail rescue is the name of the game which is very cool uh here we can see as well a screen sorry a green uh helicopter green shoulder green background green grass we don't know if it's sure what is he doing here uh okay so we have to test again once again the scientific method method we define the strategy what's going to be our queen's gambit in this case and in this case we tested very different things as you can see here in the previous slide this is very same more or less same palette of colors and here what we tested is very different one it's much more there are there is much more contrast more eye-catching right um so we tested three different concepts first one very amazing in our thoughts uh with this explosion behind the helicopter this uh orange fire um

which is very like wow what's that uh focusing on soldiers in the in the in the b where's where we can see here they are like three shoulders above the helicopter just trying to to see the enemy to see if they can be rescued uh it's getting dark so it's going to be dangerous and then um concept c we see here soldiers plus helicopters here we can see we are inside our chopper and there is a blue sky and there are other helicopters uh flying along along with us and is really cool right so variancy was the winner in the u.s with a conversion rate improvement of three percent uh which is what this one right uh variancy okay this is the same that variancy right if you remember but there is optimization uh because in the other one we have this blue sky and here is is like um it's like orange which is like the sunset which is like

explosions it's war around us uh and honestly we love the variant a which is like okay it's like you're going to the king kong island jurassic park island with your choppers with your um with your warships there is a beautiful sunset behind you and then a c is just a completely different concept from the other ones where you are seeing something like is everything is very dusty i'm alone here hoping to be rescued and there then there is a chopper coming here to rescue me my life is amazing and then even though we love the concept a our intuition is that we are going to to win with a we'd have to trust data and c was the one who was the winner with a conversion rate improvement improvement of more than 10 percent and that's why we say okay low it's very nice to your intuition is amazing but you have to trust data right uh and also

for um in this particular game of invictus games um not all not every icon works the same in every country in google play you can test different icons in different countries so in this case in united states this was in united states brazil and russia this icon was the best but in germany this icon performed much better so this means that there are different differences in the cultural environment difference in the in the users in psychology of the users this every country is completely different one from another so if every country is different you can have all the same icons all the same art in all the countries because it's not the same people so once again we tested in all the countries um and this one these ones were were the winners so our our advice is okay uh it's nice to test but don't test only one country and don't test everything in all the countries so they also don't test the same art in all the

countries in different countries test different art right um so here we can demonstrate that the number of visitors which is the blue uh the blue lines increased hugely increased hugely from 5000 to 40 000 which is uh huge and also the number of instance right and that was through heavy testing that was a magic that wasn't just thoughtful that was something very very very tested um within a period of four months uh we increased more than 800 800 instances globally in the only in the world play store and only in the united states as you can see here there is a huge growth as well going from 400 to more than 18 000 sorry 1800 and number of visitors to the to the store listing and also a huge increase of install where we were having like less than 100 to

up to almost 600 so we were able to increase more than 500 percent installs in the main country the united states and the number of users who play everyday from 500 500 to 9500 currently which is almost two thousand percent more which is amazing and also users with the game download went from fourteen thousand to one hundred two thousand um new cents currently which is uh huge so that's my the end of my talk thank you very much for your time and if you want to drop us an email if you if you think we can help you to grow like this um just write to all adult luck game lab games.com go to our website our twitter linkedin my colleague jamie and i we are going to be here um the whole week you can reach us out anytime you need us um thank you very much for your time

all right that's great i've got we've got a lot of questions we're gonna try and get through we've still got about uh we've started about 10 minutes late so you've still got some time for questions do you um can you just turn off your screen share a second yeah there we go everyone wants to see your lovely face that's beautiful yeah i'm looking like this because i have another screen yeah no no it's fine it's absolutely fine i've realized i keep moving my my um my screen i keep cutting my head off so apologies everything uh just the bottom of my face um so yeah there's a a number of questions here i'm going to try and run through a few of them uh i'll start with um i i thought i was specific on twisted brains i don't know if this is relevant or not um how do you a b test on a platform like steam or other app stores um i guess other app stores is any different from do you test each one for a week or do some platforms have an option to a b test i think it's a a broader question about a b testing outside of the specific platform you talked about the are the similar similar facilities

elsewhere uh we test everything first on on google play because it is the platform that offers you the most options for testing it's very insightful is very it's very easy to test actually if you know what you're doing it's not test it's not easy but once you know what you're doing it's very easy to test on the play store it gives you a lot of data it could give you more actually but if a good number of data is not a app store is way worse in this case than the google play store but in ios for instance you can use the the [Music] tools like for instance split metrics to test all the stuff and yeah basically we test everything will play store right now okay um you mentioned um about starting google play and then and then doing slightly less on on on ios because have quite the same tools but what what how much sort of difference do you find between the android um and ios uh users as a question here

um particularly they've mentioned us customers but i guess it's the same is there quite a variance between i mean people are people obviously there's a variance yeah yeah but um is there anything noted between android and ios users you see regularly well you know this is saying that yeah if you want reach go to well play if you want read go to ios so in google play the numbers are very um are much more grateful you know everything you test is much more grateful in terms of number but of course in terms of money as you can see here for instance an iphone is very expensive but not all android are very expensive so the kind of user is not the same [Music] so i think the most the biggest difference is that you have you are going to have a bigger reach in google play but you are going to have more revenue in you know i like that i like that reach versus rich that's a very smart i've never heard that before yeah

that's a common industry thing i really like that as a nice sign up um you mentioned uh this is a question you mentioned 800 i noticed as well like pretty sizable increases in downloads your case study there yeah sure you're not picking the worst case study ever to show obviously so that's huge i mean that's incredible and obviously testament to the stuff you guys do but you know i'm assuming not every game can necessarily get eight hundred percent i mean or thousand percent what you know what sort of baseline success should you really be hoping for is is there kind of should be going well if i can't you know double the uh the click-throughs i'm not doing it right or is it is it so variable as to not that's a very good question um actually it's like real life i mean um if you have a very small game it's easier to have more growth if you are a super successful game with me hundreds of millions of installs is much it's much more difficult to get that amount that that huge growth so if your game is just starting and you have nothing

optimized i think it's like when you start to losing weight at the beginning you see a lot of results but once you are optimized you are going to be like okay i'm eating very well i am going to the gym but i know seeing a huge so with games is the same if you are just starting and you have you didn't optimize your keywords you didn't optimize your art your you never tested anything you are going to see more um more improvement of course but once you are half array optimized you have to test and maybe one or two percent improvement in compression rate it can means a hundred of thousands of installs if you are very big yeah yeah yeah no i mean you mentioned a lot of uh i don't i'm sort of scaring through the questions to try and summarize things you you mentioned uh uh there's a lot of things you can do obviously which is you know for for quite small changes when you think about like a screenshot here or a you know an icon that has a huge impact and there's a lot of thought going into that um so i would conveying kind of two

questions here one is when should you start this strategy you know obviously until there's a game you don't it's probably not much point doing it to them but but at what point should you be should be really building this into your sort of development process and then and then how often should you be looking to reiterate obviously if you're sort of zynga or some tencent or whatever you're doing this constantly you've got a massive team if you're a smaller studio maybe or you're not as familiar with this how often should you be looking to kind of iterate um well it depends uh of course always marketing the answer it depends yeah so if you test of course we look at the data every day if you are already having a good number of downloads and you are going to you are decreasing it's because it's related maybe with the test so we we have to look to data every day but normally we recommend to test every three four weeks once per month and more or less every art asset because the more time you

it depends on the number of instances but if you are a low a small studio or you are just starting you have to get you have to get to give time to the data to be gathered to be trustful uh to be consistent so maybe three four weeks is the recommended amount of time we we recommend to wait okay no very very good i'm gonna try and get through two more two more questions anybody else so he's been around all week you can contact him through me to match from the discord or whatever so please do go to him because i i sense uh it's pretty impressive service here which you know we've just seen the scale of the impact you can have um there's a very specific question i'd like to get to from uh from attending many hyper casual games do not have call to action in their screenshots what are the pros and cons and then how does app preview video impact on ios there's quite a lot of questions in there actually but yeah one of them uh it's kind of like four things here but um is there anything i mentioned hybrid casual not having call to action i mean

i don't know that's that's that's true or not but um is it do you need an interaction i mean that seems like uh some of them have some of them don't have you have to test okay how do you know if you are if you test with ctas and with ribbons how do you know it's going to work worse than when you currently have okay you have to test you you can you cannot know it's just impossible so if your game is having a lot of downloads this is performing incredible well incredibly well without ctas you can make a small test maybe you can apart some some percentages to do a small test and then um maybe you can see some improvements maybe if you if you put some some very appealing ctas some very appealing ribbons you can see a huge increase of instance it means if you don't test you can never know uh it's impossible it's like um everything in life fair enough um yeah uh final question

i'll get some messed up so so again you know obviously you've made a brilliant case uh for why you should be doing this and you've you know very sensible business reasons uh constantly and as much as you can but say i'm a small studio i've not done much i haven't really thought about this so i'm going i'm going god why we're not doing this but i've only got you know i've got a smaller team what what would be the first thing or the that they should focus on apart from from ringing up lab cave and trying to hide them but what should be the uh what would be the one kind of main thing for like a smaller team and maybe it's an indie team that they should focus on is it all about the app icon just spend as much time on that or or something else well that's a very good question and the the short answer i think um if we are just speaking about art about assets we are not speaking about keywords about basic metrics uh i will say uh you have to measure if your app is having impressions first and then of those impressions how many people are

visiting your store and of how many people are visiting your store how many are is actually downloading the app because it can can tell you if maybe okay you are your app is is having a lot of impressions but people is not going to your to your store listing so there is something they don't like there and okay what what they don't like probably the icon probably the icon but you don't know you have to test and once okay i am receiving a lot of traffic but i think uh or maybe the the the category percent of conversion rate is huge mine is very low okay there is something you need to test because if you are getting users there and they are not downloading that they are not being convinced convinced by something maybe the screenshots maybe the keywords maybe the feature graphic uh you never know you have to it's like science what's going to work about the cockpit we all know we have to test so fair enough i i love the what the analogy of the chest and the

the science and art combined that's that's uh that's that's like if you play chess you know it's very creative but you have to measure everything what you're doing because you can get messed very quickly fantastic on that note i'm glad to say much as gracias and and uh yes it's been a pleasure hopefully i'll be able to see you uh in madrid uh at some point uh anytime anything over there to have been in spain for a while could do the could do that to do it leaving the country for a little bit i think soon um thanks for your time obviously uh uh that cave and and jose are available to talk throughout the show so please go and find him i'm now going to let my beautiful assistants push him into digital ether as we move right along with more of the track you