Curated Panel: IDFA Effect' on UA Performance & New Changes for ASO

now we're back into idfa we're back into the big question and big impact and we have our first panel of the day uh which will be coming in now we're featuring representatives from axe flyer hello games and ad quantum all moderated by app follows anna gotcha cover i've almost got that right almost certainly um anna how are you can you hear me yeah perfectly hi chris and right you yeah i'm not bad i'm not bad i'm kind of as as usual i'm surviving on uh not much sleep but a lot of coffee so i'm pretty uh pretty pumped um but yes i think your panelists where where about remind us where you are in the world i'm basically located in cyprus right now oh nice nice where you go i'm sure it's quite warm in cyprus had a i love your breaking grease i'll say some holy

on the list as well so i might pop over there as holidays open up anyway enough about my holiday plans uh and more about plans for uh dealing with the uh ever changing the ever shifting exciting world of uh changes in apple's app store and elsewhere so i'm gonna hand uh the reins to you and i'm here in the background i might come back at the end but i'm gonna let you introduce your wonderful panel uh very uh very uh of luminaries and then uh and run the uh run the chat and if you need me power i'll be here sure thanks chris and thanks for the introduction and thank you pg connect for having us today let me share my screen with the presentation yeah here we go so um yeah hi everyone and welcome to our panel session today we're going to discuss a very interesting topic which is the idk effect on the user position performance and some changes for the app store optimization and to do this um

wonderful speakers have joined us today so let me introduce them first of all we have here the senior partner development at app square eden eden please the mic is yours thank you anna thanks a lot thank you for the opportunity it's great to be here hello everyone i'm i'm in charge a caps fire for the relationship with our partners in north and south europe africa and israel um i'm based in israel and as chris uh was speaking about his holidays so today we are actually celebrating the sukkot holiday here it's a sukkot eve for israel in september actually too many too many too many two little working days but it's good i appreciate it and i embrace it so i i a quick intro into absolutely who we are as a business right i'm sorry sits as

the attribution authority in the space right and our mission is to help brands make good choices for their business and for their customers app store is fully unbiased which puts us in a position that we have no conflict of interest and that's ultimately why we are trusted by the market to make these decisions when it comes to mobile attribution performance you can see here that we're backed up by the leading vcs in the world have more than 1000 employees a lot of them were actually hired during the pandemic where we saw the opportunity to hire more mainly to our products and r d teams uh we operate across 19 offices worldwide and according to some independent studies we have more than 65 market share in this space uh all of that helps us provide our advertisers our partners accurate unbiased data innovative solutions of course for these new arising challenges and insights market trends

and offline support thank you for having us thanks for the intro eden so let's move forward also today we have here alexey who is the user acquisition growth manager from app quantum alex hello yeah hello guys uh uh you can call me just alex uh it's okay yeah uh i represent up quantum uh we are a publishing team uh like in our teams consists about 70 people so yeah we uh we already succeeded in many titles and many genres uh such a time management or tycoons or minors uh so in in our portfolio about like 15 and more games uh and also uh accept uh publishing we do code development we help we help to small teams to improve improve their games so feel free to contact us and we will help

you with your game super thanks and we also have here the executive director of hello games pavel hi guys hi guys so i think we're i'm the only one who doesn't want to share the services to anyone else so we're independent games game developer company located in cyprus so we leave we don't need the same city at the end of the day so uh company is four years old we have two hardcore mobile titles uh wesley survival and [Music] so this these are quite the westland is all the game and it's quite successful uh and i'm here to i would say to share the pain of those who need you know more uh better user retribution segmentation targeting etc uh so what else to add here so we we have

80 people in our team located in two countries and one of the things i put on the slide as well is where one of the very few [Music] game companies who who put machine learning driven monetization as uh as a key point of our our core monetization cycle so and succeed with that basically that's it thank you thanks for for the intro and actually yeah i mentioned the drone uh driving monetization well it's really exciting and i hope we can touch it partly today and let me introduce myself the last but not least my name is anna and i'm head of mobile growth consulting department at appfolo um for those who probably are not familiar with appfolo is we are uh helsinki based app management platform for app monitoring and review processing so we're actually creating a one window solution for the whole app teams to cover both market analysis store performance analysis

features for app owners and product owners and marketers of course and we also have lots of features regarding their reputation management which basically is working with really using the stores so um that's pretty for the intro and i guess it's the right time to start one thing for the audience guys please if you have any questions please feel free to drop them inside the q a section we'll definitely cover them either during our discussion or at the end in the comments section i guess we will have a couple of minutes to do this so um to start with let me give a brief recap of on the topic um so since the release of ies 14.5 which has happened on this april apple introduced a big new privacy framework called app tracking transparency which is actually att for short from that moment all devices running on ios 14.5 and above was required mobile app publishers to ask for users permission for tracking and accessing

the device identifiers which such as like idk so it cause a huge resonance in the industry as soon as it's supposed to touch almost everyone like a mobile app publishers and developers the mmp um platforms and like so on so everyone was guessing what will happen after this policy will be implemented and actually the most common um forecasts and expectations there was that during that time was first of all many marketers were predicting the allocation of all budgets that are used for the app store applications promotions to the apple searches channel following the idea that apple will definitely introduce some significant changes and improvements of these user acquisition stores the second main opinion was that the web to our promotion will take the leadership inside the user position process inside

all app teams and the third most common thing was that unfortunately the user acquisition performance is supposed to significantly drop as soon as less knowledge for the users and for the audience will most probably cause the less user campaigns accuracy so such metrics as roy ross um pretend to dramatically draw so those are some expectations that was you know on the market and today we will cover uh those expectations whether they match reality or not and before going to switch our topic to the user acquisition performance changes let us start our conversation with this really important question what actually happened inside the attribution mechanisms and what algorithms are currently using the market so eden what solution was introduced to the market uh during these changes okay thanks anna for the question um

yeah we we share the same feedbacks and concerns exactly as you mentioned um i'll take a little bit recap like the the mobile ecosystem right it sits in the midst of an ever-shifting landscape right over that past two decades mobile has evolved significantly we all see it but however in the last three years we have witnessed the pace of change even accelerating most notable is the marked increase of changes around privacy and in the way data is being shared and consumed right now despite these massive changes mobile is still booming and the question is is ios 14 different 14.5 or now ios 15. is it different or is it yet another change that we're going to successfully overcome and there's really no politically right way

to say that but mastering scad network and conversion value is a nightmare yeah here i said it now we've invested much of the last 15 months educating ourselves our customers and the market about scan we focus a lot of our attention on helping advertisers learn the bits and bytes about the bits and bytes right and to complement to complicate things even further by design limits advertisers ability to measure post install activity now we've worked closely with hundreds of the most advanced advertisers across different industries understand how they are planning to utilize conversion values and what sort of capabilities they are currently missing and one thing was clear mastering conversion values has been challenging even for the savviest of advertisers and it really comes as no surprise that the

number one feature request has been to provide a flexible conversion value mapping tool and since he announcement that you mentioned there have been 75 product releases around and and the one i'd like to mention here is what we launched earlier this month which is called the convergence studio and what is a conversion studio it's a flexible conversion value tool that allows advertisers to customize and measure multiple post install kpis now let's break it into four very simple areas now with conversion studios advertisers are no longer limited to measure only one metric what does it mean for example a gaming app you're a gaming app you can now measure game progress let's say in-app revenue and ad revenue all together okay that's number one number two we're extending the timer without diving deep

into the details i can share that advertisers can now measure beyond the given limited 24-hour activity window we can also provide them to measure flexible ranges for example to say level one to three and not measuring only one level as it was up to now and i think that the most important thing that conversion studio is also giving is to measure a funnel which is really about measuring smarter advertisers can now measure a sequence of events to get a bigger picture and how you on how users are behaving and they can do it in the most efficient way right to measure with the least bits and bytes to optimize it and that's a huge step towards scan measurability and simplicity but we are far from done and i think everyone here will agree with me it actually feels like it's just the beginning and there's more features to come more updates more products but it feels like we're in a good place to provide some clarity to the ecosystem at

this time thanks yeah for sure this conversion studio sounds as a really exciting feature as soon as uh yeah this is uh as far as understood this is the brand new thing that you just like plan to deliver or is it live already it is live since beginning of uh september wow that sounds really exciting i hope that everyone like will be able to at least try it because like it sounds like it will have more data and yeah speaking of this thanks you even for such a detailed um recap of like these solution methods and like how it's the process is working on right now um the good point to to move is that um continuing these changes and of course the attribution models are not that accurate probably it's a very interesting question whether the user acquisition performance became better or worse considering these things so um alex how do you think like what are the changes after those like um i guess um 14.5 implementation in terms of user

acquisition like sources ones inside the app quantum so like what issues you tackling like what um happens inside your company yeah thanks anna for the question i mean i i i would like to continue eden's speech that uh we already tried uh such uh features from upslayer so uh i really appreciate it and would like to say thank you to the slayer team they did a great job and so we i mean it it definitely helped us to measure uh uh uh scott uh users so yeah uh in terms of uh user acquisition so it will not be like a revelation uh that our focus shifted to android uh uh and uh what about like ios uh before 14.5 we used i mean we only focused on uh self-reported networks such as facebook google or snapchat uh after implementation of 14.5 and

other version of us ios so we like we decided mostly to focus on uh probabilistic uh attribution sources networks so uh especially it was a good like not like with a period it was a good chance to dig that into into new sources to try something new so uh and probabilistic attribution uh like probabilistic uh sources it was a help to us to keep same uh spend or increase increase spend on some of the titles uh yes what about like i i would like to say more a little bit about uh probabilistic attribution considering ios 15. uh as you know uh they as they release private relay feature in ios 15 by default it's off but we will

see what will happen with the performance on probabilistic attribution sources uh yeah so like in in general like our effective spend is increased uh and we had a chance to deep dive into new sources and like all the team analysis team and uh product team so they just uh more focused on uh one product and yeah yeah yeah thanks thanks for the comment and actually follow up your um speech can i ask you like what are these new sources because like we have also these questions in the chat so yeah if it's possible of course yeah uh yeah sure nobody's uh dsp network such a loco for example uh applauding or unity so yeah which is uh those are like the other dsp or like some um actually like solutions

that's uh it's it's not that i mean like the direct traffic channels is just like the companies acquiring the um traffic yeah yes okay got it um thanks alex ah yeah i would like to add a few points here so the first thing alex mentioned unity unity has no idea issues if you don't know that and that can be very usefully used for those who who love their traffic so from our perspective as a again a hardcore game producer for the production studio so we tried uh apple searches for sure because it was a clear clear approach and we got really bad results we stopped that uh so nothing changed there at the moment uh and before apple would introduce or not introduce it's not clear yet their own traffic platform within the ecosystem i think we'll stop for that now so as

alex mentioned we see many companies distribute their volumes to android it results in increasing cpis there for sure and uh that leaves a room in ios for for those who don't need you know who has broad broad targeting and that's who benefits most from the current situation so as from from our point from our business perspective our volumes drop down uh especially take into consideration that facebook algorithm changed late late spring and uh make it more difficult to target users and uh again it had bigger impact on horror hardcore products rather than the casual or hyper casual for sure so we suffer from that so uh facebook now is pretty pretty a tough topic uh and we redistribute most of the volumes to

google ads so that's that's that's it i think yeah i i i have a little bit more about like facebook and other networks so a web to up like uh sending uh people to the lending landing page it can be a solution as well so you can use a feature from upslayer one or one link smart script so it's uh yeah it can easily smart banners that uh that solution no no no um okay let's keep it i mean let's keep it uh that feature so i i would say like uh there is another method uh how you how you can acquire users uh by using landing pages on on that landing page you can just uh uh like uh not like just a second uh like keep all the information from users and uh track them uh on the ops layer

you don't probably you know the proper name of the solution you have for that yeah for it's yeah alex has been is referring to deep linking yeah and i wanted to say first of all that i really appreciate the feedback and it's great to hear and i can see also from the questions in the chat that there's many more questions around attribution and challenges which i'll be happy also to to assist with i can also say that regarding what pavi just mentioned we we also see uh uh the trend that uh papa mentioned about the drop i i do want to bring you know the positive part into that as well that we uh see in the last month the overall spend in ios apps rising eight percent right so we've seen like that the in-app purchases revenue right eight percent and it even surpassed the pre-atp enforce enforcement days right in the u.s for example there's been 23 percent

jump in overall consumer spend since the end of july so it's it is challenging but we see you know the industry going in more of a confident way yeah sure that's actually a very nice insight um do you have like any thoughts why maybe having probably the seasoning theme or like what's um or or just like basically the people started to understand how to acquire users in this new reality like what are your thoughts about that so i i think there's a lot of unclarity right in the market a lot of and clarity and people are trying to like uh try new things and we can see it also across different verticals and we can see for example i'm if i'm looking at it i can see that in the past several weeks uh the share of ios in-app installs among hyper-casual games increased by 15 right and uh uh in the u.s casual games also experience around 10 but core games like on the other hand mid core and hardcore saw a 10 decline

in the share of ios and total up so that's what i told you yeah so now exactly so so there's different kind of trends and in in everyone would agree like that there's unclarity in in more features and more things are coming and i hope that that will bring more clarity in numbers to write okay that's actually an interesting thing and um like following this topic regarding like summarizing that uh the thing that we just mentioned guys um like the there was allocation of budget to android and actually the overall volumes of traffic a bit decreased but if we're speaking in terms of the metrics and kpis um are there any insights like what probably some new metrics appeared um in terms of like these new changes or like performance-wise what are like what happened with the keypads like the main ones pavel will you want to start yeah yeah let me start the first to start with uh i made a let's say the poll

among many many uh medium and the bigger size game developer company and most of them suffer i mean expect not not expect but experience uh lesser loss figures than it was before so the those who need better targeting they start losing money or getting less money than it was uh i think it has has some data for that for sure if you share that will be good so for the new metrics uh since uh neither scandal's card uh is is working as expected for in our case uh it's it's not a common thing but in our case it's not it's not that good so we went uh that was traditional analytics so added a few metrics we measure on a regular basis like consent percent and then really get a deeper insight into other cohorts we we don't know uh about it's like

non-ndva payment capacity multiplayer like uh it's not it's not a secret that those who press no uh normally are smarter ones so and they have more money and they have bigger payment capacity so and we should we're making predictions uh prediction ltv ross and etc so we should take it into consideration uh plus non-consent players uh also have some other multipliers depending on the country so it's it's really it's really not a simple question that it's like 50 percent those who press no pays 50 percent more no it's it's different depending on the country but uh adding this uh this mechanism and this analytics uh allows us to to say to [Music] to refresh and restart and uh actually increase the volumes on ios and this is the approach i would recommend to those who who cannot use the

industrial solutions for whatever reasons that sounds really like a nice insight but a very complicated one in terms of like how many um like what are the job positions for working on this model i mean like the analytics or yeah it's it's it's it's first of all it's analytics so it's this i mean uh the the the key thing is you need to predict the revenue uh you you have you have your cost that's not a problem you need to predict revenue and you should do it as precise as possible so uh any model which which helps you to to get the co part revenue at the moment the actual revenue yeah is is necessary for the ua team so and this is definitely a job for analytics department not for uae or product or something like that that's a also an an interesting thing that um these knowledge and like this the actual the attributes was shifted from

the user acquisition team to analytics and actually this is a very common trend that we also see inside our customers teams and snapchat alex you definitely wanted to to add something yes yes yeah i mean uh yeah on our site we definitely have like a little bit different situation uh our uh like our genres of game is literally i mean different and uh the metrics of first three or seven days is uh uh ross or like much higher than uh on on games such as rpg or partners so uh and what what i can tell like we uh already uh could work with the day one metrics or day zero uh so yes it was it was challenge for us as well but uh we i i guess we didn't had such a struggle like uh you know like game developers from from rpg

gender head so but what i can advise like what i can say like uh like keep not like keep track track and uh set what's the metrics on days you know if you if you would like to work with the sky on rpg genre or like on other genres where you have a pretty big ltv so uh i would say like let's uh let's take uh some events on day zero which depend on which depend on your loss uh on your targets so and just track that drag that event on day zero and probably it can help you to prove to predict and uh develop curve curve model uh prediction model yeah yeah but i guess it's like this is a nice advice but definitely it's useful for all general like game genres as well as mid-core and hardcore games day zero basically means nothing for them and

they still have these uh issues but um like a very final uh question like in terms of like this part of the topic um no one actually mentioned the apple store chats and i know that for games it's actually like one of them for some of the games it's one of the most um you know like important in terms of the volumes uh traffic channel like were there any channel um changes in like in terms of the apple search has complaints yeah move on actually anna i told you we tried able to chat no success i wouldn't say there was a difference between the experience we made in the past before this era so nothing changed in my opinion interesting uh yeah what about outside i mean we just try i mean it can be like first months when we're trying uh apple search ads so i mean there is no

news okay then uh yeah if anyone from the audience has uh like any other experience like yeah let's share in the chat and moving forward like we already discussed like these uh things and um there was also some like there were some questions in the chat section and i also have this uh thing in my mind like uh we already have like these changes and what are the next step for the attribution and what like what will be the next challenges for it and how potentially we can solve them uh there is like the person in the chat is asking probably it will um somehow connect to the um add fraud features improvement so even um how do you think like what will be like these changes and how you will tackle them yeah thanks and thank you for the for the chat i see a lot of questions about challenges and attribution and and i'll try to put like something very general okay so let's take like a quick trip down the memory lane to january to june 23rd 2020

when apple introduced the ios 14 updates right everyone across the ecosystem began asking the same questions right to some we already have answers they were asking one is cad network going to be a good way to measure marketing efforts or not the answer is really both you know so far they asked and pavel mentioned it what would be the opt-in rate before what would be the opt-in opt-in rate for the a pp for the uh advanced privacy the tracking transparency for the pop-ups like we no one knew we can see now that it's much higher than expected it's around 45 percent and another question that everyone asked like if most users won't opt-in anyway right why should i even bother showing the att pop-up on my app and the answer is because this simple sample of sets of users right will

help us answer many questions about the full set of users that's like answers that we have so far but all of these were legitimate questions back then but over time it became evident that the most critical concern and i can see it here in the chat also is how can i consolidate all the different all the siloed data sources into one single source of truth and it's challenging because just quickly giving the challenge other than scad network there are alex mentioned uh different attribution methods such as id matching aggregated advanced privacy probabilistic modeling each one of them has its advantages and disadvantages right so in theory you could just choose your model and based on your specific needs and you'll get attribution but in reality that's not possible and the reason is is that scad network the data is anonymized and there's no entity that can know if the same conversions were attributed also by

other models by the way it's also the goes in reverse so without going into the technical aspects it looks like and i'm saying it very slightly that the problem might be solved and by leveraging the conversion studio that i mentioned earlier we are able now to offer advertiser to deduplicate and create unified reality a single source of truth across different attribution sources to provide the clarity now this way advertisers could maximize all the advantages right of each attribution model without compromising and that's very important on the end user privacy now many of asking what's the risk what's the cons it will require to allocate one out of the six bits which is a very pricey resource right making the conversion value a bit more limited but the bottom line having 100 certainty in the true result of the marketing effort is mission critical right and doing that

with the scan remaining fully covered and accurate mean that no one is affected right and just to sum up i believe that in this time which is full of changes it's super important it's even crucial that accuracy and user privacy continue to coexist so that's my take on attribution at this time uh yeah wonderful actually like i have nothing to add it's like super overwhelming uh answer it yeah i i actually like following your ideas so i agree with it and like actually about these uh further changes um how do you think what we can expect in future in terms of um i don't know changes in the ua like the further one not in the attribution side but also for these uh product teams like what changes we may expect pavel alex um how do you think like for for your liking in your company's perspective let me start i think with this question i like it i like to talk about the

future so in my opinion the digital privacy trend is very strong and it will continue for next few years for sure so i expect that uh it's not it's a common knowledge that google will do the same thing like like apple did at the end of the year if i'm correct so uh i expect that non third party companies would have granular data anymore in the very very near future so the only consumer of this kind of data would be government authorities for you know you know where we're living so this is this is the reality this is the world so uh i would say to sum it up like in mid-term less than a year the granularity targeting will disappear at all no it won't it won't be available anymore so that's that's for sure again is a big threat for for

all applications that requires precise targeting so that would be very hard time for for any kinds of products like this and leaving the room for for quite broad uh broader products uh that's what i think and also one point to mention here because it's it's it will affect uh let's say market structure at the end of the day so the the battle royale between epic and apple so it's it's come is is happening now and uh it's it's very likely that uh the facebook would be one of the biggest hotseat beneficiary of of the result that's that's how how i feel it so the prices will the prices net will go up and this is the main source of the company income so but on the other side the gaming needs to also benefit from it because uh the more the more portion of the revenue will be granted to the gaming industry

uh the more would be considered invested in the advertisement giving more more uh outside keep giving uh leave it into more higher bids with the same rows and uh oh and games industry will consume part of the the marketplaces the physical goods setting ups and the service ups which which will not get the any benefit out of this uh this happened in uh battle i would say so this that's what i see in the in the nearest future one year actually like super nice points and like following the um the point that you just mentioned regarding this granularity reality and then think that probably in the future we won't have this data anymore for sure considering the google privacy oriented policy that is um i don't like two or three months uh time decision of implementing so i like the phrase that someone told me that before

the user position managers take the sniper wings to shoot and find the veils you know and now we took the shotgun and just like this is um perfectly for from from my opinion uh describes like this new reality that we have in terms of the user acquisition campaign structure and actually i like that you mentioned that um that fight between apple and epic because um despite the things that um like if we just like ignore that this is the apple but the thing is that we are the witnesses of something huge that is happening now um actually gusting is here to ask like whether they will implement these third party payment solutions even how do you think like relapse fire will track all these payments and like whether we'll be there any like difficulties for you to um tragedies actually yeah so so just to continue what pavel has mentioned i also like there's no question google will follow uh apple's way and and the way it's going from

now you you mentioned sniper to to shotgun it's going from device level to aggregated level and all adjustments should be done that way there will be much more solutions in the market there will be much measurement but to turn the wheel back to where it used to be that that won't happen if you're asking me if it's good or worse it's not that's the reality and the reality is that we're going to a more privacy centric approach and pablo said a few years i think that's the thing to go right that's how the the world would look like and we would have to find the the ways to help the brands allocate the budget in the right way by having not a very narrow device level but really more of an aggregated view on their different activities nice yeah alex do you want to add something here i i i guess guys already told like i mean i totally agree with them uh so yeah google or already on that fast to

the privacy things so uh but like i i would say that it's it's i believe that its effect not only like on mobile acquiring users on a web as well for example like they can uh create uh ecm as like google can create pcm uh private uh click measurement as well so and we we will see only on a high level high level attribution like campaign level or like geo level so yeah yeah i mean totally agree great so guys we also are a bit running out of time and i actually have um a like last question for you so let me just like share my screen back for the content details so guys um we have actually a lot of questions english in the inside the uh q a part so uh for our audience if you still have any uh

anything to ask please feel free to connect us and um eden alex uh and pawel if you have um just one magical wish for the apple and google uh what would you wish for uh for anyone to change or to to to introduce for them what would be your suggestions here let me start again so i would love i would love to see detailed uh app ranking algorithm i mean uh most most of the bigger companies probably know it how it works uh roughly but it would be lovely to see the details you're really requesting yeah i know it's impossible it's impossible but i would love this yeah and very nice i i definitely would uh say the same uh eden alex what what would be your wishes for that i i wish go back to the past time

but like uh seriously uh i mean i wish to be to be scott and future google think be more flexible uh be more flexible and uh or advertisers could set all the like or delete all the timers watch what they have yeah i mean more flexible uh flexible feature nice eden and so first let me thank you anna for this great panel and hosting it was amazing um and you know on top of this holiday spirit i mean i'll make a wish and um uh you know my wish is really not google once they start implementing the more privacy-centric approach and add new features is to involve as many

people from the ecosystem to hear comments to hear concerns to hear uh different kind of approaches and based on that make the decisions but i believe that transparency is a key factor to make people engage and succeed also with these kind of changes very nice and i you know guys while i'm asking because a couple of years ago i'm still i still have the same wish i was wishing that apple will introduce their a b testing solution for the creators etc so this year or probably the early beginning of the next year it will finally happen so i will i hope that your wishes also um will happen like probably in the next year or some someone else so just like uh i want to thank you as well for uh this informative and super fruitful discussion i hope everyone liked it and um once again thank you pg connect for having us today it was pleasure to be here hope that next year

we will meet offline yeah thank you i thought it was incredible i've been kind of really i've been uh being gripped with this um i i have to say i don't think you're going to get your wish about apple rolling back to the blessing yeah but everything else i'm hoping uh i hope thing comes through it certainly keeps us uh keeps us living in interesting times but thank you guys so much that was really really good to see some open frank debate about about about issues which affect all of us in the industry but um i wish you a a uh i wish you i hope it's a it's a profitable an exciting q4 for you whatever system we're working under and uh thanks uh thanks so much for lending us your time today so uh there's loads of questions in the past in the um in the q a so guys if you've got a question you didn't get answers quite a lot to get covered please reach out to pablo or edan or anna or alexa and i'm sure they'll do their best to uh to help you one way or another i'm now going to send you into digital ether thank you so much and a great job

thank you and great job mastering these difficult guys