Optimizing Graphics for Low-end Mobile Devices | Tommy Krul

hi everyone uh today we are going to talk about a topic which is really important and that is how to optimize player experience on mobile devices and this is something which i have been struggling with from a very long time and if you are building a game for mobile you know how big of a challenge this is so to discuss this topic with me is tomikrill he is co-founder of super evil megacorp chief technology officer and principal architect of a proprietary game engine super evil mail cop is known for their debut game when glory which was featured on apple stage uh i think iphone 6 and then also featured in iphone x commercial unleash so welcome tommy thank you thank you thank you for inviting me yeah and tommy is also an advisor in lila games and we have been having multiple discussions on the same topic so directly jumping on the topic what are the key issues of getting a good performance and

good visual fidelity on mobile devices which you watch from the start like fps poly budget what are those things which you look into from the start of the development well so let me first talk a little bit about why the com why the problem is actually complicated right um especially if you come from console gaming which is where i came from uh before starting super evil mega corp i worked at a company called guerilla games uh on on killzone another game they made horizon zero dom probably some people are familiar with it and i was on the technology team there i was also for a brief stint at white games and worked on league of legends so but the majority of my background is actually in console games and in console games you have a very fixed platform that you're optimizing for you know exactly what the hardware is gonna be you know exactly how you know how fast it's gonna run how hard you're gonna push it exactly what type of shader complexity you can run and so when you move to mobile

development and most people probably listen to this this podcast would know that not all of that goes out of the door you basically have no real point of reference anymore and it's a little bit back to the days of pc game development if any one of you worked on that where you know especially for a windows game you can have any cpu any gpu any monitor uh all the way from really low-end cpus to very high-end cpus and gpus and i think the problem on mobile is actually even worse um not only because there's a vast uh an even vaster range of performance characteristics from one phone to another there's also a lot of additional elements that actually affect performance so um you have different territories uh different countries that just you know where the majority of phones are

different categories of phones uh you know southeast asia is known to have a lot more lower end devices for example and yet they are a vast very large part of your like likely player base um so that is important to support some countries uh bias very heavily to certain brands right if your game needs to come out in china um there are you know certain phones that are mostly only used in china that are still that are incredibly popular there that you might not have access to here and then even if you have all the right devices different devices have very different trade-offs in terms of uh you know what what the hardware is capable of so some manufacturers um they will put extremely high resolution displays like very high dpi uh on a phone because

you can say you can print on the box like hey this phone has this many you know uh this this high dpi or like you know twice as sharp as an iphone and there's androids out there that have you know much higher fidelity displays than maybe your current like top-of-the-line iphone but at the same time um you know like that is not always tied to uh to hardware that is equally powerful right so you have a very high end display but a very low-end uh cpu and gpu and that combination of elements is actually extremely complicated to optimize right right um and lastly like and this is actually something that only recently i i realized um it's not just the combination of hardware also actually the country where you deploy

sometimes has an effect so the same device in two different countries for example if the climate is incredibly different right if you if you have a phone uh if you have a user who you know is in the tropics or a country where it's incredibly hot that phone will be thermally throttled all the time right whereas if you're in a country with like a mild climate this is not a problem wow yeah i think this is something which many people don't think of like personally when we target a device we always look at the configuration of those devices but not the geography like you know the thing you said thermal condition of a country that's very unique uh yeah so this brings brings me to the next question which is uh now that you have so many devices and i think the difference between console and mobile is the you know variety of devices you have so in ios that is still limited but in android you have like thousands of devices so when you are just starting the development how do you plan like which device to target and uh like where do you set the base

criteria it's it's it is a hard problem it's a hard problem personally for us we we do use i do use ios as a as my default skew and i think different studios have different approaches for this um i use ios as our reference sku right because it is more stable because i have fewer number of devices that i'm optimizing towards i can sort of categorize it i can say okay an iphone 6 class phone is my low spec why when a iphone 12 maybe is my high spec and i can use that as a reference frame and it's it means basically the same to any team in the world to any other engineer i can just tell them go and get this phone and use this issue means that go that phone at your high spec um i like to do that and i think that's convenient um other studios will argue like well android most likely is going to be

a much larger part of your install base and uh when it's a larger part of your install base maybe that is what you should be using as a as a reference for me the complexity with that is is because there is so little consistency from one device to another i find it very hard to use these as a as reference devices and it it just makes it hard to to optimize for it yeah then uh in that case how do you uh uh like do you plan for android later on or do you like you know uh set for example if you're just starting your development right now and your game is going to come out in a year so do you choose a decent device of today that tomorrow you know after year maybe this will be a low end device or do you start with low end device today itself that's also a really good question um we did different things for different games so in vainglory i think we weren't as as strategic around that what we ended up doing for vainglory was you know we we build and optimize the

game initially for ipad then for iphone and then we operated the game as an iphone exclusive for a while before we started porting to android i think even though we intellectually knew that android was going to be a very large part a very important part of our user base i don't think we gave it enough thought early on in the process so for our current game so we currently work on a game called catalyst black which which which shares a lot of like uh similar aspects to being gory it shares the same characters or same camera same perspective basically same type of gameplay it's a bit more action oriented um a bit easier to get into but it's still a similar game experience and what we did here and i i'm actually quite happy at how that worked out is that very early on in the development cycle we looked at the top 100 most popular devices for gaming and we ended up categorizing

these devices in like sort of like performance buckets right so that we had a good representative set of of devices in uh you know our low end in our mid and maybe our high end and our ultra high end and we use these devices as our uh call it you know reference devices for qa for performance testing for for all these things right right uh so one thing also stands out about you guys is you instead of choosing or any third party engine like you know unreal or unity and there are like lots of discussions going around which one is better for mobile which one is like you know better for console but you just wrote your own engine so what was the reasoning behind it or did you evaluate those insurance and then made a call yeah um it is a question i get asked very often and i think the true

the true nucleus of the answer is really that i did it because i could and i wanted to and if you're going to start your own company and go all the trouble of all the blood sweat and tears why don't you just do the thing that you're passionate about in my case my my core skill is uh is writing game engines right so um and i had been writing game engines and i had wrote the game engine that we actually ended up using for the company even before we started the company right so i didn't ride the game engine as part of the company it was a basically a project that i had in development for for a while and that i decided to sort of bring into the company um but i do think that there is there are good reasons to write your engine i it's definitely not something that i think is is right for every team or every every engineer but there are some good reasons for it um one of them which at least applied to our case is that um if you think back

when we started vainglory the industry looked very very different um i don't even think that unreal was available on mobile yeah unity was there but was pretty immature uni has made great strides in the last couple years but when vainglory came out it wasn't a very mature product yet it was it was getting there but there were a lot of issues um and really the industry at large uh you know i i like to joke and i'm not sure that it's 100 true but like it's almost true um most of the games that were out but when vainglory came out were match free games like they were casual 2d games puzzlers that kind of stuff and um

and you could do that stuff perfectly well on on unity but if you wanted to truly push the envelope you had to go to a custom engine um and our engine was written um you know with with all the sort of best practices of console engines in mind um all the types of optimizations all the the features that you that you would think of you know uh multi-core ready um we were also one of the first engines to be fully ported to metal when when apple came out with a metal api um you know when when we did a demonstration of that on stage for apple unity still had i mean they had a beta or like even alpha release of unity that you could download if you were a particular developer that you could use to port your game to to metal but it was it was very basic and it was very unstable and it was not

you know a good product jam we could do that we could we could adapt ourselves to any sort of new device that would come out cpu that come out maybe a new keyboard a new input mechanism on your phone um or a new graphics api and immediately uh adapt to it right and that for us um you know made a huge change and made a huge difference it's also easy to forget but you know when vainglory came out it was a fully 3d uh you know high visual polish uh game running 60 frames per second on an iphone 4. right like that i mean and i i i have a lot of pride in that we were able to pull it off and that came from a perspective of of really being able to optimize at the very lowest level for for what these phones were capable of doing and we were running maybe like you know 50 animated characters a large map

all unique textures over the map hundreds and hundreds of particle effects and at least at the time for that generation of devices if you wanted to pull that off yeah not only was your only option unity i don't believe that at that time you could have built that game on unity in the in the way that we were able to and so although it's definitely not something that is right for every team if you have technology that makes a difference uh or you have an idea that you can't execute well on an existing engine i i i think that there's a there can be a great value into you know building it for yourself yeah i think you brought up a very important point which is uh it was the context at which you decided to make an engine so you know i remember those days when i played vainglory for first time it was just wow and then you know as a developer you always wonder how did you achieve that because at that

point of time i i remember it was like coco studio for 2d games and then unt was still pretty new people would write their own opengl libraries on top of you know some existing engine to just get something out of it so that's why vayne glory kind of changed the perception that hey you can achieve this quality on mobile device so that if you wanted that if you wanted to execute that vision you didn't have any other tools but you know writing your own so you know uh when you are building a game building game itself is quite complicated you know you have to operate so many things now that you are building an engine so this is specifically for people who want to build something on their own they want to write your own tool how do you manage uh the development cycle like did you first complete your engine or did you made only the parts which were required for a particular feature of that game and then you know the parallel development kind of thing um also a really good question actually it's a little bit of both so what ended

up happening is uh i had written different parts of a game engine multiple times over my career usually i would write something at work not quite happy with how it worked out because you know timelines constraints you know you know other technology other libraries that we had to use at the studio and i would go home and i would go like now that i've seen a to z how this system works if i would redo it from scratch i would i would change i would keep this because that work worked really well we changed this i would change this or change this and that would work so much better and that is to this day how i still work uh i actually think that it's a it's a good recommendation for any any engineer you know build something once and then build it again and build it again and only the third time that you build it will it actually be any good because sometimes you need to build the system in order to understand what you don't understand yet about

building a society but in this case i had written call it all the core components uh of the game engine and actually what had happened i had written most i want to say most i would i'd call it like 50 of a basic game engine at some point in my career and that was the moment i decided to quit my job and i basically spent a year full time writing another 30 of the game engine so that after that year this was unfunded before we had a company but after that i felt i had 80 of the game engine um and and that was basically the moment where i probably thought at the time it was 100 of a game engine but it really wasn't but it was close enough that i felt confident to start a company at that point

and the rest of the game engine was kind of written as needed so but but i got it to the point where you could make a basic game with it before we started the studio uh and i think that was the right thing to do like it's very rare that you have the luxury to while you're paying everyone a salary and why you are sort of like on the clock for producing a product you know if at that point you're still writing major parts of a game engine you are probably you know you're in a problem you have a problem because you know this stuff takes time yeah um but in our case we had done or i had done most of the game engine before we started the company um but the reality is also like you i don't think you can ride a full game engine without writing a game that lasts like i say that you that i got to the 80 mark but it probably felt like it was 100 mark because you

don't know what that last 20 is until you start building something with it and and clearly now it's many years later and there's still parts of the game engine that we're writing simply because you uncover more and more systems that you would like to have the other thing that we i think did right was we did choose the game and the game mechanics to fit the technology that we had right so if if we had decided to make a game that had tons of cinematics uh you know in-game cinematics we didn't never have the game engine to do that like even to this day we don't have the infrastructure to build incredibly uh you know elaborate in-game cinematics um but then it's fine the games that we're making don't require them yeah so when you when you make when you use a proprietary game engine you optimize towards uh the specifics of the problem you're

trying to solve right and i think actually like to to bring it back to sort of optimizing for mobile game development it's the same thing uh you optimize towards the specifics of the game that you're building on and you know this is definitely the case uh when you have a proprietary game engine because you can you can go so low and you can go so deep into the specifics of what you're optimizing for but even when you're using a unity or unreal you will want to be very strategic with what you're optimizing for right um and depending on the game that you're making the content that you're producing the type of optimizations you end up doing are very very different right if you have a 2d platformer yeah you know like that is a very different set of optimizations to worry about than if you have a first like a yeah first person 3d shooter game right um you know yeah

that's very interesting because uh there is a fine line between engineering and over engineering and you know building a game engine itself is so challenging that you can over engineer lots of stuff and the definition of game engine is like you know do you want to write audio engine right now or you will write it when you need it so that's why you know uh what made sense while you were talking about it is like you are building game engine for a game so imagine if you didn't have a game and you were just building game engine then you have you don't have a milestone what to do and the thing is like building a game engine as a hobby project i highly recommend it to anyone on this call or is listening to this it is a fun exercise like i think it is a great you know way to hone your skill but it's also a project that can go on for years decades and many people that have written a game engine you know they've been working on it for 10 plus years and as long as you're not forced like being forced into a product is kind of like a mixed blessing it is

frustrating because you have to take shortcuts and you have to finish certain parts and you can you can't write that ultimate system that maybe you're dreaming of but it's also a forcing function to to uh to force you to finish certain things and to force you to solve certain problems even if it's not the ideal solution and therefore like i think in our case like our engine is a mixture of both there are systems that i wrote maybe 10 years ago now that i had the time for to truly build an incredibly elaborate um system that is very sophisticated that is very advanced that you know that i don't think any other mobile game engines including um will do and we can use those in our pipeline and in the types of games that we make but there's other parts of our game engine that are incredibly basic and they are kind of basic for a reason namely we don't need anything that is more sophisticated and actually killing something more sophisticated would be cost more time to build it also more

complex to maintain and the other thing that many people don't realize about complex systems is is that if the requirements change for that very complicated system adapting that complicated system is harder than if it's a simple system and so sometimes trying to write this perfect you know complicated solution that tries to cover all your bases the solution that tries to cover all your bases often doesn't cover all the bases and therefore like if a new basis is introduced that you didn't think of even though you thought you had you're in a much bigger problem whereas maybe if it's like a throwaway simple script that automated like a couple of steps uh you know you can just rewrite it start again at that constraint write a new system maybe completely different approach to doing it okay um yeah and we have many examples of that so

so for example you know vainglory really didn't have a collision system we didn't need it there was there is no collision in the game like it's it's all like you know uh rts like you know event based collision you select an enemy and you shoot a projectile to that enemy and you know there's a timing issue involved um but you know like there is you know the the the if it's a targeted ability the ability is always going to hit right um so maybe the the user has that has you know two seconds or like half a second to bring up a shield but they can't walk away to avoid pulling right and in the cases where we did have projectiles you know we did very simple just like you know very simple system which we we call the the physics system but it was really just a flat array of colliders and a flat array of projectiles and everything you just like intersect you know n times n every single interaction because at any given moment

there is maybe 10 projectiles and you know 10 uh colliders and that's it right so you can write something incredibly simple for catalyst black which is much more of a shooter game we have upwards of you know 30 40 characters on screen all of which have the ability to shoot at you know a very high maybe 10 bullets per second so it you know at any given uh and very large maps so at any given moment there is a very large number of projectiles on the screen and we needed a dedicated physics system and even there i ended up writing we ended up writing a physics system that is not what for example would come out of unity or unreal where you know your go-to solution maybe to say like okay well i'm just going to make physics halls i'm gonna put it in the physics world and i'm gonna use you know either bullet or built-in physics for one of these

engines and i'm just gonna you know simulate that in 3d in our case for example we could make 2d approximations of our projectiles and colliders because the gameplay is essentially being emulated on 2d surface so you know those calculations were also infinitely faster to do and we could all like paralyze them and there's a lot of specific optimizations we could make because we knew the type of game we were making it is not the physics system that solves every physics problem in the world i cannot stack two blocks onto one another but boy can i run a lot of like bullet intersections yeah that's a true agile way of approaching development so i i had another question which is uh if if you are building for mobile you have to make sure your game is performant and you know you are taking your performance you know at all the stages of development but when do you start looking at performance like you are just starting

development are you always critical about what systems you are writing or would that be overkill in the start or you plan like hey let's let's get all these features done and then we'll do iteration on performance or something like that yeah i think it's a little bit of both um i would always i think it's always good to engineer with performance in mind so never make decisions that will later prevent you from common sense good practice uh optimizations and the trick there is in knowing what those how those systems need to work right so if you ride a collision system because you are riding a shooter game it is important that at the beginning you know how other games have solved this and have solved this efficiently and that if you know that most likely

later you need to optimize it you write that system such that it is easily optimizable later if you don't add interactions to it or interfaces to it that are later that will later prevent you from making the optimizations that you do but given that so given that you're not doing dumb early on that that gets you in trouble later i would say first get to the point where you have the game by and large that you want to make don't make it pretty make it playable make it run right um use a couple of simple devices as your uh as your reference point so in our case vainglory was the you know i think uh uh iphone 4 min spec and iphone 6 uh high spec it might even have been the early days we ran on a uh was like an iphone 3

3gs or something like that like yeah so have your uh and in my case i use ios for that because it's nice and stable and i i know what to expect i know what cpus are in there with gpu's in there with memory is in there um but i would take sort of like a low bar and a high bar and and make sure that you know that those are reasonable expectations um and get yourself to a playable prototype that you can prove out the idea that you have because your game is most likely going to change substantially through its development cycle spending a ton of time optimizing that very early phase uh is not useful but once you start creating content you need to do this a with a very deep understanding of what this content should be you know how are you making your

particle systems uh or are your artists creating particle systems with incredibly complicated shaders are them or with extremely high uh you know polygon budgets are they're making 3d models with like you know single model 100 000 polygons right um probably not a good idea um once you start creating your content be sure that you know your budgets even if they are like ballpark budgets uh how you know your budget is also a bit of a it's a great question i think you know it comes with experience it comes with understanding and knowing reference games that have done similar things as you it comes with knowing your technology which i mean in my case i happen to know because i wrote it but you know it might be harder with unity or unreal in that case it might just come from your experience writing other games in unity on unreal kind of like knowing where the bottlenecks are

and then i mean in our case what we ended up doing uh you know for catalyst black we we did that um catalyst black was quite different so our current game is quite different from vainglory where the glory looked very pretty but everything was baked out early on and i did this very specifically because we were dealing with iphone 3 gs and iphone 4 which are fairly underpowered devices uh actually the worst offender at the time was the ipad 3. the ipad 3 was barely any faster than an ipad 2 but because it had a retina display it had four times the pixels and so it was a horrendous device to optimize for it was very very hard because it just like we had to i think average out like one or two cycles per fragment per pixel that you're rendering and anything over that

would just you know it would just we would no longer be able to run 60fps the the interesting thing is actually if you look at if you if you have an old ipad 3 lying around you'll notice that even in the operating system if you're just browsing it apple in in some cases can make the device run 60fps as well their internal teams had similar problems getting the frame rate up to par because it was such a mismatch of gpu and display um for catalyst black we went the other other way for keller's black rather than having everything pre-baked which which works really well because everything is fast and you can make really pretty graphics right like your your graphics are as pretty as you can just bake them um the big downside is that it's incredibly complicated from a authoring point of view we had tremendous trouble building new levels and building new game modes because

everything was so static it was the amount of effort and i really give credit to the artist working on it like the amount of effort to build a pretty level although it was very pretty at the end it was a vast amount of works like many many months of work just to build a new level because everything had to be flattened everything had to be uh uh merged together we would merge all the geometry of the level together in like big blocks so like it wasn't like a tree and a model of a tree and a model of a you know a fence or or a rock it was like big blocks of like maybe 20 or 30 by 30 meters that was a single mesh with everything baked in there the ground the walls the cliffs the vegetation a single mesh and single drop um for catalyst black we're making because authoring new levels was a an authoring new game modes is a very important part of the gameplay experience

we ended up opting doing a lot of real-time techniques lots of real-time shadows real-time dynamic ambient occlusion real-time lighting uh real-time reflections fairly high-end stuff and of course that stuff is way harder to optimize and i give credit to we have a extremely talented uh graphics engineer that that joined the team uh uh last year and when we brought up our android port we although we've made builds and made sure it wasn't horrible we had mostly tested on like mid to higher end devices and and even the mid devices when we ran the game the first time we're running at like 15 frames per second and we had to go through a lot of optimization to get that to the expected 60 fps uh and then we had a category of like low end devices that were truly running like you know three to

five frames per second that we have to get up to at least a acceptable 30 fps or 25 fps and there was a tremendous amount of effort to do that in this case actually our our android performance we did fairly late in this case also because we were still building a lot of these real-time technologies of real-time uh real-time lighting real-time shadowing real-time reflection ambient occlusion all these things came online somewhat later in the life cycle of the game for a long time we were basically just using the vainglory pipeline uh but without any of the pre-baking which meant the game just didn't look very pretty because the lighting was very basic yeah uh and so we started in this case optimizing the android build fairly late but and this comes back to the point i was making earlier we made absolutely sure that we didn't do anything

obviously stupid so everything that we had implemented earlier even though we maybe had not made the optimizations yet we knew what the optimizations would be and at least within a certain error bar i was pretty confident that we would be able to get it up there yeah and then there's always the mother of all optimizations which is just to like you know crank down the resolution of your screen yeah which you know is a very coarse uh tool that you have but it works incredibly effectively right so you know luckily we didn't have to do much of that but that was sort of my my fallback like if everything goes to i would be able to reduce the resolution of our game and at least make the game run on lower end devices yeah yeah at an okay framerate right i mean there is no one solution you know for for every every problem and these are something which you do and then you know what works best for

your game so that's why you know there are so many uh it's like really a pretty challenging problem to solve so this brings me to the next question that is uh how do you q and test uh against performance so like there are so many devices there are ios devices android devices then as you also mentioned that in in the start you were only testing against your mid-level or high-end devices but at some point you have to switch to you know the devices where you can support your game and now you have a list of 50 devices where you want your game to work so like you know how do you like manually test those or do you automate some of those tests like how do you approach this automatic uh automated what we ended up doing is actually we we had a so i already said that we ended up finding sort of the list of 100 most common devices yeah we ended up pairing that down to i think 14 or 15 devices that spent the spectrum that span all

the different cpu and gpu combinations that we were aware of and also that kind of covered the performance spectrum and and the first phase of our optimization pass was literally those devices we put them in a excel sheet uh i think horizontally and vertically we had a couple of locations within the game and then for each location within the game we had the build number and we were just qa i would just ask our qa guy like okay build you know build the game this build revision number for these 15 devices tell me what the frames per second is on this location and then we would make some changes and we would do it for the next build and that is you know like some devices at that point were already running 60 fps some were like 30 fps some were like 15 fps and at that point uh we started optimizing it such that we got all that whole set of 15 devices

to our i wouldn't say we didn't get them into a great place but we got them into an acceptable place and at that point we made the pretty uncomfortable uh decision to go life with this build in our public beta so basically we brought in uh our android players at this point the beta had only been out uh we're still in beta by the way on catalyst but at this point in the development cycle we had been out in beta for ios for a while but not on android and it was at this point when we got those 15 devices to an acceptable level uh that we brought in our android players and of course immediately what happened is a whole ton of devices didn't work we found many more that were performing poorly and then it was a combination of just listening to our player base and the moment that we identified a

device as particularly problematic either because it was crashing or because the performance was bad we would buy that device and just add them to our testing pool the other thing that is kind of interesting that we did is we instrumented the game right before this build went live because i did not expect those 15 devices to truly cover you know android i mean like of course i hope to pray that you know the game would go out it wouldn't work well enough but of course you know that it's never how that works out so before the build went out what we ended up doing is um uh we instrumented uh the build uh which means that we collect um you know i think we do different things and we can control it remotes so in different phases of the project we've had it in a separate setting or different settings but like some percentage of our player base every maybe 30 seconds

we we capture four or five seconds of fps data including the min the max the medium the variance we also grab the amount of memory that is being used on the cpu amount of memory is being used on the gpu we get the gpu uh model we get the cpu model we get the number of cores um if possible we get the temperature that the phone is running on like a whole bunch of things and we for this percentage of people we use essentially an a b test for this so that a certain group of players is in this profiling group and another group of players if we don't collect these these datas of course everything is anonymized and we submit that to our analytics system and we track this for a certain percentage of our player base and with that we can then run uh analysis on all that and see what we are doing

and there were some really interesting insights from that uh so for example uh i completely missed the whole category of gpus uh in all our testing and i think maybe this was just because we were biased looking at western games when we made our when we selected our test devices but there is a small number of the android devices that use your power vr gpus and although in theory i knew that because almost everything is mali and andrino like in my experience that is basically 100 of the market but it really isn't depending on the country that you're looking at right and we have some countries where we had a very significant percentage of users on some combination of uh of hardware that included android power vr and those power vr users basically the game was always crashing so initially the problem was like why

actually we noticed that none of our power vr users when we plotted our retention against our uh before uh performance and install base we saw that we had a big group of people on power vr that were zero percent retaining and and and either running very very low frame rate or you know right probably just immediately crashing and this came out of our analytics data because i just had never accounted for power vr hardware in any of our qa but also in none of our profiling even once we got it to not crash it was still performing incredibly bad because we had not defined any sort of default settings for this combination of hardware and therefore we would like just pick some random arbitrary mid setting default that really did not fit uh a lot of those funds right yeah uh i think two important points which

you discussed with me last time when we were actually uh discussing the same topic one was what you guys have done is you have list of devices and you have categorized them in a group and then you have configurations which you download on the device so if you know since it's single binary running on multiple platforms uh how do you control like this is the setting ideal setting for this group of device so that's that's just one thing i wanted to add in this conversion this could be useful for people it's like you know group your devices have configurations and then you know uh on the load of the game first load you can just you know pull that perfect combination for that particular hardware so i mean you can do that um and this is i guess highly dependent on like what engine you use or what technology you use in our case all our content is converted in a way so we actually don't pull down specific content for specific configurations all the one binary runs on all the way from like the lowest end to the highest end

there are some benefits to that there's also obvious downsides to that right if you are of a very low-end phone there's no need to download all the high-res textures right um and we've definitely thought about and and i would still consider in the future basically having a high-end patch that catches assets with their highest end settings for certain devices so that you know everyone downloads the same basic binary and some high-end devices you know pull down an additional set of assets that have has higher resolution textures maybe higher quality sounds higher quality video and you can make you can even make this optional right if someone doesn't have a whole lot of storage on their phone you can you know ask whether or not they actually want this in our case the way it currently works is we have all these different configurations and we have a configuration file that we ship with the with the build but the content is the same we are just

able to tune the content in different ways with either shader leds um which that is a big part of the the performance of course right like if you and this is very easy on uh this is very easy to mess up on uh on a on an off-the-shelf engine because every feature that you have is so accessible right you know if i want to write i don't know a post effect that is a depth of field so you know you can focus your camera on something and something that's very far away is out of focus our game engine currently doesn't do that we could add it we don't need it because you know we have a top-down game but this would be a deliberate choice for us to add it and in that deliberate choice we would deliberately think about the performance implications of adding that feature if on the other hand you're an off-the-shelf engine all of this is built in so your artist just goes in or even you go in and go like oh this looks really great turn this

feature on oh it looks really great with like six dynamic lights turn all the six dynamic lights on this level looks so much better with the ambient occlusion fullscreen m inclusion journey i'm an inclusion on and before you know it you have an incredibly expensive game that looks beautiful but also you've never had to truly think about the implications of how you're gonna make this whole thing perform and and how you're going to pull off all these visual effects together how they work together and how they will work performantly you know in our case uh we can generate different shader leds uh automatically generate shader leds so we start with a graph description of the surface of a shader and and in our convert process we can build different shader variants from that basic ideal shader and in that convert process we can say like okay well actually on low end devices rather than three

dynamic lights actually you're only going to take one dynamic light into account right um if for certain category of objects if your specular texture has very little detail in it we can analyze the texture come to the conclusion that there's almost no detail then for your low low bitmap just leave the specular texture out there or just assume that it's black right and and skip the calculation right so we can we can tear down uh or maybe on low end we say like you know what don't do any normal map calculations doesn't look as good but it may be fine right like if you run a seven year old phone especially if this is a rock that is just a prop in the level who cares that it has normal maps you're just running past it right yeah and so so we can we can automatically generate different shader leds and that's one way we are uh tuning in the performance and with this configuration file we can just categorize our devices and say like okay

well uh this type of device falls in this led category other things we do is we switch certain things on and off not just in terms of shaders but also in terms of full screen effects uh in terms of what resolution we render at how we composite things uh what full screen post processing we run uh all of these things we can show a tune depending on the pocket now this way of tuning performance for us works well enough i definitely don't think that this is the gold standard of performance because coming back to what we discussed earlier especially on android it is incredibly hard to bucket uh different devices so most studios including us we end up bucketing by gpu or by gpu and cpu combination um actually the worst that you can do is to bucket

based on phone model because i mean some of you guys will be aware of that but even the same model on android the same model phone will be able to come in many different hardware configurations sometimes completely different brands of gpus on the phone that is that has the exact same name you might walk into the store you might buy a particular phone and you might go home and you're fine your phone might have a molle cpu uh gpu rather than an andrea and you wouldn't know even by the serial number you wouldn't know right um and this is of course extremely hard to do so rather than bucket based on phone model most people would bucket based on gpu or gpu and cpu combination but even that is tricky because you know we for example talked about you know if you live in an extremely warm climate that phone will be thermally throttled 100 of the time yeah and when the phone is

thermally throttled it's running at 40 power or 60 power right so if you make your game run 60 fps and now it's running on 60 of the power now all of a sudden it runs like 35 fps or 40 fps the huge drop um or worse it goes back and forth right the player starts playing has a ton of fun run 60 fps 10 minutes later it's running at 40 fps um so i don't think that there is a gold standard of how to do this um some sort of dynamic bucketing i think would be uh even better right like if you can keep track of your performance characteristics of your of your game and dynamically sort of reclassify maybe you start with the default but but if your performance is not not up to it you could sort of like essentially degrade

the performance settings over time uh you know depending on you know whether the phone is running particularly poorly uh we would definitely consider doing something like that and there's definitely cases where something like that would help a lot um another thing to just do is just expose the quality settings right like it's basically back to the pc days where when you had a game the first thing you would do is you go to the settings menu and you you know tune the settings of your game to what to your particular pc um and especially on android that that is still something that might be good because uh you know some devices might require that because it's just very hard to exactly pinpoint correctly where they fall the performance vector and also some some users just have different preference right some users will say like oh i want my game to be really pretty and i'm okay if it runs 45 50

fps as long as i have all the reflections and all the shadows but the other player is like hey i am competitive i turn everything off i just want a straight 60 fps i never want my phone to get too hot so that i can conveniently play it anymore i'm okay switching all the features off just make it run really really fast so different players also sometimes have different preferences yeah i think it comes down to like how much can you do and some of these optimizations you can do after the game is live so now you know that these are the uh this is the user base and now i will just take this bucket of users try to improve performance for them through this dynamic dynamic configuration and then you can keep doing that you know keep bucking your live players so uh like people who are listening to this they might think like hey man this is so challenging like you know there are so many attributes and so many scenarios for performance optimization on mobile and that is true but it's like you know uh be agile like you know build the game test it and then know

your audience know your players because there are there might be 100 standards in the market but those might not be relevant for your game and the type of game you are building so really uh useful insights tommy which you shared so this brings me to the last question which is actually not related to this topic like performance optimization but this also comes from your experience of building a studio from scratch so like you know you uh you came up with an idea of a game and then now you are you are assembling a team and then you start working on the game so what advice do you have for companies or even within the companies when they are setting up a new team for a game what what learnings did you uh have while building this team from sketch for this game oh man like where do i start um so many so so many or maybe the most uh critical mistakes which people can avoid so that could be um okay let me just go over a couple of them what would i tell myself my previous self um

a common silicon valley wisdom which i now truly understand is you will need more time and more money than you think when you start this project so whatever amount you raise and whatever amount of time you've given yourself even when you padded it is not enough it's just not enough and you're gonna face that problem and you will have to deal with that problem um i remember i think that we were actually lucky in this regard it's a funny end ago when we raised our first round for super evil we went out to raise 750 000 and we thought we could get the game done in eight months and our first investor being ex infinitely more uh seasoned than we are was like came to us and he was like don't take

just because it was from him that we were raising the 750 000 he's like don't take the 750 take a million you will need you will need a million minimum and we thought like oh you know why would we do this extra dilution you know but he was right and we needed more than that like we you know we ran out of money by the end and you know although i think we ended up we had a game in beta after a little under a year which i still think is very it's pretty impressive we were a very small team maybe five or six people at the time built the game basically from scratch including the engine including all the art uh all done in-house no outsourcing um just through hard nail blood sweat and tears hard work blood sweat and tears um but we did have to raise more money like we were we did not have the budget to then launch the game and we were lucky that you know we had a game that we could put

in beta and that showed promise and people were liking it but had the game been further behind had the game not resonated yet with players i don't think we could have raised more money and super evil would never have existed um so that would be one you're you're gonna need more time right you're gonna need more money uh and therefore there's also there's no time to lose don't spin your wheels with things that are not important like get get to that game that you want to build because getting to that point will prove to you all the things that work about the games but more importantly all the many many things that don't work about the game and that you still have to change and fix and you know if you get to the point of of of discovering the areas of the game or the product or the technology that don't work by the time you ran out of like 80 or 90 of your money you have a very big

problem uh other learnings building teams is hard building teams is incredibly hard hiring is incredibly hard uh there's not just like hiring the right skills but also hiring the right personalities um evolving those personalities over time is incredibly hard the type of people you need early on in a company is there are very different than the type of people you need at a mid-sized company and again different from if you're a larger size company or when you start your second game this was definitely a very painful phase that we went through not too long ago when we when we transitioned from vainglory to catalyst black you know like when you do your first game it's a little bit like the first the first album that that a singer makes right you put your heart and soul in and you know exactly what you're making right like you had all your years of working towards that one game you had in your high

in your head all the founders are aligned around this one particular game this one vision that you all had what if you make the game and it's not as fun as you thought it would be and now your three founders or you know maybe a team of like 15 people all these people signed up for that original game that you said was gonna be great and that's the game that they spent their you know their energy on that their their their their passion on and and that's the game they signed up for making if now you go like actually you know this game didn't work really well i mean you guys are working on a shooter game right what happens if you guys are you know we try to shoot your game and it doesn't quite work we're going to make an asynchronous game and like half the team is like well i i really want to work on shooters what the man i thought we're going to make shooter games and maybe or maybe you say like i want to keep making shooter games because that's all the gameplay that we've made and your business partner is like but we look at the you know the the metrics and you know the progression part of our game works

incredibly well and people are loving it but the shooter part no one really cares about and this is not a part of the core mecha like it turns out that this is not as part as important as important a part of the experience as we thought it would be and that misalignment then like trying to reinvent yourself incredibly hard incredibly hard um other learnings and i could go on for a while i mean yeah this is something like pretty challenging for everyone but the two points which you discussed uh those are like real challenges because no matter how [Music] close planning you do and no matter how much you know you define breakdown your problem but you always miss a target even a smaller target like you know you plan a sprint for two weeks and most probably you will not finish everything in two weeks so think about you know plan for a year so that's that's really a challenge which i can imagine like people facing and the other one is like hiring we are facing the same challenge you know hiring against culture

uh hiring against the belief system because when you are just starting everyone has to believe in the same mission so it has to be more than a job for for the people who are whom you are hiring for the yes right so thank you so much tommy like this has been really helpful and lots of things discussed uh this is not an easy topic to cover in in one podcast but this definitely gives some idea about like how people can approach this problem and uh plan for it so i hope so i hope it was useful right thanks thanks for being here tommy thank you so much yeah my pleasure