Lessons learnt from converting parts of Apex Construct from PSVR to mobile Gear VR

[Music] so hi guys thank you for coming this afternoon Pavlov Riley I'm director of developer ecosystems arm and having me Christopher benjaminson city of pastoral games so grab this and hope that I can use it so we're going to talk about how we managed to create some console experiences on mobile using APIs construct which is first our games latest title so I just wanted to give you some context of you know what we are here I just want to give you a quick talk about quick chat about arm so we are you may know arm about the company who makes processors for mobile devices we are pretty much if you have a mobile phone in your pocket chances are the processor inside is powered by arm we also quite big on on mobile GPUs so

our multi TPU is about 50% of the of the mobile market and last year we shipped about 1.2 billion GPUs so obviously gaming is quite an important part of what we do and when we were looking at gaming experiences we were but this time last year we were thinking so what would it take to bring a console like experience to mobile what would it be what would it take what would it be involved so around that time we were talking to Christopher and we found that they had the same design vision so we decided to work together and did a great project that we're going to talk about Thank You Pablo so first well games is a video game studio in Stockholm Sweden and APIs construct Opik of this presentation is our first title before we jump in to show and talk about how we got it around on mobile VR does have a quick look at game

I really hope you're ready for this [Music] [Music] okay so a peace contract is a game that is built for the yard from the ground up and what we mean with that is that we require motion controllers and positional tracking for your head for the game to be played you used your hands when interacting with the environment and you also use your hands when using the bow and arrow our combat mechanics we mix high paced action with slower paced combat no space exploration and puzzle solving and it's

being launched on ps4 and PC so why are we at fast games interested in mobile VR there also lies in our technical vision in that we want our players to be able to play immersive experiences wherever they are whenever they want and not be bound to a specific location what's really interesting with mobile VR is that it has the potential to offer the same experience that we have on PC and and console there's obviously a lot of ways to accomplish this but we believe that low-power CPUs and GPUs or the best option right now and the thing that we're kind of waiting for is positional tracking for heads and hands so this apex construct on mobile even a feasible it is soft world title built for PC and console and we require motion controllers and procedural tracking but if you look at the

performance range that you have in the game going from a base piece for up to a high-end PC that range is already quite broad and we had already built a lot of tech and techno technologists you handle that range and scale between the the different platforms so we felt that adding mobile to the mix should definitely be be doable and also mobile CPUs and GPUs are really quite powerful so the the gap might not be as big as one might think so for us this product was a first reality check we heard the game it was done it was running what would it take to get it to run on mobile VR what kind of sacrifices would we have to make would we be comfortable we're doing those sacrifices what bottlenecks would we run into would it be similar to what we saw when optimizing for for console lots of questions but all through this we had this working a theory that even

if the actual fidelity when comparing screenshots side by side would be different it would be lower on mobile they perceived fidelity inside the headset would be similar and that is really all that map that matters as I said we've built a bunch of tools and technologies to help no scaling the games I'm going to mention a few of those first up is sub levels you can think of sub levels as layers in Photoshop it allows us to divide a level into separate files so different people can work on the same level at the same time we then have an automated build process that merges these parts together depending on which quality target that we are going for so this allows us to have much more detail on a higher end that for instance thing we do on on lower end next up is bucket streaming and it's not really a streaming system in the way that we load and unload data at runtime it's more a way for us to

control exactly which assets that are active at any point in time so as the player plays through the world we enable and disable different things and the theory here was that we could save a lot of performance by making sure that the engine only had you know consider as few assets as possible and this was really big help to us next up is quality set selectors it works on the same premise as bucket streaming but on a single asset so it allows us to control no parts of an object I should be enable or not depending on the quality target so if you take an affair as an example we generally have much more particle systems running on PC than we do on on mobile in epics we use a modular mesh set to create our environments it's an awesome and great

flexible way to author content but it comes with the cost with a lot of drug holes unless you can batch them efficiently and again instead of relying on the engine to do that for us we create the tools that gave the power to our artists that they could use their domain and context knowledge and make the decisions of what and which object that should be combined and patched together and this allowed us to reduce draw calls a lot I'm not going to go into to physics let's just say they can save a lot of precious CPU time by making sure you have sane colliders for objects right so when we got involved this project we had to do a fair amount of content optimization on mobile and in arm we have a number of tools that we we use for this purpose we have a called Emily DP to suite that includes an an offline compiler that allows you to compile your

shaders and predict performance we have a Mali graphics debugger which is a debugger and profiler for the for GPU calls and on on your give you API calls then we have stream line which is a GPU CPU profiler that lets you see Hardware contours and on a timeline so when we looked at at this problem first of all what you try to do is to profile the content to understand where is the issue which assets are giving you problems what which frames are the heaviest and then what you profile the content you understand the where the issues are and then you decide how you're going to optimize it and optimization can be quite a creative process because basically you can do pretty much anything anything you want in this particular case we were trying to optimize the content in a way that didn't affect the the visual experience so everything we did was trying to remove load on the

hardware that was not contributing to the visuals on the on this on the game so on one hand with these mesh simplifications in taking the the mesh load from about 2.4 million triangles to 1.4 million triangles so that's about 40% down and that reduce our our vertex load significantly the shaders the default shaders on the game where were standard surface shaders with a big lightning so that was quite heavy for mobile again so we replace that with optimized versions that as you can see pretty much didn't impact the visual fidelity and then we draw calls is a really significant issue of mobile particularly in high quality mobile gaming like this so for to reduce the number of drug calls we looked at occlusion culling so removing rendered objects that would not actually be in displayed on the screen because they are occluded and unmerged some meshes to reduce the

number of turkle's even even further so the other aspect of the project was optimizing the content on one hand but then actually how to replicate the game the gameplay experience or on a mobile device with the constraints we have on a gear VR headset so one of the key issues we had this of the gear VR doesn't have tracking head head tracking so we implemented our own version of head tracking Wheesung AR core and the Samsung phone sensors we did the project integrating all of that into the engine and we have the resources at the end of the article if you they talk if you want to have a look at it it's all on our website how we did it on source code and it's quite neat and the result is is like this this is vanilla one of our developers using the head control just on the owner gave we are using the front or the rear facing camera on the on the phone and it works very well for the for

the demo it's probably not you could do better with dedicated hardware and obviously we get in there but it does it does a it did the job so this is what it look like in the end on the left hand side you have PlayStation VR on the right hand side you have gear VR or or on version of the of the game and you can see the visuals the assets the whole scene is is identical the gameplay is is also very similar so there are not I think the differences are very very small the only difference is when you come to this this point where you have to actually shoot because we only had one controller in gear we are we couldn't do the born arrow effect so we had to shoot the evil robots instead of throwing arrows of them but apart from that I think we were very pleased what's coming up in the future to help us get even closer to where we really want to be Vulcan is a low level

API if you know familiar with it is is going to over time replace OpenGL ES Mobile API for particularly on Android and it has a number of advantages over hourglass it improves the efficiency of the CPU by making better use of multi-threading on mobile devices it improves the number of the use of draw calls there are things like multi-view particularly important on on a mobile VR that helps reduce the Drakh whole account even further coming up in version 1.1 and there there are other bits and pieces on things like 48 of rendering and so on so in conclusion from our point of view I think there's pains has been really positive we got very close to what we wanted I think looking into the future things I Balkan and obviously better Hardware more powerful hardware and

all-in-one headsets including dedicated hardware for things like inside out tracking and uncontrolled and so on are going to make a huge huge difference I mean from our perspective again coming back to the point about you know the perceived fidelity and this project really showed us now you can really reduce a lot of stuff and can no really create a simmer civic spaces or mobile that you can on on PC and an console you just need to know your target hardware and how to massage it the right way has so to speak and really pay attention to performance all along the way and they can create some some really cool stuff so a lot of these learnings are obviously going into our future titles and if you want to know more here are some sub resources and online suppose I project the word we did the tools we've used and so on that's all it's all available online so go have a look at it that's it he's eager going to

say you have time for questions but you're eager no problem here hi thanks for his talk he's in your experience the mobile tracking using the phone camera good enough for accurate head tracking or are there any problems regarding for example lighting or some other Amir manufactures I will say that as published said it was a great experiment but it's not on the same quality level that you have with a dedicated hardware so in other if de vive ously much better than that yes I think what we were trying to do is to prove the point that it can be done and and one of the reasons we try to do it that way is because if you think about a our core and all of the you know the api's and the vision system is adding load to the hardware so it has to manage all of the gameplay all the visuals plus on top of

that the input from a high resolution camera to do a job like that so it wasn't interesting for us whether we would be able to fade it and do the whole gameplay at the same time and we could and a follow-up are you aware of third-party positional tracking hardware such as anti latency which is a sensor that you can adjust to gear VR yes have you developed in moving this we haven't yet an away from faster games perspective we are kind of waiting for know the stuff that comes out of the box that's what we ought to know targeting prior primarily but who knows for me it is after doing this it's definitely a possible option yes Thanks more questions maybe put myself here so I can see if there I do have a question

while they think about it Oh market wise because you were used to work with PC and console and you had worked with VR before but now mobile I understand what do you think is gonna be the impact business-wise on the market with your new with the new release I mean this wasn't a release of the game this was just a experiment to see if it was possible to do but if you ask me what I think about mobile vo then I think that's it's definitely going to be you know the type of hardware or the type of video that's gonna hit mass-market the quickest it has the potential of being and cheapest but also being more accessible so I'm a strong believer in a mobile VR sorry actually sorry wow I wanted to play it I'm afraid you can't it's it was just a demo we've shown in a few in a few places we showed this demo our TTC and few other events but it's

not published and it won't be published did you guys tested with public to get feedback do you have feedback about the project yet so the people we got in those events is it's very positive I think people will particularly like as a specific size the mobile aspect of being completely untethered so you can move around you not have to necessarily look to you in the case of a PSP art you don't have to look at the screen so you can move it face backwards so it's a lot easier all the freedom that you don't have on a tether device and I think that's really very important you can't underestimate how much of a difference makes any other questions yes a second hi and thanks for the talk how did you coat the unity sublevel system because it was in one scene right and how was the sublevel it's it's

basically built around additive scenes so we have our own framework kind of loading and unloading nothing's [Music]