Building the Stage for an Immersive Theater VR Experience

[Music] okay well it's a pleasure to be here please turn off your cell phone sounds they asked me to tell you I stopped good afternoon I am hi Megan Saloth Vallejo I am 3d artists at tequila works and I'm here on behalf of the art team and I come to tell you our experience in these unannounced be our project that we are currently developing tequila works it's a game developer studio located in Madrid and established in 2009 by son of the former employees of these are entertaining miraculous team Peter Studios or Sony between others 2017 is

going to be a remarkable year for tequila works due to sexy brutal and drive releases and now with Game Stop as partner gang trust the company is developing its first PR project the team has been very small since the beginning and nobody had previous experience in VR and this is the whole team last summer visiting the local museum and taking references Madrid the Machine of Romanticism and this is the current team last week the game core is developed in the studio as well as the environment and considered characters motion capture animations and audio they are all hot sourced these are the tools that we use and those three on the right side are the platform that targets in 12 months we have to quickly create a an island

big mansion on top of it and a lot of furniture elements and assets all without outsourcing support our rates and previous experiences are quite different as well as our workflows and on asset creation in general but the good thing about the small teams is communication and something that we need a lot on this revolutionary challenge my previous experience based on games with hyper-realistic this way style confront you I learned how to build first-person shooter environments with a high level of detail and physical base textures I was lucky to be part of the cinematic team at remedy so I could work on close to the drama directors and the animators and go through the motion capture process in Gears of War 4 I work on the multiplayer map son of them and I faced several

optimization as tasks in order to run the game at 60 frames per second on Xbox one so I arrive to the killer and they told me that I will be working on a real experience that is a first-person shooter but without weapons and a higher immersion with characters based on real actors and motion capture animations and environments that seem to be running at 60 frames per second on a ps4 I was happy and excited to join this team and this particular project a project that is not a game that the players goal is not about participating on the story it's about discovering it this is an immersive VR theater experience directed by Raul Rubio and written by Rob Diaz calm as I said a mansion at the beginning of the last century with an

American neo-colonial design avoiding a victorian baroque approach these are the first sketches well the story script was in progress still and later the environment gets shape from the script both of them untouchable since motion capture sessions finished the layout never changed so reproduction was hard useful here a comparison of an old concept and the current state of the of the environment we try hard to respect the original ideas and focus only on the addition of details here an example of the layout first the concept art and the first past of the lighting in game we use several paint overs and interpret action because they give us a quick and refreshing idea of how the environment is going to look and everything evolve slowly literation the iteration final

result here richer in details and lighting the art direction here we have two early 3d concepts used on the demo we consider our chosen art direction to be placed in the middle of cartoon and realistic and so the project aimed for a consist look with real sage colours lighting this physical lighting is the most valuable element on the final add since material and textures support this rendering method but the level of detail are not actually realistic we want to create comfortable and calm atmospheres and we just prompt the player to to be focused on the character and this be our experiences ask and deserve nice immersive environments this chosen art direction is not only artistic it's also

consequence of BR constraints 60 FPS zone on ps4 is gives us a lot of limitations and headaches so the process starts stylizing the textures and shapes getting lower resolutions x + triangle cone a correct physical base lighting and materials to the resin we achieved the desirable atmosphere in the project we qualify the noise as an onion you can easily get dizzy with VR and one of the main reasons is that noise and where our direction asks to delete it in every single texture mmm this decision was easier to take at the beginning since performance was not not allowing us to use any anti-aliasing method but recently we could add at least the temporal anti-aliasing so we are less worried about this issue

concept our Oscar questa our director and lead artist was the first member of the team well not the first member of the art team and he was the only artist during several months working on own concepts and the layout of the environment he used sometimes this is symmetric perspectives of his artworks and I think that in a very successful way our confessed artists aleeah's sent it she feels comfortable with paint overs on top of office screenshots and his ideas are many times necessary for us in order to progress on the room's iterations he also enjoys the use of 3d code and its scope and rendering features he quickly creates 3d shapes apply smart materials and paint overs are even easier as demonstrated on

all these concepts we are facing high contrast between the different locations in the map and now the environment art my first task was the development of the assets creation pipeline and this is an example from real picture references we design an asset with softer shapes noiseless optimized as you can see on the textures we aim to use horizontal and or vertical tile and information and if required we work with non square files the goal at the end is is to use the minimum textures resolutions always keeping a balanced and proper Texas radio between 512 and 256 pixels per meter that's I think it's very important for us basically or textures are clean with correlate correct I'll video values and low contrast roughness is richer and information but always avoiding that

pixel grainy noise and normal Maps takes they take an important position with clean height values on surfaces and baked information on on a small objects you can see all the small objects that they are packed on the top left corner and unimportant narrative objects we push the quality and time effort of the textures trying not to use styling textures this time I decided to use overlapping UVs symmetrical pieces so I could work on the want edges and contact areas without big texture resolutions the cubital map the baked map is a big support and here but the visibility of this details must be subtle to keep up balance with the other assets and environments you can see for yourself that roughness metalness and being a Christian and they are pack on a single texture that colorful one we decided to

add the ambient occlusion here since we will not have any real time and in occlusion in game therefore we always use a neutral gray as a specular and you can finally check that the mesh corners are rounded but the normal map doesn't have any high poll information baked this is because I used on this asset the medium poly workflow this workflow has been used on island in isolation and star citizen between others there are games with big objects weak acids and surfaces it's based on the creation of soft bubbles for the corners avoiding the hard edges and a low poly look and we realized that we are not only saving textures here and locals with this method but also getting a better pixel resolution we have this way less seems more tiling textures better silhouettes

and faster modeling times maybe employer flow on the left and typical high poly baked on the right side and after all this high poly a normal map tech is not needed with a slight movement had a head of movement in VR you can easily spot the LEDs that they are swapping although we have higher amount of triangles we decided not to use LEDs resulting in memory savings at the end here one of my favorite assets a bit one that I made with only one key texture and and a good texture density from my point of view the best workflow is the mix of both low poly and medium poly workflow spend the minimum time possible modeling high poly parts and use tempers and edited normals as much as you can on big elements like the support of the telescope here and

this typewriter was made by one of the artist Jose Randall on these cases where the object is a small and full of tiny details the medium poly workflow is not suitable due to the triangle count besides without using LEDs we get another challenge for the GPU a lot of triangles and in a single pixel so this workflow must be used with caution and this is related to lighting more or less without LEDs will also save memory with less light Maps to bake prepare and light maps are crucial if we want beautiful environments and with all these limitations so we take them seriously we create a second UV channel manually and and we give them the maximum quality possible here the light

complexity view the lighting artists worst nightmare and then dynamic lights are maybe the most expensive feature on on this project but they are necessary for the character lighting this kind of lights work on a different Channel than the environment and it's is a challenge for undress him if I were lighting in lighting artist to place them all around the map without overlapping on each other so character art there are several several main characters in the game and a direction needed to be clear enough to they were an in-depth study of all of them these are the oldest concept designs is unbelievable how how they have a ball I'll show you the progress of this particular character first considered based on the original idea there were also some 3d concepts and tests in zippers and at the end after knowing the

actors involved on dimension capture sessions the final design based on them this was possible thanks to these concepts and the references and continuous feedback from Oscar they are collector and solid annum the company in charge of the captor and the animations a French company the actors were also scanned in 3d to help a lot in the in the design process and with motion capture we are we are dealing with realistic body and facial facial animations the models and textures were outsourced 300 character artists from different part of the job different parts of the world did the job any video game production that deals with these working methods they faces a common issue that is the uncanny valley

in VR this is even a more relevant issue due to the immersion the player is there is close to the character looking at he or see from different angles and distances so we need to avoid the player to feel uncomfortable and well distal eyes art direction relief this problem achieving our balance with the environments something I want to mention here is the use of the Unreal Engine tool sequencer it's a good choice for us because we can easily detect animations on the viewport and how they are working with environment of the player we are developing an experience we have to focus on the narrative animations and characters but this time we also have a different player one that will be intriguing the VR device bite

for example goes everywhere you can hide anything what if the player decides to jump or what if he or she decides to lie on the floor and if the player tries to see what is inside the word close to Quadro for example we saw him the the darkest black for this video we need to make collisions for every asset and this collision will also be related to the teleport feature we need to be in constant communication with the game designer Pablo a latitude well here you can check the teleport feature how is working and clearly the moment the character is crossing into the into the player the built-ins softly to black as you can see is for these kind of issues that the corridors tend to be thicker than this one so I'm talking now about the scale it

has been an interesting element of the on this adventure we check the asset size many times before it's done on the undeterred you can appreciate that that the environments are like huge then you put on the glasses and the proportions are now almost okay at the end the main objective is not to bump up against the characters we have lately some good examples of Kings with incredible cinematic sequences no matter if they are first-person shooters or third-person shooters on our gameplay we don't have cameras and our single our points of view the player is free to go and decide from where he would enjoy the story so as well as in theaters we need to create larger environments even in saturated and we aim to create like comfortable comfortable spaces without obstacles

another B are related concern the textures as in vitamin artists we are worried about those tiny details on the assets but is disappointed when when the VR blurriness and lower resolution don't let you appreciate the props when they are too close to the camera well we do you right now we have to decide what is worth it and to find more pleasant pleasant designs and we are all glad of using the negative myth tools for the textures creation with perforce linked to them we can share all the new materials and effects in seconds and well on a stylized games like this sharing base materials between the artists like wood generator we believe his suitable for achieving like a good consistency between the different assets

in the ring a feature created by Norman s car technical artist the rain is made out of a single static mesh I guess we do don't appreciate water drops but well you have a better representation on the right side pivot point positions for each raindrop are stored as paired attributes and they say there make sure that they followed the camera properly everything is handled in the GPU to free up the CPU from updating the raindrops every frame and at the texture is also used in order to avoid the rain to clip indoors we take that the texture to reposition the three Leafs on the exterior floors this tool was created by Mary open meadow lead programmer I'm glad you're here on

this talk because he is currently having another talk in another room he's also another speaker well the ranking with some consequences for example the water drops on the glass but before the technical artist found another issue at the time and real for glass solution was very expensive and it requires to compute the entire PBS later to to achieve strong reflection in the glass that reflection however was a cube map with no parallax correction and a less physically correct but way cheaper and better-looking solution was to completely drop the PBS shader sampling the cube map in an elite shader and do the parallax correction their technical stuff having more control over this also Louis to to sample two different cube maps for the glass window so a different

cube map was so was in depending on on which side of the glass you were looking through one more time substance designer allowed me to click create a procedural texture taste and the water drops the ring is one of the special effects we have thinking but all the performance issues and limitation makes impossible to use of the use of real-time simulations or particles so both to FX artists working on our projects among tramper and Simon Clements choose image choose the vertex shader animations and almost every situation for example the five sparks how the rain is affecting the character temporarily not only the hair also the the skin or the clothes and more limitations this

time with the expensive Paloma treat like the effects artist managed to use planes with transparency at the faith assault distance at a short distance but we have then say their complexity issues that you can see on the right side that's why we environment artists avoid and almost forget about the using of alpha and all aware textures on the assets the platforms something we enjoy a lot was designing the controllers for the different PR platforms that was the Bible one here's the oculus one all of them with a soft cyberpunk influence and all materials and as you can see we we added more functional features to them like a clock on our last area meter related to the teleport future and well the artists this and announced projects

and I'm not allowed to show more stuff or talk about the story but I can say that the story is full of science fiction and this has been a very fun experience for us the artists you catch the objects with your hands and you can easily get hypnotized by the light reflections this is also appropriate for the inclusion of aesthetics so you can hide them on the assets and select challenge the player to find them and well as conclusion VR is a beautiful experience and a perfect place for admiring and so in our own creations thank you very much [Applause] if you have any question

hi hi so thanks for the talk so I noticed you talk about a lot about optimization but you seem to have soft shadows on your characters isn't that it's quite heavy for yeah yeah yeah well he was asking about the soft shadows and the dynamic dynamic like the known characters when I was talking about them I said that it was maybe the most expensive the most expensive feature in inking and as you can see on that particular slide so you hear those three in the left side you have all those overlapping dynamic lighting it was just for this demonstration but that's impossible for VR 200

yeah it should be all like the right side green and yeah that's why I said that is the nightmare for the lighting artist because in the moment everything it has values of lagged Maps South Lighting's and like and in the moment you don't put any dynamic lighting on every single light emitter you see that the character is too dark and is not it's not a fitting on environment and so can you predict roughly where the player will be at some point in the story yeah exactly I wouldn't I I was not talking too much about the mechanics or they how they came design was bad eh yeah the the characters they were all the actors were all recorded on this motion capture session and and they were all like following work they were all following

the story and so yeah we know exactly in every minute where they are they are not moving with any that's why this is an experience and this is a theater experience we know exactly what where are they going and what are they saying well how much time they are staying in every room so yeah we we are always using this sequencer and we are checking where where to put all the lights all the dynamic lights thank you very nice art style thank you any other question well thank you very much you