Democratizing Real-Time Filmmaking Technologies for Indie Game Developers

[Music] thanks everyone for being here virtually this is definitely a very interesting times and a little bit about myself my name is Normand and I'm the CEO of glass box we're a company that specializes in making real time production tools and we have a global team with offices in Germany Los Angeles Australia and in Asia today I'm going to be talking about the democratization of virtual production for indie game makers which is brought about through a convergence of cinema and gaming technologies and you may have seen some of this in the works of the large projects such as The Lion King where they made use of game engine technologies and filmmaking equipment to make a animated film almost as if they were making a real film and more recently we have projects such as the mandalorian that broke new ground on

how to combine real-time engines that are traditionally made for video games to transform how film and visual effects are made but before we go into that I kind of want to take a step back and think about the history of cinematic gaming we being with this term for a long time now and we as an industry doesn't really have an agreement on what cinematic gaming really is meant to be and here are some examples of attempts to making games cinematic and this includes certain things like using live-action in live-action cinematics like in the case of Resident Evil and Red Alert but then there's also the idea of cinematic gaming being more about the content and the presentation and the subject matter like is the case with Detroit become human as well as other games like the lass of ours read their

Redemption The Witcher and got a war but strictly speaking I don't want to think about what makes games cinematic and also the evolution of the technology that are enabling cinematic gaming so adopting a very specific look at cinematic cinematics we're just talking about game cinematics here so here we have an example of Sarkar war which is pre-rendered produced using the same tools that are available for animation studios and visual effects studios and this was in 1998 and now in 2015 for StarCraft the legacy of the void we have Blizzard also using tools and technologies available to animators and visual effects studios but they're able to create a much much higher visual quality because of the evolution of the technology and the

evolution of the tools and also the evolution of the artists themselves but here's an in really interesting intersection of real-time cinematics or in engine cinematics where we're actually making use of game engine technologies to render basically game footage but to equality that approximates the pre-rendered cinematics of yesteryear so now I want to talk about cinema when we talk about cinematic we have to talk about how cinematic references the idea of cinema and in the in the area of cinema there is the technology called virtual production and it really is about making use of technology to emulate certain aspects of filmmaking instead of using a physical camera to shoot a subject a physical subject on a physical set we're

now using virtual camera to shoot virtual actors on a virtual set and the beginning of that comes from things like photo real digital humans virtual cinematography and performance capture such as is the case with Final Fantasy in 2000 and 2001 Polar Express Beowulf then also the film that made all of this prolific is avatar and when we're looking at avatar we're looking at an entirely new way of making phones and here is an example of a virtual camera and this virtual camera is basically virtual reality without a headset it'll allow the director to look into the screen as if they're looking onto a a world that doesn't exist then there's performance capture where through the use of technology with digitizing the performance of an actor and not just how

they move but also their emotion and the third piece of this puzzle is real-time visualization where in order for the director to see what what is being shot they needed a way to interactively render the images so that the director could see it as they're being generated and the combination of these technologies is one main Avatar possible and this technology very much still persists today where we're using the combination of cinematography performance capture and real-time visualization to drive a new generation of virtual production take for example battle angel leader where this is very much kind of the latest in how this technology was used where we're using a performance capture technology to digitize the performance of the actor and then we're shooting in

with a virtual camera and combining with virtual sets and this is especially sophisticated because we're also emerging real-life elements into it so it's not just virtual actors on virtual sets it's the virtual and the real combining together to make a seamless home now that we looked at cinema I want to take a step back to looking at games and for a while I think people have been toying with making use of gaming technology to make film and this really began with machinima where players are making use of gameplay footage of gameplay technology to record content that is not games so we have some really well-known examples like reverses blue and the series of highly popular shorts that made by valve for the promotion of Team Fortress 2 as I touched on before this technology has

definitely evolved in the last decade in such that here we have two comparisons of the same same scene this both of these games a matter of honorary allied assault and cold duty World War two are both depicting down the Normandy landing but on the Left we have one generation of technology whereas on the right we have an entirely different generation technology capable of producing visual qualities the likes of which that was just simply not attainable previously so where does that leave us well now that the technology behind real-time rendering has led to an exponential increase in the computing power available to us and we have real-time rendering that are that are approaching the level of pre-rendered and animations from last decade we're

sort of looking in an integrated an area where the filmmaking tools that were somewhat originally meant for very niche productions beginning to be applicable for a broad range of uses and here is here are two examples of some of the tools that were maned in game engine dedicated to using game engine as a filmmaking tool that maybe a lot of us haven't really come across or use ourselves this is where it leads to can the next and the final topic I want to talk about which is the convergence of technology where technology meant for virtual reality and broadcast is coming together to create what we now called real-time visual effects and going further than that we're now making use of game engine

technologies to render virtual images that we directly apply to cinematic production to animate the animation production to commercials and more and the latest advances of using virtually rendered images on LED walls to facilitate what is now called in-camera visual effects and some of the most recent advances in real-time visual effects is in the production of animated features directly using game engine technology and also in advertisement but perhaps the most influential is the use of real-time technology to render what's now called in-camera visual effects where in combination of display technology we are displaying the virtual world together with all the live-action components so that the virtual visual effects is a part of the camera plate so

the crew can walk away from set with near-final footage with minimal work in post-production in a bid to dramatically shorten the time it takes from conception to final frame and the best example of this is perhaps the mental or in where they use a very large LED volume to kind of create the effect of surrounding the film crew inside of this virtual world that doesn't exist and then when you're looking through the camera it is almost as if everything is together in one place but not everyone has access to Mandalorian tech and ultimately the real changes comes from the advances in latest technology making the great ideas from the decade ago cheaper faster and more available and that leads to the topic of democratization and democratization we're talking about affordability

availability and the usability of the technology like for example we see this general trend of highly specialized hardware becoming increasingly more available and highly specialized tools becoming increasingly more available and generally we're kind of moving from something that would cost millions to something that would cost thousands take for example over here we have a specialist real time renderer that was bespoke to the project all the way through to something that was running on a consumer device and likewise here on the one hand we have highly specialized camera tracking in-camera robotics through a motion capture technology and on the low end we have a consumer centric augmented reality and virtual reality technology that approximates the capabilities of

the tools that used to cost tens of thousands of dollars and this is also true for performance capture where we're talking about highly studio specific pipelines on the one hand professional tools in the middle and also performance capture using consumer centric devices like the AR kid on the lower end and for example of how this new democratized virtual production technology is being used by independent studio to create content here is infinite reality who is working on the latest project Mora [Music] [Music]

well Reb is essentially a giant robot military trauma we're a global arms race for building robots sparks World War three even though there's a lot of these amazing giant robots it's very important that the story remain about the people and the crews of the giant robots so the seamless integration between visual effects and the story including the robots is is very very important since partnering up with fonkoze Studios and Vaughn Davis we've been able to take some of his miniature creations and digitize those for the game engine the real-time visual effects pipeline in the Unreal Engine has been a game changer for us you know we can build these effects and particle Sims right out of the box and we can see these things in real time so once we have the board's

all worked out we're going to plan our performance capture with the combination of glass box dragonfly camera system will take face wear for our facial performance capture perception or on studio suit to do our motion capture we can put out hundreds of shots in a day to experiment with what direction we want to take our storyboards and our final edits this would have taken weeks before animating cameras by hand and trying to set up our shots but because of these virtual production tools that changes everything and we can do this and add a big benefit of using real time for me is how fast that we can put scenes together shots together you know with our basic VFX pipeline it works the same we get the animation out the models out but now we're not going into this post-processing

part of the pipeline it's as we're creating it we're actually putting it into the scene and we're seeing it come to life immediately special thanks to Dean and his team for allowing us to use the materials from the more a project and if you have any further questions sorry I can't take orience questions given the circumstances but you can visit our website and send us an email if you have any further questions thank you and best of luck