How Film, Games & Social Media are Merging Using AR PG Connects Digital #2

up next we have another speaker um raphael from the surreal digital twins who will be discussing how film games and social media are merging using ar and we should have them ready oh yeah we've got them coming in hi everybody it's great to be here and i want to talk about how films games and social media are merging using ar technology in today's time first we'll talk about myself so i'm the head of ar and film for so real digital twins we create um the perfect digital twin which allows you to it's a high resolution scan and here you can see which one is the real one which is the fake one and i'm gonna dive into this topic in a

bit why this matters so much in this case here on both sides the right one is the real one the high-res digital twin is a ct scan and it allows for both the interior and exterior to be to be digitized which is pretty pretty remarkable here's an example of that you know the absolute high resolution perfect topology that can be used in games and ass and mobile devices as well i also run a company called superba ar we create augmented reality applications for brands and mobile and also social media and my personal background comes from the movies over a decade in films mainly on blockbuster films such as iron man avengers and a lot of 3d previous virtual effects blowing things up designing action sequences so here's the topic and why it matters

is that we now have three different industries that are basically all coming together they're all converging we have the film side who traditionally was practical just till uh you know maybe 20 years ago everything was practical and on the visual effects side the game set was always digital but it couldn't never match the quality of film it was always just not as good looking and everything and social media sort of this the new medium where the user is actually content creator but of course until recently wouldn't really dream of um having access to the quality that the film people have or even the game people yet that's completely changing and all three are coming together so when film and then around the early 90s started to go from practical effects to digital with jurassic park and terminator 2 a whole new set of tools started to emerge which is the creation of 3d computer software and of course that's when you start with crude models and it started

more simple and it's still remarkable how these these films hold up and this is really important to understand this history i believe because computers back then were so slow and the whole process took so long and yet the quality that was achieved is really remarkable and this matters because if you go fast forward to our social media people now and youtubers and influencers and everybody with a mobile phone you have much more computing power than people have back then so it's an important reference point to remember where did film start and where is social media now again just an example of practical effects you had to build full-scale dinosaur you know move it with animatronics and robotics and uh yeah it was a tall order so same challenge that has been around for years is the use of of the green screen you know it's still obviously probably today that's about to change and here's where

we get the conversion of the game industry helping the film industry for most of the time of visual effects of the last 20 years everything is done in post-production and it's done fairly slowly meaning render farms to generate the final image takes a long time a lot of computers and the game industry whose was always about real time but couldn't really quite match the quality of film got better and better with unreal engine and also unity and as we now with real-time ray tracing can re achieve a quality all of a sudden the film industry is very interested in game engines and this is where we are now we got uh it was a demo of real time ray tracing just last year and i'm sure you guys heard about it but this is a computer game generated image it's not a post render and it it can rival now um you know render farms and offline

renders which which is incredible so here the conversion of how the the game industry is starting to penetrate the film industry the social media influencer now all of a sudden is a storyteller and is starting to leverage basically real-time visual effects if you go back to jurassic park the the dream of just tracking real time onto a face would have been unrealistic it would have been an absolute pipe dream and yet here we are and based on your phone you can now create visual effects in real time and anywhere you are a thing that started to develop in the last couple of years in the last decade is virtual production it allows a director to scout a virtual set walk around basically the camera and scout and look around and see a set before it's built and uh yeah you see the connection the connection is to the to the cell phone

and the ipad now to be able to scout look around and everything so for 10 years the step-by-step came along um and we were built usually motion capture and you can see um you know the uh up on the picture you can see those uh two markers or it was used with motion capture you know there was no ar kit there was no ar you had to use motion capture cameras to track such a device here we are in the world of ar today you know just everybody can using a mobile device can be uh can do more than what a director was able to do scouting uh just just a couple years ago you can place objects in the real world and you can place move things around see the result right there record it and share it another pipe dream basically for films just in recently and here you have the power to do the same thing if you guys have watched the mandalorian

maybe you've heard about it why it's a breakthrough in filmmaking the crucial part is is that the disappearing of the green screen so if we look back the other picture i showed just a little bit a while ago with the matrix everything is a green screen we're now entering the stage where the green screen is disappearing thanks to game engines and so where all of a sudden you can actually have the background and the set extension in the stage in real time using led walls and you can change the sunset with the click of a button you can swap out the whole environment where you're at the game engine is really changing the way movies are made and basically uh let them do things that never been possible for thanks to real-time technology and that's all thanks to the guys from the game side and here we are basically today yeah the ability same for you in real time being able to see and learn

and place elements it puts you it's a reminder it puts you basically in a position the user the position that was unfathomable for directors and also game designers not that long ago and where does that bring us to it all brings us together that all of a sudden we can enter worlds we've never been able to to see before but we're using tools that are all borrowed from different industries all coming together and all as a result all these disciplines are merging game film social media and all starting to use the same tools and also starting to face the same problems so key element is the user all of a sudden can now become part of the story you know you can be batman you could be indiana jones and obviously uh that changes the game for brand storytelling it changes the game for for the first time it's not always about somebody else

if you look at traditional advertising you're usually looking at a image of somebody else it's a model or a campaign and now thanks to ar and with all the tools that have come before it from the film and the game industry all of a sudden you can now be part of the campaign or even maybe the center of a campaign and you obviously see the result right there we can change the game in shopping marketing and also in viral storytelling because you're telling the story and you may share that one um now of course where we are today we also can see where all this is going we got a combination or a new segment that's evolving from all this and that's where a new language basically has to be developed till now we're basically only lived in the medium of a 2d world

and we still do which is the world of a flat screen as human beings we are we're 3d people we're 3d the world around us is 3d but the world of information is flat and we scroll through the phone page 7 page you know google scroll and swipe but now all of these disciplines games film and social media that have come before now have to tackle the next challenge which is understanding the space around you and then bringing us actually to a real 3d world that combines the virtual world and the real world but all the disciplines and all the knowledges from each one are needed to make that happen film world the games and social media so let's go back to talk about what are the same challenges that are all now prevalent in in this industry number one is realism everybody wants it

to make it uh look as real as possible which which is a challenge where do you start do you have a uh you know did you scan an object is it just a photo it needs to be digitized do you have to build it from scratch it's a big challenge always is in film games and also on social media the ease of use 3d tools are complicated it's still not easy for somebody uh to just uh create digital assets many people now can use photoshop many people can use apps to maybe liquefy their face and to facetune and adjust things but 3d is still very hard there's so many elements that have to go into it especially if you want to do something that's a bit more advanced the same thing is also um the same thing is also the communication

you know is is a big issue the more complicated it's getting how can you communicate with your clients or with the people you're working with what exactly you need 3d has a unique language of like are we talking about uv textures are we talking about blend modes are we talking about animated particles volumes all these things do all these elements combine how do you talk to each other can people from new industries understand this and speak the same way the next challenge we all face together film games social media storytelling how do we involve the user how do we how do we use these tools to to tell a new story how do we use all these possibilities given to us to make something that's compelling and has never been seen before and can we now go around that is exciting and we all face the same challenge that it's all

digital content okay so it really means we gotta everything has to be available first to work with it and how do we match that where we need the content so in the essence we all start the same challenge how do we blend the virtual world with the real world how is that achieved and uh that's the challenge we face today and uh that's sorry we're excited that we we're part of that the help of creating the perfect digital twin and there's obviously many more possibilities nowadays how they can be achieved and volumetric capture is coming where people can be recorded live in motion that's already available it's just still very expensive so many more things ahead and many more challenges ahead as the amount of data just really really is growing and here we have it what does it look

like do we have cameras who can digitize the whole world and understand where everything is at in real time do we have that in the cloud they're called a or cloud and where we then who can pull this right over 5g so many more challenges ahead it's important to go back again we think okay where did computer graphics come from in the 80s and 90s how far have we come and and the key part is doing all of this in real time it's not imposed it's going to happen all at the same time that's where the game changer is so where do we go from here the key disciplines and elements that all the three um all the three industries so to speak phase is you take advantage of machine learning and ai can things be learned and improved and understanding patterns understanding faces

taking through thousands of images and and being able to help you and how can that also help to achieve real time i believe that 5g will be a game changer because maybe we can move in the world of ar we can move rendering off the device into the clouds and your device is only the uh the camera basically to look at the importance of creating digital twins uh how do we digitize it how can we make it easier how can we um capture people it's going beyond the photograph and how do we all store it in the cloud and make it accessible so that could be used in film production on the set maybe a game can also pull right from the real world and of course in augmented reality to do information overlay right there so thanks so much for listening i'm happy to take any questions and um

brilliant thank you for that that insight raphael that was uh that was really cool to listen to i i don't know a whole lot about ar it's not really my my area so it's nice to uh to sit and absorb this stuff we do have a couple of questions coming in if you're happy to stick around and answer them um we've got almost all of the ar enabled mobile games have i've played caused the device to run incredibly hot what is causing the processor to work so hard and how can we avoid cooking hands and draining batteries yeah the issue is with ar it's just using way too many elements together at the same time and because you got to run the gyroscope at the same time you get around all the sensors of the phone at full blast to understand where you're at any other app if you just use the camera you're choosing just one element but also you're using them continuously so to make that better it's obviously up to the experts it's apple and google to to to figure that out

but it is it is a big challenge and um yeah if you go to 5g then at the same time we're going to stream the data over and upload it at the same time so that's not going to be easier but it's a challenge absolutely okay has anyone tried using vr to create 3d models like this there's a link that i can't send you um i've been teaching 3d for years and it hasn't become easier for anyone to learn over the past 10 years but many improvements had been made 10 years prior now there are many more tools which helps improve the results but don't make it easier to create yeah yeah it's a good thought there is some people who now work in vr i personally can't be that long in vr that you know like usually 3d works takes hours and hours so i admire those people but you will find 3d artists who model and sculpt in in vr now and um and also in scouting or film sets some directors use vr headsets to walk around

for me personally it's just still an issue to be able to be in there really really long like could you read in there eight hours you know and be also removed from everybody else but it does certain people do do like to use it and i agree with you it has not become easier it's just more options more tools but it's not that much easier okay um i have a question back from when you said when you were speaking about influencers getting more involved with uh with ar and i think um there's this very new trend of um companies using ar to create virtual influencers um do you think that that i guess real life flesh and bone influencers are it should be kind of observing this and observing brands creating um these perfect influencers virtual influencers that can uh basically say whatever a brand wants them to say um are they more likely to be implemented in marketing over actual influencers do you think that's a

will be derogatory for them yeah it's quite surprising to me because i always thought that the influencer is all about the person that it's like you like this person and that's why you want to hear what they have to say or their their sort of personality is shines above all the virtual influencer to me suggests the opposite but um i don't know if that's more like a sort of a spoof on society a little bit like look we can get somebody to do anything um and maybe what could be a virtual influencer we could maybe do things nobody else could for real but just a straight coffee a copy and then it's virtual i yeah i i would love to hear what are people fascinated with that much yeah it's interesting because i think there's such a an importance of authenticity and influence and marketing and like up front if you know that a virtual influencer is just programmed then there's no authenticity up front there it's an interesting one for sure

i would say that everybody talks about authenticity but everybody knows autisticity is is a complete illusion you know and that's and people are people are more okay with it than we think you know that goes from the instagram photos who everybody knows is retouched and color corrected and optimized two they know it's probably a brand deal that they're posting about but still yeah i think people probably are not okay with it and then in private they're it's just so overwhelming that we just take it yeah yeah i agree um someone has asked people tend to turn off the ar functionality off in current games uh in pokemon go for example everyone i know disables the ar visuals as a while after a while because it doesn't affect the gameplay how do you make the ar functionality call to the gameplay experience sure um i think the issue is people are still not that much used to ar and it's still you know it's it's just more it requires more work

excuse me you'd walk around and look around there's more work than just maybe tapping and all that i think that's one the other thing is just it will take time for people to maybe get used to this and also find it more interesting you know i think once your phone understands the surroundings around you and you use this a lot more and maybe also in other devices um when that becomes the new standard that's when it will stop but are still new and um there's still also sort of a question of maybe i'm going to say surveillance but you know you're looking at things and everything do you want that i still think there has to be steps taken both in the technology but also in user behavior and if we just go back social media you know there was a time when people would say i'm gonna post everything on instagram and my food people would have laughed at you they're like nobody's doing that you know you're the outcast and then here we are and you know you can see that at every restaurant so things can flip they can flip over time and fairly quickly so

it could happen there as well um i think i think pokemon go is a really good example of a game that has brought um ar kind of into the mainstream a little bit and it's attached to such a big ip that people are trying out um is what other kind of things do you think will make ar more accessible in that kind of way to people i really think that shopping is going to change it the if you're the ability to see elements on yourself and be able to try products virtually see how you look and already maybe get feedback from friends i i think is the big game changer both in mobile but also in maybe stores that you could walk in and on the screen you could really see yourself and get an idea what they could look like yeah that's something i've looked into actually i think that especially like ikea made this thing where you can put furniture into your house to see what it looks like before you buy it and i think things like that are really cool um someone has asked do you think the need to physically move limits physically move limits the implementations of ar having to walk

around the world or even get up and walk around a table seems to be a big hurdle for some gamers how do you encourage people to get up and game yeah i think it's it's changing you know it's new people are very uh um accustomed to just sit on the couch and just move their fingers and it's it's a change and it's not necessarily comfortable in every case and there's case where it's absolutely should not be done i think it's not the right way to go you know and then there's others where yeah it will make sense you know and it will make it more interesting over time it's a succinct answer um with the advance of ar is there a clear marketing use that has already started to be exploited ikea for example do you think that ar will allow games to have a new monetization model with product placement in a way that is less disruptive for gameplay yeah i think so i think um the same way ar is evolving as a shopping and planning tool it should evolve exactly that way that

um the key part here might be that you could design a game that overlays over the real world and uh i don't know let's say let's take a motorcycle maybe uh you your job is to find a motorcycle outside and um it will recognize it and now it overlays a virtual motorcycle over it and you can do a game play with it and and now it's a sponsorship by the bike mic manufacturer or something like that uh that's a completely still very untapped and huge potential for sure absolutely and we've got do you think the consoles are going to get more on board with ar sony opted for vr with psvr but we're still waiting on microsoft's xr headset what do you think will happen there yeah absolutely i think we're at the very beginning we're like in the vhs age still of um you know of of ar if you look at the headset what it is and um you know the pipe dream it is it's a tony stark reading glass and has ai and sees everything we're

for sure not there and and it's going to take many more years to get there but i think a key part will be that we're going to get off the device you know and that's where 5g will will make a huge difference you if you're going to build a computer on your headset to see and compute yeah you're going to build a massive computer in your head and just be small but if it could be off the device or partially of the device to help then all of a sudden the headset could be very light and things might change correctly do you see um vr and ar as competitive technologies are they part of the same kind of family to you in my opinion they're completely different i i always i'm amazed that they're mentioned in the same sentence i think they have very little in common in the end i think the vr community is way too focused on the hardware constantly because they have to it's like which headset does it work you know does it have a cable or not and the ar world is about the opposite it's all about right here we're doing it together and

let's add very selective elements so for me they should not be mentioned in the same sentence okay yeah that makes sense i guess they are if you're very deep into them when you work in those spaces they are very different um one thing that i just thought about is that i remember when the the pc vr headsets came out a couple years ago and they they were kind of passed off as a little bit of a novelty that people had one and you played on it for a little while and then you stopped and you put it down and there was no real real reason to go back to it um is there any way that you can kind of make people see vr headsets in a different way because they have such a high price tag and they're quite you know they're not super accessible to a certain market and they're not really something that you would sit and do for hours at a time i suppose is there a way for to encourage people to buy into that market yeah i i don't think so i think the way to go is if you want to experience vr your best bet is go to a local vr arcade which are popping up everywhere

you can do it with your friends you don't have to buy the headset and you might even have a backpack on and and you really get to see the power what what vr is it's not necessary alone as you do with your friends and you're walking and you're in an empty room but in the virtual environment there might be fire under you and you're actually really really scared that's where you can see the real power of vr and they will have the latest tech and you have to worry about it and i just don't think the idea of vr equals cell phone is correct you know not everybody needs a vr headset and should and that's what the vr community obviously wishes for but i don't think it's there but the vr arcades are really well done and and worth a visit and um and i would try that cool awesome um well that is some cool insight i'm going to wrap it up there raphael but thank you so much for coming in and uh sharing your insight with people and thank you for the presentation i hope you enjoy the rest of your pgc

you