Characters to Get Immersed In: Creating the Cast of 'Blood & Truth'

Okay, good afternoon everybody. Welcome to my talk. I'm Toby Haynes. I'm the lead character artist at Sony London Studio. Um just before we get going, I just need to remind you that you'll get an email about 1/4 of an hour after this talk. Um and if you could give any feedback, that would be absolutely brilliant. Okay, let's get going. So, this talk is on the development of the characters for Blood & Truth. This is the new action shooter for PlayStation VR. Blood & Truth has been developed by Sony London Studio. Our primary goal is to make groundbreaking VR games. We released VR Worlds in August 2016. This was conceived as a set of experiences to introduce players to PlayStation VR. Anybody played VR Worlds? Good. And did you play The London Heist? Excellent.

The London Heist was by far the most popular of these games. We were able to analyze data in the months after we released VR Worlds, and it was twice as popular as the next most popular experience. Now, this might have been down to just people enjoying shooting things in VR, but the feedback we got from play test reviews, social media, and forums told us that players found the character interacted character interactions really immersive and fun. As you can see from this selection of quotes, the experience of having characters stood before you in VR and interacting with them is very powerful. Simple things like a character handing you a cell phone and being able to take it out of their hands have real impact in VR cuz you're you're in the same virtual world as them, and you're physically performing these tasks. So, to take these learnings from The London Heist and have them form the pillars of our next game became the main driving force behind Blood & Truth. How far could we push the envelope on character interaction and player immersion.

This generation of VR is still finding out what works and what doesn't work, but there are already a lot of rules and commandments about what you should and shouldn't do that have been put forward by various people. Two of the big ones relating to characters have traditionally been realistic characters don't translate well in VR, and players don't like their avatar to be to depict someone else. Let's start with realistic characters. The consensus being if you make believable charac- making believable characters risks straying into uncanny valley, unsettling players, and ruining the immersive experience. To avoid this for the London Heist, we had stylized characters. Now, that worked for this that tone of experience, which was larger than life and over the top, but we didn't want that style to become set in stone. And once we'd finished VR Worlds, we knew we wanted a grittier, more serious tone for the next project. So, we were keen to investigate more realistic characters. We felt they could work in VR, we just needed the right conditions.

And making highly realistic game characters is very subjective. People are hardwired to judge virtual humans with every ounce of scrutiny evolution has given us. And in VR, players are naturally even more critical of realistic characters cuz we're essentially fooling them that we inhabit the same world, and there's no filter. Added to this, the higher frame rate we have to maintain for VR means we can't use the same game memory budgets reserved for regular console characters. But, what if the solution actually lay in the advantages we have in VR? If we could create characters realistic and compelling enough to be believable and immerse the player through their interactions with them, they'd be less inclined to dwell on any perceived flaws. And if we gave the player the starring role in the story and missions, so they were never a detached observer, Again, the focus would be on the unfolding drama rather than whether the eye shade or skin shade was absolutely perfect. With this approach, we felt

confident about exploring more realistic characters. Now, the second rule. Players don't like their avatar to depict someone else. The theory being that a VR experience will be spoiled if a human avatar is a different gender or race to yourself. To avoid this, there should be no determiner of race or gender visible on the avatars. This is why the player wore leather gloves throughout the London Heist, but we ended up with weird situations like sitting in a pub wearing leather gloves, which is a bit odd even on a cold day. Now, this makes sense if the premise of your game is you're meant to be you experiencing something amazing or magical. But for a game like Blood & Truth, where you're a James Bond, John Wick style hero, escapism and role-playing is the whole premise. Arkham VR, for example, is cool cuz you get to be Batman. I'd already mentioned giving the player the starring role to immerse them. And Blood & Truth's key pillar is be the action hero. So we are very focused on

convincing people they are this person and letting them revel in it. So it seemed natural to have the hands of our hero represented in the avatar. The most important thing to for us was the player gets to feel they are Ryan Marks, SAS soldier, embroiled in London's criminal underworld. Scanning and photogrammetry. Making realistic game characters is challenging and time-consuming. More and more developers developers are utilizing photogrammetry and 3D scanning to do this with impressive results. And when it came to making realistic characters of Blood & Truth, we thought it would be great to cast, costume, and scan real actors to play the key characters. Essentially, create digital doubles and use these same actors for the mocap sessions. The game is set in modern-day London, so we could easily cast suitable actors and buy appropriate costumes. As an initial test with Blood & Truth, we downloaded free photogrammetry humans

from the web and reviewed them in the PSVR headset to see if they held up to scrutiny. This enabled us to see highly realistic characters in VR without spun spending months making full characters. They actually read really well, which was great. Problem was, we're a central London studio with limited space, so it was impossible to build and maintain a permanent photogrammetry capture stage on site. So, we decided instead to look at handheld 3D scanners. These could be used straight out of the box. We'd simply book a meeting room with suitable lighting and get scanning. So, I did some research and organized a demo at the studio with Artec scanners. The character team and I scanned a bunch of clothes and heads so we could assess the detail that we could capture. We also scanned some poses for clothing deformation. The results were really encouraging, so I did a pitch for our leadership team, and on the back of that, we were able to buy an Artec Eva and a Spider scanner.

Now, we moved into pre-production and began looking at R&D and workflow. We did some tests to simulate deformation on clothing. The idea was we'd scan people in various poses, capture the changes and folds, and use these in game to make the clothing look more realistic. Here's a video of the first test we made using normal map blends and mesh blend shapes. So, there's no cloth sim going on here. It's all blend shapes and normal map blending. The next step was to create a full benchmark character model. We scanned me in a suit, not for my dashing good looks, but Like, was a good test to see if we could create an accurate digital double. From this, we created a fully textured and rigged character. We reviewed it in the headset again, and it looked great. A first real validation of the process we were taking. It also served to define mesh and texture budgets and timings on how long these characters would take to make. Then, I wrote a detailed production

guide going through the workflow from start to finish. This guide was then adopted by and added to by the character team, and we could send it out to outsource as we brought them on board. Initially, we planned to handle facial rigs and blend shapes in-house, and we worked with the animation and the rigging teams on initial tests. We were getting some really good results, but it soon became clear this was going to be a monumental task to get the level of quality we wanted for all our key characters. This led us to seek a third-party vendor who specialized in this field to partner with us. What the facial test did achieve, though, was a good base head mesh and UVs, which was low poly enough for our budgets, but still looked really good in the headset. We could get a third-party vendor to adopt this for all adopt this topology for all our key char- sorry, for all our characters. Character tiers. Before production, we decided to split the character into two tiers based on their level of fidelity.

Tier one was for key characters with speaking roles who the player would interact with in drama scenes. Tier two were for characters the player would encounter during missions. Let's talk about tier two first. These were mostly male NPCs split into different factions such as the street gang and the militia fighters. Each faction had four to five gameplay classes within them with names like heavy and maniac. We received concept art to use as character look guides and briefs from design on the role of each gameplay class, which also needed to inform their look. These are some of the street gang concepts. They occupy a derelict tower block in the game where you have to infiltrate and rescue one of the key characters. And this is the private security faction. These are highly trained soldiers you encounter later in the game. We decided the best way to get as a much variety as possible was to create these characters using a modular system. So, upper, lower bodies, and heads, and

headgear, and hair could be interchanged to create many MPCs from relatively few pieces. We hired a costume buyer to source clothing to match the concepts and use some of our own. You may notice this turn up later on. I sourced and purchased the military gear from various websites, mostly airsoft ones. And I contacted companies such as Bates Footwear, Crye Precision, Mechanic and Mechanics to license army boots, clothing, and gloves to add more authenticity to the private security faction. And we licensed this awesome Ronin ballistic helmet from Devtac in Japan. We also brought a full set of riot armor. I tried to order this from China originally, but I discovered I needed a government permit to do so. I was about to give up when I found a place in the UK that had a set for sale. It's very cool. All the tier two clothes were scanned in the studio with the Artec. They were adjusted to fit a single scan body to

use as a kind of mannequin to dress with the scans. The post uh we processed the scans in Artec Studio and then in ZBrush we adjusted them to fit over the male body exactly. And then began cleaning them up. I could do an entire presentation on just scan clean up best practices, but I won't bore you with that right now. With this method we could use the same rig for all the tier two characters. We also had a pool of female assets created as well for peds and civilians that also were fitted to a single body and rig. For the tier two heads, we scanned some of our team with the RTX. Basically, anybody we could find who looked intimidating. And we bought some scanned heads from 1020 the 1024 web store to add some further diversity. We ran these heads through our pipeline, so they all used our base head topology and UVs. We handled the facial rig for tier two characters in-house cuz we only needed to be fairly basic for barks and eye tracking.

Here are some Maya grabs of some of the finished tier two modular parts. So, this is a couple of the street gang upper bodies. And here's a couple of the private security pieces as well. We later added some accessories to the drop leg character on that guy, so he could carry brass bags or shotgun bandoliers to add a bit more variety. Tier two characters came in between 30 and 40,000 tries. Our creative services department made logo and logos and designs to add color and believability to the clothing of the tier two characters. Now, tier one. As I said before, tier one were key characters with speak- speaking and dramatic scene roles. We received concept art and biography to use as guides. The next stage was auditioning and casting actors. Once cast, we got their measurements, bought costumes, and got them in for fittings. Unlike the tier two characters, they would be exact digital doubles, so their

bodies and rigs were unique. After searching for a suitable vendor to create the tier one character heads, facial rigs, and blend shapes, we chose three lateral in Serbia. They adopted our head topology and were pleasantly surprised how well it held up, and we were really pleased with the results. Here are some of the heads before we added hair, makeup, etc. James Anson, our lead tech artist, wrote us some cool eye and skin shaders and lighting tweaks such as being able to procedurally generate shadows um from eyelids and using bent normal maps to remove lighting artifacts. The first step with Tier 1 was to scan the actors in yoga gear or tight-fitting clothing at the studio while doing the wardrobe fitting. We create a base body for each actor in a rig pose much as we did for the Tier 1 characters and females. Male and female bodies, sorry. This base body was then used as a ref model to dress with the clothing scans. And it also worked as a representative placeholder for animation to use for

mocap previews. Originally, we were going to scan the Tier 1 characters in costume at the studio, but as we were taking them over to 3Lateral to do head scans, we decided to scan their clothing over there, too. The main reason for this was for the clothing deformation scans I spoke about earlier. We'd scan actors in a rig pose that matches the base body, and then a set of poses to capture clothing deformations for fold to cap- to then create fold and crease normal map blends and blend shapes. We decided on a small set of poses based on the most regular movements and deformations we were likely to see in the game. Some of these poses are hard to maintain while capturing with handheld scanners, so using 3Lateral's photogrammetry capture stage made a lot of sense. Let's talk about hair. Getting to hair look getting hair to look good in the headset is a challenge. Alpha is very expensive, and poly cards can get very heavy geometry wise. And the headset resolution can cause scintillation or shimmer on hair, which

can be distracting to players. Luckily, Tier 2 haircuts lend themselves to being non-alpha as the hair was either gelled back, partly shaved, or crew cut. For the Tier 1 hair, we created a wig mesh with no alpha on it that that would define the big volumes of hair, and then we'd clad this with alpha cards. Again, James wrote some cool shaders that blend card intersections and would offset hair strands to simulate depth. He also worked with the graphics programmers on techniques to minimize any scintillation or shimmering. Here is some of the Here are some examples of the finished Tier 1 characters taken straight from the engine. This is Michelle, who's your adopted sister in the game played by Clarissa Clayton. This is Deacon. He's your SAS buddy played by Brian Larkin. And this is Nick, your brother, played by Jay Taylor wearing a familiar leather jacket.

And this is Tony, the ruthless head of the rival sharp firm, played by Stephen Hartley. Tier 1 characters came in between 40 and 50,000 tries. And here's an example of one of the finished clothing deformations. This is one of the character's t-shirts, and this is how it looks in motion. VR offers unique challenges, but also unique opportunities. In VR, the player is the camera and by that definition free to look at whatever they like. I spoke earlier about trying to keep the players focused on their role in the story, but with a free The free camera does give us opportunities to put nice

details on characters that hopefully players will notice. The avatar, for example, the avatar hands, for example, are something we expect players to scrutinize. So, if you hold them up and flex your fingers, you'll see wrinkle maps and mesh blend shapes changing, so knuckles and tendons appear on the skin and the skin wrinkles realistically. We also created blood flow maps to simulate color changes in the skin. So, for example, the knuckles whiten when the hand is drawn into a fist. Adding details on characters are also great storytelling devices, especially in VR. Easter eggs that allude or give hints to the history or backstory of a character. Adding details players that will hopefully notice and wonder about their meaning or significance. Here are some examples on the tier one characters.

Kyla, who's one of the main adversaries in the game, has this skull and flag motif on her leather jacket and these warlike biblical quotes tattooed on her arms. This alludes to her covert US military past and her somewhat precarious state of mind. Deacon, your SAS buddy, has these tattoos on his arms. Members of the SAS sometimes have tattoos of the regiments they were recruited from. In Deacon's case, the Royal Marine Commandos with this dagger design and the Latin motto "Acta non verba". How's everybody's Latin? Does anybody know what that means? Actions, not words, which is a very Deacon-like trait. And on his left arm, he has the flag of St. Andrew as the actor who plays him as a very proud Scotsman. When we first meet Deacon, he has injuries from being interrogated. When you're reunited with him later in the game, we wanted to give the player some idea of the passage of time. So, his injuries have only partially healed. The bruises are a deeper colored a deeper color and he has steri-strips on some of

the bigger cuts. We wanted to give Michelle a sleeve tattoo. We found out the actress, Clarissa, who plays Michelle, has Filipino ancestry. So, we did a traditional tribal tattoo. And your brother Nick wears this silver ring, and you're given it later in the game. And if you hold it up closely, you'll see it has blood and truth inscribed in Latin on the circumference. We work with the tech and tools team to create a character assembler and viewer that runs in our editor. We can load various tier two modular parts and build a character from those. And we can pick colors and change the roughness on clothing using RGB masks exported from Substance Painter. We can also change the build of characters to make them thinner or more muscular. These characters can then be saved as prefabs for the designers to use. And we can also use this character

Sorry. We can also use this viewer to review characters in VR. The headset is key to getting assets signed off with the art director, animators, and designers. There's a really nice tactile quality to reviewing characters in VR. It really feels like you could reach out and touch them. And being able to lean in and check out details is really cool. Thanks for your time, everybody. I hope that's given you some insight into Blood & Truth characters. Before questions, I just want to show you a bit of footage from the game. So, you're the first people outside of the studio to see this. And just to warn you, there's a bit of strong language, so uh brace yourself. Right. Still no sign, then. I swear with that awful headset, he's starting to captain or something. What have we got? Tony's been approaching some of our people. Most of them just told him to [ __ ] off. Someone's helped him.

Find out who. Safe houses? All good. Warehouse is secure. Scott's on his way to the courtyard now. He'll take care of the rest of it. And Clint, someone needs to tell Sarah and the kids. We're on it. You sure they're looked after? Our American bird. What do we know? She's military. Professional. No wonder you [ __ ] yourself. Yeah, you're not wrong. No one knows anything about her. It's weird. You can't Hold on. Go ahead. We need to talk about what we do next. Give him a slap and tell him to [ __ ] behave. He's just trying it on because dad's gone. No, he's gone way too far for that. He's not going to start a mob war. He's not big enough. Well, he clearly thinks he is big enough. We need to push back hard or we're finished. Tony's got to go. Got it. Thanks. We know where Tony's going to be tonight. Private party at his casino. Great. You might as well be on [ __ ] Mars.

Yeah, he'll be surrounded by his best people. And the last thing he'll expect is me turning up. No way, Ryan. You are keeping well out of this. It could work. No, it's madness. Could you do it? I can do it. Did you hear what I just said? You are not getting involved. But I'm already involved. I could be in and out of there before anyone knows what's happened. I deal with situations and people a lot heavier than Tony Shaw. Ryan's right. What the [ __ ] are you thinking? Tony won't stop here. We know that. Mom, Ryan's trained for this. Anyone else, you wouldn't think twice. But it isn't anyone else, is it, Nicholas? It's your brother. Okay, okay, fine. At least we know where Tony will be tonight. We can keep an eye where he goes afterwards and then pick our moment. Why risk waiting? If we've got a chance to take him down, we should do it. All right. But the first sign of trouble, Ryan, we get you out of there. Course. What weapons we got? Let's have a look.

So, it goes without saying, thank you. Goes without saying that needs to be experienced in VR, but even that moment where um the characters are talking amongst themselves and then you speak and everybody just turns to look at you when you're in the headset, it's a real kind of everybody's attention is suddenly on you and you are suddenly just brought into it. It's a Yeah, I'm really looking forward to you guys playing it. Um do you have any questions? Please step up to the mic. Um got some t-shirts here if anybody uh wants to ask a question, you can grab a t-shirt. Thank you everybody. I am on here, yeah. Not only for the t-shirt, I promise, but um I I stepped in just a few minutes late, so I apologize if this was addressed. Uh but in London Heist for PSVR Worlds, those characters were already, especially for the time, really impressively uh detailed. Were you using any of the

scanning approaches uh for those characters back then or is this a entirely new pipeline? Totally new pipeline. Yeah, yeah. The The London Heist ones were I mean, you know, we'd be doing high-detail sculpts in uh in ZBrush, but it was all from scratch. So this was a I'd used photogrammetry at um a previous studio when I was working on Formula 1 to do Formula 1 drivers, but this was doing it for all characters. Um so yeah, it was it was a new new pipeline altogether. Got you. And then for the scene that we saw at the end there, was that uh uh motion captured? Yeah. Yeah, yeah, absolutely. Yeah. Awesome. And then uh for the the purpose of just going into this obviously more complicated uh approach, uh do you think this is uniquely um kind of key for VR because of the immersion and how close you are, or is this just general uh more generally uh kind of the future of all game production, would you say? Um I'd I'd like to think what we're doing is going to be a step step forward than that. Until you try it, but when people do try

it, it's it is you know, the feedback we get from people trying it, it's it's really immersive and it feels like a step forward. So, not only the drama scenes, but the missions as well. So, yeah, I'm I'm excited about it. So, I hope we're pushing the medium forward. Great. And sorry, just one last quick one. So, for the last scene, was that it was motion capture, but were they simultaneously acted or individually? Yeah, they were they were acting together and it was facial capture as well. Cool. Thank you. Hello. Um I'm curious when you're dealing with conversations dialogue between the playable character and the NPCs, did do you encounter any lessons or non-obvious uh lessons when when the playable character speaks, how does the player feel like it's their body doing the talking and not just some other spectre in the room? With the voice. a it's a good question and it's one we we thought about a lot whether to make

the kind of the character a kind of silent Gordon Freeman or whether to actually give him a voice. And I think to reinforce that idea that you're playing this person and to make the player think they are Ryan Marks, it felt natural to have his voice cuz then you are you know, to us it feels like we're we're making the player this person. You have the role-playing side of it which I spoke about. Is um is his voice the first voice you hear in the game to like familiarize the player with like, oh, when you hear that voice, that's you? Or do you hear a lot of other people talk for a while and then just his voice comes up? Or or is or is do people just understand and there isn't really any necessary special considerations? I think there's one voice before you. Um which is a a chap called Carson who's basically in a kind of interview with you and recapping events. So, he comes in, introduces himself, and then you get to speak. So, yeah, that's

that's the first time you hear your own voice. Thank you. Okay. Um so, real quick question is the uh voice uh of the main character, is that modulating any way to like because people's voice when they actually talk sounds different in their when it's in their head? If that makes sense, so do you do you modulate the voice at all for the main character to kind of like, you know, change it ever so slightly so it sounds like it's I I'd have to get one of our audio guys in here. I imagine they're um I nearly pushed the whole lectern uh off here. It's not actually attached to the stage. That would have been a brilliant way to finish. Uh very rock and roll. Um yeah, you'd have to ask our audio team. I their attention to detail always blows me away, so they may well be doing, you know, how do you hear your own voice in your head? I know what certainly when I hear this back, I'll probably go, "God, I sound weird." But, um yeah, they might well be doing that, but I haven't got an answer for you, I'm afraid. Thank you, sir.

Anyone else? Well, thank you, everybody. Cheers.