Lessons Learned from Creating a Virtual Reality Engine

okay so welcome everybody and thank you for joining me for my talk virtual insanity lessons learned from creating virtual reality engine so let's get started okay so the agenda for today I'm going to take you through the horror engine which is the technology that this talks based on so I'm going to be talking about what it is and what it does and then the inspirations behind it so the developers and the existing applications that informed it development and then I'm going to give the technical overview of the Minimum Viable Product so just on the pinning the key functionality of Weider's so then that should give a great background to the lessons learned so I'm hoping that if you're here because to know more about creating your own technology that

you should have a very at least by the end of this talk have the confidence to go away and get started so hello yes I am Liz McCrory I love dinosaurs first and foremost and I'm an educational evangelist for unity but I'm yeah but I'm actually a super recent graduate too and so I graduated from a master's last year in computer game software development so all the lessons that I'm sharing with you today are super fresh so and I'm also Prince William scholars so that's a partnership between bathroom Warner Brothers so if you are wanting to continue into higher education in the entertainment industry I definitely look at the initiatives they offer because that's a new talent because it's a game changer for me so so the horror engine what is it so it was the focus on my research project of my masters and it's an overall concept so

it's a set of ideals for what would be a fully formed engine that we may all be familiar with and it was informed by my research in to virtual reality and harvest and how the mechanics of virtual reality can really lend themselves to the horror genre and I like to define it as a prototype which shows the potential of social first reality through the use of an interactive social screen so to add a little bit more color to that I'll go through what inside it whoo and one sliding unsurprisingly virtual reality inspired it so last year was the consumer release of virtual reality we've finally welcomed it into our homes and it has us now so no scope so we saw the commercial release of the psvr for example and the

device and we've also got mobile VR at our fingertips so as this all became more commercially available I also became more available to developers and educational institutions and so with me starting my masters at the crest of this message new wave of VR there was so many sort of amazing experiences being made and not just in the gaming industry but like from TV and also sort of like educational and financial sectors so there was a massive wealth of inspiration and they're being created to me to get to grips with as I did my research so it was often said the virtual reality was a very isolating experience so you're there in your headset and you're goofing around to do whatever you're doing and your friends can only kind of watch you and gasp and what we mean with the commercial release of VR that this idea has been addressed

so there's like the rec room for the vibe and the playroom VR for the PlayStation VR which is addressed this more inclusive and social idea of virtual reality so when I was doing my research the biggest area of inspiration was a company called triangular pixel that didn't if you've heard us and they did a talk last year too and they use this idea of a social screen interesting with the term coined by Sony and so what this social screen does as you can see here and if you have a play on virtual reality the social screen gives you a window into that virtual reality world so you can have access to information about what's going on so this game here is called smash hit plunder plunder and so you're sort of in a dungeon and you're going around collecting treasure

and smashing things up in the process and they found that by using this social screen and by giving information to the players outside of VR that the players in VR may not be privy to such as a timer or a leaderboard for example it just really organically created a more inclusive virtual reality experience I just found that concept super interesting and then finally harvest so I could talk about horror forever so I'm going to stop myself right there but I went around asking friends and other developers what scared them because horror super subjective and a lot of different things scare a lot of different people but I spoke to Lizzie at word and she told me what that her company curiously had created this little tech demo of what scared her and so you were in a house and you had an

object in the house and every time you weren't looking at the object it was outside your field of view its position updated so it was kind of like we've been followed like those creepy angels from Doctor Who and I just absolutely loved that and I saw this is scaring her and it would be so cool if I could be the one that's placing that creepy angel and scaring her and so that's really where the idea for the horror engine came to life so going back to the original definition them so it's the prototype it's proof of concept for social and more inclusive virtual reality through the use of an interactive social screen so the idea that while one of you is in VR another one of you and a separate machine can manipulate that world and place objects as a name to scare your friends because

I'm terrible like that so why did I create it other than the obvious well as I said earlier I started University at this peak of this amazing new way so virtual reality in itself isn't a new technology but this new wave this new iteration of it is it was just the perfect time to get involved and as the developers this new witness of it really fascinated me which meant that I naturally wanted to get a better understanding of it I was one of those kids that couldn't just have a radio I had to take it apart and put it back together to understand why the radio is radio so I could have used existing plugins an existing engine to create this experience but as a programming student I really want to create that code myself I want to write it myself get an understanding of it and not just

take for granted and at face value what this plug-in was telling me it did and I think it's developers it's super important to understand more the technologies we're using every day so we're able to leverage it and just make better experiences and finally giving back so yes but reality's ever-evolving and we're finding out a lot of new developments but actually our industry is like that as a whole and I think it's really great that anybody can contribute that and it's important to do so to sort of keeping potential of what we're all making so now on to the technical overview of the Minimum Viable Product so I start out with this big goal that by the end of my master's I'd have made this engine and but actually that doesn't just happen overnight or in a very short space of time so what I did

was I started off with the minimum viable product for what I wanted to create so this is like the minimum functionality that you need in order to validate what you're creating so I split this into three areas which was the virtuality integration the networking and the rendering so this is what it what it built up to be so everything was built from the ground up in C++ libraries using the visual studios integrated development environment the IDE for the rendering now use DirectX 11 for virtuality integration I use the C++ SDK so mutters I use Windows sockets libraries for networking so there's communication between two machines and I use direct input for player movement so

let's just take a little moment to consider what this Minimum Viable Product is compared to why overall wanted to create so you'll see straight away that it's missing quite a bit there's nothing for example for collision or audio and you'll also see that everything is set in stone so there's no abstraction for it pull to allows for the integration of another very reality headset or rendering API and also setting in stone like this really limits the release platform so there's all these considerations that you would take for granted having used an engine so the MV P van the Minimum Viable Product is the minimum functionality before concerning itself with all the out intricacies that you're going to have to consider when you make your final project and just weird words by the

island from doing it this way rather than using plugins is that when you use a plug-in more often than not the person who's creates it has an appreciation of the pipeline's of the engine or going to integrate it into so sometimes you don't have to alter too much what you already have to be able to accommodate it or get it to work how you would expect or you want it to whereas if you build up a product in this way you really have to have a deep understanding of what you're using so just because the Occulus SDK supports DirectX 11 just by integrating and referencing that in your existing DirectX 11 project means it's going to work as expected or at all there's a process of identifying the codes that you need and integrating that together

appropriately and so the minimum variable product is a concert look like this so on one machine the product would load as virtuality experience so this is where the SDK with you oculus SDK and and then on the mother machine you would have this interactive social screen so the player with business on this machine could use the grid the grid system to be able to place the objects and the network libraries were used to facilitate this data transfer between the two machines so while the play with the interactive social screen could place the objects and it would send the geometry type and position to the VR what machine which would manifest it in the VR world the VR world was always sending back the position of the player in VR so that the player with the social screen could more strategically place those and you'll see the twin ping to impede

kind of ascetic going on that's my doing and so in reality a lot like this so this was the first time that I showed it to anybody actually that you can see the 2-way machine communication that I've talked about the dk2 there and then the picture on the left words from concept art and by my friend whose concept art is called Bradley and he mocked up a number of scenarios for watch the MVP could be that the mechanics of it could be useful so he came up with the idea of like a detective game where you could place clues and move the clues and that kind of thing so is this it was really cool so this is the reason you're all here so the lessons learned so I learned a lot doing this some good some bad and so

what I've done is I've taken all that and I've tried to condense them down into what I think with a key important part I'm hoping will be useful to some of you so put them in three areas so one is pre-development so things to consider before you even get started then development is early on in development and iteratively throughout and then finally just more personal lessons learned about just being a developer and looking after yourself and all these are going to sound really obvious as well but actually when you get into that development headspace they quickly go out the window so it's worth remembering them so clear goals yes it sounds obvious what do you want what you're creating to do but actually it goes beyond that and it's about what you want to achieve from it too so is there something that you want to collaborate to or something that you want to learn and so for me yes it was an academic

project and I wanted to create this multi this multiplayer virtuality experience that's what I wanted the technology to do but I wanted personally with the ability to learn more fundamentally have this reality code worked so then that leads quite naturally onto resource so you've identified what you want to do but now you need to think about how you're going to do it and they'll often be things that you forget so if you're not at university you need to sort of think about the software and also the licenses and not just the development software is there anything that you need outside of fire there's also the consideration that if you're using virtual reality you're going to have to think about what headsets you ever beiie is what has available but also the hardware to run that hit those headsets so for me my machine at home didn't suffice and I had to make sure I was in the computer labs at university all the

times we have support the oculus and this also extends to any research journals or articles because if you're working in something fairly new there might not be the resource that you are used to having if you use an established engine for example and what you already know because I think it's really important to at this point to address your strengths and play to them so for example I'd already had experience with networking and rendering so that allowed me to more fully as applied myself to learning these virtual reality elements so if you have a project that you already have that you wanted to build on or with strengths that you really want to play to it's a good idea to get started that way Oh like I'm backwards how do you forward oh no am i spoiling it I've missed despised that when I roll

with it roll with it so rethought research so this is going to start and very early on in your project but it's going to carry on iteratively throughout and so this is adding color to what you're creating so the inspirations part of this talk was my research it was existing products existing processes and developers that inspired what I was creating and this also it went into like research journals and articles as well and then starting small again seems obvious but this is why I took this overall concept and started off with the Minimum Viable Product what I did then when I had the Minimum Viable Product is I broke each concept down further so for each of those elements I broke it down until I couldn't break it down any further and this sounds mental because I would have a million casts to do but actually

having tasks that took two minutes or two days or two hours with a great way to keep having that constant positive reinforcement that I was going in the right direction and I think that's really important for momentum in morale when you have a big project ahead of you so how can you do this this is really dependent on developer or team or individual I know some people who've used burndown charts in Excel or just kept a diary or written on the whiteboard every day and cleared it down and started again the next day but for me I use Trello because I thought it was a really great visual representation of what I was creating I could color code each of the areas of the MVP and as you can see with this I could tacking a tick list this is what I mean about that constant positive reinforcement I could break each task down even further so I always had that visual representation that it was going in the right way

code design so when I set out on this nice tablet the Minimum Viable Product I was like this is fine I've been prototyping those bits in separate applications separate projects I just throw them all together because that's the sensible thing to do and boiler alert it's very quickly crashed and burned and was a nightmare so don't do that what I ended up doing was having to go back and think more about thinking more smartly about my development process and some advice I was given which I used and I recommend it to have a visual flowchart of what you're creating so what I did was I took each object to each element of my project and I popped it down on a piece of paper and so I very quickly built up a road map of the areas that I needed for my project and

once I had all these pieces of paper and I could move them around I can very quickly identify the data flow between them I may need what objects needs to be aware of each other any commonalities and the relationship and then move them around until it made sense and only then did I transfer that into code so to illustrate that a bit further I'll just give a really high-level overview of the networking element of the project so execute driven so so I needed a way to be able to control the application if it was lodged in virtual reality but also if it wasn't so my first piece of paper was this reality integration so let's say that translated into code I called this VR application so then I needed to think about what that class would need

in order to work so then I started thinking while it's going to need the virtuality integration that's another piece of paper there's going to have to be some player class that's another piece of paper the geometry that would be a selection more pieces of paper and so it's starting to and then I think about the other side the non virtual-reality side and I'd repeat the process and so I could quit clearly see quite early on that there's going to be commonalities between these two applications even though a few classes even though they were fundamentally going to be doing the same thing rendering geometry and taking note of the player so when I had these classes defined I started to think about how they could be aware of each other when they needed to be aware of each other and how to facilitate that so what you may find again if you use engines is

that it's usually a good at will and in like code constructs it's a good idea to have an overall class that has an appreciation the visibility for everything that you're creating so in this example we could call it the manager so the manager could have an instance of or a reference to every other class within your application so it instantly is able to access the state of it and facilitate that communication between all the other classes it could for example detect when a headset is attached to the machine or trigger the virtual reality classes implementation to to get started so then if I look a little bit more at this area of code we use so like I said both these applications share attributes and usually when that happens it means that there's that we're rewriting it the same or similar code and so to reduce that we usually have a base class which

holds that generic functionality but which still allows the derived classes to be able to utilize their overrides that and also means their individual elements and behavior and this is likely to change even when you've got all your pieces of paper laid out and you translated it to code as you find new elements that you want to add to it or areas that you sticking points you come across that you think consider it will start to change but having that roadmap that visual representation already mapped out just makes it a lot easier to be able to identify where you're going to be putting in these new resources or changing it so then testing I found from very early on it was important to have testing from very early on in the project and intuitively throughout especially with virtual reality it was really important to have as wide a test base as I could because

virtuality was affecting a lot of different people in different ways and I also found because I had my head and virtual reality all the time I was more forgiving of it so it's really important to keep testing on that wider audience so you don't get caught in that trap of sort of getting you through it and just going with it usability was a big one for me because I remember when I first showed my projects people and it was like hey look at this it was basically just that and it's super shiny and they just went yeah but how do I use it and I was like like I've been sat in my little cave really hammering on those technical elements with no consideration for how the wide world would use it like maybe I just assumed they were psychic and so it very quickly became apparent to me that if you're what you're creating isn't intuitive to use then people just really aren't gonna take the time to get to figure out how to use it so these more

personal points I actually heard a talk with an engine programmer to meet aghori who just taught last year now don't often hear the iterative and again that obvious but I think a really important point out so the first is to pace yourself so I find I start developing mid-morning like 11:00 a.m. and then I look up and look at the clock and it before a.m. the next and like I hadn't gone out I hadn't eaten anything I had a cup of tea I hadn't spoken to another human being and this was usually when I was trying to figure out how to solve a bug or implement some things before I went to bed but what I found is that if I took a step back and had a break I went and got something to drink and I ate properly I went for walk or I spoke to someone that I'd had the headspace to be able to go back to that problem and approach it in a lot more open-minded and different

angle and it just was great for productivity again when you're on any sort of project you learn what you're good at and what you enjoy but you also learn what you're not very good at and what you hate and I found that really sort of take it took me a long time to be able to turn that into a positive and identify it as areas for improvements but also as a way of identifying what I really enjoyed and I think it's really important that if there's something you enjoy to take that passion and run with it and celebrate you so like I said about ticking off those mini achievements and share and ask your friends and your peers and share it on social media to like social media is a great way to meet that immediate response is back and feelings for the response to what you're creating from the active development community and ask there's no such thing as a stupid question

and even if you think the person is the smartest person in the world they don't know everything and asking is all part of the learning process and nobody ever stops learning so don't be afraid to ask and also this is important so believing yourself if you're doing and if you're doing some things that you're emotionally invested in yourself in and putting a lot of your time in you can often get to sticking points where you think you know I can't do this anymore I'm not smart enough for this I'm not good enough for this I can't do it but if you talk to the development community you'll find that everybody feels that way it's supernatural so you can do it and you just have to keep high so in summary set your goals clear please do code design just please remember you're a human being and collaborate share and celebrate and thank you for joining me today and I hope that there's been something useful for you to take away and I'm going to be in the wrap-up rooms

if you have any questions I have summary PDFs of all the lessons learned par or you can just throw a treat at me but thank you for spending your time with me today I really appreciate it [Applause]