Application of triple-A dev skills to healthcare and other non-gaming markets

excellent well I will try to keep this interesting and brief for whoever is staying because I know it's almost lunchtime so yeah my name is Peter lund I'm the CEO of a company called super genius we are in Oregon City the very end of the Oregon Trail for anybody who grew up with an apple - and we started our life so anyhow very quickly the the folks who look at gaming market sizes think that VR is going to be big soon they have sliced open the proverbial chicken and looked at the entrails and they think it's going to be worth a lot of money who knows whether this is correct or not but we do know that we're entering a new part of human-computer interaction and that it's going to be interesting and the other thing that's interesting about it is that even though

the vast majority over the next couple of years of this industry is likely going to focus on games and entertainment there's a very large part of the industry that is going to be things other than what we all do right now and so if you think about these see changes in technology that we've experienced in our lifetimes when we went for example from HTML to mobile there were a lot of web teams out there that could fill the gap and start making mobile applications the only people that can make 3d software are us and they're real time developers and there are not that many of us so that leaves us in very interesting and strategic places as people who make these sorts of things and we need to start thinking about where we want to fit in the market so supergenius is our company we started as an external development company way back in in 2008 we mainly did art and we expanded our capabilities into the full development stack over time and it's

been interesting Bart DNA is certainly in triple-a video gaming that's that's what we do we make good real time VR and AR here's some of the stuff that we've done with some of the larger hmd manufacturers and we like to keep at the very front and of where the technology is going working on interesting products to keep our really interesting craftspeople happy and as we were getting into virtual reality in particular we became really interested in the way VR talks to our brains because it talks to our brains in a way that is completely different from any media that humans have yet created when we move into VR our brain actually registers it as a memory of a place that we visited when you watch a film you have a memory of watching a film when you see a play you have a memory of seeing that play but when you go into VR

you have visited a place and it registers in your mind is a place that you visited and we know how powerful this is from all of those goofy little YouTube videos that we watch of people attempting to climb up a rock wall in VR and losing their hold and falling flat on their face it's because their brain talked to their body and believed that they were falling down and so we became very interested in how powerful this is what can we do with this connection between our media and our technology and our minds and how far can we push this and so we became very interested in the world of health care and therapeutics and as we started to dig around we realized that the folks who were doing best we're different than we were here are two examples of the types of therapeutic real-time that exists in the world right now this is a game that is created for specific types of neurological and memory related conditions it is the first piece of

software that is currently being assessed by FDA as a prescribed Abul substance which is absolutely fascinating over here we have a pain therapy it's it's used for burn victims and if you are ever unfortunate enough to suffer severe burns wound care is absolutely excruciating it can last for 45 minutes two hours if you're severely burned this comes from a company that that decided that maybe VR would be useful for distraction based pain therapies and so they created this little environment where you can shoot the bawls little teddy bears with stuff like this they were able to decrease the amount of narcotic pain intervention necessary by 20 to 30 percent which is huge and so looking at what was out there in the market and looking at the stuff that we do we noticed a pretty significant difference and the reason is the folks who are going into these

fields are medical technologists they don't know real-time it is not their world they know the potential value of it and they're trying to assemble these teams and they're trying to make things but we felt that we could do a little bit better that perhaps there was a model come on now there we go that that would better use all of the skillsets that already exist out there in the world and our idea was that the best therapies and the best ways to help people with our technology are to bring together the Triple A developers with the medical experts and the biomechanical engineers this had been done but no one had thought to talk to the Triple A folks and maybe they're doing it over at oculus or HTC but it's not public yet and so we started to look around for people who are doing things that interested us and happily got to work on some interesting stuff because of this guy right here dr. Albert Chi I know you guys are doing the best you can

thank you so Albert she is a trauma surgeon at OHSU that's the level 1 trauma center in Portland where we are and dr. Chi does lots of trauma surgery but he came from Johns Hopkins University where he specializes and specialized in a new type of surgery called targeted muscle reinnervation now when I first got in touch with dr. Chi it's because he was 3d printing pediatric prosthetic limbs if you think about kids they go through prosthetic limbs the same way they go through shoes and prosthetics traditionally are extremely expensive and so dr. Chi was was 3d printing these functional prosthetics and I emailed them said hey you know we've got amazing 3d modelers in their downtime we to make these prosthesis better just make them you know somebody can have a steampunk princess arm or an Iron Man arm let's let's keep the functionality there but let us do the art and design and we sat down and chatted for a bit

and he said this is fascinating and I love your idea but I can do one better we need to talk about this the modular prosthetic limb the modular prosthetic limb is the most advanced prosthetic in the world right now the US military spent something like a hundred and seventy million dollars on this thing it has fully articulated fingers you can attach it to an upper limb at any point of amputation so you can just have the hand you can have it above the wrist elbow all the way up to the shoulder oh it's Troy hello Troy the modular the modular prosthetic limb is an amazing piece of technology and it functions directly from brain output so the thing that dr. Chi does is to rewire the human body to allow it to talk to the modular prosthetic limb and when we move to this should I just tell you when we're moving to the next slide is that maybe I'm hitting the laser is there just a delay

yeah I've been hit anyway whatever I'll keep hitting the green top button it'll be fun so yeah dr. Qi does this thing called targeted muscle reinnervation he takes the nerves that used to serve the lower portion of the limb that no longer exists and redirects it into muscle groups that are also not needed for the lower limb and what happens is that the lower limb remaps onto the upper limb where TMR has been completed and so for people who have had this surgery if you poke them in their upper limb where TMR has been done they're actually feeling you poke them in their palmar and their index finger their whole limb has remapped their arm and it also allows them to control these prosthetics there we go this is Johnny Matheny he lost his arm to cancer several years ago he was the first person or one of the first people to get TMR in the United States and he's the

first person in the u.s. to osteo integration which means he has a piece of metal that runs straight into his humerus this is a much better way of attaching a prosthetic than than just having a harness because even if you have that prosthetic tightly attached to your stump it's still going to move around this allows him to manipulate the arm naturally he has the natural weight and feel of the arm in his hand it's removable so he can change the batteries and he can do all sorts of interesting things with it for example you know he can leave the arm over there and still flip you off from the corner with his arm we use mile electric bands to capture signals coming out of his muscles and interpret it turn it into kinetic movement the MPL has all sorts of different movements and functionalities including distinct finger movements he can hang loose you can turn the thing around he can turn this entire hand around because again

when your arm is a robot you're not limited by the meat that we all have it's quite interesting and johnny is learning to play the piano which is just mad by the way we do live in the future it's right now we're inventing early-stage Luke Skywalker arms it's a hell of a time to be alive friends the the arm functions off of just basic linear regression analysis you have to train yourself to use one of these things and it can take a long time to learn how to control one of these I've attempted to do it it is not easy it's not particularly intuitive and traditionally what you'd do if you were being fitted with one of these arms is to go into the clinic see a representation of the arm on a flat screen and then attempt to make it move in the way you want it to move so he'll be sitting there you've got your Myo bands you don't have an actual arm you see a robotic looking thing on a screen it's like okay I want you to flex your

elbow up and flex it down so we're asking people to a remember what it was like to control a limb that either they've never had or haven't had for a while be recreate this kinetic movement in a different way than there used to and then see attempt to map themselves in this movement onto a 2d screen so when dr. Chi and I started talking we started talking about what we could do with virtual reality and we decided to make a virtual reality prosthetic trainer and that's what we've been working on for the last couple of years so there's dr. Qi and Johnny Justin as it happens was working on a cowboy from overwatch when Johnny came into the office and the cowboy on overwatch has a robotic left arm Johnny's originally from West Virginia and so as Johnny would say well you didn't get a cowboy but you did get a hillbilly with a robotic arm johnny has

a distinct sense of humor so I think we do have a video of Johnny using there it is so we we created we recreated the modular prosthetic limb in VR Johnny uses this limb exactly the way he uses his actual prosthetic out in the world he's controlling it with his minds and thoughts and this is pretty darn good considering the fact that he's not having any sort of haptic feedback here to manipulate these devices the benefit for Johnny is that when he is wearing a prosthetic like this one his mind body actually Rhian habits the limbs so he feels like it is his this limb here the NPL does have sensors in the fingertips and on the palm that will allow Johnny to directly feel things and he's going to have a surgery later this year that should allow him to feel some basics

like texture pressure and perhaps hot cold so we're going from from motor tracking using the brain to direct sensory feedback in into the human brain which is going to be absolutely absolutely fascinating so a couple of takeaways here and a couple of places where where we're going to go next these types of conversations have allowed us to think differently about the work that we do we're still a video game studio we're always going to be a video game studio because that's where you hone your skill set and you hone your craft but as a craftsperson in this industry you might find a time where you're a little bit tired with you know making games for a buck here and there or maybe you want to do that thing and do something else and so we're now at the stage where we've been talking to the the Applied Physics lab at Hopkins dr. chi we're thinking about how we can

use these devices for specific and targeted therapies what we're going through patent cycles right now and and we're really really excited about what the next couple of years are going to hold it's not going to keep us from focusing on games because we have to have that focus in order to be the best at doing stuff like this you guys can do it too so good real development real-time development it's valuable in it's rare as we said and it's only going to become more rare in the next couple of years keep doing that thing most industries are not familiar with what we do yet and most of us in our industry are not talking to those industries conversations need to happen there are people on your team right now who are interested in all sorts of things they might be interested in Legos or botany or history or whatever the case may be I guarantee you there is a use case in there somewhere find some experts in that thing we don't know jack about

trauma surgery or really robotics we know a lot more about it now it's been a lot of fun find somebody that can augment what you're interested in and go talk to them and see what you can do together thanks very much thank you I think it was actually awesome and there was there's a particular doctor that works in the UK who was doing something very similar with the when the apps you talked about before and I believe he was as I talked to him but I understand that the tests that they did by having that kind of distraction therapy in an ice world was even more effective yes and there has been an arguments has been it's it's more effective than upping dose yes which is yes after yes down in particularly the burn victim side of things and and a fact that we have that physiological response Bible Inc being put into a virtual world of cold is to me magical literally magical it really is and the thing that's really interesting about the way we're doing

pain therapy right now is that it's it's distraction based and the use cases are pretty obvious you know you've been burned so you should be in a world that is full of ice and penguins and throwing little snowballs at them right what is the mechanism that causes the pain to diminish we don't know yet neuroscience has to research this are there specific targeted things that we can do we know that near therapy works for people who've had amputations it doesn't work quite as well if they're experiencing telescoping which is when the the phantom limb is just sucked into the stump and it feels like your whole arm has been compressed in into the end of the stump and you can't you can't do that in mirror therapy the same way we can do that in VR we can show you your hand right here at the end and you can pull it back out and so these are the types of targeted therapies that were interested in exploring and then I think science needs to come back and figure out what the specific mechanisms are so

that instead of just kind of the shotgun approach of yeah let's put you in a cold world and see what happens maybe we can do something even better so that we can bring those numbers up well attacking and this particular doctor also does a lot of remote work so he could is using hey aren't they our help with remote surgery so it's there's this practical kind of realities there's also these things that we're talking about with games and and I think also one example I loved was to do with the anxiety that kids were having over injections yes vaccinations and a particular game I think was Spanish team I can't remember did is amazing experience where effectively it's a be walking around and then they they tell you they're going to sting you mm-hmm and that's when you do the injection and again there's just like massively it changes the outcomes and the willingness to get that injection which I wish my son had had access to when he was that age yeah sorry I'm rambling on so people you know please let me have other questions so yeah we've got we've got plenty of

time for questions and few people so we can be we can be conversational yes hi Peter so nice to meet you um one question they have for VR is that so many people have thought of it as accessible because it's so expensive and now you guys this next level amazing thing that's happening like is that in your radar right now of how to make it more accessible for like everybody no and and here's why they're there there are two reasons which isn't to say that we're not thinking about that thing because we're thinking about it all the time the the cost of good VR is coming down really drastically so we know that oculus is gonna release the quest pretty soon it's going to be tethered 'less standalone room scale this is great and they're probably going to release it at a price point around what you'd expect to pay for a console so the cost is going down the the systems that we're using here of course are old-school rifts you're gonna need that $2,000 laptop you're gonna need the rift and then of course we have to hack as well anyway the reason why we don't worry about it in this

world is that we are in the world of health care which has completely different pricing models and so if we think about a pharmaceutical if you think about the actual cost and the potential cost of taking a painkiller long term versus you know spending $500 for a device you can send home with a patient and then then they can check back into the company and then you know $100 a month for a licensing fee or even $500 a month for a licensing fee in the United States I know everybody isn't from the US but our health care system is terribly exploitive and so you know here you can get away with all sorts of stupid things and hopefully that doesn't last too much longer but that's the way it is now so there are ways for us to get around it I think it is still a concern when it comes to take home therapies that a lot of people are doing right now you know meditation physical therapy things like this we're focusing on targeted clinical therapies testing

them out in healthcare environments and then possibly getting them out into the world and I think by the time have something that's ready to ship and is approved by FDA the cost for the devices will be low enough that's a breach I strongly recommend the NHS your profits sadly too many of our folks want your system I don't exactly know why but also think 2020 is an interesting year obviously there's a nice synergy with vision and 20/20 vision so I suspect there'll be a lot of branding people wanting to get new types of device out obviously we've got the quest we're going to know that ps5 is coming it's gonna have a strong VR focus to it I suspect end of the year we'll see something for Apple I don't know but that's my belief so I think we are getting to stage where if if if the pundits is talking about 2023 is a just a tipping point for visual computing which is mmm a lot of people seem to be

saying my suspicion is that we need to start seeing some of those consumer level devices coming out I've been saying for a couple of years that we're in the Pocket PC palm pilot stage of what mobile devices are now that light that's where we are inferior I'm that old too any other questions so I'll add one because I think it again I think this is really critical so I'm I've been talking to people in the mid-21st there is their use they're quite advanced use of what they call simulator training yes and I think that's not just about military is also about kind of any blue light services the level of ability to do to put people into situations and train is incredibly powerful but the systems they tend to use are incredibly outdated they're using the same game engine that was used for operation flashpoint back in like 30

years ago with it and that's what they've bought for the whole of NATO Oh bless and know bless and yet you know so you imagine meeting in a room with you know a bunch of these military types looking like I do scaring the hell out of them but it's but what I'm saying saying all this is because I think there's a wider question and health care is a very key area and I particularly because there's a lot of positive more out moral and and kind of you know human values that comes out of working nature it but what do you think is we're not open enough about how we integrate as an industry in games with other walks of life is there a general problem there ABS absolutely absolutely yes there are reasons why we're not in these areas and it's because we don't like them and because we look down on folks who sell out and go and work in these parts of the industry we all know this the thing is

when you take that stance and it's you can make an ethical argument for it there's a moral reason why you're not there you're not at the table and you're not bringing your ethics to the conversation and so that is the thing that is so important and why we need to go out there and have these conversations that sit at the table because we know a the best way to implement these systems be were gamers and storytellers we think about ethics and stories and the way people talk to each other and all of these moral problems all the time and because we tend to be knowledgeable about these things we've built this this universal system of ethics that is not perfect but it's also better than the status quo in many of these industries so I would encourage people to set aside their their unwillingness to go and have these conversations bring your preconceived

notions but be honest about them and go and talk to people who are very very different than you I think that's one of the things that we one of the mistakes that has been made in the last couple of years is saying you know this is so wrong or this is so distasteful that I'm just not going to talk about it or talk to these people that has to stop you have to go have the conversations so that you can change the way things are otherwise you can't and I've kind of that a little bit further issues I think unless we have those conversations we can have our own ethical arguments in our games as effective you've abrogated your responsibility to be an ethical actor acceptor absolutely correct yes I think what we've found really interested by talking to these MRD folks is you know again you're partly influenced by working with Tony on reality clash you know having the idea of thinking about actually this is how that particular group thinks and operates if I'm going to be able to make a realistic game

dealing with that conversation there's actually a whole bunch of really interesting stuff that they have to do so artillery people have to go through a whole series of checks and balances and ethical decisions and operational criteria and they have to actually think about the impact of the collateral damage but we don't ever deal with collateral damage in traditional you know shooters and combat games all we do is try and get the biggest possible explosions yes we never add the actual moral implications that real military people have to think about so the underlying point is that if you talk to people who know different things than you and think differently than you you might learn something a bit