[Music] i work at a place called mount sinai it's a hospital for those of you who haven't come across it before um it's a hospital system in new york city there are eight hospitals several hundred clinics tens of thousands of clinicians and and millions of patients um that go through mount sinai every single year and my role at mount sinai is to run a place that we call the abilities research center and we really uh built the abilities research center because of a couple of uh horrifying statistics and i'm gonna i'm gonna run those by you in rapid fire just to give you a sense of where we come from um the first one came out in forbes magazine in 2016 and it was about health technology companies and they said that 98 of digital health startups or zombies what they meant by zombies was that 98 of digital health
startups uh were failing before they got to touch a human being so you know on average these these companies were raising a few million dollars apiece and then um 98 of them would die before they actually got to help someone um when you think about the fact that uh the health technology industry right now is around 85 billion dollars a year and i think the nih budget is around 35 billion dollars a year that's a tremendous waste of money um on technology that could be actually be helping people so that was the first thing that really got our attention the second thing that got our attention was that in the pharmaceutical industry we've heard of this concept of the valley of death this idea that to take an idea in healthcare from the moment that you conceive of it the moment that you're like i've got something and i want to i want to develop it further all the way over to
the point where it's actually in common clinical usage um that period of time is 17 years in the united states and this is not um we're not talking about medicine here we're talking about an app we're talking about you know um a wearable we're talking about very very common technology and it takes this long to cross the valley of death um obviously there are a lot of issues with this um first and foremost who who is still using the same phone that they were using 17 years ago and in fact who had a phone 17 years ago let's start there this is the problem with integrating technology into healthcare so the mission of our lab and our space is to take technologies and rush them through clinical trial protocols get them in front of the fda have have the right conversations and get them into the hands of clinicians as quickly as as humanly possible and is responsible to do so
and ever since we've started the lab we've had a lot of clinical trials starting up coming out of the the lab and they range from everything from the the simplest of mobile app technologies i'm just going to give you a quick example uh this is a technology that has been the name has been changed it's now called well path but it is a app for suicide prevention um where uh what we have shown is that when people who are at risk abuse of suicide use this app um they can re they can experience a 35 reduction in negative emotions that bring on suicidal thoughts within two minutes of interacting with the app so we're talking about point of care emotion regulation so we're rushing this through clinical trials um as we speak um and and working with padme it's called well path yep um and uh it follows the uh path technique problem adaptation therapy by a a psycho a psychologist called demetrius kiosas
so this would be an indication of a you know low-tech technology that we're working with right on the other end of the spectrum this is a technology called a stent road this is developed by a neurosurgeon at mount sinai um and what we're looking at right now is an endovascular stent something that ordinarily goes into your blood vessels and holds them open but it has electrodes on built into it so this technology will go into the superior sagittal sinus which is the blood vessel you're seeing it go into now and as um as it is withdrawn um it will heal into the blood vessels so you're about to see what happens over a few period a period of a few days once it's healed into the blood vessel it can actually record brain activity uh very very similar to something like eeg but internally so there's no sort of sensors getting knocked off or or external noise affecting the recordings once you're recording eeg
we can um implant it into someone with say als who can only communicate with eye movements and rather than having them communicate at a rate of two words per minute because they need to look at the the letter that they want hover over that letter and then wait for the hover to initiate a click they can look at the letter that they want think click and the system will click for them accelerating their uh their communication rate to about 10 words per minute so that's the goal of a project that we'll be starting at the end of this year in our first and human trials with this technology so that's kind of the scope of the projects that we work on all the way from you know mobile apps all the way out to to implantables injectables um and and really fun technology i really think we got a cool job because we just get to play all day long with with fun fun tech but today's talk is about pain so um one of the projects that i'm really so
proud of and excited about is some of the work that we're doing with pain unless you've sort of not been paying too much attention to the global conversation recently pain is a big problem um especially in western countries um especially as it pertains to the opioid crisis and the way that we manage pain currently especially chronic pain and i wanted to um i wanted to take a real big step back because typically when i'm talking about the role of virtual reality and pain i you know i sort of have to sort of run through it but i figured um today we could take a little bit more time just to talk about how virtual reality might help someone's pain and we can start with uh the neuroanatomy uh 101 so we have areas of the brain that are focused on producing movement and sensing things that are coming in from the outside world and what we've known about the brain for a little while is that the sensory and motor areas of the brain are actually really well organized you can get quite
a nice map of the body from the amount of brain tissue that is assigned to each body part and what we stand to notice is if you happen to use a certain body part more more brain tissue will be assigned to that particular body part and so in most people um what you end up with is you know a funny little guy like this which is your your motor homunculus where um what you see is how how many how many bits of brain brain tissue is sort of assigned to each body parts and they're represented by being larger in in the picture uh if you have more brain tissue assigned to it um and up until around the 70s or 80s maybe even the 90s in in some of the soil moving hospitals this map was thought to be static it never changed throughout your life this is this is you for the rest of your life but what we started to see was actually it did change so
it had the potential to get better it had the potential to get worse it had the potential to always be dynamically changing depending on what sort of trauma you were exposed to or what sort of skills you were learning and practicing so when we think about this in the context of pain what starts to happen is let's say for instance you have a you you have an injury to your hand and you have chronic hand pain as a result of that injury what starts to happen with your motor and sensory representations is not only do they start to react to the the hand pain in a way that you start avoiding movements you start using the hand less you start to learn to use the other hand for instance because you know that using this hand will hurt but as well as that you also start to fear using the hand so a really great researcher uh by the name of uh david butler actually showed that uh and and his colleague laura mosley showed that um
when we when they put a whole bunch of chronic pain patients into a mri scanner and asked them to imagine using the painful hand and then they had a whole bunch of blinded radiologists reading the brain scan and saying and trying to diagnose the patient all the radiologists said oh you brought me phobia patients because what they were seeing were the fear centers lighting up so not only do they start to see fear but they start to see loss of that hand representation they they actually described it in their research as blurring the hand representation starts to get very sort of confused neurons from from other representations like your leg or your arm start to take over your hand representation and what that results in is this weird dysmorphia where you don't really know where your hand is what we know about chronic pain patients now as if they have chronic hand pain even the task of looking at a magazine and telling you if they're looking at a left or a right hand is harder than it is for an average
individual without hand pain so their whole relationship with their painful limb gets thrown out of whack and so once we started to think about what we could do to to fix this problem a whole bunch of clever people started to realize that um the nervous system can be tricked in lots of ways um to to uh to rebuild a relationship with the hand um we have this we have this whole network of neurons that are affectionately called mirror neurons some people like that name some people hate that name but effectively what these mirror neurons are are a network of neurons that will fire and get excited whether you're performing a task or just watching someone perform a task so what that means is just the act of observation or just the act of imagination imagining you're observing something can actually trigger a whole net network that is similar to the
network that is triggered when you're moving which to some seemed like a really clever way a really clever back door into the nervous system if someone who was in so much pain that they couldn't even think about moving and so a really brilliant scientist by the name of vs ramachandran came up with this idea of mirrorbox therapy and so his idea was for people who were missing a limb so they had what was called phantom limb pain in their missing limb this this sensation that the missing limb was still there maybe it was itchy maybe it was cold maybe it was in a cramp but the point was it was there it was hurting or being very very annoying and there was nothing that they could do about it and so what vs ramachandran did was he said okay well i'm going to put your your uh your missing limb into a box and i'm also going to put your intact limb over here
and on the side of the box there's going to be a mirror and when you look in the mirror what you're going to see is the mirror image of your intact limb and i want you to start scratching at it i want you to start stretching it whatever it does to give you a relief and it was elegant it was wonderful it was simple and it really really uh in you know a thousand studies out there now showing it really really does help a lot of people who have phantom limb pain and other forms of chronic pain um but you know we like to be difficult so um it doesn't help everybody and we still had a problem because at mount sinai we we treat a lot of people with um chronic spinal cord injury and people with spinal cord injury have this this uh condition around 80 percent of people with with chronic spinal cord injury also have a condition called chronic neuropathic pain this is a very very upsetting intractable form of pain
that is basically your injured nervous system telling you that there is something wrong with one or many of your limbs but there's actually nothing physically wrong so what these people go through on a daily basis and we see thousands of individuals with this this issue is at all times 24 7 um their their brain is telling them that their leg is in a block of ice or is on fire or is unbelievably itchy or there's bugs crawling on it or or just throbbing and there's nothing they can do about it it doesn't respond to pain management so opioids don't work um they they have other sorts of drugs like gabapentin they have a varying a varying rate of response um and less than 30 of people respond to to medications for neuropathic pain and mirrorbox as much as it would be great if we could help these people with mirror boxes
they like the basic requisite for mirrorbox therapy to work which is the ability to move another limb that looks like the limb that is hurting so that was when we actually started thinking about maybe we could use virtual reality for this um and it was you know it was a pretty simple idea it seemed logical right you know um so we started playing around with with virtual reality and we we started creating a bunch of hand environments and leg environments for people with chronic neuropathic pain and you know i've got to say you know humble beginnings i'm going to show you guys some pretty embarrassing uh virtual reality environments here but um you know these were just real simple things that people told us that they missed doing um playing silly games with a sibling or a friend playing cards driving a car and we used the um cheapest 360 degree camera that money could possibly buy and we started recording all of these
experiences for people so that they could throw on a headset look down and after years and years of not having arms or legs moving actually have this experience of their arms and legs moving and what we've seen has been really staggering so what we've seen in a cohort that keeps growing so now we're at about 35 or so individuals who have worked worked with us on this protocol these are all individuals who have had a spinal cord injury for at least five years they've been diagnosed with chronic neuropathic pain this entire time they haven't changed anything about their medications for at least six months so we know that their symptoms are stable and what we're seeing is that there is on average a 40 reduction in their symptoms within 10 minutes of exposure to the virtual reality environment so this has been a really impactful intervention for them um just off the bat um and so we're really excited about this because even with a crappy
environment we we can do something and i might i'm allowed to call it crappy you guys aren't allowed to call it crappy but i am um and you know so one of the things that we see is as a really strong potential here is that this is woody he's one of our like um uh serial um volunteers for every single project that we've ever run and one of the ideas is well any smartphone google cardboard and let's get this content on youtube and off you go so with that in mind um i want i want everyone to uh take this down we've got a whole bunch more crappy virtual reality environments on youtube right now on on the petrino lab channel we're running clinical trials on the east coast so if you know anyone on the east coast who has a spinal cord injury who has chronic neuropathic pain send them to send them our way and i'll put all of our contact details at the end of the talk
if you know anyone else who is not on the east coast who wants to try something out that is non-invasive not a medicine and pretty easy to try out just direct them to this youtube channel get them a google cardboard and you know the internet will do the rest and you know fingers crossed um so what we're doing right now is now that we've seen some really powerful initial responses to the virtual reality we're trying to delve into why the virtual reality works how we could make better with virtual reality environments how we can collaborate with people to make really high quality virtual reality environments and how to keep it sustainable and all of those things one of the main open questions that we have right now um about virtual reality about virtual reality which we are currently testing out in in the next clinical trial that we begin in may is why is it working so i gave you a whole bunch of theory um about why i think it works
but and i hope i haven't biased you because there's there's another reason why it might work so the first theory is of course maybe we're rebuilding so you've got we've got our homunculus with a blurred hand representation and maybe over time as you're watching the virtual reality environments and and you're starting to take it all in maybe over time that homunculus is being completely repaired and and that person has regained a healthy relationship with their limb and they're going to be okay but there's also another reason that that it might be working and it could just be distraction so it could just be that we're creating uh you know a home away from home for our viewers where they just get to think about something other than their pain it's a very novel experience and therefore they feel this relief um in our initial pilot trials what we did was in a repeated in a crossover design study we actually exposed people both to
um the distracted vr and the somatic vr the the so the vr where they were actually watching hands do things as well as what we called uh distractive where they couldn't look down and see a body at all but they were kind of like a floating eye going through an aquarium and things like that what we saw in our initial data was that both of them caused significant pain relieving effects the somatic environment caused more of a pain relieving effect but it wasn't significant yet so the next study is to actually expand that and and see if we can uh see how things um peace out over time the other thing that we hypothesize is that the distraction virtual reality might just be kind of like a band-aid and it might stop working over time so if you just keep showing someone distractive uh content maybe after a while they're like oh yeah well virtual reality is kind of old hat to me now so i need something even more immersive or more extreme or
something like that whereas with the somatic environment maybe you actually really are rebuilding a relationship with that with with their hand or their leg or their uh you know painful body part the other thing that we're looking into in a lot of detail is trying to understand um how certain psychometric properties of the people who are viewing the vr actually impact their response to the vr so again in the pilot study that we've done what what we saw was that individuals who um who rated high on a scale called the immersive tendency scale has anyone ever heard of the scale by the way a couple of people yeah it's really interesting it's this scale that actually tries to delve into how likely you are as a person to be drawn into a virtual reality environment it asks hilarious questions like you know if you're walking through a room and a tv is on how likely are you to stop and get drawn into the tv you know how scared do you get after
scary movies you need to sleep with the light on or you just forget about it immediately and from that it tries to get this sort of measure of how easy it is to immerse you what we saw was that people who scored high on the immersive tendency questionnaire overall had better responses to the vr but we also saw on a separate questionnaire which was the presence questionnaire this is a questionnaire that says regardless of all that immersive tendency stuff just how in the moment were you in the one in the virtual reality environment we showed you and uh in that case and this effect was stronger than the last effect people who scored high on the presence questionnaire um had much stronger effects to the virtual reality environment so again what we're trying to do is understand a how realistic do we need to get to hook everyone in or how unrealistic do we need to get because actually one of the things we've been playing around with is like rather than trying to perfectly match skin tone and
uh masculinity or femininity of a hand and hairiness and all of these sorts of things maybe we just give everyone a blue alien hand and so you know every everyone is a you know at a sort of similar level so these are things that we're playing around with to see um what matters what doesn't and how quickly we can scale this experience to to sort of draw everybody in um and uh it's been a really you know interesting experience um trying to understand how to explore agency within a virtual reality environment and we really hope that um this is something that we will be able to scale out to lots of people who are in pain and i can't stress it enough lots of people who are in pain at the lowest possible cost because the populations that we're dealing with in this case don't have a lot of money to spend on um on on solutions like this so that's why even even sometimes we get pushback when we're talking about a google cardboard that's the sort of demographic that we're dealing with so we're really
trying to uh push into ultra low cost solutions for this because we really need it right now so thanks everyone for listening um this is all of the work that i've been doing is supported by a lot a lot of amazing people it's not just me doing all the work um and uh all of those things are where you can find us um we're best at instagram um we're okay at twitter we're not so good at facebook i think we're too cool for facebook now but you know don't tell them but um but thanks a lot [Music] and i think we've got some time for questions yes i have a question yeah i know everybody probably has their pet condition that they would now like you to address so i have pretty severe scoliosis and of course everybody says nobody knows what causes it can't possibly be cured um all you can do is adapt to it
can you think of a way this could be adapted to scoliosis because it affects millions and millions of people and it's a it's one of those absolute dead ends in terms of the medical profession yeah um so the question was um about a clever clever ways that we can we can treat scoliosis my my experience with scoliosis i don't think it's a dead end so much as it is um one of those things that um education and advocacy for real hard work doesn't get enough of a um you know it doesn't get enough of a look in there's a lot of different causes of scoliosis so and when we're you know particularly remiss we call it idiopathic because we don't know why it's happening it doesn't actually mean that it has no cause it just means that we don't really understand why what's going on and currently the best thing that we have for it is to do very targeted very personalized exercise therapy for scoliosis to
strengthen the muscles that are going to you know sort of push the spine in the other direction to work on posture to create environments where people are constantly being reminded to draw themselves up and and do all the right things for for their body um but the the problem is people lose interest in interventions like that very very quickly because um it's not a silver bullet and when i say people i often mean physicians and clinicians because what they really want is to be able to say here it is here's the pill now go away and get better um but that's not the case with scoliosis it's a lifelong journey and it and it needs to be addressed um consistently over time but so can you see it can you imagine a vr solution something that would help people do that i i think um more than vr i i don't even think we need to go as far as vr i think we just need uh um you know rip connect but we need like motion capture systems that give people continuous feedback about
how they're um how they're moving walk them through exercises this is a perfect example of a a condition that would benefit from having daily exercises in front of a motion capture camera that tells you exactly if you're doing the exercise correctly and if you're maintaining you know good posture and good form and how cheap are those cameras right now well they just got discontinued so they used to be about 250 bucks but now they're uh you know on ebay like 800 bucks or or a thousand bucks if you're lucky but um connect said they are coming out with a connect three so we'll you know cross our fingers and if if the microsoft gods are listening you know we see how we go thank you okay no worries uh this is sort of amazing because i'm currently developing a real-time motion capture virtual uh cinematic visualization system but um we're deliberately taking an approach that does not use uh vr headsets uh partially for inclusivity partially well there are many reasons but i personally don't like to sweat in a vr
goggle but um we are deliberately rendering the body that you see in a an abstracted impressionistic way so that we don't generate negative uh feedback loops of you know idealizing ourselves in ways that are not really healthy um but i wondered would you comment do you think all of this sort of research is equally applicable to a non-vr but still somewhat immersive system i absolutely do i i and one of our [Music] one of our control conditions is actually viewing just in a 2d screen because if we can do this in 2d let's do this in 2d absolutely i totally agree i would love to talk to you about possibly being a consultant if you're yeah later thank you so much on your website do you have information about patients like what are the requirements to be a patient yes so the um on the on the website um you can link to all of the clinical trials the the vr clinical trial is already up on clinicalclinicaltrials.gov
which will tell you exactly how to um uh you know who who uh what our recruitment criteria are and who can be who can be a patient so you can be functioning like i have a spinal injury i take medicine every day all the tingling burning but i'm not in a wheelchair that's fine that's fine anyone with a diagnosed spinal cord injury who experiences neuropathic pain welcome in the trial thanks hey i was just curious if like player interaction has been like an accelerator or something that you've uh experimented with or seen oh decide that again sorry uh player interaction has been like an accelerator for uh rehabilitation have you noticed any differences uh player interaction as in like uh um oh i see what you're saying yes sorry um so all of the trials we've done thus far have been passive uh visualization simply because we were we wanted to create the most inclusive experience for
everyone with spinal cord injury and that we have a lot of people with spinal cord injury who can't move um but we have done a lot of work with uh gamified environments in the past and we've seen really positive results in improving motor function and and things of that nature in fact we've seen them outperform conventional care thank you so these have all been a lot better questions than mine but mine's more on fda regulation and moving forward with fda regulation i know that eventually if you're planning to get this into a patient market in the hospital how would you move forward with the fda regulation if you haven't already and if you have yeah so i think it's a great question i i think we've always got to be mindful of these sorts of things you know when it comes to regulation of technologies like this it really does just come down to um a what are you charging for it so in a
case where we're not charging anything for it um you know and and it's not particularly high risk intervention it's just watching videos take care give good instruction and you know don't do anything foolish when it comes to actually charging for something like this so let's say we wanted to start a business um i think that uh just by creating content i think you know my if there's anyone out there listening who wants to create a business out of this do vr netflix and don't make any claims about it don't tell people it's going to you know revolutionize their pain management but but do put it out there if you have a hundred people sending you in videos saying it's completely changed their life put those on your website and don't make any claims yourself and and off you go again it's it's not a
very um uh you know high-risk thing that we're talking about here and there should be no reason for um the fda to try and block you creating a repository of content of limbs doing things that that can be viewed from a third person point of view so long as you're not assigning any sort of therapeutic intent to it once you get enough pilot data to actually show that you're making a difference different story go to the fda get an indication because then insurance can cover it then you know then you can actually start to get doctors prescribing it and and all those sorts of things and and that's where you can you know start to scale more rapidly but in the early stages i it's just content you're just making content yeah uh if you have time afterwards i'd also love to talk to you a little bit more about that sure thank you hi um kind of made me nervous uh so i'm from the fda
[Applause] i'm part of a new initiative for going towards medical mixed reality and so actually we have a collaboration with medstar rehabilitation we actually last week just talked to um the the pt's uh working with patients with spinal cord injuries and one thing that they mentioned is you know for the patients who are younger it's so easy to get them in vr um but for you know like the um they said they had a 56 year old woman who they put her in vr and in five minutes she came out and she says that was great i loved it but that was her vr experience you know they offered to put her in some different kinds of games but she you know she was like no that that was good that was my vr experience so what would you recommend how can we get
more patients in vr um yeah cool so first of all i just want to say that uh for the last 10 minutes i've just totally blacked out so i don't know anything that i just said it can't be can't be held responsible for it um but secondly i think it's a great question and i think um this there's no silver bullet for any condition um least of all pain um i was i was having a conversation with a vc the other day about another pain intervention that we're running through clinical trials in my lab and the vc said i only want to invest if it's definitely going to work on at least 80 of pain patients and i said well then don't invest in anything in pain because that's just not a reasonable thing to experience and and we have we have the same in um all aspects of rehabilitation whether it be robotic rehabilitation or um gamified rehabilitation with motion
capture or or just regular old you know conventional otpt some people respond other people don't some people have preferences go with their preferences um the way that we we really like to compare it is athletic performance in our lab we work with a lot of high performance athletes when i have a world champion athlete that doesn't want to do a certain exercise i find a different exercise for them i don't say oh well stick with it because i'm i'm the coach that doesn't fly and we should keep that frame with all of our patients because um when they're telling you this isn't doing much for me well what that is telling me is i need i know i need thousands of hours of intentional work to get a get a satisfactory outcome and if they're not feeling it from the first session i got to do something else with them because i know right there and then it's not going to work okay well we're out of time okay we'll go gather in the hall thank you everyone thanks for listening
Auto-generated captions from YouTube — names and numbers may be misheard.
Game Developers Conference · 2026
Comments