The Making of the Xbox Adaptive Controller

[Music] so I'm Bryce Johnson I'm the inclusive lead for Microsoft devices I work on both Xbox and surface devices I'm kind of work on our hardware I'm Erin must infer I'm an occupational therapist I work at Craig hospital and if you're curious why I'm here I will tell you in a minute so this is the Xbox adaptive controller I'm sure some of you seen it if you haven't seen it please go to our demo hall go to the expo hall in South Hall we've got them set up there you can kind of see it in action we can show you it as well we like to call it Zack not all of us do I do I like to call back Xbox adaptive controller exact less is less of a mouthful you know and this is some of the friends that like play well with it the one important thing about that we found with this device is that the having a rich ecosystem of

peripherals is really vital so where are we going to start wait I had something and then I put it down oh here it is we're going to start with this this is a traditional Xbox controller as we like to now say traditional Xbox controller classic we love this thing I personally think that we make the greatest controllers your opinions may vary but I think they're the best but if you look at this controller through the lens of exclusion it's very possible that we need to recognize where this device can leave some people at and the thing is this device makes a lot of assumptions about how you use it right all controllers do it assumes that you have two hands to hold it right it assumes that your thumbs have a fluid range of motion and that you can hit all these buttons all right how many buttons are

on a controller does anyone know anyone have a guess actually I'd love to hear some guesses just call some numbers out someone's thought us before it's in 17 yes there's 17 buttons on an Xbox controller that you hit with two thumbs and your two index fingers right that's a lot of density that's a ton of density that's more dense than a keyboard so the other thing that we assume about this device is that we assume that people have the endurance to be able to hold it all day long right and that's something that was really important for us to recognize early in our process that while we loved this device that this was the bear this is how we approach inclusive design at Microsoft recognized exclusion exclusion happens when we solve problems using our own biases as

Microsoft designers we seek out those exclusions and use them as opportunities to create new ideas and inclusive designs that's how we think about it the next one learned from diversity human beings are the real experts in adapting to diversity inclusive design puts people in the center from the very start of the process and those fresh diverse perspectives are the key to true insight and then the last one solve for one extend amenity everyone has abilities and limits those abilities designing for people with permanent disabilities actually results in designs that benefit people universally constraints are a beautiful thing so so those are the principles we live by and I've already talked about recognizing exclusion with our controllers so this is where we start the next process which is learned from diversity this is sergeant Josh price we met Josh in 2015 when him and CEM Jones came to

the Microsoft campus to work with us and our one-week hackathon and it was it was so enlightening to kind of work with Josh and Ken because you can kind of see josh is an amputee actually and what type of amputee is Josh I don't I mean I thought oh my goodness hello Josh looks to me I don't know him but he looks like he isn't above the elbow amputee meaning he does not have an elbow joint he's missing everything below that right right so you can see that like Josh basically has a joystick that he puts under where his elbow would be and then he uses an elite controller on his hand and then he has an array of buttons that he gets in front of him the reason why this was so important and so impactful for us was that you know we got to really see what Josh's life would be like if he couldn't play video games and we have a lot of veterans on our Xbox Hardware leadership team and they got to see what it would be like if Josh

couldn't play and I will say like in you know it really fundamentally changed like how they looked at controllers it was super powerful so powerful that in 2016 we went back to the hackathon we kind of went away for a year we kind of we had some ideas kind of pirkled percolating an Xbox Hardware his leadership team basically assigned us three interns to go into this 2016 hackathon to have a new version the prototype to basically kind of iterate on what we did in 2015 2015 was really hacky 2016 was a little bit better you know we had we had something and I'm gonna play a video that's of Rachel who is one of our interns [Music] people use a ready-made adapter to the

Xbox one controller so that if they want to externalize the inputs for their own custom use for their physical needs they can easily just grab it off the shelf and then plug in some buttons and it's right there ready for them to use so that's Rachel and if you can hear that music it's because I have minecraft on in the background of my PC and I really am digging it okay yeah yeah super intuitive yeah so actually I was maybe I should for this one in all due respect to our partners but no you know what it's fine it'll be fine um so after 2016 we started reaching out we worked with warfighter engaged we had some ideas we started reaching out to other nonprofits and charities we started working with AbleGamers and if you who's familiar with AbleGamers a lot of people in here great a lot of people aren't so I'm going to play this video and please indulge us it's really important for us

to like work with these partners and I really want to make sure that we give them their due for over a decade AbleGamers has been on a mission to bring inclusion and improved quality of life for people with disabilities through the power of video games and in 2018 we had the honor of helping many incredible gamers gaming accessibility doesn't really have a one-size-fits-all solution however and sometimes it takes a couple tries to get it just right as was the case with Ajay Ajay has cerebral palsy a condition that makes holding and usually standard control are difficult the first control we gave AJ wasn't a perfect fit so we went back to the drawing board to make him the perfect controller which ended up being a much better fit and got AJ back in the game destroying his opponents in the process we found out that AJ's fiancee Regina a gamer who also lives with cerebral palsy had taken the first controller and now they're able to play games together in ways that they were unable to before these stories like these that inspire us to create more and all of this was only possible thanks to the love and support of the generous able gamers like you or with us every day the belief that gaming is for everyone together we're changing the world so everyone can

gain [Music] sorry minecraft sorry we'll get there we'll get there you just might have to give us a minute to get back to it okay sorry one second oops no slide snafu because of minecraft music where were we so the one thing I will say about AbleGamers AbleGamers has a service that they offer developers called player panels if you want to get people with disabilities viewing the things that you make and I cannot stress enough how important that is the one

thing we always say about the adaptive controller is that we designed it with the community we didn't design it for the community it's a very important distinction AbleGamers player panels were invaluable to us when we did a beta of the hardware for the first time Microsoft did an external beta of hardware we typically don't need to go outside of our employee pool to do betas of hardware we have hundreds of thousands of employees so but when we did this we know we we knew we needed to and AbleGamers gave us 20% of our they recommend 20% of the people who are in our beta it's very very valuable for us for over a decade able to skip that we already watch that so next next organization we worked with special effect we talked about AbleGamers there in the US special effect is in the UK I think what we learned from special effect was that and and all the partners we worked with that everyone had goals that aligned everyone

wanted to basically make people be able to play but their approaches and how they went about it were all different and when we were creating the Xbox adaptive controller we needed to create a device that not only like worked for one organization but worked for all of them and then even beyond those organizations we had to think about who is going to use it so the thing about special effect because you're in you K you can kind of like they can kind of get in a car they can put an occupational therapist and a technician in a car they can go to someone's house they can build them a controller and they can leave you can't really do that in the US the geography the US doesn't allow us to like have a mystery machine that kind of goes around building controllers for people really cool really cool super cool so that's that's how special effect works and I'm going to just show you Oh before I show you the video the one thing that was also super powerful about special effect is they're really good at sharing the work that they do online so their Instagram feed before we actually met them in person their Instagram feed was

basically a huge source of inspiration for us [Music] [Applause] it's been yet another amazing year from all of the special effects thank you and it's always fun to hang with those people so oh now we're gonna get into it so another one of the charities that we work not charities well we're a non-profit Hospital we're a hospital that's what's I think the biggest difference and so this was still in 2016 we met Erin and Patrick here and I will

say that from our very first sort of call which was this one I'm a total creeper I did not know he was recording it so from her very first call it was apparent and amazingly wonderful that the approach that Craig Hospital had was unique it was different than AbleGamers it was different than special effect it was different than more fighter and that's what made it so valuable to us yeah and you know I will get into why I guess why we're unique in a bit but this first call for me was it was kind of a proving ground like in terms of just feeling out at this point I didn't know anything really about the project but it was just a conversation about accessibility in gaming and what we were doing and the problems that we were having and so for me this was this was pretty cool to start having a conversation with someone in the industry and to be kind of speaking the same language in terms of devices and

workarounds and things that we were using because especially in the hospital setting sometimes you're taking your best shot in your best guess and you're hoping that you're on the right track I did stock special effects to Instagram page as well so that's kind of where some of my ideas came from too but so that's kind of what this meant for me yeah yeah and I mean I think for a lot of people on the team - because Craig's in Denver it was it's actually the closest to us yeah I'm up in Seattle and so we would send a lot of the team there and so for a lot of for many members of our team it was their primary and kind of initial experience it was their first sort of foray into this world so they were really affected by it yeah and that's something I probably took for granted and it took me a while to kind of see that impact because I'm I'm in a hospital every day so there are a lot of things that are like quote unquote typical day for me that the team at

Microsoft probably didn't hadn't even been exposed to so I think it raised a lot of good questions in terms of situations or design ideas even that they might have to account for in a hospital environment so I'm kind of showing so I'm showing what I can show you well we originally told the folks at Craig Hospital and you can see that some of the things we're talking about our copilot does anybody know what copilot is some some folks ian does just good copilot is on Xbox one and Windows 10 it's the ability to take two controllers and tell the system that their one controller right it was one of the things that we did really early on to address this notion of like that controllers are meant to be used by this like this so we saw that if we took two controllers and we we made them mirror each other on the console all of a sudden someone could use controllers like this you can imagine that me having another one so they could split the inputs apart and at

the same time we knew we were going we were in the process of developing the Xbox adaptive controller and we do really want people to use the Xbox adaptive controller with the traditional controller so yeah that's how we kind of got into copilot there so I want to kind of go back to this bit just to kind of reinforce what we talked about earlier when it comes to inclusive design so recognize exclusion you know we talked about that in the beginning we needed to recognize that this was the challenge right this was the barrier learned from diversity we worked with not only Craig Hospital with who are an excellent partner we worked with AbleGamers special effect also warfighter engaged our last charity that we worked with and when we were developing this product was the cerebral palsy foundation they are a wonderful partner so it was very important to work with them all we do believe in nothing about us without us and then this last bit solve for one and

extend to many so we're gonna get into this look this one a little bit more kind of as we could go on but the idea is that when we first started pitching this to our engineering team they kept asking us what the primary use case is the adapted controller was what's the primary use case and we really didn't have a good answer because we knew that the diversity of needs was so great so what we eventually kind of came to was this idea that this is the primary use case our traditional controller is still the primary way that most people will play games on xbox but what we needed would the adaptive controller was a device that tackled all the alternate use cases everything else and that's what we went is said about to do so I want to I want to kind of show you some of the bits of our journey here after the hackathons we had a pretty big pivot

into what was the form of the device and that there's a number of reasons for it but it was enabled basically because we knew that copilot was going to be a thing we knew that you could always use a traditional controller in conjunction with the adaptive controller and that gave us a lot of freedom to basically explore forms that were quite unusual as you can see the device is probably not what you think of when you think about a controller so these are sketches from the industrial designer Chris Kujawski kind of as we started to go down this path so I'm a little parched if you can't see on the back of the device here there's like we said there's 17 buttons on a controller there's 19 ports on the back of this device there so there's one for every button on the controller plus two extras this was how we determined how big this device was

going to be once we got to a point where we were really a ting we started from a place of basically we knew that there was going to be this many ports on the device and it basically determined how wide this needed to be after that we started to think about well what are we gonna do with all this space right because at the time we were thinking about this kind of more like a hub than I think a controller so we started to think about adding buttons and we started to explore not only adding the buttons that you need to all the buttons that are on the adaptive controller or what you need to use the system shell on Xbox one so if people are wondering if these are directional or capacitive in any way they're not they're actually too big buttons they're a and B and they're they're meant for four people who basically can't hit small buttons did you want to say anything about this part

in terms was just the design of it yeah I mean I think Bryce hits on the fact that that you guys were initially thinking of it as a hub and and that allowed a lot of flexibility in the device and that certainly gives me a lot of creative freedom when I'm trying to match it up to someone but including some of the buttons and using that space on the top of it functionally in a way that people can actually have access to buttons and inputs has been super helpful without saying too much I will say that that there was a lot of discussion around that yeah right and and a lot of opinions around that and and things feedback we gave them on different ways to do it that they definitely heard but then the understanding that there are always reasons in a design process why certain things work certain things don't but I think it just generated a lot of discussion like a lot of collaborative dialogue about what are we going to do with the top this control so you can also see from these these photos of these Pali jets

you know we would we would write on them we would sketch on them we would kind of figure it out you might notice that the corners of these are actually fairly sharp which we got feedback on pretty quick from Aaron I did say I was pretty opinionated about some things I've been I've been working in occupational therapy for ten years and that's given me a lot of time to to develop feelings and to learn how to express them appropriately but while that was a really sharp sleek looking design having those sharp corners for the people that I work with that was not gonna work both because maybe because of fatigue they were gonna have to rest the side of their hand on those corners and maybe because of whatever impairment they have they might not have feeling in the side of their hand that they're resting on and said I can actually lead to skin breakdown blisters actual wounds just from resting in a space or rubbing out of space and not knowing it so I was pretty vocal about that and you know we

needed to hear that because up until this point we were really kind of assuming that people wouldn't interact with it a lot but once we did really kind of understand that there was a lot of value in interacting with the device you know we started to think about how we could make it or more ergonomic more more better better for playing with and one of the things that they came up really early on with was the idea of sloping the device so you know these early Pali Jets it was pretty boxy we sloped the device so that you could rest your hands on the table and just kind of lay your fingers on it or your hands or whatever and you wouldn't have to like kind of get over this this edge and it was tricky to do because these buttons on this thing are actually pretty big they take up a lot of vertical room the ports on the back take a lot of vertical room too so we couldn't make the whole profile the device thinner because at the back we basically need it so we had we came up with the slope so this one is

pretty far along I mean this is pretty much what we kind of came up with in the end this form I think what's really interesting about what Aaron sort of came to us with was like I said we didn't we didn't assume that people would interact with the lot but when we started looking at it we started to see things like well let's move those buttons like view and menu and the Xbox button out from between those two buttons because you might accidentally hit them as you're going between these two big buttons we just increase the overall whitespace of the device and it was really important because sorry I really you know I always joked that these buttons aren't for fingers even though they work really well with fingers we thought of this place where it was like you could rest your hand here and just kind of rock it back and forth to get at these two buttons and it felt really good like it felt like this place of like where we we couldn't accidentally hit things I think another thing and this one will be a

little bit harder to see but I would do my best you know I talked about copilot I talked about using these devices together we really tried to make it fit so that you could in essence hold this and use these two big buttons while you're holding a traditional controller and we're going to get into an example of that in kind of a minute so yeah one of our researchers at the hospital his name's or sorry one of our researchers at Xbox when he was visiting the hospital he noticed that a lot of the the corners were dented and scratched and there was and it's a new facility yeah we just expanded like the year before they started coming so yeah it's got a lot a lot of love on our walls and you know we asked why this was the case and the reason is is that when you get someone who's like new to driving their power chair you know they're gonna make some mistakes we're gonna hit a wall run over toes hit walls

scratch up the mirror and the elevator because they're trying to turn their chair around and don't have a sense of space so yeah so when we thought about how this would affect the Xbox adaptive controller we put some mounting in this system in the system so you can actually mount this controller to any number of things if you go to the expo hall you can see some of the mounts that are there but we envision basically people possibly mounting this to their wheelchair and once we saw that like oh they're gonna run into a wall possibly with this thing attached to it it actually made us go back to the drawing board and make sure that we kind of beefed up the mechanical engineering of this of this device so not only is there a tripod mount there's a different type of mount the a MPs hole pattern and that's used in a things like Ram mount and these are attached to a structure inside the controller that RF is

actually very very strong we went back and redid it to make sure that if you know you did happen to run into a wall your controller wouldn't legs snap off you're you're don't try that though I didn't you know I tried that did you one of the things that was also amazing about this journey was that we got to bring our packaging designers along with it and when Scott was doing the research he would basically go to someone's house and he saw them open a coke can with their teeth because they just didn't have the mobility and so it basically created one of the strangest design rules for our packaging team like one of the principles of the design of the for the controller packaging was no teeth so to just kind of go through I'm gonna kind of back up and go through this one again oops there we go

so you just need a finger to basically pull the tape off in the mailer opens up inside there's another tape loop which you can pull up and then this cloth strap is exposed the controller sits on a card that also has a loop that you can pull out of the device you can slide it out and then the last thing that is in the packaging is the USBC cable which is in that little loop but there's no ties on it you can just shake it and it kind of falls out and the that the controller sat on is the QuickStart guide so we're very extremely happy with how this packaging turned out and the team is so excited it's it's really changed how they think about packaging so as we develop packaging for future Microsoft devices you know we're really going to see what we can take from this where it's appropriate and kind of have it affect our packaging kind of all out so one thing that I

didn't think I would have expected to be as important as it was which in retrospect sounds silly is the form and the visual design of this device we were very focused on empowering people and what we found when we went out into the world was that and and I think this totally makes sense in when you when you put it now that we have the opportunity to look at it this way people didn't want a device that looked too medical right they wanted something that looked like what their friends were using so we really made sure that the Xbox adaptive controller felt like a member of the Xbox heart family the width of the device remember I said before the width of the device is determined by the ports it's also matches the Xbox 1s console so

the width of the controller and the 1s console are the same it has rounded edges it fill it it follows our visual design pattern it looks great I think a lot of things when it comes to assistive technology they tend to look they tend to look either overly medical overly homemade or they can be look really childlike you know I don't know if you've seen a lot of assistive technology but they yeah yeah colorful and a little little infantilizing I think from that side of it and then and then yes also overtly medical because they have to be medical grade and they've just looked different and people really struggle with that yeah and so the fact that we created this device to look like it was part of the of the Xbox family was was something that we heard from the community and we really appreciated and we took it but the fact that when we launched the

device that right away we got tons of feedback about just how non-medical it looks and how it looks like it's part of the the family was actually really really gratifying to us and it's really taught us a lot about how we think about assistive technology right I mean some people have told us well you mass manufacture these you have an opportunity to do that I think that's partially true lots of assistive technology is mass manufactured that doesn't mean that it is it isn't medical ok so you know I think one of the things that's really been interesting is this idea of like how do we create a device that doesn't stigmatize you from using it doesn't other you for using this device I was super important to us yeah and I so I think that's something that I see really frequently in my work so I'll take a moment to kind of just talk about Craig Hospital and what I do there and why why Microsoft would even think about approaching a hospital on a project like that because there are a lot of really

great nonprofits out there who are who are in the gaming world and addressing gaming but Craig hospital is it's a rehabilitation hospital we treat people who have had either spinal cord injury or a brain injury or both honestly and what a rehab hospital is is basically after you have had a change you know an injury after you've been in kind of the acute medical ICU stages of a hospital you get kind of like medically stable but because of that change in your function you still may not be like strong enough or able to go home and resume things so a rehab hospital we really kind of work with people partner with them to help them learn the skills that they can take with them and and kind of go home and be as independent as possible and do the things that they wanted to do before so I'm an occupational therapist and that's kind of the bread and butter of my work is understanding what's important to a person and just helping them develop the skills or use the tools to do that again I think that's the easiest way to

describe what no tias and I work in assistive technology which means a lot of times I'm looking at a person's ability to use a computer to use their phone again all these things we take for granted to control their environment to use a communication device that they can no longer speak and we did those things like really well at Craig but we didn't have anything for gaming when I first started there and so you're gonna get kind of an origin story inside of an origin story here so I'm gonna talk a little bit about like why I do gaming I actually worked with a patient five years ago that when he came to the hospital his big goal was to be able to play video games again he'd had a spinal cord injury he could he's what we would call a quadriplegic or tetraplegic meaning that he couldn't move his legs anymore I couldn't if it's in his trunk he could move his arms but he didn't have any fingers anymore so he's essentially you know working in a world of kind of gross motor movement and he was really struggling with rehab and it was because he was a gamer and at

the time we didn't have the tools to meet that need and he he saw that it was very apparent and it affected his participation so those are the kind of patients that get referred to me in the tech lab when there's not a solution they usually come to me and just through the process of working with this person and understanding his need and being really honest and learning from him and helping him get back to gaming we saw kind of the power in that and his ability to to participate in something he loved to relate to his friends in his online gaming community and that really translated to his overall rehab so from that one need we built a whole program because we saw the importance of it which it's kind of an inclusive design thing right Yeah right right parallels so that was kind of our start to it and about six ish months into it I think is what when Microsoft approached us and I was pretty skeptical in the beginning again when you work in a specialty

hospital doing a specialty area assistive technology you get approached sometimes by people who have a project a passion project and want to do it and you see a lot of projects you know and it's a waste of your time it's a waste of your resources and there are always reasons and it's but it also is really disheartening to expose that potential to a patient and then take it away right so that's kind of how I came at it and that's kind of why I was probably a little a little skeptical in those initial calls but those calls kind of evolved into some focus groups brice mentioned the team coming out to the hospital that was really interesting having a whole team kind of come in and kind of examine our space and have those conversations and what I think was really interesting was what they saw in addition to what we said so so again I take for granted that I work in a hospital so there's things I don't think about I don't really see the dents in

the walls anymore because it's my day-to-day so for the team to observe that and then come back and really ask questions about that was really interesting and then of course we were part of the beta testing with our patients and that ended up being kind of a really cool thing to offer like if you can imagine being in a hospital being in a uncomfortable position adjusting to you know a major life change and just dealing with that and then all of a sudden not only do we want you to play games but we want to want you to test this cool new thing that's not even out yeah and you're not even allowed to talk about it so that that provided some really really interesting and good motivation for our patients and I think helps them see what they could contribute and that their voice still mattered in that regard yeah so I will talk about these pictures a little bit because these are our patients as my hospital I think yeah the upper orientation wise upper left and

lower right those two pictures were in the official announcement video for the xbox adaptive controller and they show two very different people with two very different abilities and that's pretty typical for our hospital and so it diversifies our use cases I guess in that regard but again these guys are there at a game night we host game nights weekly and they're able to try the controller and really get a good feel for it and use it to work towards their therapy goals as well so the guy on the bottom you can see like he really has to work to look up at the screen while he's playing like it's really hard for him to divide his attention between the two activities so that's like a really good therapy goal he also was unable to communicate and was using a device to talk and he had press buttons to navigate that device so this controller actually helped him build skill and capacity to use this communication device and actually eventually to drive

a wheelchair which we weren't sure he was going to be able to do the guy on the top was actually transitioning between using a power wheelchair to using a manual wheelchair like he was developing this strength to do that and again you know the big joysticks he's using the buttons all the reaching he's having to do is a great way to work on that strengthening because if you have someone all of a sudden wheeling around a hospital a manual chair all day it's really tiring and then actually the guy on the bottom was transitioning also between using a power wheelchair but with his mouth right so using a sip-and-puff or a straw he would blow into drive his wheelchair to a hand control to drive his wheelchair and so the controller actually gave us the ability to have him work on both of those skills and kind of transition between them which was pretty cool okay this is a photo of one of our game nights and it's also a photo of maybe a different way to use one of

these right you know Bryce talks about assumptions we assume we have to hold on to this thing but a lot of my people cannot hold on to this thing so the gentleman in the picture he can rest on the table and then he can kind of use his hands on top of it use his knuckles the side of his hands to get to the joystick he can't get to the buttons on the back so that was probably a little bit of a limitation before we had the adaptive controller but now with the adaptive controller we could actually break those two buttons out and put them somewhere where he could reach them and then the guy next to him was had been one of our patients for a while it actually was kind of a good example of how people could just start to use gaming as a way to like relate to each other and like have some common ground while they're in a hospital this is like a pretty heavy set up this is probably not something you guys imagined using it bar I you know I think when we talk to everybody we knew that we couldn't

imagine everything that'd be used for we kind of got to that place yeah so this this patient had had a surgery as a result of his injury that required him to be on what we call flat bed rest for like 12 weeks so if you can imagine not being able to sit up or get out of bed for 12 weeks right that's that's pretty boring especially in a hospital so the adaptive controller really gave us the flexibility to put things within his reach which actually helped him maintain his strength while he was on flat bed rest for that whole time the glasses he's wearing I again it's probably something people don't see the glasses actually have a mirror in them so that the TV's in front of him he could still see that and that's a pretty low-tech workaround and a little safer than suspending a TV and overtop of someone to be honest but we actually would bring him down to game night in the hospital bed wheel it all the way down so that he could like get to know his roommates unlike the other people in the hospital

and again the flexibility of the controller allowed us to really put things where he could reach them change them you can see he's actually using a switch in his mouth that he bites to do one of his button presses he's got a button on his chest that he can head and then he's using a joystick with his left hand so that's so important I mean just for his overall quality of life and his ability to just kind of get through that moment his final setup looked a lot different because he got to the point where he could he wasn't up in a chair he was driving a wheelchair and the controller gave us that flexibility to mod it so you know those are some really extreme kind of cases and I mentioned before that last principle inclusive design solve for one extend too many so we started this device from that place of like solving those extreme cases but what we found along the way were some really interesting use cases about how to use it for people who might have a less severe condition so this is my friend cherry ray she's not here so I'm gonna like basically just make fun of her and

when this goes up on the vault it'll be there forever so hi cherry so you know this is a picture of her and this is how an idea that she had that kind of came to her for how she can use the controller stick presses are really hard for her it is hard for her to use to basically maneuver thumbsticks and use them as buttons at the same time happens for a lot of people so we mentioned copilot before the ability to make two controllers seem like one so what cherry can do is she can take the adaptive controller she can make these two big buttons into stick presses and then she can basically just use a controller and instead of like instead of pressing down on the stick she can just let her hands drop and let gravity hit these buttons she can lean on them in different ways you can kind of move it forward and back so you know that was really powerful for us and when we talk with people who have like RSI or arthritis or repetitive other repetitive

stress conditions you know we would hear stories like I play destiny for like three hours a night and then my fingers are on fire but those three hours are the most important three hours of my day because those are the three hours that I spend with my friends so I ration that time I am NOT casual about how I use that time so we were always just like well if you can change the movement from like this to something else maybe we could give you four hours right that was the whole point how do we give someone more time to play so I'm going to transition a little bit into how do I design for the Xbox adaptive controller so we'll get this question a lot from game developers how do I design for the Xbox adaptive controller and here is my answer

please don't so what does that mean okay I don't I'm a designer I don't really believe in prescriptive guidance I'm not a fan I don't like people coming up to me and telling me what to do I like being told what the problem is right so when we're talking about creating a games for people with limited mobility I'd rather people ask these questions apply it to their game and I believe that if they they ask these questions their games will be better not only for people who use the adaptive controller but for people who use like a typical controller for people who use mouse and keyboard for people who use other input devices I think we are getting to a place where we can't design for just one device modality anymore so I don't kind of want to put out into the world do this for the adaptive controller so what

are my questions here they are I'm going to go through some examples we'll kind of get back to these in a bit can your controller remapping be is unique is your game play we have robust controller remapping on the Xbox adaptive controller you should still do it in your game there's a lot of reasons for it what unique game assists can tune your game play for individuals would secondary action facilitators hurt gameplay can you have fun with simple controls and I use that word fun in particular because I think a lot of the times when we're talking with our when we're talking about the games that we create we think about mastery we think about being able to compete and I'm using the word can we have fun is precision vital to progression and my example there is sniper versus shotgun our simultaneous actions required this one I'm going to be a little prescriptive please try not to do simultaneous actions it's very hard for people can in accurate timing

be forgiven Ken timing be flexible so we got minecraft here we've got a little creeper coming out of the water Minecrafts a really interesting example for me because minecraft has you know some really decent controller remapping there are more actually actions in minecraft controls than there are buttons on a controller so that they have to give you the ability to choose what you want to do there are some things you can't do in our typical controller but for me what I really like is this tiny little accelerator which is auto jump right there's now all of a sudden when I'm going up this hill I don't have to keep going a hey you know I don't have to do that repetitive action it's a tiny accelerator that doesn't really affect gameplay but makes the world of difference to someone who's hitting this button I'm going to talk

about fortnight here's the accessibility menu in fortnight I am NOT going to talk about anything in the accessibility menu what I'm going to talk about is the feature that they have on console which is sprint by default so we talked about simultaneous action the fact that I can go here and say yes I'm going to sprint by default I don't have to hold that stick down to run I'm just always gonna run has opened up this game for people immensely I know some people who play fortnight with the adaptive controller they cannot use left stick we have a feature on the adaptive controller that lets you assign stick movement to a button so her name is Mellie she lives in Germany she puts a button between her knees and when she presses that button her character runs forward so on a first-person shooter or in a game like fortnight her character just runs forward she uses right stick to steer she doesn't strafe she doesn't go backwards she just goes for it but the

fact that she doesn't have to do a simultaneous action hits print by default is a big deal one stick two buttons all you need to play inside I love this game I love the idea of this you know that it's you have such a rich thing you have such rich gameplay with just the simple controls I wasn't very good here I forgot what I had to do and then I remembered I sped it up a little yeah all that two buttons this is my youngest daughter in front of an xbox adaptive controller and a monitor demonstrating something we have in a facility we have a Microsoft called the inclusive tech lab it's rocket League with no hands so she has a rock band pedal that she's using for accelerate and you can see that her knee is between what's two buttons sort of mounted to the desk so she pushes on that rock band

pedal and she steers the car by hitting these two buttons it's very digital like digital digital digital so her her movement is very specific the other thing that you can see is by her head she's hitting a little button so I said no hands I didn't say all feet so she's got that button there and that's her jump button is that little head button and we like this in the lab because it shows it shows a very simple control the great thing about rocket League one of my favorite things about it is in their controller remapping they have controller remapping which is contextual to their game so you can do acceleration in Reverse you can map it to a thumb stick so it's very possible to play rocket league with just the thumb stick you might not be able to do everything but again can you have fun you might not be able to master it but can you have fun with the simplest of controls [Applause]

[Music] we have to talk about Celeste Celeste gives you the option to turn on assist mode when you kind of come in now the developers of Celeste said that they didn't believe in it's this mode but I really really thank them for putting it in anyway and Celeste SAS this mode is wonderful it lets you do things like turn down the speed of your game you can turn on infinite stamina air dash is you can make infinite and then you can turn on invincibility this might not be the game that you envisioned but for a lot of people this really opens up this amazing title to them it's super wonderful that they do I like here - is I can totally Superman 3 level let's talk about shadow of the Tomb Raider here's their accessibility menu there's some

wonderful work in there there's fantastic work that they did one of the things that I really like is that you have the ability to change weapon aiming from left stick or from right stick to just left stick so you can do a lot with just the left stick it's great one stick kind of being able to play the other thing that they did which was really really tuned to their game and just wonderful was the idea of of having different types of difficulty you know for the types of action you're playing like combat difficulty versus exploration difficulty versus puzzle the fact that they have these assists and they're so tunable really makes this game really great so I just kind of want to get back to this and your controller me mapping B is unique is your gameplay you know rocket League good example that what you unique game assists continue your gameplay for individuals Shadow the Tomb Raider has some awesome assists really great stuff in there with

secondary action facilitators her gameplay auto jump in Minecraft things like that would it really changed your game a ton can you have fun with simple controls not necessarily compete rocket League inside things like that it's precision vital the progression I didn't show an example for that but I think you'll kind of get it simultaneous actions required one of the great things in that shadow of the Tomb Raider accessibility menus it allows you to kind of tone down your QuickTime events even though it's something they're kind of known for they've given you a lot of options for kind of getting by them this last one is tricky the the example is basically Celeste in the ability to slow the game down but one of the things that we always kind of hear from people is that they don't necessarily want the game to be easier they just want to have more time and I think people with cognitive like cognitive limitations can kind of just need an extra second yeah just to kind of figure out navigate process and react without dying yeah so

I'm really want to kind of do a demo and then but I also really want to make sure I have time for questions actually maybe I could do a demo later if someone wants to see it I have this button here but maybe we can line up for some questions and I'll try to do the demo at the same time does that cool anyone have questions we would asking you to go to the mic if you can't go to the mic just shut your question out we'll repeat it no no question no question oh here we go I think the first ones the hardest right oh boy okay fight it's a softball I want to know where you can get them where you can get them you can get the Xbox adaptive controller at any Microsoft store Microsoft stores online so if you go to the the Westfield Center like just like a block from here and the Microsoft Store they have them and are they available in other countries or Z so we're avail I don't have the countries listed off the top of my head but we announced in Mexico City that we're working on expanding them to more of the world so we're going to have announcement the future about that

sorry but like right now right now it's like US Canada most of a lot of parts of Europe you know I think kind of typically what you would expect Thanks yeah please uh I just wanted to how you got from you know getting the product out to Super Bowl commercial I don't know sorry um you know I think I think what I can say is that the response has been we've had heaps and heaps of accolades it is super appreciated we have been overwhelmed and we just we love this story and we love that people love this story so the way that we start got to the Superbowl commercial was

basically we were going to do a holiday commercial so reindeer games was our holiday commercial that's where you guys met Owen and you know we had a lot of fun doing that commercial we got to work with the community we got to work with not only Owen but we got to interview an audition a lot of players for that for that role and when reindeer games the holiday ad did so well basically the team kind of went oh let's let's really amp this up and so we already had a group of kids that basically were like that were that were in the audition sort of we that were in our audition process so we basically went back and talked to them all and I think the one thing that I super appreciate what the the team did was that even though we had a story to tell for the holiday ad for the Superbowl ad they really doubled down on making sure that the kids talked and that we got to see you know these kids and they could talk with their own voices and that was something that was super important to us Ioan it's kind of

related to the first question so if anyone in the room is interested in building their own input devices to plug into the into the adapter controller what advice would you give them are there any resources available for them so we do have an input spec oh sorry the question was do we have a spec to be able to enjoy on input devices I will say that buttons are a very simple thing you can take a headphone cable cut it off cut cut like the headphone cable off take the wires touch them together and it'll fire the button that's what an open switch is right you're completing the connection so we do have a spec online that allows you to design and build your own buttons as long as well as potentiometer based devices for analog controls like triggers and joysticks I would encourage people to do that just as a selfish person in healthcare who's looking for more inputs and maybe lower-cost inputs like please

take a crack at it and let us know what you have yeah sorry at this side okay yeah so it's really great work with the controller and the thing that I'm wondering is how are you spreading this information to other healthcare like I'm from Norway so personally in in Norway get to know how to actually use this controller yeah I think that's a great question and a question that we we hear a lot and I think every time these guys put that on another commercial my inbox gets flooded with questions right so I'm doing a lot of talking you know for so for those that are here there you go Microsoft had does have some resources online as Bryce can speak to what you guys have online and then I know some of the nonprofit's also are starting to put out videos and and information well as

well so just reach out to the community yeah I mean I do think that this is this is new ground for a lot of us you know and you know I think there there's a lot that we can all do to kind of put this information out into the world but yeah I would just encourage you to kind of keep it keep at it you know keep engaging with us keep engaging with people with disabilities you know we're all gonna learn you know if you think about how long it took to get to form of what a modern controller is and now we've like opened up something that's like you know III just want to I just there's a lot that we can explore yeah I am wondering bout if you guys have plans for integrating or allowing this to be used as a controller with Windows mixed reality ad sets or VR in general buca's like I know some games you use controllers and like I've helped try to

give demos of VR games to some people in wheelchairs and with paraplegics and it they they love seeing these worlds that are typically like five or ten feet away through a small screen there in the world and having a controller like this is so fundamental to that system but I don't know how your teams are currently talking okay Windows 10 and Xbox one don't see the Xbox adaptive controller any differently than this they're the same thing so if you're mixed reality game you can work with it with a traditional Xbox controller it will work with the Xbox adaptive controller thank you over here so I'm gonna just first say thank you for making this because you actually touched one of my friends

lives with us and thank you so much for doing that but so my question is kind of from the research side of things so with a lot of this unfortunately is my area of human-computer interaction but I I know that there's a lot of research into input devices and the therapeutic effect that they have and I've not yet seen any papers published on this and you know are they're gonna be put papers published on the therapeutic effects of this and also could you elaborate a little bit more on I'm working in healthcare and medical research is slow yes if there's okay it's slow there's there's a lot of rules about hurting or not hurting and who you can work with them you can't so I do envision things will come out if you do a lit review of just gaming and therapy you're gonna you already get a ton of things some of them are really old things right now talking about really

old devices and that's I think because of the amount of time it takes to do research but stay tuned it's coming yeah awesome thank you I want just before I ask a question I just want to give people a quick demo twin-stick 3d world game minecraft two six one two look around one two one two you know move I've set this button with the adaptive controller to be like a shift key it allows me to change what button values are and even allows me to swap sticks so when I do this demo and you see me do this now I'm looking now I'm moving now I'm looking now I'm moving right just so you mean you see me watch so you know what's going on yeah which side were we on I guess this side sigh yeah so I was just wondering how you arrived at the decision especially with the massive enlargement of the buttons how did you arrive at the decision to have a standard saw his d-pad on the adaptive controller so the d-pad is actually a hundred and fifty percent bigger than our one on our control our regular controllers I mean I will say that you

know we knew we wanted a bigger one I think that in my personal opinion I think that we could have though the d-pad is the one thing I wish we could kind of go back and look at the most but yeah it's a hundred and fifty percent it was it was really about symmetry we didn't expect we wanted to make sure that people had something that they could navigate the - and that was primarily what it was was intended for thank you welcome hi I know that you talked about hey don't design specifically for the controller don't yeah okay so I was thinking though if let's say if people did start creating something like alt ctrl G DC or Nintendo's lateral or not steel battalion would you be concerned about the standards of the control are being made overcomplicated again or do you think it's gotten the all controlled GDC question it in I will say I really did

try to get something in there this year I just want to ran at a time but so when I say don't design for the controller I don't I mean if someone wants to create something unique the last thing I ever want to do is someone who makes platforms is tell a game designer how to design anything right like so if someone wants to create something unique with this device I am happy to do that I just I get I get concerned when people want to break a game down in a simple way that is far too focused on this device and what I mean by that is that if you made something that would work with this device it would also make games with a typical controller better for people with limited mobility as well okay okay so just think broad or not specific yeah but I think your idea of I mean one of the things that came up really early on was that someone built a joystick that uses potentiometers that they could stand on and they literally lean on it like a pogo stick and it would go like you know and that and that will work you

know when people ask me I think one of the things that we need to start talking about soon like Ian kind of suggested was the the maker community right and so when people ask me when people don't quite get what this thing is and they're like kind of entrenched in gamer culture I basically go hey you could build a wicked arcade cabinet with this thing and then they get it like it's really kind of it cool so kind of an opposite question do you have any intentions of making a standard peripheral that could be used by somebody who has no prior knowledge of how to set up a gaming controller or how to operate something like this so so the question as I understand it when you say this you mean specifically the Xbox adaptive controller yeah I mean things like buttons that you didn't necessarily have to make out of a headphone right yeah yeah yeah so I mean I will say that that's one of the areas that we need to work on we wreck

that and it's and the challenge is is that as you saw from our examples today there's a huge range of ability so while when someone comes up to me or Aaron or someone who's basically in this field we can typically quickly come up with a rig that probably works for them but we certainly recognize that that that might not be the easiest thing for people so we need to tell better stories of how people can use this device because I see I know that now that the commercials out you know people get how kids with one hand can use this device but that's not that's not a broad enough group of individuals right so we need to tell those stories better we need to continue to basically give people this inspiration about how they can build devices with this and and you know for us this is the start of the journey we're trying to figure out what's next cool thanks come on play more minecraft

so yeah so movement Auto jump when I hold this it's jumped right but when I shift so when I shift all of a sudden this button is now or it isn't [Music] oh yeah now it's mine so think of that like not only are we swapping the sticks with the shift button we're swapping what buttons are and that's a great way to reduce the number of buttons you might need you know yeah so we use that demo and then the other nice thing about the big buttons is that you know you can hold this thing and still hit these big ones like while you're holding hey we got another question I will remove your question yeah you guys were leading this on the Windows 10 xbox front cuz it seemed you said it's seen as a regular Xbox controller has there been any talk about collaborating with the other

platforms or phones and Android iOS yeah no so when we launched the Xbox adaptive controller or when we announced it Phil said that we would share our knowledge with the world we are completely open to working with other companies I cannot speak for other companies but what I can tell people is that we share all this knowledge I run a facility at Microsoft called the inclusive tech lab I've had all kinds of developers that are not Microsoft game developers you know actually Robert spoke at one of our events today and he's a from Wizards of the coast Wizards has come by the lab a bunch of times we are very open to sharing what we know with the community but I can't speak for any other kind of development house but actually you know so I you know I can't talk to them are you guys doing it's coming thanks

for coming thank you yeah you know it well no asking you fun no one's kicking us out yet I'm not seeing anybody yelling at us are you yeah right I'm sorry it was it was just another question related to the maker community yeah so you mentioned one thing like the giant oversized controller I mean already there's been tons of people making loads of really cool interesting stuff so just wondering if you have any favorites any favorite from the maker community yeah well I mean the one thing that I think for many of our teams that was really tricky was all of the charities that we worked with our makers so like you know when we when we would start to talk with marketing about well we're gonna do a maker beat next like you know they were like what and they didn't quite get it and they didn't quite realize that like you know people with disabilities are like the original life hackers like let's get real yeah right like so all of these folks we are

here to learn from them and kind of build off them do I have favorites those endless guys are here I could talk about their stuff I'm really excited about their work voluminous guys are doing some amazing stuff with prosthetics and there's you know really great potential for how we can think about it cool that's all thank you thank you [Applause]