Kinect Disconnect - How NOT to Do Motion Control - Extra Credits

Some of you may remember that way back in season 1, we talked a little bit about motion controls. We talked about what Microsoft was trying to achieve with the Kinect and what it might become if we stepped past the idea of simply trying to make Wii games on a fundamentally different device. And now, 18 months after its initial launch, now that we've had the opportunity to experience the Kinect as a community, and now that we've gotten the chance to work with the device both from a player perspective and a development perspective, we felt like maybe it was time to return to the topic and discuss the most salient thing we've learned about the device and what it might mean for motion controls in general. Now, back in season 3, we discussed a principle called the uncanny valley. This is turning into quite the little nostalgia trip. Just for a quick refresher, the basic concept is that as something becomes more human, we tend to like it more and associate with it better until it gets close to being human but still has noticeable differences. Then we tend to be creeped out by it and revile the thing. The idea comes from robotics. You know how when you see a cute Wall-E-like robot, you

feel that it's charming and quaint and with his little eyes and oh, it thinks it's people. But when you see some of the fleshy robots at the Tokyo trade shows with their thousand-yard stares and their rubbery lips, you find them disconcerting or downright even though they're a lot more human-like than the plastic hunk that is Asimo. That's the uncanny valley and it happens a great deal in games. When your hyperrealistic marine or assassin ends up clipping into geometry or just taking actions that make no physical sense for a human being like running in place or into a wall over and over, the whole scene often feels unsatisfying. When you're playing a tune-shaded game on the other hand, humanoid characters can get away with all sorts of odd behavior and it won't knock you out of the experience. But enough recap. The important thing we found out is that this also happens in controls and Kinect is the uncanny valley of input devices. You see, as human beings we're sort of evolutionarily hardwired around being tool-using animals. That and language are pretty much the things that define us. So, as tool-using animals, we've developed something that I'm just going to call kinesthetic projection. You ever notice how when you're driving a car, you say, "I'm going to turn right." or

"I was clipped by a drunk on the highway." rather than "I'm going to turn the wheel to the right in order to make my car turn right around this corner." or "My car was clipped by another car being driven by a drunk while we were driving our cars on the highway." You do this because you view the car as an extension of yourself. Your mental process actually is, "I'm going to turn right." You don't think about all that other stuff. That's kinesthetic projection. The idea that we can see tools as an extension of ourselves. That we can project our movement out onto the tools we use. It's actually an incredible adaptation and a mind-blowing mental trick when you think about it. Well, this sort of kinesthetic projection is a well-known phenomena amongst game designers. If you've ever been playing a game and the controller just sort of disappears and you in some ways lose the separation between you and the character on the screen. In some ways you become the character. You start to think, "I'm going to turn right." rather than "I'm going to turn my character right." That's kinesthetic projection. The moment where the controller sort of falls away, where you stop thinking about the piece of plastic in your hand and it feels like the character on the screen is doing what you're thinking rather than following a

series of commands that you've punched into an input device. That's the moment when that character has become an extension of yourself. But with Kinect, we lose that. You see, the uncanny valley happens when something's close enough to human for it to register in pre-existing mental slots. Slots that we know incredibly intimately and can tell even the tiniest deviation from the norm. So, rather than registering something as "Ooh, cute robot thing." there becomes a point where your brain registers it as human but with something horribly, horribly wrong. This sort of thing is what happens with the Kinect. With the controller, the actions are abstracted enough that we can understand them as tool use actions and then mentally begin to project onto them, entering a state of kinesthetic projection and losing the controller entirely. But with the Kinect, an odd thing happens. Because all the actions are happening in your body, you run into the problem of your body knowing how those actions are supposed to go. We all know that you don't run by pressing a stick forward, but that's tool use. We can get past that. We also all know that you don't run forward by running in place, or sliding one foot forward. But, those actions are similar enough to what we do understand as running that we can't get out of that jarring sensation that this just doesn't feel right. And

this follows for almost all the actions on the Kinect. If there's any way we can understand doing an action as a human being, even if it's one that we don't often do, like say firing a bow or something, all the little things, like the lack of haptic feedback, prevent us from ever getting past the fact that we're using an input device in the way that we do with a controller. So, by making controls more realistic, we've actually made them feel less real. But, the very fact that these controls tend to be immersion-breaking leads to a fascinating new vector in design. The games that I believe are the most successful for the Kinect seem to start with inherently engaging physicality, like Dance Central. In the past, we never had to worry about whether using the controller was in and of itself fun. We just had to make sure that the translation between the inputs on the controller and the actions on the screen were smooth enough that a player could achieve that state of kinesthetic projection, and just forget the controller was there. But, now with these full-body movement controls, design may have to start with what is it fun to do physically as a human being? The control scheme itself has to be fun and engaging, because you are going to notice it. This type of controls-first design is something pretty new to our

industry, and may, in many cases, be why so many Kinect games fall short. Traditional developers, beginning design traditionally from systems, mechanics, theming, or narrative, will have to backfill the controls that are best for the game they're building, which has always worked in the past for good developers, because with a traditional controller, they had to find something highly functional rather than inherently fun. But, now, even a good game with a functional control scheme may feel cumbersome to the player if it forces them to do an action which feels silly or unnatural in some way. So, Kinect has reached the uncanny valley of input devices, and we're left understanding that there's a lot more we have to do to make physicality engaging than simply putting together a good control scheme. This isn't to say that we should stop experimenting with the Kinect, or that this type of full-body motion control is a bad idea by any means, though. I think there's a lot of untapped territory, and I'm almost certain that that become important for a number of applications, even if games don't end up being one of the first places to use them well. So, does the Kinect broaden what we can do with design? Absolutely. But, without that inherent kinesthetic projection that comes from a well-crafted tool, creating an immersive experience is even harder than before. See you next week.