A Character in Your Hand: Puppetry to Inform Game Controls

Puppetry offers a useful framework for designing character controls in virtual reality, argues this study from Georgia Tech researchers Nitsche and McBride. Puppets and game characters share a position between imagined and factual worlds, and players project themselves into both, making traditional puppet manipulation a promising model for VR interaction. The authors note that VR currently lacks a standard control scheme, and that practical constraints such as tracking volumes of roughly 5 by 5 meters on the HTC Vive, along with untracked legs, complicate character design. Earlier puppetry-inspired interfaces, including Octodad's cooperative Bunraku-style mapping and various Wii and Kinect projects, provide relevant precedents.

The core contribution is the Archiving Performative Objects project, which works bottom-up from traditional puppet control mechanisms drawn from the Center for Puppetry Arts archive: rod, marionette, and hand puppets. Using Kaplin's distance model as a design logic, the authors mapped these onto HTC Vive controllers. Their central finding is that VR controllers excel at position and orientation but lack the finger-level precision that detailed puppet manipulation demands. In response, they developed tailored schemes: grab-based rod puppet control permitting multiple rods per controller, a marionette layout that preserves the dual-control structure while adding spatial navigation and a detachable controller, and a hand puppet scheme that assigns each controller to a finger with a touchpad governing head movement.

The authors conclude that drawing on established puppetry traditions rather than conventional game controller logic can yield richer and more expressive VR interaction. They are candid about limitations, most notably the absence of haptic feedback, which leaves physical weight and balance to be conveyed only visually. The work is presented as ongoing, and the resulting schemes may prove harder to learn than familiar game conventions.