Multimodal Multiplayer Tabletop Gaming

Digital tabletop interfaces offer a superior environment for co-located multiplayer gaming compared to traditional desktop and console setups. By leveraging the physical affordances of a shared horizontal surface, digital tables facilitate collaborative interaction through shared views, simultaneous input, and enhanced workspace awareness. The core thesis posits that integrating multimodal gesture and speech input into these tabletop environments creates a more engaging, visceral, and socially connected gaming experience that bridges the gap between the rich, specialized interfaces of arcade games and the versatility of home systems.

The research establishes that effective collaboration relies on workspace awareness, which is maintained through intentional gestures, consequential communication, and verbal alouds. Empirical data indicates that multimodal interaction—combining speech and gesture—is significantly more efficient than single-mode input, resulting in 10% faster task performance, 36% fewer errors, and a 23% reduction in spoken words. Furthermore, users show a strong preference for multimodal interfaces, with 95% of participants favoring them over pen-only alternatives in map-based tasks.

To demonstrate these principles, the study implemented a multimodal, multi-user wrapper for two commercial single-player games: the real-time strategy game Warcraft III and the simulation game The Sims. By utilizing a multi-user touch surface, the system allows players to interact directly with game elements using natural hand postures, such as grabbing, pointing, and fist-stamping, combined with spoken commands. To manage the transition from single-player to multi-user environments, the system employs a micro-turn-taking wrapper that regulates input streams. Informal observations confirm that this approach fosters a more collaborative and entertaining experience, successfully repurposing existing single-player software into a dynamic, social gaming platform.