Extending the Serious Game Boundary: Virtual Instructors in Mobile Mixed Reality Learning Games

This paper proposes that mobile augmented reality learning (MARL) games, in which learners wear see-through heads-up displays and interact with a 3D animated virtual instructor, can deliver personalized, context-aware education that is both engaging and pedagogically sound. The authors argue that such virtual instructors can adapt to learning styles, emotion, and pedagogical technique, and can guide players through real-world quests anytime, anywhere, and at their own pace. The work situates this concept within Milgram's mixed reality continuum and within prior research on mobile augmented reality systems (MARS), while noting that research specifically demonstrating virtual-instructor-led AR learning remains sparse.

The central contribution is the Context-Aware Augmented Reality System (CAARS), a software and hardware architecture originally developed for manufacturing training and extended to MARL environments. The hardware component is a lightweight optical see-through headset with a camera, speaker, microphone, inertial sensors, and roughly eight hours of battery life, which transmits images and positional data wirelessly to a game service. The software side is organized as a service-oriented architecture with visual, human-computer interface, and training subsystems. The training subsystem uses software agents and pedagogical models, such as scaffolding that gives novices more step-by-step guidance than experienced learners, and the virtual instructors are generated from the PECA (Pedagogical Embodied Conversational Agent) product line architecture, which supports multimodal input including gesture, speech, and optional physiological biosensor data.

The paper is primarily a design and architecture description rather than an empirical study, and it presents no quantitative evaluation, user testing, or learning-outcome data. Its scope is conceptual and technical, with prototype figures illustrating the goggles and animated instructors, and it identifies future work including empirical research studies with learners of all ages and integration with a massively multiplayer online role-playing game prototype to share virtual instructors across augmented and virtual reality environments. Applications are proposed for K-12, graduate, and postgraduate education, though these claims remain untested within the document.