Inclusive AR-Games for Education of Deaf Children: Challenges and Opportunities

This paper examines the opportunities and challenges of designing inclusive augmented reality (AR) games for educating deaf and hard-of-hearing (DHH) children, a topic the authors identify as under-researched relative to AR learning games more broadly. Its central argument is that effective inclusive AR games should be built on visual, sign-language-centred design, and that developing them requires an international, multi-expert collaboration. The work combines a scoping review of prior AR studies involving deaf people, drawn from Google Scholar searches conducted between December 2021 and May 2022, with an appropriateness analysis of AR input and output techniques adapted from a unified method for universally accessible games. The authors also draw on experience from two Portuguese projects that produced bilingual mini-games and a deaf-accessible video game.

The review finds that prior work has explored AR for DHH learners in areas such as sign language vocabulary learning, real-time captioning and transcription, and signing virtual characters, while noting that written text is only partially accessible given variable spoken-language competence among DHH children. Drawing on a broader literature review of AR learning games, the authors observe that such games tend to emphasise collaboration, quizzes, puzzles, and location or image recognition as their most common mechanics. The appropriateness analysis concludes that sign recognition and image or object tracking are the most suitable inputs for DHH players, while local signed videos are the ideal output, with animations and images as appropriate supplements.

A key contribution is the identification of five dimensions that shape inclusion for deaf people: cultural, educational, psychosocial, semantic, and multimodal. The authors argue that automated signing avatars and automated sign-to-speech translation conflict with several of these dimensions, and they propose a civic intelligence approach in which trusted, qualified deaf signers and teachers contribute signed video explanations of concepts. The paper concludes that such a game must be co-designed with DHH children and experts through a user-centred, international process, a conclusion grounded in the principle "nothing about us without us."