Using In-game Analytics to Explore Learning Dynamics of Information Literacy in a Social Media Simulator

This study presents an in-game analytics framework for evaluating how players learn to identify fake news within Quacker, a social media simulator embedded in the educational video game Julia: A Science Journey. The authors argue that conventional assessment methods such as pre- and post-game surveys and focus groups cannot capture the non-linear, moment-to-moment learning dynamics of game-based learning, nor the informal play that happens outside classrooms. By logging detailed player interactions through the Unity Analytics platform and combining them with content metadata, the framework aims to reveal how players actually reason through disinformation tasks, with the broader goal of clarifying debates over whether fake news "inoculators" build resistance or merely foster scepticism.

The analysis draws on 3,565 quack interactions across 436 runs from a sample of over 112 players, collected between September 2022 and April 2023. Findings indicate sustained engagement, with an average of about four runs per player and many continuing to pursue the maximum score after a three-star result. Disinformation tied to political opinions was misclassified far more often than pseudoscientific or COVID-19-related falsehoods, with one politically framed quack judged as factual by over half of players. Players also appeared to use source identity as a heuristic, trusting official health organizations and distrusting suspicious accounts. Reading time correlated with content complexity and inversely with performance: higher-scoring runs showed reading times roughly half those of failed attempts.

The authors acknowledge that the reading speed relationship may reflect repeated play, with experienced players learning to recognize disinformation quickly, rather than a direct effect of reading behaviour, and they call for further research to disentangle these explanations. They also note that fine-grained analytics require access to a game's source code, which limits applicability to commercial titles, though mods may partly offset this constraint. The work positions in-game analytics as a promising, though still developing, tool for studying informal game-based learning at scale.