Developing an Evaluation Framework for Analysing Educational Simulation Games

Slyman, Gillies, and Lytra present the GADDIE model, a game evaluation heuristics framework designed for educational and entertaining simulation games. Their central argument is that existing heuristics such as HEP, MDA, PLAY, and GameFlow are either isolated in scope, repetitive in their criteria, or fail to integrate human-computer interaction, usability, and educational perspectives within a single structure. The authors seek to address this gap by offering a more holistic instrument for assessing games that must be both engaging and pedagogically effective.

The framework is grounded in qualitative evidence drawn from two case studies: forty professional critic reviews of six MMORPG-style games and three focus groups involving 24 university students. Across these sources, gameplay clarity, meaningful feedback, rewards and risk incentives, and a compelling narrative emerged as the primary drivers of engagement. Conversely, unclear objectives, repetitive or unvaried gameplay, weak story appeal, and usability problems such as poor feedback, bugs, and confusing interfaces produced boredom, frustration, and disengagement. Game story, mechanics, aesthetics, usability, and social or payment features were identified as the principal dimensions shaping player experience.

These themes were synthesized into a structured model organizing evaluation criteria across three stages: Game Analysis (20 heuristics), Design and Development (9 heuristics), and Implementation and Evaluation (13 heuristics). The authors note, however, that the model has not been quantitatively validated, so its utility rests on the interpretive coherence of its qualitative foundations. The study's scope is limited to a small, university-based student sample and a specific set of MMORPG-style titles, which bounds the generalizability of its conclusions.