Traditional outcome-focused evaluations of educational gaming simulations often fail to capture the underlying mechanisms that drive learning, treating the simulation as a black box. To address this, a theory-oriented logic model approach provides a more robust framework by mapping the causal relationships between game features, learner characteristics, and environmental factors. This methodology shifts the focus from simple end-state measurement to a comprehensive input-process-outcome structure, allowing for a deeper understanding of why specific learning outcomes succeed or fail.
The application of this model to the business simulation SIMGAME, involving 468 students and 37 teachers, demonstrates its practical utility. The evaluation revealed statistically significant improvements in content knowledge, with an effect size of 0.46, alongside high levels of user acceptance. Regression analysis conducted on a subset of 207 participants indicated that input and process variables—specifically prior knowledge, teacher motivation, and the perceived appropriateness of the simulation—accounted for 56% of the variance in cognitive learning outcomes. These findings underscore the critical role of instructor preparation and learner readiness in the efficacy of digital simulations.
While logic models offer a superior, holistic perspective on educational environments, they are best utilized as tools for exploratory analysis rather than rigid hypothesis testing. The complexity of these learning environments necessitates large sample sizes and careful methodological design to account for both intended and unintended outcomes. Ultimately, integrating learning psychology and stakeholder feedback through a logic model framework provides educators and developers with actionable insights that traditional metrics overlook, facilitating more effective design and implementation of simulation-based learning tools.
