Comparing the Student Engagement with Two Versions of a Game-based Learning Tool

This study compares student engagement with two versions of Tower of Questions (ToQ), a peer-quizzing game-based learning tool, one implemented as a web application and the other as an Android mobile app. Conducted during the online-only phase of the Covid-19 pandemic, the research recruited 121 first-year programming students at a Canadian university over a four-month academic term in 2021, with 75 assigned to the web version and 46 to the mobile version. Engagement was measured through weekly active users, towers (questions) created, and towers successfully solved, while a post-study survey captured user experience ratings. Participation was incentivized with a bonus of up to five percent of course marks for students posting at least three questions per week.

The findings reveal no significant difference in the number of weekly active users between the two versions, indicating comparable overall reach. However, students created significantly more questions on the mobile app, with a higher median (431 versus 385 towers) and a large effect size (Cohen's d of 1.39). Conversely, students in the web group solved significantly more questions than their mobile counterparts, with a median of 314 versus 214 towers solved and a large effect size (Cohen's d of 1.53). Survey respondents rated both versions favorably, with average satisfaction scores of 3.65 for the app and 3.86 for the web version on a five-point scale, though response rates were modest at roughly 40 to 53 percent.

The authors attribute the differences to platform characteristics: question writing may be favored on mobile devices for convenience, while the web interface may better support reading lengthy multiple-choice questions and consulting course materials when answering. The study concludes that both platforms effectively engage students in practice-based learning and recommends that educational game designers consider offering both web and mobile versions so students can choose the format that suits each activity.