Playful Testing: Designing a Formative Assessment Game for Data Science

"Playful Testing" presents Beats Empire, a music-industry management game designed as a formative assessment of data science skills for New York City middle school students, and argues that games can capture computational thinking that standardized tests often miss while building student agency. The book's central thesis is that play-based, stealth assessment can make data collection, storage, visualization, and interpretation engaging and connected to academic subjects, provided that the design balances fun, measurement validity, and usefulness to teachers. Its scope spans participatory design work with NYC teachers and students, pilot classroom implementations in a seventh-grade computer science class, and the collaboration between the research team and the commercial developer Filament Games.

The design process is documented in considerable detail. Early rock-themed concepts were abandoned after student surveys showed little interest, leading to a music-production framework grounded in culturally sustaining pedagogy. Teacher workshops revealed that Art, Music, and Health educators were often more enthusiastic about integrating data concepts than History and English teachers, and that the game fit best in subjects that directly overlap data analysis. An evidence-centered design process narrowed the assessment to four targets, and the team repeatedly traded off open gameplay against assessable evidence, favoring prediction mechanics and premade datasets over complex player-generated data systems.

Empirical findings from pilot logs and think-aloud cognitive interviews with 18 students identified three player groups and varied data-use strategies, showing that players' reasoning shifts over the course of a game and that absence of logged evidence does not imply absence of ability. In classroom trials, students engaged strongly with the game but mostly used bar graphs and rarely adjusted data collection frequency despite understanding the tradeoffs, while a teacher-facing dashboard built on non-punitive, non-comparative design principles received a high usability score. The authors conclude that the game succeeds as an engaging stealth assessment, while acknowledging limits in short play time, incomplete coverage of collection and storage skills, and insufficient evidence yet that it meets goals of identity affirmation and critique of industry inequities.