How to Evaluate Serious Games Concepts: A Systematic Prototyping and Testing Approach

This paper examines how a systematic, multi-stage prototyping and testing approach can balance learning and entertainment in serious game development, using the case of E.F.A., a game teaching occupational safety and health to managers in the social service sector. The development process alternated between paper prototypes tested by eleven media didactics experts and one occupational safety expert, and a digital prototype tested by 60 members of the target group in online sessions followed by group interviews. The authors investigate how the target group experienced the game and its learning effect, how well feedback from the target group aligns with earlier expert testing on paper, and how feedback varies across user groups and prototype types.

The target group's responses were markedly divided. Some praised the playful approach, the jungle setting, the clear presentation of content, and the knowledge repository, and reported retaining specific facts such as fire protection and accident documentation. Others found the game childish, criticized long dialogs, awkward controls, poor orientation, and an inappropriate metaphor for fatal accidents, and questioned whether knowledge gains would last. Using deductive content analysis across 15 predefined evaluation categories, the authors found that feedback diverged most on first impressions, playing time, remembered knowledge, dialogs, and transfer of learning into the workplace, while mini-games and certain task aspects showed greater consistency with expert findings.

The study concludes that expert testing with low-fidelity prototypes is effective for assessing didactic concepts, mini-games, and general task complexity early in development, whereas criteria such as playing time, dialogs, and perceived fit with the target group are better evaluated with the target group and higher-fidelity prototypes. The authors acknowledge limitations: the findings derive from a single case study, the sample sizes were small, and group interviews may have encouraged conformity and socially desirable answers. Nonetheless, the work offers a practical framework for matching evaluation criteria, test groups, and prototype fidelity in serious game design.