Technical University of Darmstadt

Realizing a Mobile Multimodal Platform for Serious Games Analytics

The StoryPlay Multimodal platform establishes a sophisticated framework for evaluating serious games by synchronizing granular gameplay logs with diverse multimodal data, including front-facing video, audio, and mobile sensor readings.

• The StoryPlay platform enables remote, non-invasive serious game evaluation by synchronizing granular gameplay logs with multimodal data like front-facing video, audio, and mobile sensor readings.
• By reconstructing sessions directly through the game engine instead of using resource-heavy screen recordings, the system achieves efficient, objective analysis of player affective and cognitive states.
• The architecture consists of three core components: a mobile Capturer for data acquisition, an Observer tool for qualitative logging, and a C#-based desktop Replayer for synchronized analysis.