Evaluation of Empirically Collected Feedback from a Simulation Game for Digitalised Production

This study evaluates feedback from a game-based simulation developed by Fraunhofer IPA in Stuttgart to help manufacturing employees transition from lean production to digitalised production. The authors surveyed 117 participants across three focus groups, namely manufacturing company employees (59%), works council representatives (27%), and future qualified employees, meaning students (14%), after workshops held between December 2018 and October 2021. The questionnaire used Likert-scale statements covering learning outcomes and acceptance, along with workshop organisation and motivation. The authors are explicit that the non-random, self-selected sample limits any conclusions to the respondents themselves, so the findings cannot be generalised to all potential participants. The data collection was also disrupted by Covid-19 restrictions, which reduced workshop frequency.

Overall responses were favourable. Around 96% of participants reported interest in the simulation's topic, roughly 94% felt adequately supervised, and more than 95% would recommend the game to others. Participants also broadly agreed that the game clarified what Industry 4.0 means and where its impact lies, though agreement was weaker on whether it prepared them for future challenges, with about half of manufacturing employees and works council representatives expressing satisfaction. Statements on understanding Industry 4.0's lean foundations and its effect on paper-based processes produced the greatest divergence, with work council representatives notably more sceptical about reduced paperwork than other groups.

Across the focus groups, students consistently gave the most positive ratings, with the majority selecting "strongly agree" on most items, while works council representatives were the most critical of the content. Manufacturing employees and works council members rated the execution of the workshops positively but were more reserved about the new learning content. The authors attribute these differences to students encountering concepts they had studied but not yet experienced in practice, while experienced employees may have had less to gain from familiar material. Planned future work includes collecting more data, comparing focus groups across countries, and evaluating an expanded version of the simulation.