Can ChatGPT Match Experts? Comparing Input for Serious Game Development

Generative artificial intelligence, specifically ChatGPT versions 3.5 and 4.0, serves as a viable supplementary tool for serious game development, capable of producing design inputs that are qualitatively and statistically comparable to those provided by human experts. By analyzing 148 expert suggestions alongside 1,455 AI-generated ideas across the domains of learning content, game mechanics, gamification, and accessibility, the research establishes that AI can effectively reduce development workloads. However, the utility of these models remains highly sensitive to specific variables, including the underlying model version and the application of persona-based prompting, which often yields inconsistent results.

While the AI successfully replicates standard design insights, it lacks the nuanced contextual understanding inherent to human professionals. Human experts uniquely contribute complex narrative structures and identify critical pedagogical concerns, such as potential cheating vulnerabilities, that the models fail to address. Furthermore, the reliance on persona prompts—such as instructing the AI to act as a game designer—does not guarantee superior output and can, in some instances, hinder the statistical reliability of the generated data.

Ultimately, the integration of generative AI into the serious game design process requires a balanced approach that prioritizes human oversight. Developers should utilize multiple model versions to cross-reference outputs and avoid over-reliance on specific prompting techniques. Because the AI functions most effectively as a collaborative partner rather than a replacement for professional judgment, maintaining human expertise remains essential for ensuring the educational integrity and functional depth of serious games.