I Want To See How Smart This AI Really Is: Player Mental Model Development of an Adversarial AI Player

Players develop mental models of adversarial artificial intelligence through iterative engagement, a process that relies on distinct cognitive strategies to interpret and predict machine behavior. By analyzing gameplay sessions of the drawing game iNNk, research reveals that these models are shaped by two primary dimensions: focus and style. Focus describes whether a player approaches the AI through a top-down perspective, relying on prior knowledge and expectations, or a bottom-up approach, which prioritizes immediate sensory data and gameplay feedback. Style further categorizes the interaction as either systematic, involving deliberate hypothesis testing, or reactive, characterized by spontaneous adjustments to AI actions.

The longitudinal development of these models involves a continuous cycle of assimilation and accommodation, where players refine their understanding by comparing AI performance against human benchmarks, historical gameplay data, and external system expectations. While players employ diverse strategies to decipher AI logic, the frequency of concept generation does not necessarily correlate with the accuracy of the resulting mental model. Instead, the effectiveness of a player’s model is determined by how they process discrepancies between their predictions and the AI’s actual output.

These findings offer a practical framework for game designers to influence player learning and engagement. By intentionally structuring game conditions, feedback loops, and AI representations, developers can guide players toward more accurate interpretations of adversarial systems. Strategies such as incorporating reflective in-game prompts and leveraging productive failure allow designers to steer player exploration, ultimately enhancing the depth of the interactive experience. This approach shifts the focus from merely creating complex AI to fostering a transparent, intuitive relationship between the player and the machine, ensuring that the adversarial challenge remains both engaging and cognitively accessible.