Exploring Apprenticeship Learning for Player Modelling in Interactive Narratives

This research explores the application of Apprenticeship Learning (AL) to model player behavior within Interactive Narratives (INs). The primary objective is to develop artificial agents capable of mimicking specific player styles and decision-making processes, which can assist game designers in personalizing experiences or identifying potential narrative gaps before product launch. By utilizing Inverse Reinforcement Learning (IRL), the study seeks to derive reward functions from human gameplay traces without requiring manual definition of goals, thereby capturing the sub-optimal and dynamic nature of human decision-making in partially observable environments.

The methodology employs Receding Horizon Inverse Reinforcement Learning (RHIRL) to analyze 36 gameplay traces generated by 23 anonymous players of the interactive fiction game Anchorhead. The researchers modeled the game as a Markov Decision Process, incorporating nine parameterized actions and various state representations, such as location, inventory status, and plot point progression. The study tested different configurations of step horizons and noise parameters to evaluate how well the learned policies replicated the behavior of specific player groups, categorized by their game endings and player profile factors like persistence and familiarity.

Key findings indicate that RHIRL effectively learns action sequences that mirror human behavior, with the best performance achieved when accounting for the inherent noise and sub-optimality of human play. Specifically, configurations favoring intention learning—characterized by larger step horizons and lower optimality assumptions—outperformed those focused on strict imitation. While the model successfully captured behavior for certain player groups, the study notes that performance varied based on the complexity of the game path, suggesting that future iterations should incorporate larger datasets and hybrid models to further improve generalizability and robustness in diverse interactive narrative settings.