Can We Detect Non-playable Characters' Personalities Using Machine and Deep Learning Approaches?

This study investigates whether machine learning and deep learning can recognize the personalities of non-playable characters (NPCs) in visual novel games from their dialogue, using the Myers-Briggs Type Indicator (MBTI) as the typology. The motivation is practical: companies designing gamified situational judgment tests for personnel selection want NPCs whose personalities are consistent and realistic, which may strengthen the validity of team-cohesion assessment. The authors position the work as a preliminary, hypothesis-generating investigation, noting that no prior research had applied natural language processing to NPC personality detection.

The dataset comprises roughly 12,600 character speeches drawn from nine visual novel games, grouped into 6,200 labeled documents by character and game chapter, with 170 NPCs labeled through a fan-voting platform. Five conventional classifiers (logistic regression, support vector machine, naive Bayes, decision tree, random forest) were compared against a convolutional neural network, using bag-of-words features and 10-fold cross-validation. Logistic regression achieved the highest average F1-score among the traditional methods (about 0.713), while the binary CNN reached an average accuracy of 0.720. A 16-class CNN classifier achieved an accuracy of 0.728 and an F1-score of 0.715, outperforming the overall binary results. Per-trait performance varied, with the strongest single result being a random forest J/P classifier at an F1-score of 0.784.

The authors acknowledge several limitations: class imbalance across dichotomies, a modest dataset size, possible semantic overlap between training and test documents from the same character, and the absence of hyperparameter tuning. They also note that the MBTI's academic validity is contested. In a preliminary use case, the models helped writers revise dialogue on two learning games to better reflect intended personalities. The study concludes that traditional and deep learning methods are viable for NPC personality recognition, though differences among the top approaches are not statistically significant, and that further work with richer data and alternative typologies is needed.