Methodology for Emotional Analysis of Video Game Footage

This study proposes and demonstrates a methodology for quantifying the emotional expression of esports players using publicly available livestream footage rather than laboratory measurement. Its central argument is that facial emotional expression can be extracted at scale from video recordings and correlated with in-game performance, offering a low-cost route to studying the emotional load of competitive gaming. The authors frame this as a proof of concept, motivated by the limited empirical understanding of how esports affect player emotion, cognition, and wellbeing despite their rapid growth in participation and viewership.

The method combines Affectiva's AFFDEX facial coding software, integrated through the iMotions platform, with recordings of Rocket League streams downloaded from Twitch. Seven streamers, four male and three female, were selected from the top pages of the platform during summer 2020. Individual matches were annotated from the "GO!" signal to the end of the game clock, and seven emotion channels (attention, engagement, anger, surprise, fear, contempt, and sadness) plus joy were averaged per game and normalized. The analysis is limited by its small sample, its reliance on streamers who interact with audiences rather than competing seriously, and the absence of a controlled setting.

The principal findings are that average engagement declined by roughly 1.51 percent per game across players, a pattern the authors attribute either to fatigue dulling facial expression or to improved decision-making efficiency. Emotional expression was judged to track gameplay outcomes, though correlations between specific emotions and score differential were weak. Notable shifts in joy and sadness appeared around transitions between winning and losing streaks. The authors conclude that continuous emotional monitoring could eventually inform performance and wellbeing feedback, and they recommend future work incorporating eye tracking and EEG under controlled conditions.