The Wisdom of the Gaming Crowd

The primary objective of this research is to establish a baseline for the application of natural language processing (NLP) techniques to video game reviews, specifically those hosted on the Steam platform. By analyzing large-scale user-generated content, the study aims to provide tools and methodologies that assist players in making informed purchasing decisions, help developers address customer feedback, and enable researchers to better understand player experiences. The thesis posits that while game reviews are uniquely subjective and informal compared to other consumer goods, NLP can effectively extract features, determine sentiment, and identify spam.

The methodology involves scraping review data via the official Steam API, which allows for the collection of high-volume datasets including review text and metadata such as helpfulness and humor ratings. The research covers three distinct analytical projects: feature summarization, sentiment analysis, and spam detection. For feature summarization, the authors utilized a modified High Adjective Count (HAC) algorithm on the five most-played games on Steam at the time, achieving relevance rates between 53.3% and 66.7%. Sentiment analysis was conducted using Naïve Bayes and Support Vector Machine algorithms on four selected titles, finding that prediction accuracy is generally higher for games with a clear majority of positive reviews.

The spam detection project categorized deceptive content into untruthful, duplicate, and non-review types. By applying Positive Unlabelled Learning to a dataset of over 33,000 reviews, the researchers successfully identified approximately 15% of the sample as spam, a figure consistent with broader e-commerce industry trends. The study concludes that while existing NLP algorithms are effective, they face challenges regarding game-specific context and the current lack of labeled datasets. Future research is encouraged to focus on creating specialized, labeled game review corpora to improve the accuracy and generalizability of automated analysis tools.