From Non-Paying to Premium: Predicting User Conversion in Video Games with Ensemble Learning

This study by Yokozuna Data, a Keywords Studio, applies survival analysis to predict when non-paying players in free-to-play games become paying users (PUs). It uses data from two Japanese role-playing card battle games, Age of Ishtaria (AoI) and Grand Sphere (GS). The sample comprised 30,000 AoI users and 10,000 GS users, with daily activity records collected between January 2015 and February 2017 for AoI and between June 2017 and May 2018 for GS. Only players who logged in at least two days were included. Three approaches were compared: Cox regression, random survival forest (RSF), and conditional inference survival ensembles, with RSF also tested with competing risks. Models predicted time in days, game level, and cumulative playtime before conversion.

The authors report that all models were very good at detecting potential PUs, with low false negative and false positive rates. Ensemble models outperformed Cox regression across most validation metrics. The RSF model yielded slightly better lifetime and level predictions than conditional inference ensembles, but performed significantly worse for playtime. Including churn as a competing risk did not outperform the other models and had no clear positive impact. The authors also note that all models tended to predict higher conversion levels than observed.