Level the Level: Balancing Game Levels for Asymmetric Player Archetypes With Reinforcement Learning

This research addresses the challenge of automating game balance for asymmetric player archetypes by utilizing reinforcement learning (RL) to modify tile-based level designs. By focusing on procedural content generation (PCG), the study aims to eliminate the need for extensive manual playtesting when balancing games where players possess disparate abilities, such as varying movement capabilities or victory conditions. The methodology involves training RL agents within the Neural Massively Multiplayer Online (NMMO) environment, using a refined action space that reduces complexity to accelerate training convergence.

The study evaluates the effectiveness of this approach by testing four distinct agent archetypes against a base agent. Performance is measured by the proportion of levels successfully balanced compared to random search and hill-climbing baselines. Results demonstrate that the RL-based approach consistently outperforms both baselines across all tested archetypes. Specifically, the model successfully compensates for inherent imbalances by strategically placing resources and obstacles to equalize win rates. However, the data indicates a clear correlation between the degree of initial asymmetry and the difficulty of the balancing task; as the disparity between player archetypes increases, the number of training steps required for convergence grows, and the model’s overall accuracy in achieving balance decreases.

The research concludes that while RL is a highly effective tool for procedural level balancing, it faces limitations regarding unintended outcomes, such as the generation of levels where neither player can win. Despite this, the refined action space significantly improves training efficiency, providing a scalable framework for future game development. The findings suggest that while level design can effectively mitigate player asymmetry, highly unbalanced starting conditions remain a significant hurdle for current algorithmic approaches.