A Machine-Learned Framework for Automatic Content Generation, Evaluation, and Critique

This doctoral proposal, written by Adam Summerville at the University of California, Santa Cruz, argues that procedural level design should move away from human-authored rules and heuristics toward machine learning systems that learn directly from existing creative artifacts. The core premise is that people struggle to introspect accurately about their creative reasoning, making hand-formalized design constraints difficult for lay designers to produce. Instead, the proposed approach asks designers to curate positive and negative examples, and lets models infer the underlying style, playability, and structure of game content.

The proposed end-to-end framework has three components: learning entity properties and annotating 2D game levels (ideally extracted from video), training generative models to produce new levels stylistically matching the inputs while also learning likely player paths, and using the trained models to suggest edits, critique levels by flagging overly bland (high-likelihood) or incoherent (low-likelihood) sections, and visualize probable player routes. The author reports promising early results on generation in The Legend of Zelda and Super Mario Bros. domains, with preliminary work on annotation that falls short of human quality, and initial exploration of player path analysis using YouTube playtraces.

The document is candid about limitations: generalization across games remains incomplete, complex level topologies are not well handled, the video-based pipeline is still under development, and the likelihood-based critique is described as initial work requiring further investigation. Methodology is largely demonstrated through prior published systems rather than a new quantitative evaluation within this text.

Looking forward, the author aims to lower barriers for casual and indie developers by offering a machine-learned "QA department and design partner in a box." The document also identifies graph-structured learning as a crucial unsolved problem, noting that neural methods have progressed from scalar values to images to sequences but still lack strong methods for graphs, which would be needed for richer level topology modeling.