Finite Automata Extraction: Low-data World Models as Programs from Gameplay Video

Finite Automata Extraction (FAE) provides a neuro-symbolic framework for synthesizing executable world models from minimal gameplay video data. By leveraging a custom domain-specific language known as Retro Coder, this approach transforms raw visual input into transparent, editable programs rather than relying on opaque black-box neural networks. The primary thesis posits that symbolic representation offers superior interpretability and generalizability compared to traditional deep learning models, particularly in scenarios where training data is severely constrained.

Evaluations conducted on classic titles such as Pac-Man and River Raid confirm that FAE achieves higher precision in environment modeling than established neural baselines like GameGAN. By extracting a comprehensive sprite dictionary and formalizing game mechanics into structured code, the method facilitates rapid design iteration and provides a human-readable foundation for game logic. These findings indicate that symbolic extraction is a viable alternative for reverse-engineering game engines, offering a level of clarity that purely neural approaches currently lack.

Despite these advancements, the methodology faces specific technical constraints that define its current scope. The system struggles to accurately process complex sprite transformations, such as rotation or varying sizes, and lacks the capacity to model time-dependent or stochastic behaviors effectively. While these limitations restrict the current application to deterministic, sprite-based environments, the success of FAE in low-data settings marks a significant step toward more efficient, explainable, and modular world modeling in the gaming industry. Future developments focusing on expanding the domain-specific language to handle randomized logic and more complex visual assets will be essential for broader industry adoption.