LAMSADE

Monte Carlo Search Algorithms Discovering Monte Carlo Tree Search Exploration Terms

Automated discovery methods for Monte Carlo Tree Search exploration terms represent a significant shift in algorithmic design, moving away from manual, theoretically derived formulas toward machine-generated optimization.

The Mathematical Game

This analysis evaluates the performance of various game tree search algorithms applied to automated theorem proving within the Metamath formal language environment.

• A hybrid search strategy combining Product Propagation with a modified PUCT bandit function outperformed all other tested methods, solving 144 theorems.
• The hybrid approach successfully surpassed the 141-theorem performance ceiling that had previously limited other search algorithms in the Metamath environment.
• Product Propagation demonstrated superior efficiency for automated theorem proving, particularly when operating under strict time constraints.

Batch Monte Carlo Tree Search

This analysis examines the implementation and optimization of Batch Monte Carlo Tree Search (MCTS) for complex two-player games, specifically focusing on the game of Go.

• Batching neural network inferences for MCTS on GPUs increases throughput by 26-fold when using a batch size of 32 compared to sequential processing.
• Separating the search tree from the transposition table improves both computational throughput and the accuracy of search results in complex two-player games like Go.
• The µ FPU (First Play Urgency) heuristic outperforms constant or best-mean FPU methods for guiding exploration during tree searches.