GAMA: Generative Agents for Multi-Agent Autoformalization

Multi-agent simulations facilitate the exploration of interactions among both natural and artificial agents. However, modelling real-world scenarios and developing simulations often requires substantial expertise and effort. To streamline this process, we present a framework that enables the autoformalization of interaction scenarios using agents augmented by large language models (LLMs) utilising game-theoretic formalisms. The agents translate natural language descriptions of interactions into executable logic programs that define the rules of each game, ensuring syntactic correctness through validation by a solver. A tournament simulation then tests the functionality of the generated game rules and strategies. After the tournament, if a ground truth payoff matrix is available, an exact semantic validation is performed. We evaluate our approach on a diverse set of 110 natural language descriptions exemplifying five $2\times2$ simultaneous-move games, achieving 100% syntactic and 76.5% semantic correctness in the generated game rules for Claude 3.5 Sonnet, and 99.82% syntactic and 77% semantic correctness for GPT-4o. Additionally, we demonstrate high semantic correctness in autoformalizing gameplay strategies. Overall, the results highlight the potential of autoformalization to leverage LLMs in generating formal reasoning modules for decision-making agents.