With the rapid advancement of Large Language Models (LLMs), LLM-based autonomous agents have shown the potential to function as digital employees, such as digital analysts, teachers, and programmers. In this paper, we develop an application-level testbed based on the open-source strategy game "Unciv", which has millions of active players, to enable researchers to build a "data flywheel" for studying human-like agents in the "digital players" task. This "Civilization"-like game features expansive decision-making spaces along with rich linguistic interactions such as diplomatic negotiations and acts of deception, posing significant challenges for LLM-based agents in terms of numerical reasoning and long-term planning. Another challenge for "digital players" is to generate human-like responses for social interaction, collaboration, and negotiation with human players. The open-source project can be found athttps:/github.com/fuxiAIlab/CivAgent.
View on arXiv@article{wang2025_2502.20807,
title={ Digital Player: Evaluating Large Language Models based Human-like Agent in Games },
author={ Jiawei Wang and Kai Wang and Shaojie Lin and Runze Wu and Bihan Xu and Lingeng Jiang and Shiwei Zhao and Renyu Zhu and Haoyu Liu and Zhipeng Hu and Zhong Fan and Le Li and Tangjie Lyu and Changjie Fan },
journal={arXiv preprint arXiv:2502.20807},
year={ 2025 }
}