SmartPlay: A Benchmark for LLMs as Intelligent Agents

SmartPlay: A Benchmark for LLMs as Intelligent Agents

2 October 2023

Tom Michael Mitchell

Papers citing "SmartPlay: A Benchmark for LLMs as Intelligent Agents"

18 / 18 papers shown

Title
Humanizing LLMs: A Survey of Psychological Measurements with Tools, Datasets, and Human-Agent Applications Wenhan Dong Yuemeng Zhao Zhen Sun Yule Liu Zifan Peng ... Jun Wu Ruiming Wang Shengmin Xu Xinyi Huang Xinlei He LLMAG 55 0 0 30 Apr 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong X. Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Yu Jiang ALM ELM 84 0 0 26 Apr 2025
ZeroSumEval: An Extensible Framework For Scaling LLM Evaluation with Inter-Model Competition H. A. Alyahya Haidar Khan Yazeed Alnumay M Saiful Bari B. Yener LRM 60 1 0 10 Mar 2025
AppAgentX: Evolving GUI Agents as Proficient Smartphone Users Wenjia Jiang Yangyang Zhuang Chenxi Song Xu Yang Chi Zhang Chi Zhang LLMAG 82 1 0 04 Mar 2025
Ad-hoc Concept Forming in the Game Codenames as a Means for Evaluating Large Language Models Sherzod Hakimov Lara Pfennigschmidt David Schlangen ELM 53 0 0 17 Feb 2025
BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games Davide Paglieri Bartłomiej Cupiał Samuel Coward Ulyana Piterbarg Maciej Wolczyk ... Lerrel Pinto Rob Fergus Jakob Foerster Jack Parker-Holder Tim Rocktaschel LLMAG LRM 106 10 0 20 Nov 2024
SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluation Jingxuan Chen Derek Yuen Bin Xie Y. Yang Gongwei Chen ... Liqiang Nie Yasheng Wang Jianye Hao Jun Wang Kun Shao LLMAG 38 5 0 19 Oct 2024
Learning to Ask: When LLM Agents Meet Unclear Instruction Wenxuan Wang Juluan Shi Chaozheng Wang Cheryl Lee Chaozheng Wang Cheryl Lee Youliang Yuan Jen-tse Huang Wenxiang Jiao Michael R. Lyu LLMAG 24 8 0 31 Aug 2024
Testing and Understanding Erroneous Planning in LLM Agents through Synthesized User Inputs Zhenlan Ji Daoyuan Wu Pingchuan Ma Zongjie Li Shuai Wang LLMAG 40 3 0 27 Apr 2024
How Far Are We on the Decision-Making of LLMs? Evaluating LLMs' Gaming Ability in Multi-Agent Environments Jen-tse Huang E. Li Man Ho Lam Tian Liang Wenxuan Wang Youliang Yuan Wenxiang Jiao Xing Wang Zhaopeng Tu Michael R. Lyu ELM LLMAG 77 32 0 18 Mar 2024
Towards Unified Alignment Between Agents, Humans, and Environment Zonghan Yang An Liu Zijun Liu Kai Liu Fangzhou Xiong ... Zhenhe Zhang Fuwen Luo Zhicheng Guo Peng Li Yang Liu 24 4 0 12 Feb 2024
Plan, Eliminate, and Track -- Language Models are Good Teachers for Embodied Agents Yu-Chih Chen So Yeon Min Chase Davis Ruslan Salakhutdinov A. Azaria Yuan-Fang Li Tom Michael Mitchell A. Bovik LM&Ro LLMAG 70 32 0 03 May 2023
Can Large Language Models Be an Alternative to Human Evaluations? Cheng-Han Chiang Hung-yi Lee ALM LM&MA 209 559 0 03 May 2023
Generative Agents: Interactive Simulacra of Human Behavior J. Park Joseph C. O'Brien Carrie J. Cai Meredith Ringel Morris Percy Liang Michael S. Bernstein LM&Ro AI4CE 215 1,701 0 07 Apr 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 215 2,232 0 22 Mar 2023
BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments S. Srivastava Chengshu Li Michael Lingelbach Roberto Martín-Martín Fei Xia ... C. Karen Liu Silvio Savarese H. Gweon Jiajun Wu Li Fei-Fei LM&Ro 135 152 0 06 Aug 2021
Grounding Language to Entities and Dynamics for Generalization in Reinforcement Learning H. Wang Victor Zhong Karthik Narasimhan 76 53 0 19 Jan 2021
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 275 1,561 0 18 Sep 2019