Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents

3 February 2023

Xiaojian Ma

Papers citing "Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents"

50 / 254 papers shown

Title
WALL-E: World Alignment by Rule Learning Improves World Model-based LLM Agents Siyu Zhou Tianyi Zhou Yijun Yang Guodong Long Deheng Ye Jing Jiang Chengqi Zhang LLMAG LM&Ro 17 3 0 09 Oct 2024
EVOLvE: Evaluating and Optimizing LLMs For Exploration Allen Nie Yi Su Bo Chang Jonathan N. Lee Ed H. Chi Quoc V. Le Minmin Chen 16 5 0 08 Oct 2024
MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agents Junpeng Yue Xinru Xu Börje F. Karlsson Zongqing Lu 32 0 0 04 Oct 2024
Open-World Reinforcement Learning over Long Short-Term Imagination Jiajian Li Q. Wang Yunbo Wang Xin Jin Yang Li Wenjun Zeng Xiaokang Yang OCL VLM 47 1 0 04 Oct 2024
SEAL: SEmantic-Augmented Imitation Learning via Language Model Chengyang Gu Yuxin Pan Haotian Bai Hui Xiong Yize Chen 27 0 0 03 Oct 2024
ToolPlanner: A Tool Augmented LLM for Multi Granularity Instructions with Path Planning and Feedback Qinzhuo Wu Wei Liu Jian Luan Bin Wang 45 5 0 23 Sep 2024
InteLiPlan: An Interactive Lightweight LLM-Based Planner for Domestic Robot Autonomy Kim Tien Ly Kai Lu Ioannis Havoutis 26 2 0 22 Sep 2024
Can VLMs Play Action Role-Playing Games? Take Black Myth Wukong as a Study Case Peng Chen Pi Bu Jun Song Yuan Gao Bo Zheng LLMAG 27 10 0 19 Sep 2024
Assessing the Zero-Shot Capabilities of LLMs for Action Evaluation in RL Eduardo Pignatelli Johan Ferret Tim Rockäschel Edward Grefenstette Davide Paglieri Samuel Coward Laura Toni 30 2 0 19 Sep 2024
LLM-Agent-UMF: LLM-based Agent Unified Modeling Framework for Seamless Integration of Multi Active/Passive Core-Agents Amine B. Hassouna Hana Chaari Ines Belhaj LLMAG 30 1 0 17 Sep 2024
Hackphyr: A Local Fine-Tuned LLM Agent for Network Security Environments M. Rigaki C. Catania Sebastian Garcia LLMAG 21 3 0 17 Sep 2024
Improving Interface Design in Interactive Task Learning for Hierarchical Tasks based on a Qualitative Study Jieyu Zhou Christopher MacLellan 23 0 0 17 Sep 2024
BattleAgentBench: A Benchmark for Evaluating Cooperation and Competition Capabilities of Language Models in Multi-Agent Systems Wei Wang Dan Zhang Tao Feng Boyan Wang Jie Tang LLMAG ELM 31 2 0 28 Aug 2024
HiAgent: Hierarchical Working Memory Management for Solving Long-Horizon Agent Tasks with Large Language Model Mengkang Hu Tianxing Chen Qiguang Chen Yao Mu Wenqi Shao Ping Luo LM&Ro LLMAG RALM 24 3 0 18 Aug 2024
Perceive, Reflect, and Plan: Designing LLM Agent for Goal-Directed City Navigation without Instructions Qingbin Zeng Qinglong Yang Shunan Dong Heming Du Liang Zheng Fengli Xu Yong Li LLMAG LM&Ro 24 8 0 08 Aug 2024
Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks Zaijing Li Yuquan Xie Rui Shao Gongwei Chen Dongmei Jiang Liqiang Nie 49 18 0 07 Aug 2024
Caution for the Environment: Multimodal Agents are Susceptible to Environmental Distractions Xinbei Ma Yiting Wang Yao Yao Tongxin Yuan Aston Zhang Zhuosheng Zhang Hai Zhao AAML LLMAG 22 16 0 05 Aug 2024
Visual Grounding for Object-Level Generalization in Reinforcement Learning Haobin Jiang Zongqing Lu LM&Ro 25 2 0 04 Aug 2024
Semantic Skill Grounding for Embodied Instruction-Following in Cross-Domain Environments Sangwoo Shin Takehiro Matsuoka Youngsoo Jang Moontae Lee Kazuya Yoshida 38 0 0 02 Aug 2024
AgentGen: Enhancing Planning Abilities for Large Language Model based Agent via Environment and Task Generation Mengkang Hu Yixiao Wang Can Xu Lingfeng Sun Chensheng Peng T. Hannagan Nicola Poerio Saravan Rajmohan LM&Ro LLMAG 60 15 0 01 Aug 2024
Enhancing Agent Learning through World Dynamics Modeling Zhiyuan Sun Haochen Shi Marc-Alexandre Côté Glen Berseth Xingdi Yuan Bang Liu 47 3 0 25 Jul 2024
VideoGameBunny: Towards vision assistants for video games Mohammad Reza Taesiri C. Bezemer VLM MLLM 33 2 0 21 Jul 2024
LLM-Empowered State Representation for Reinforcement Learning Boyuan Wang Yun Qu Yuhang Jiang Jianzhun Shao Chang-rui Liu Wenming Yang Xiangyang Ji 24 7 0 18 Jul 2024
Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systems Tamer Abuelsaad Deepak Akkil Prasenjit Dey Ashish Jagmohan Aditya Vempaty Ravi Kokku 39 23 0 17 Jul 2024
Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI Y. Liu Weixing Chen Yongjie Bai Xiaodan Liang Guanbin Li Wen Gao Liang Lin LM&Ro SyDa AI4CE 48 47 0 09 Jul 2024
Aligning Teacher with Student Preferences for Tailored Training Data Generation Yantao Liu Zhao Zhang Zijun Yao S. Cao Lei Hou Juanzi Li 42 1 0 27 Jun 2024
OmniJARVIS: Unified Vision-Language-Action Tokenization Enables Open-World Instruction Following Agents Zihao Wang Shaofei Cai Zhancun Mu Haowei Lin Ceyao Zhang Xuejie Liu Qing Li Anji Liu Xiaojian Ma Yitao Liang LM&Ro 30 11 0 27 Jun 2024
Towards a Science Exocortex Kevin G. Yager 74 0 0 24 Jun 2024
Data Augmentation of Multi-turn Psychological Dialogue via Knowledge-driven Progressive Thought Prompting Jiyue Jiang Liheng Chen Sheng Wang Lingpeng Kong Yu-Hu Li Chuan Wu 41 0 0 24 Jun 2024
VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought Gabriel H. Sarch Lawrence Jang Michael J. Tarr William W. Cohen Kenneth Marino Katerina Fragkiadaki LLMAG 31 8 0 20 Jun 2024
Look Further Ahead: Testing the Limits of GPT-4 in Path Planning Mohamed Aghzal E. Plaku Ziyu Yao ELM 24 6 0 17 Jun 2024
From Text to Life: On the Reciprocal Relationship between Artificial Life and Large Language Models Eleni Nisioti Claire Glanois Elias Najarro Andrew Dai Elliot Meyerson J. Pedersen Laetitia Teodorescu Conor F. Hayes Shyam Sudhakaran Sebastian Risi AI4CE LM&Ro 35 2 0 14 Jun 2024
Security of AI Agents Yifeng He Ethan Wang Yuyang Rong Zifei Cheng Hao Chen LLMAG 29 7 0 12 Jun 2024
LLM-Craft: Robotic Crafting of Elasto-Plastic Objects with Large Language Models Alison Bartsch A. Farimani 76 6 0 12 Jun 2024
MobileAgentBench: An Efficient and User-Friendly Benchmark for Mobile LLM Agents Luyuan Wang Yongyu Deng Yiwei Zha Guodong Mao Qinmin Wang Tianchen Min Wei Chen Shoufa Chen LLMAG 40 12 0 12 Jun 2024
Transforming Wearable Data into Health Insights using Large Language Model Agents Mike A. Merrill Akshay Paruchuri Naghmeh Rezaei Geza Kovacs Javier Perez ... Shwetak Patel Jiening Zhan Tim Althoff Daniel J. McDuff Xin Liu LM&MA LLMAG AI4CE 35 8 0 10 Jun 2024
VillagerAgent: A Graph-Based Multi-Agent Framework for Coordinating Complex Task Dependencies in Minecraft Yubo Dong Xukun Zhu Zhengzhe Pan Linchao Zhu Yi Yang 35 11 0 09 Jun 2024
GameBench: Evaluating Strategic Reasoning Abilities of LLM Agents Anthony Costarelli Mat Allen Roman Hauksson Grace Sodunke Suhas Hariharan Carlson Cheng Wenjie Li Joshua Clymer Arjun Yadav ELM ReLM LLMAG LRM 33 16 0 07 Jun 2024
Open Grounded Planning: Challenges and Benchmark Construction Shiguang Guo Ziliang Deng Hongyu Lin Y. Lu Xianpei Han Le Sun LRM LM&Ro LLMAG 22 1 0 05 Jun 2024
A Survey of Useful LLM Evaluation Ji-Lun Peng Sijia Cheng Egil Diau Yung-Yu Shih Po-Heng Chen Yen-Ting Lin Yun-Nung Chen LLMAG ELM 24 12 0 03 Jun 2024
Evaluating Uncertainty-based Failure Detection for Closed-Loop LLM Planners Zhi Zheng Qian Feng Hang Li Alois C. Knoll Jianxiang Feng 44 6 0 01 Jun 2024
Learning to Discuss Strategically: A Case Study on One Night Ultimate Werewolf Xuanfa Jin Ziyan Wang Yali Du Meng Fang Haifeng Zhang Jun Wang OffRL LLMAG 46 5 0 30 May 2024
LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence Zhuoling Li Xiaogang Xu Zhenhua Xu Sernam Lim Hengshuang Zhao LM&Ro 32 2 0 27 May 2024
Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agents Zhengliang Shi Shen Gao Xiuyi Chen Yue Feng Lingyong Yan Haibo Shi Dawei Yin Zhumin Chen Suzan Verberne LLMAG 47 14 0 26 May 2024
STRIDE: A Tool-Assisted LLM Agent Framework for Strategic and Interactive Decision-Making Chuanhao Li Runhan Yang Tiankai Li Milad Bafarassat Kourosh Sharifi Dirk Bergemann Zhuoran Yang LLMAG 22 5 0 25 May 2024
AutoManual: Generating Instruction Manuals by LLM Agents via Interactive Environmental Learning Minghao Chen Yihang Li Yanting Yang Shiyu Yu Binbin Lin Xiaofei He LLMAG 34 1 0 25 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 64 41 0 23 May 2024
Token-wise Influential Training Data Retrieval for Large Language Models Huawei Lin Jikai Long Zhaozhuo Xu Weijie Zhao 42 3 0 20 May 2024
(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts Minghao Wu Jiahao Xu Yulin Yuan Gholamreza Haffari Longyue Wang Weihua Luo Kaifu Zhang LLMAG 114 22 0 20 May 2024
Meta-Control: Automatic Model-based Control Synthesis for Heterogeneous Robot Skills Tianhao Wei Liqian Ma Rui Chen Weiye Zhao Changliu Liu 32 2 0 18 May 2024