Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents

3 February 2023

Xiaojian Ma

Papers citing "Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents"

50 / 254 papers shown

Title
RePLan: Robotic Replanning with Perception and Language Models Marta Skreta Zihan Zhou Jia Lin Yuan Kourosh Darvish Alán Aspuru-Guzik Animesh Garg LM&Ro LRM 27 26 0 08 Jan 2024
NovelGym: A Flexible Ecosystem for Hybrid Planning and Learning Agents Designed for Open Worlds Shivam Goel Yichen Wei Panagiotis Lymperopoulos matthias. scheutz Jivko Sinapov GNN 42 2 0 07 Jan 2024
On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS) Vishal Pallagani Kaushik Roy Bharath Muppasani F. Fabiano Andrea Loreggia K. Murugesan Biplav Srivastava F. Rossi L. Horesh Amit P. Sheth 18 35 0 04 Jan 2024
LLM-SAP: Large Language Models Situational Awareness Based Planning Liman Wang Hanyang Zhong LLMAG 23 2 0 26 Dec 2023
LARP: Language-Agent Role Play for Open-World Games Ming Yan Ruihao Li Hao Zhang Hao Wang Zhilan Yang Ji Yan LLMAG LM&Ro AI4CE 22 16 0 24 Dec 2023
CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Update Zhi Gao Yuntao Du Xintong Zhang Xiaojian Ma Wenjuan Han Song-Chun Zhu Qing Li LLMAG VLM 26 21 0 18 Dec 2023
Learning adaptive planning representations with natural language guidance L. Wong Jiayuan Mao Pratyusha Sharma Zachary S. Siegel Jiahai Feng Noa Korneev Joshua B. Tenenbaum Jacob Andreas LM&Ro 13 21 0 13 Dec 2023
MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception Yiran Qin Enshen Zhou Qichang Liu Zhen-fei Yin Lu Sheng Ruimao Zhang Yu Qiao Jing Shao LM&Ro 15 39 0 12 Dec 2023
Building Open-Ended Embodied Agent via Language-Policy Bidirectional Adaptation Shaopeng Zhai Jie Wang Tianyi Zhang Fuxian Huang Qi Zhang Ming Zhou Jing Hou Yu Qiao Yu Liu LLMAG LM&Ro 19 1 0 12 Dec 2023
ThinkBot: Embodied Instruction Following with Thought Chain Reasoning Guanxing Lu Ziwei Wang Changliu Liu Jiwen Lu Yansong Tang LRM 22 8 0 12 Dec 2023
Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planning Zhiting Hu Tianmin Shu LLMAG LM&Ro LRM 102 34 0 08 Dec 2023
LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystem Yingqiang Ge Yujie Ren Wenyue Hua Shuyuan Xu Juntao Tan Yongfeng Zhang LLMAG 12 27 0 06 Dec 2023
Imitating Shortest Paths in Simulation Enables Effective Navigation and Manipulation in the Real World Kiana Ehsani Tanmay Gupta Rose Hendrix Jordi Salvador Luca Weihs ... Alvaro Herrasti Ranjay Krishna Dustin Schwenk Eli VanderBilt Aniruddha Kembhavi 22 12 0 05 Dec 2023
Creative Agents: Empowering Agents with Imagination for Creative Tasks Chi Zhang Penglin Cai Yuhui Fu Haoqi Yuan Zongqing Lu LM&Ro LLMAG 49 20 0 05 Dec 2023
LLM-State: Open World State Representation for Long-horizon Task Planning with Large Language Model Siwei Chen Anxing Xiao David Hsu LM&Ro 16 5 0 29 Nov 2023
Embodied Multi-Modal Agent trained by an LLM from a Parallel TextWorld Yijun Yang Tianyi Zhou Kanxue Li Dapeng Tao Lusong Li Li Shen Xiaodong He Jing Jiang Yuhui Shi LLMAG LM&Ro 10 34 0 28 Nov 2023
See and Think: Embodied Agent in Virtual Environment Zhonghan Zhao Wenhao Chai Xuan Wang Li Boyi Shengyu Hao Shidong Cao Tianbo Ye Gaoang Wang LM&Ro LLMAG 11 31 0 26 Nov 2023
Towards Responsible Generative AI: A Reference Architecture for Designing Foundation Model based Agents Qinghua Lu Liming Zhu Xiwei Xu Zhenchang Xing Stefan Harrer Jon Whittle LM&Ro AI4CE LLMAG 29 11 0 22 Nov 2023
Applications of Large Scale Foundation Models for Autonomous Driving Yu Huang Yue Chen Zhu Li ELM AI4CE LRM ALM LM&Ro 46 14 0 20 Nov 2023
An Embodied Generalist Agent in 3D World Jiangyong Huang Silong Yong Xiaojian Ma Xiongkun Linghu Puhao Li Yan Wang Qing Li Song-Chun Zhu Baoxiong Jia Siyuan Huang LM&Ro 23 131 0 18 Nov 2023
JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Models Zihao Wang Shaofei Cai Anji Liu Yonggang Jin Jinbing Hou ... Zhaofeng He Zilong Zheng Yaodong Yang Xiaojian Ma Yitao Liang LLMAG LM&Ro 17 95 0 10 Nov 2023
Human-Centered Planning Yuliang Li Nitin Kamra Ruta Desai A. Halevy 11 1 0 08 Nov 2023
Everything of Thoughts: Defying the Law of Penrose Triangle for Thought Generation Ruomeng Ding Chaoyun Zhang Lu Wang Yong Xu Ming-Jie Ma Wei Zhang Si Qin Saravan Rajmohan Qingwei Lin Dongmei Zhang LRM 25 59 0 07 Nov 2023
Active Reasoning in an Open-World Environment Manjie Xu Guangyuan Jiang Weihan Liang Chi Zhang Yixin Zhu LLMAG LRM 8 10 0 03 Nov 2023
Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game Zelai Xu Chao Yu Fei Fang Yu Wang Yi Wu LLMAG 21 77 0 29 Oct 2023
Conditionally Combining Robot Skills using Large Language Models K. Zentner Ryan C. Julian Brian Ichter Gaurav Sukhatme 12 1 0 25 Oct 2023
Open-Ended Instructable Embodied Agents with Memory-Augmented Large Language Models Gabriel H. Sarch Yue Wu Michael J. Tarr Katerina Fragkiadaki LM&Ro LLMAG 11 18 0 23 Oct 2023
Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worlds Sipeng Zheng Jiazheng Liu Yicheng Feng Zongqing Lu 29 29 0 20 Oct 2023
ToolChain: Efficient Action Space Navigation in Large Language Models with A Search Yuchen Zhuang Xiang Chen Tong Yu Saayan Mitra Victor S. Bursztyn Ryan A. Rossi Somdeb Sarkhel Chao Zhang LLMAG 29 52 0 20 Oct 2023
Video Language Planning Yilun Du Mengjiao Yang Peter R. Florence Fei Xia Ayzaan Wahid ... Pieter Abbeel Josh Tenenbaum L. Kaelbling Andy Zeng Jonathan Tompson PINN LM&Ro 89 83 0 16 Oct 2023
Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlook Ming Jin Qingsong Wen Yuxuan Liang Chaoli Zhang Siqiao Xue ... Shirui Pan Vincent S. Tseng Yu Zheng Lei Chen Hui Xiong AI4TS SyDa 31 116 0 16 Oct 2023
Theory of Mind for Multi-Agent Collaboration via Large Language Models Huao Li Yu Quan Chong Simon Stepputtis Joseph Campbell Dana Hughes Michael Lewis Katia P. Sycara LLMAG 20 59 0 16 Oct 2023
LLaMA Rider: Spurring Large Language Models to Explore the Open World Yicheng Feng Yuxuan Wang Jiazheng Liu Sipeng Zheng Zongqing Lu LLMAG LRM 11 15 0 13 Oct 2023
Octopus: Embodied Vision-Language Programmer from Environmental Feedback Jingkang Yang Yuhao Dong Shuai Liu Bo-wen Li Ziyue Wang ... Haoran Tan Jiamu Kang Yuanhan Zhang Kaiyang Zhou Ziwei Liu LM&Ro 33 45 0 12 Oct 2023
Tree-Planner: Efficient Close-loop Task Planning with Large Language Models Mengkang Hu Yao Mu Xinmiao Yu Mingyu Ding Shiguang Wu Wenqi Shao Qiguang Chen Bin Wang Yu Qiao Ping Luo LLMAG 36 33 0 12 Oct 2023
Cross-Episodic Curriculum for Transformer Agents Lucy Xiaoyang Shi Yunfan Jiang Jake Grigsby Linxi "Jim" Fan Yuke Zhu 8 4 0 12 Oct 2023
GROOT: Learning to Follow Instructions by Watching Gameplay Videos Shaofei Cai Bowei Zhang Zihao Wang Xiaojian Ma Anji Liu Yitao Liang 83 26 0 12 Oct 2023
CoPAL: Corrective Planning of Robot Actions with Large Language Models Frank Joublin Antonello Ceravola Pavel Smirnov Felix Ocker Joerg Deigmoeller Anna Belardinelli Chao Wang Stephan Hasler Daniel Tanneberg Michael Gienger LM&Ro LLMAG 39 34 0 11 Oct 2023
LLMs Killed the Script Kiddie: How Agents Supported by Large Language Models Change the Landscape of Network Threat Testing Stephen Moskal Sam Laney Erik Hemberg Una-May O’Reilly 26 16 0 10 Oct 2023
Self-Convinced Prompting: Few-Shot Question Answering with Repeated Introspection Haodi Zhang Min Cai Xinhe Zhang Chen Zhang Rui Mao Kaishun Wu KELM LRM ReLM 20 8 0 08 Oct 2023
Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyond Liang Chen Yichi Zhang Shuhuai Ren Haozhe Zhao Zefan Cai Yuchi Wang Peiyi Wang Tianyu Liu Baobao Chang LM&Ro LLMAG 14 41 0 03 Oct 2023
SmartPlay: A Benchmark for LLMs as Intelligent Agents Yue Wu Xuan Tang Tom Michael Mitchell Yuanzhi Li ELM LLMAG 21 60 0 02 Oct 2023
Reason for Future, Act for Now: A Principled Framework for Autonomous LLM Agents with Provable Sample Efficiency Zhihan Liu Hao Hu Shenao Zhang Hongyi Guo Shuqi Ke Boyi Liu Zhaoran Wang LLMAG LRM 23 33 0 29 Sep 2023
Suspicion-Agent: Playing Imperfect Information Games with Theory of Mind Aware GPT-4 Jiaxian Guo Bo Yang Paul D. Yoo Bill Yuchen Lin Yusuke Iwasawa Yutaka Matsuo LLMAG 13 40 0 29 Sep 2023
AdaRefiner: Refining Decisions of Language Models with Adaptive Feedback Wanpeng Zhang Zongqing Lu LLMAG 13 6 0 29 Sep 2023
Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution Chrisantha Fernando Dylan Banarse Henryk Michalewski Simon Osindero Tim Rocktaschel LLMAG ReLM LRM 12 174 0 28 Sep 2023
Reasoning about the Unseen for Efficient Outdoor Object Navigation Quanting Xie Tianyi Zhang Kedi Xu Matthew Johnson-Roberson Yonatan Bisk LRM 53 9 0 18 Sep 2023
MindAgent: Emergent Gaming Interaction Ran Gong Qiuyuan Huang Xiaojian Ma Hoi Vo Zane Durante ... Zilong Zheng Song-Chun Zhu Demetri Terzopoulos Fei-Fei Li Jianfeng Gao LM&Ro 99 61 0 18 Sep 2023
Find What You Want: Learning Demand-conditioned Object Attribute Space for Demand-driven Navigation Hongchen Wang Andy Guan Hong Chen Xiaoqi Li Mingdong Wu Hao Dong 14 14 0 15 Sep 2023
Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculum Shen Gao Zhengliang Shi Minghang Zhu Bowen Fang Xin Xin Pengjie Ren Zhumin Chen Jun Ma Zhaochun Ren LLMAG CLL 27 35 0 27 Aug 2023