Title
Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyond Liang Chen Yichi Zhang Shuhuai Ren Haozhe Zhao Zefan Cai Yuchi Wang Peiyi Wang Tianyu Liu Baobao Chang LM&Ro LLMAG 394 55 0 03 Oct 2023
SmartPlay: A Benchmark for LLMs as Intelligent AgentsInternational Conference on Learning Representations (ICLR), 2023 Yue Wu Xuan Tang Tom Michael Mitchell Yuanzhi Li ELM LLMAG 521 105 0 02 Oct 2023
Berkeley Open Extended Reality Recordings 2023 (BOXRR-23): 4.7 Million Motion Capture Recordings from 105,852 Extended Reality Device UsersIEEE Transactions on Visualization and Computer Graphics (TVCG), 2023 V. Nair Wenbo Guo Rui Wang J. F. O'Brien Louis B. Rosenberg Dawn Song 250 9 0 30 Sep 2023
HarmonyDream: Task Harmonization Inside World ModelsInternational Conference on Machine Learning (ICML), 2023 Haoyu Ma Jialong Wu Ningya Feng Chenjun Xiao Dong Li Jianye Hao Jianmin Wang Mingsheng Long 148 15 0 30 Sep 2023
Motif: Intrinsic Motivation from Artificial Intelligence FeedbackInternational Conference on Learning Representations (ICLR), 2023 Martin Klissarov P. DÓro Shagun Sodhani Roberta Raileanu Pierre-Luc Bacon Pascal Vincent Amy Zhang Mikael Henaff LRM LLMAG 239 88 0 29 Sep 2023
Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents Marco Pleines Matthias Pallasch Frank Zimmer Mike Preuss OffRL 264 9 0 29 Sep 2023
Unsupervised 3D Perception with 2D Vision-Language Distillation for Autonomous DrivingIEEE International Conference on Computer Vision (ICCV), 2023 Mahyar Najibi Jingwei Ji Yin Zhou C. Qi Xinchen Yan Scott Ettinger Drago Anguelov 213 45 0 25 Sep 2023
An In-depth Survey of Large Language Model-based Artificial Intelligence Agents Pengyu Zhao Zijian Jin Ning Cheng LLMAG 185 36 0 23 Sep 2023
Text2Reward: Reward Shaping with Language Models for Reinforcement LearningInternational Conference on Learning Representations (ICLR), 2023 Tianbao Xie Siheng Zhao Chen Henry Wu Yitao Liu Qian Luo Victor Zhong Yanchao Yang Tao Yu LM&Ro 371 97 0 20 Sep 2023
Guide Your Agent with Adaptive Multimodal RewardsNeural Information Processing Systems (NeurIPS), 2023 Changyeon Kim Younggyo Seo Hao Liu Lisa Lee Jinwoo Shin Honglak Lee Kimin Lee 305 10 0 19 Sep 2023
A Data Source for Reasoning Embodied AgentsAAAI Conference on Artificial Intelligence (AAAI), 2023 Jack Lanchantin Sainbayar Sukhbaatar Gabriel Synnaeve Yuxuan Sun Kavya Srinet Arthur Szlam LM&Ro LRM 184 10 0 14 Sep 2023
Language Reward Modulation for Pretraining Reinforcement Learning Ademi Adeniji Amber Xie Carmelo Sferrazza Younggyo Seo Stephen James Pieter Abbeel 189 39 0 23 Aug 2023
ProAgent: Building Proactive Cooperative Agents with Large Language ModelsAAAI Conference on Artificial Intelligence (AAAI), 2023 Ceyao Zhang Kaijie Yang Siyi Hu Zihao Wang Guanghe Li ... Xiaojun Chang Junge Zhang Feng Yin Yitao Liang Yaodong Yang LM&Ro LLMAG 319 112 0 22 Aug 2023
AgentBench: Evaluating LLMs as AgentsInternational Conference on Learning Representations (ICLR), 2023 Xiao-Yang Liu Hao Yu Hanchen Zhang Yifan Xu Xuanyu Lei ... Yu-Chuan Su Huan Sun Shiyu Huang Yuxiao Dong Jie Tang ELM LLMAG 512 482 0 07 Aug 2023
Pre-Trained Large Language Models for Industrial Control Lei Song Wei Shen Li Zhao Jiang Bian LM&Ro AI4CE 153 22 0 06 Aug 2023
Learning to Model the World with LanguageInternational Conference on Machine Learning (ICML), 2023 Jessy Lin Yuqing Du Olivia Watkins Danijar Hafner Pieter Abbeel Dan Klein Anca Dragan LM&Ro SyDa 267 71 0 31 Jul 2023
WebArena: A Realistic Web Environment for Building Autonomous AgentsInternational Conference on Learning Representations (ICLR), 2023 Shuyan Zhou Frank F. Xu Hao Zhu Xuhui Zhou Robert Lo ... Tianyue Ou Yonatan Bisk Daniel Fried Uri Alon Graham Neubig LLMAG 650 807 0 25 Jul 2023
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming-Hsuan Yang Fahad Shahbaz Khan VLM 416 151 0 25 Jul 2023
Towards A Unified Agent with Foundation Models Norman Di Palo Arunkumar Byravan Leonard Hasenclever Markus Wulfmeier N. Heess Martin Riedmiller LM&Ro LLMAG OffRL 237 70 0 18 Jul 2023
Is Imitation All You Need? Generalized Decision-Making with Dual-Phase TrainingIEEE International Conference on Computer Vision (ICCV), 2023 Yao Wei Yanchao Sun Ruijie Zheng Sai H. Vemprala Rogerio Bonatti Shuhang Chen Ratnesh Madaan Zhongjie Ba Ashish Kapoor Shuang Ma OffRL 254 20 0 16 Jul 2023
RL-ViGen: A Reinforcement Learning Benchmark for Visual GeneralizationNeural Information Processing Systems (NeurIPS), 2023 Zhecheng Yuan Sizhe Yang Pu Hua C. Chang Kaizhe Hu Huazhe Xu OOD OffRL 277 34 0 15 Jul 2023
Brain in a Vat: On Missing Pieces Towards Artificial General Intelligence in Large Language Models Yuxi Ma Fangqiu Yi Song-Chun Zhu ELM ALM 165 8 0 07 Jul 2023
Goal Representations for Instruction Following: A Semi-Supervised Language Interface to ControlConference on Robot Learning (CoRL), 2023 Vivek Myers Andre Wang He Kuan Fang Homer Walke Philippe Hansen-Estruch Ching-An Cheng Mihai Jalobeanu Andrey Kolobov Anca Dragan Sergey Levine LM&Ro 392 38 0 30 Jun 2023
Large Sequence Models for Sequential Decision-Making: A Survey Muning Wen Runji Lin Hanjing Wang Yaodong Yang Ying Wen Kai Zou Jun Wang Haifeng Zhang Weinan Zhang LM&Ro LRM 260 47 0 24 Jun 2023
Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented TasksNeural Information Processing Systems (NeurIPS), 2023 Maxime Chevalier-Boisvert Bolun Dai Mark Towers Rodrigo de Lazcano Lucas Willems Salem Lahlou Suman Pal Pablo Samuel Castro Jordan Terry VGen 342 303 0 24 Jun 2023
SPRINT: Scalable Policy Pre-Training via Language Instruction RelabelingIEEE International Conference on Robotics and Automation (ICRA), 2023 Jesse Zhang Karl Pertsch Jiahui Zhang Joseph J. Lim LM&Ro 431 22 0 20 Jun 2023
Semantic HELM: A Human-Readable Memory for Reinforcement LearningNeural Information Processing Systems (NeurIPS), 2023 Fabian Paischer Thomas Adler M. Hofmarcher Sepp Hochreiter 287 17 0 15 Jun 2023
LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023 Peng Xu Wenqi Shao Kaipeng Zhang Shiyang Feng Shuo Liu Meng Lei Fanqing Meng Siyuan Huang Yu Qiao Ping Luo ELM MLLM 293 228 0 15 Jun 2023
ChessGPT: Bridging Policy Learning and Language ModelingNeural Information Processing Systems (NeurIPS), 2023 Xidong Feng Yicheng Luo Ziyan Wang Hongrui Tang Mengyue Yang Youssef Attia El Hili D. Mguni Yali Du Jun Wang 258 67 0 15 Jun 2023
Behavioral Cloning via Search in Embedded Demonstration Dataset Federico Malato Florian Leopold Ville Hautamaki Andrew Melnik OffRL 153 3 0 15 Jun 2023
Toward Grounded Commonsense ReasoningIEEE International Conference on Robotics and Automation (ICRA), 2023 Minae Kwon Hengyuan Hu Vivek Myers Siddharth Karamcheti Anca Dragan Dorsa Sadigh LM&Ro ReLM LRM 234 14 0 14 Jun 2023
Language to Rewards for Robotic Skill SynthesisConference on Robot Learning (CoRL), 2023 Wenhao Yu Nimrod Gileadi Chuyuan Fu Sean Kirmani Kuang-Huei Lee ... N. Heess Dorsa Sadigh Jie Tan Yuval Tassa F. Xia LM&Ro 244 352 0 14 Jun 2023
Value function estimation using conditional diffusion models for control Bogdan Mazoure Walter A. Talbott Miguel Angel Bautista R. Devon Hjelm Alexander Toshev J. Susskind DiffM 139 8 0 09 Jun 2023
Embodied Executable Policy Learning with Language-based Scene SummarizationNorth American Chapter of the Association for Computational Linguistics (NAACL), 2023 Jielin Qiu Mengdi Xu William Jongwon Han Seungwhan Moon Ding Zhao LM&Ro 147 9 0 09 Jun 2023
OMNI: Open-endedness via Models of human Notions of Interestingness Jenny Zhang Joel Lehman Kenneth O. Stanley Jeff Clune LRM 419 51 0 02 Jun 2023
Heterogeneous Knowledge for Augmented Modular Reinforcement Learning Lorenz Wolf Mirco Musolesi OffRL 210 0 0 01 Jun 2023
Active Vision Reinforcement Learning under Limited Visual ObservabilityNeural Information Processing Systems (NeurIPS), 2023 Jinghuan Shang Michael S. Ryoo 281 0 0 01 Jun 2023
LIV: Language-Image Representations and Rewards for Robotic ControlInternational Conference on Machine Learning (ICML), 2023 Yecheng Jason Ma William Liang Vaidehi Som Vikash Kumar Amy Zhang Osbert Bastani Dinesh Jayaraman LM&Ro 218 177 0 01 Jun 2023
STEVE-1: A Generative Model for Text-to-Behavior in MinecraftNeural Information Processing Systems (NeurIPS), 2023 Shalev Lifshitz Keiran Paster Harris Chan Jimmy Ba Sheila A. McIlraith LM&Ro 320 96 0 01 Jun 2023
Thought Cloning: Learning to Think while Acting by Imitating Human ThinkingNeural Information Processing Systems (NeurIPS), 2023 Shengran Hu Jeff Clune LM&Ro OffRL LRM AI4CE 440 37 0 01 Jun 2023
Bigger, Better, Faster: Human-level Atari with human-level efficiencyInternational Conference on Machine Learning (ICML), 2023 Max Schwarzer J. Obando-Ceron Rameswar Panda Marc G. Bellemare Rishabh Agarwal Pablo Samuel Castro OffRL 342 142 0 30 May 2023
ChatGPT-powered Conversational Drug Editing Using Retrieval and Domain Feedback Shengchao Liu Zhenghao Hu Yijin Yang Chengpeng Wang Ling Liu Hongyu Guo Chaowei Xiao LM&MA KELM AI4MH 231 42 0 29 May 2023
Ghost in the Minecraft: Generally Capable Agents for Open-World Environments via Large Language Models with Text-based Knowledge and Memory Xizhou Zhu Yuntao Chen Hao Tian Chenxin Tao Weijie Su ... Lewei Lu Xiaogang Wang Yu Qiao Zhaoxiang Zhang Jifeng Dai LLMAG LM&Ro 259 285 0 25 May 2023
Voyager: An Open-Ended Embodied Agent with Large Language Models Guanzhi Wang Yuqi Xie Yunfan Jiang Ajay Mandlekar Chaowei Xiao Yuke Zhu Linxi Fan Anima Anandkumar LM&Ro SyDa 453 1,167 0 25 May 2023
Augmenting Autotelic Agents with Large Language Models Cédric Colas Laetitia Teodorescu Pierre-Yves Oudeyer Xingdi Yuan Marc-Alexandre Côté LLMAG LM&Ro 232 36 0 21 May 2023
Multimodal Web Navigation with Instruction-Finetuned Foundation ModelsInternational Conference on Learning Representations (ICLR), 2023 Hiroki Furuta Kuang-Huei Lee Ofir Nachum Yutaka Matsuo Aleksandra Faust S. Gu Izzeddin Gur LM&Ro 388 140 0 19 May 2023
Semantically Aligned Task Decomposition in Multi-Agent Reinforcement Learning Wenhao Li Dan Qiao Baoxiang Wang Xiangfeng Wang Bo Jin H. Zha 180 11 0 18 May 2023
Learning Video-Conditioned Policies for Unseen Manipulation TasksIEEE International Conference on Robotics and Automation (ICRA), 2023 Elliot Chane-Sane Cordelia Schmid Ivan Laptev 196 25 0 10 May 2023
Games for Artificial Intelligence Research: A Review and PerspectivesIEEE Transactions on Artificial Intelligence (IEEE TAI), 2023 Chengpeng Hu Yunlong Zhao Ziqi Wang Haocheng Du Jialin Liu AI4CE 226 20 0 26 Apr 2023
Contrastive Language, Action, and State Pre-training for Robot Learning Krishan Rana Andrew Melnik Niko Sünderhauf 130 15 0 21 Apr 2023