Title
SERL: A Software Suite for Sample-Efficient Robotic Reinforcement LearningIEEE International Conference on Robotics and Automation (ICRA), 2024 Jianlan Luo Zheyuan Hu Charles Xu You Liang Tan Jacob Berg Archit Sharma S. Schaal Chelsea Finn Abhishek Gupta Sergey Levine OffRL OnRL 575 90 0 29 Jan 2024
True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learning Weihao Tan Wentao Zhang Shanqi Liu Longtao Zheng Xinrun Wang Rui Hu OffRL 220 34 0 25 Jan 2024
SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning CapabilitiesComputer Vision and Pattern Recognition (CVPR), 2024 Boyuan Chen Zhuo Xu Sean Kirmani Brian Ichter Danny Driess Pete Florence Dorsa Sadigh Leonidas Guibas Fei Xia LRM ReLM 299 527 0 22 Jan 2024
CivRealm: A Learning and Reasoning Odyssey in Civilization for Decision-Making Agents Siyuan Qi Shuo Chen Yexin Li Xiangyu Kong Junqi Wang ... Zhaowei Zhang Nian Liu Wei Wang Yaodong Yang Song-Chun Zhu AI4CE LRM 360 30 0 19 Jan 2024
Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning Yiqi Wang Wentao Chen Xiaotian Han Xudong Lin Haiteng Zhao Yongfei Liu Bohan Zhai Jianbo Yuan Quanzeng You Hongxia Yang LRM 265 144 0 10 Jan 2024
Language-Conditioned Robotic Manipulation with Fast and Slow ThinkingIEEE International Conference on Robotics and Automation (ICRA), 2024 Minjie Zhu Yichen Zhu Jinming Li Junjie Wen Zhiyuan Xu ... Yaxin Peng Chaomin Shen Dong Liu Feifei Feng Jian Tang LM&Ro 208 24 0 08 Jan 2024
Object-Centric Instruction Augmentation for Robotic Manipulation Junjie Wen Yichen Zhu Minjie Zhu Jinming Li Zhiyuan Xu ... Yaxin Peng Chaomin Shen Dong Liu Feifei Feng Jian Tang LM&Ro 322 22 0 05 Jan 2024
QUAR-VLA: Vision-Language-Action Model for Quadruped Robots Pengxiang Ding Han Zhao Wenxuan Song Zhitao Wang Zhenyu Wei Shangke Lyu Ningxi Yang Donglin Wang 448 56 0 22 Dec 2023
MinePlanner: A Benchmark for Long-Horizon Planning in Large Minecraft Worlds William Hill Ireton Liu Anita De Mello Koch Damion Harvey Nishanth Kumar George Konidaris Steven D. James LM&Ro 214 4 0 20 Dec 2023
Auto MC-Reward: Automated Dense Reward Design with Large Language Models for MinecraftComputer Vision and Pattern Recognition (CVPR), 2023 Hao Li Xue Yang Zhaokai Wang Xizhou Zhu Jie Zhou Yu Qiao Xiaogang Wang Jiaming Song Lewei Lu Jifeng Dai 214 57 0 14 Dec 2023
Vision-Language Models as a Source of Rewards Kate Baumli Satinder Baveja Feryal M. P. Behbahani Harris Chan Gheorghe Comanici ... Yannick Schroecker Stephen Spencer Richie Steigerwald Luyu Wang Lei Zhang VLM LRM 280 50 0 14 Dec 2023
LiFT: Unsupervised Reinforcement Learning with Foundation Models as Teachers Taewook Nam Juyong Lee Jesse Zhang Sung Ju Hwang Joseph J Lim Karl Pertsch OffRL LRM 223 11 0 14 Dec 2023
Foundation Models in Robotics: Applications, Challenges, and the Future Roya Firoozi Johnathan Tucker Stephen Tian Anirudha Majumdar Jiankai Sun ... Brian Ichter Danny Driess Jiajun Wu Cewu Lu Mac Schwager LM&Ro AI4CE LRM VLM 244 269 0 13 Dec 2023
MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active PerceptionComputer Vision and Pattern Recognition (CVPR), 2023 Yiran Qin Enshen Zhou Qichang Liu Zhen-fei Yin Lu Sheng Ruimao Zhang Yu Qiao Jing Shao LM&Ro 285 76 0 12 Dec 2023
DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physics Zhiao Huang Feng Chen Yewen Pu Chun-Tse Lin Hao Su Chuang Gan 252 5 0 11 Dec 2023
Toward Open-ended Embodied Tasks Solving William Wei Wang Dongqi Han Xufang Luo Yifei Shen Charles Ling Boyu Wang Dongsheng Li AI4CE 178 5 0 10 Dec 2023
The Generalization Gap in Offline Reinforcement LearningInternational Conference on Learning Representations (ICLR), 2023 Ishita Mediratta Qingfei You Minqi Jiang Roberta Raileanu OffRL 370 22 0 10 Dec 2023
DARLEI: Deep Accelerated Reinforcement Learning with Evolutionary Intelligence Saeejith Nair M. Shafiee Alexander Wong 115 0 0 08 Dec 2023
FoMo Rewards: Can we cast foundation models as reward functions? Ekdeep Singh Lubana Johann Brehmer P. D. Haan Taco S. Cohen OffRL LRM 207 4 0 06 Dec 2023
LLM as OS, Agents as Apps: Envisioning AIOS, Agents and the AIOS-Agent Ecosystem Yingqiang Ge Yujie Ren Qingfeng Lan Shuyuan Xu Juntao Tan Zelong Li LLMAG 207 38 0 06 Dec 2023
BEDD: The MineRL BASALT Evaluation and Demonstrations Dataset for Training and Benchmarking Agents that Solve Fuzzy TasksNeural Information Processing Systems (NeurIPS), 2023 Stephanie Milani Anssi Kanervisto Karolis Ramanauskas Sander Schulhoff Brandon Houghton Rohin Shah 270 7 0 05 Dec 2023
Creative Agents: Empowering Agents with Imagination for Creative TasksConference on Uncertainty in Artificial Intelligence (UAI), 2023 Chi Zhang Penglin Cai Yuhui Fu Haoqi Yuan Zongqing Lu LM&Ro LLMAG 331 26 0 05 Dec 2023
Quality Diversity in the Amorphous Fortress (QD-AF): Evolving for Complexity in 0-Player Games Sam Earle M. Charity Dipika Rajesh Mayu Wilson Julian Togelius 189 1 0 04 Dec 2023
Visual Encoders for Data-Efficient Imitation Learning in Modern Video Games Lukas Schäfer Logan Jones Anssi Kanervisto Yuhan Cao Tabish Rashid Raluca Georgescu David Bignell Siddhartha Sen Andrea Trevino Gavito Sam Devlin 424 6 0 04 Dec 2023
Planning as In-Painting: A Diffusion-Based Embodied Task Planning Framework for Environments under Uncertainty Cheng-Fu Yang Haoyang Xu Te-Lin Wu Xiaofeng Gao Kai-Wei Chang Feng Gao DiffM 164 11 0 02 Dec 2023
Deciphering Digital Detectives: Understanding LLM Behaviors and Capabilities in Multi-Agent Mystery GamesAnnual Meeting of the Association for Computational Linguistics (ACL), 2023 Dekun Wu Haochen Shi Zhiyuan Sun Bang Liu LLMAG 142 30 0 01 Dec 2023
See and Think: Embodied Agent in Virtual EnvironmentEuropean Conference on Computer Vision (ECCV), 2023 Zhonghan Zhao Wenhao Chai Xuan Wang Li Boyi Shengyu Hao Shidong Cao Tianbo Ye Gaoang Wang LM&Ro LLMAG 356 52 0 26 Nov 2023
Robot Learning in the Era of Foundation Models: A Survey Xuan Xiao Jiahang Liu Zhipeng Wang Yanmin Zhou Yong Qi Qian Cheng Bin He Shuo Jiang AI4CE LM&Ro 400 47 0 24 Nov 2023
An Embodied Generalist Agent in 3D World Jiangyong Huang Silong Yong Xiaojian Ma Xiongkun Linghu Puhao Li Yan Wang Qing Li Song-Chun Zhu Baoxiong Jia Siyuan Huang LM&Ro 271 287 0 18 Nov 2023
Offline Data Enhanced On-Policy Policy Gradient with Provable GuaranteesInternational Conference on Learning Representations (ICLR), 2023 Yifei Zhou Ayush Sekhari Yuda Song Wen Sun OffRL OnRL 200 8 0 14 Nov 2023
JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023 Zihao Wang Shaofei Cai Hoang Trung-Dung Yonggang Jin Jinbing Hou ... Zhaofeng He Zilong Zheng Yaodong Yang Xiaojian Ma Yitao Liang LLMAG LM&Ro 345 151 0 10 Nov 2023
ADaPT: As-Needed Decomposition and Planning with Language Models Archiki Prasad Alexander Koller Mareike Hartmann Peter Clark Ashish Sabharwal Mohit Bansal Tushar Khot LM&Ro 230 140 0 08 Nov 2023
Active Reasoning in an Open-World EnvironmentNeural Information Processing Systems (NeurIPS), 2023 Manjie Xu Guangyuan Jiang Weihan Liang Fangqiu Yi Yixin Zhu LLMAG LRM 226 13 0 03 Nov 2023
A Definition of Open-Ended Learning Problems for Goal-Conditioned Agents Olivier Sigaud Gianluca Baldassarre Cédric Colas Stéphane Doncieux Richard J. Duro Pierre-Yves Oudeyer Nicolas Perrin-Gilbert V. Santucci AI4CE 481 18 0 01 Nov 2023
Graph Agent: Explicit Reasoning Agent for Graphs Qinyong Wang Zhenxiang Gao Rong Xu AI4CE 120 10 0 25 Oct 2023
Steve-Eye: Equipping LLM-based Embodied Agents with Visual Perception in Open Worlds Sipeng Zheng Jiazheng Liu Yicheng Feng Zongqing Lu 247 45 0 20 Oct 2023
Eureka: Human-Level Reward Design via Coding Large Language Models Yecheng Jason Ma William Liang Guanzhi Wang De-An Huang Osbert Bastani Dinesh Jayaraman Yuke Zhu Linxi Fan A. Anandkumar 264 460 0 19 Oct 2023
Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning Juan Rocamonde Victoriano Montesinos Elvis Nava Ethan Perez David Lindner VLM 313 130 0 19 Oct 2023
AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents Jake Grigsby Linxi Fan Yuke Zhu OffRL LM&Ro 300 44 0 15 Oct 2023
LLaMA Rider: Spurring Large Language Models to Explore the Open World Yicheng Feng Yuxuan Wang Jiazheng Liu Sipeng Zheng Zongqing Lu LLMAG LRM 218 23 0 13 Oct 2023
Octopus: Embodied Vision-Language Programmer from Environmental FeedbackEuropean Conference on Computer Vision (ECCV), 2023 Jingkang Yang Yuhao Dong Shuai Liu Yue Liu Ziyue Wang ... Haoran Tan Jiamu Kang Yuanhan Zhang Kaiyang Zhou Ziwei Liu LM&Ro 265 78 0 12 Oct 2023
Cross-Episodic Curriculum for Transformer AgentsNeural Information Processing Systems (NeurIPS), 2023 Lucy Xiaoyang Shi Yunfan Jiang Jake Grigsby Linxi "Jim" Fan Yuke Zhu 135 9 0 12 Oct 2023
LightZero: A Unified Benchmark for Monte Carlo Tree Search in General Sequential Decision ScenariosNeural Information Processing Systems (NeurIPS), 2023 Yazhe Niu Yuan Pu Zhenjie Yang Xueyan Li Tong Zhou Jiyuan Ren Shuai Hu Jiaming Song Yu Liu 329 20 0 12 Oct 2023
GROOT: Learning to Follow Instructions by Watching Gameplay VideosInternational Conference on Learning Representations (ICLR), 2023 Shaofei Cai Bowei Zhang Zihao Wang Xiaojian Ma Hoang Trung-Dung Yitao Liang 300 37 0 12 Oct 2023
PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm Haoyi Zhu Honghui Yang Xiaoyang Wu Di Huang Sha Zhang ... Hengshuang Zhao Chunhua Shen Yu Qiao Tong He Wanli Ouyang SSL 517 54 0 12 Oct 2023
RoboCLIP: One Demonstration is Enough to Learn Robot PoliciesNeural Information Processing Systems (NeurIPS), 2023 Sumedh Anand Sontakke Jesse Zhang Sébastien M. R. Arnold Karl Pertsch Erdem Biyik Dorsa Sadigh Chelsea Finn Laurent Itti OffRL 194 112 0 11 Oct 2023
Lemur: Harmonizing Natural Language and Code for Language AgentsInternational Conference on Learning Representations (ICLR), 2023 Yiheng Xu Hongjin Su Chen Xing Boyu Mi Qian Liu ... Siheng Zhao Lingpeng Kong Bailin Wang Caiming Xiong Tao Yu 183 86 0 10 Oct 2023
GEAR: A GPU-Centric Experience Replay System for Large Reinforcement Learning ModelsInternational Conference on Machine Learning (ICML), 2023 Hanjing Wang Man-Kit Sit Cong He Ying Wen Weinan Zhang Jun Wang Yaodong Yang Kai Zou OffRL VLM 177 4 0 08 Oct 2023
Language Agent Tree Search Unifies Reasoning Acting and Planning in Language ModelsInternational Conference on Machine Learning (ICML), 2023 Xiaoxiao Sun Yang Yang Michal Shlapentokh-Rothman Haohan Wang Yu-Xiong Wang LRM AI4CE LM&Ro LLMAG 383 310 0 06 Oct 2023
Towards End-to-End Embodied Decision Making via Multi-modal Large Language Model: Explorations with GPT4-Vision and Beyond Liang Chen Yichi Zhang Shuhuai Ren Haozhe Zhao Zefan Cai Yuchi Wang Peiyi Wang Tianyu Liu Baobao Chang LM&Ro LLMAG 378 55 0 03 Oct 2023