Robot Learning in the Era of Foundation Models: A Survey

24 November 2023

Papers citing "Robot Learning in the Era of Foundation Models: A Survey"

48 / 48 papers shown

Title
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey Y. Wang Shengqiong Wu Y. Zhang William Yang Wang Ziwei Liu Jiebo Luo Hao Fei LRM 74 7 0 16 Mar 2025
Is Your Imitation Learning Policy Better than Mine? Policy Comparison with Near-Optimal Stopping David Snyder Asher Hancock Apurva Badithela Emma Dixon Patrick "Tree" Miller Rares Ambrus Anirudha Majumdar Masha Itkina Haruki Nishimura OffRL 75 1 0 14 Mar 2025
dARt Vinci: Egocentric Data Collection for Surgical Robot Learning at Scale Yihao Liu Yu-Chun Ku Jiaming Zhang H. Ding Peter Kazanzides Mehran Armand 68 0 0 07 Mar 2025
Kaiwu: A Multimodal Manipulation Dataset and Framework for Robot Learning and Human-Robot Interaction Shuo Jiang Haonan Li Ruochen Ren Yanmin Zhou Zhipeng Wang Bin He 29 0 0 07 Mar 2025
Multi-Scenario Reasoning: Unlocking Cognitive Autonomy in Humanoid Robots for Multimodal Understanding Libo Wang LRM 29 0 0 08 Jan 2025
D-RMGPT: Robot-assisted collaborative tasks driven by large multimodal models Matteo Forlini Mihail Babcinschi Giacomo Palmieri Pedro Neto 24 1 0 21 Aug 2024
Foundation Models for Autonomous Robots in Unstructured Environments Hossein Naderi Alireza Shojaei Lifu Huang LM&Ro 34 0 0 19 Jul 2024
Object Segmentation from Open-Vocabulary Manipulation Instructions Based on Optimal Transport Polygon Matching with Multimodal Foundation Models Takayuki Nishimura Katsuyuki Kuyo Motonari Kambara Komei Sugiura DiffM 19 0 0 01 Jul 2024
CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Mastery Xiaoshuai Song Muxi Diao Guanting Dong Zhengyang Wang Yujia Fu ... Yejie Wang Zhuoma Gongque Jianing Yu Qiuna Tan Weiran Xu ELM 39 10 0 12 Jun 2024
Learning-based legged locomotion; state of the art and future perspectives Sehoon Ha Joonho Lee M. van de Panne Zhaoming Xie Wenhao Yu Majid Khadiv 36 15 0 03 Jun 2024
What Foundation Models can Bring for Robot Learning in Manipulation : A Survey Dingzhe Li Yixiang Jin A. Yong Hongze Yu Jun Shi Xiaoshuai Hao Peng Hao Huaping Liu Fuchun Sun Bin Fang AI4CE LM&Ro 62 2 0 28 Apr 2024
BlenderAlchemy: Editing 3D Graphics with Vision-Language Models Ian Huang Guandao Yang Leonidas J. Guibas 23 3 0 26 Apr 2024
A Roadmap Towards Automated and Regulated Robotic Systems Yihao Liu Mehran Armand 28 0 0 21 Mar 2024
Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulation Chrisantus Eze Christopher Crick SSL 68 11 0 11 Feb 2024
The Essential Role of Causality in Foundation World Models for Embodied AI Tarun Gupta Wenbo Gong Chao Ma Nick Pawlowski Agrin Hilmkil ... Jianfeng Gao Stefan Bauer Danica Kragic Bernhard Schölkopf Cheng Zhang 22 15 0 06 Feb 2024
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis Yafei Hu Quanting Xie Vidhi Jain Jonathan M Francis Jay Patrikar ... Xiaolong Wang Sebastian A. Scherer Z. Kira Fei Xia Yonatan Bisk LM&Ro AI4CE 21 54 0 14 Dec 2023
Foundation Models in Robotics: Applications, Challenges, and the Future Roya Firoozi Johnathan Tucker Stephen Tian Anirudha Majumdar Jiankai Sun ... Brian Ichter Danny Driess Jiajun Wu Cewu Lu Mac Schwager LM&Ro AI4CE LRM VLM 24 136 0 13 Dec 2023
MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations Ajay Mandlekar Soroush Nasiriany Bowen Wen Iretiayo Akinola Yashraj S. Narang Linxi Fan Yuke Zhu Dieter Fox LM&Ro 69 96 0 26 Oct 2023
Investigating the Catastrophic Forgetting in Multimodal Large Language Models Yuexiang Zhai Shengbang Tong Xiao Li Mu Cai Qing Qu Yong Jae Lee Y. Ma VLM MLLM CLL 66 75 0 19 Sep 2023
Multimodal Foundation Models: From Specialists to General-Purpose Assistants Chunyuan Li Zhe Gan Zhengyuan Yang Jianwei Yang Linjie Li Lijuan Wang Jianfeng Gao MLLM 107 221 0 18 Sep 2023
Real-time Vision-based Navigation for a Robot in an Indoor Environment Sagar Manglani 16 3 0 02 Jul 2023
CorNav: Autonomous Agent with Self-Corrected Planning for Zero-Shot Vision-and-Language Navigation Xiwen Liang Liang Ma Shanshan Guo Jianhua Han Hang Xu Shikui Ma Xiaodan Liang LM&Ro LLMAG 71 4 0 17 Jun 2023
Scaling Speech Technology to 1,000+ Languages Vineel Pratap Andros Tjandra Bowen Shi Paden Tomasello Arun Babu ... Yossi Adi Xiaohui Zhang Wei-Ning Hsu Alexis Conneau Michael Auli VLM 73 297 0 22 May 2023
Programmatically Grounded, Compositionally Generalizable Robotic Manipulation Renhao Wang Jiayuan Mao Joy Hsu Hang Zhao Jiajun Wu Yang Gao LM&Ro 104 30 0 26 Apr 2023
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond Jingfeng Yang Hongye Jin Ruixiang Tang Xiaotian Han Qizhang Feng Haoming Jiang Bing Yin Xia Hu LM&MA 123 593 0 26 Apr 2023
Emergent autonomous scientific research capabilities of large language models Daniil A. Boiko R. MacKnight Gabe Gomes ELM LM&Ro AI4CE LLMAG 101 115 0 11 Apr 2023
Toward Human-Like Social Robot Navigation: A Large-Scale, Multi-Modal, Social Human Navigation Dataset Duc M. Nguyen Mohammad Nazeri Amirreza Payandeh A. Datar Xuesu Xiao 44 29 0 27 Mar 2023
On the Utility of Koopman Operator Theory in Learning Dexterous Manipulation Skills Yunhai Han Mandy Xie Ye Zhao H. Ravichandar 16 17 0 23 Mar 2023
Rotating without Seeing: Towards In-hand Dexterity through Touch Zhao-Heng Yin Binghao Huang Yuzhe Qin Qifeng Chen Xiaolong Wang 96 53 0 20 Mar 2023
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing Xiaozhe Ren Pingyi Zhou Xinfan Meng Xinjing Huang Yadao Wang ... Jiansheng Wei Xin Jiang Teng Su Qun Liu Jun Yao ALM MoE 53 59 0 20 Mar 2023
Foundation Models for Decision Making: Problems, Methods, and Opportunities Sherry Yang Ofir Nachum Yilun Du Jason W. Wei Pieter Abbeel Dale Schuurmans LM&Ro OffRL LRM AI4CE 87 148 0 07 Mar 2023
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages Yu Zhang Wei Han James Qin Yongqiang Wang Ankur Bapna ... Pedro J. Moreno Chung-Cheng Chiu J. Schalkwyk Franccoise Beaufays Yonghui Wu VLM 77 249 0 02 Mar 2023
Towards Versatile Embodied Navigation H. Wang Wei Liang Luc Van Gool Wenguan Wang LM&Ro 32 20 0 30 Oct 2022
Visual Language Maps for Robot Navigation Chen Huang Oier Mees Andy Zeng Wolfram Burgard LM&Ro 140 337 0 11 Oct 2022
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 208 2,413 0 06 Oct 2022
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 240 1,070 0 05 Oct 2022
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models Ishika Singh Valts Blukis Arsalan Mousavian Ankit Goyal Danfei Xu Jonathan Tremblay D. Fox Jesse Thomason Animesh Garg LM&Ro LLMAG 109 616 0 22 Sep 2022
LM-Nav: Robotic Navigation with Large Pre-Trained Models of Language, Vision, and Action Dhruv Shah B. Osinski Brian Ichter Sergey Levine LM&Ro 136 430 0 10 Jul 2022
RFUniverse: A Multiphysics Simulation Platform for Embodied AI Haoyuan Fu Wenqiang Xu Ruolin Ye Han Xue Zhenjun Yu Tutian Tang Yutong Li Wenxin Du Jieyi Zhang Cewu Lu AI4CE 21 11 0 01 Feb 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Learning to Act with Affordance-Aware Multimodal Neural SLAM Zhiwei Jia Kaixiang Lin Yizhou Zhao Qiaozi Gao Govind Thattai Gaurav Sukhatme LM&Ro 10 15 0 24 Jan 2022
TEACh: Task-driven Embodied Agents that Chat Aishwarya Padmakumar Jesse Thomason Ayush Shrivastava P. Lange Anjali Narayan-Chen Spandana Gella Robinson Piramithu Gökhan Tür Dilek Z. Hakkani-Tür LM&Ro 142 179 0 01 Oct 2021
Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasets F. Ebert Yanlai Yang Karl Schmeckpeper Bernadette Bucher G. Georgakis Kostas Daniilidis Chelsea Finn Sergey Levine 147 212 0 27 Sep 2021
BEHAVIOR: Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments S. Srivastava Chengshu Li Michael Lingelbach Roberto Martín-Martín Fei Xia ... C. Karen Liu Silvio Savarese H. Gweon Jiajun Wu Li Fei-Fei LM&Ro 127 117 0 06 Aug 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,735 0 24 Feb 2021
PackIt: A Virtual Environment for Geometric Planning Ankit Goyal Jia Deng 20 11 0 21 Jul 2020
Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning Khanh Nguyen Hal Daumé LM&Ro EgoV 167 148 0 04 Sep 2019
Speaker-Follower Models for Vision-and-Language Navigation Daniel Fried Ronghang Hu Volkan Cirik Anna Rohrbach Jacob Andreas Louis-Philippe Morency Taylor Berg-Kirkpatrick Kate Saenko Dan Klein Trevor Darrell LM&Ro LRM 237 444 0 07 Jun 2018