Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections

17 November 2023

Lihan Zha

Yuchen Cui

Li-Heng Lin

Minae Kwon

Montse Gonzalez Arenas

Andy Zeng

Fei Xia

Dorsa Sadigh

ArXiv PDF HTML

Papers citing "Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections"

32 / 32 papers shown

Title
LangPert: Detecting and Handling Task-level Perturbations for Robust Object Rearrangement Xu Yin Min-Sung Yoon Yuchi Huo Kang Zhang Sung-eui Yoon 21 0 0 14 Apr 2025
RAIDER: Tool-Equipped Large Language Model Agent for Robotic Action Issue Detection, Explanation and Recovery Silvia Izquierdo-Badiola Carlos Rizzo Guillem Alenyà LLMAG LM&Ro 69 0 0 22 Mar 2025
KUDA: Keypoints to Unify Dynamics Learning and Visual Prompting for Open-Vocabulary Robotic Manipulation Zixian Liu Mingtong Zhang Yunzhu Li 49 0 0 13 Mar 2025
SRSA: Skill Retrieval and Adaptation for Robotic Assembly Tasks Yijie Guo Bingjie Tang Iretiayo Akinola Dieter Fox Abhishek Gupta Yashraj S. Narang 41 0 0 06 Mar 2025
A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards Shivansh Patel Xinchen Yin Wenlong Huang Shubham Garg H. Nayyeri Li Fei-Fei Svetlana Lazebnik Y. Li 83 0 0 12 Feb 2025
Vocal Sandbox: Continual Learning and Adaptation for Situated Human-Robot Collaboration J. Grannen Siddharth Karamcheti Suvir Mirchandani Percy Liang Dorsa Sadigh 33 0 0 04 Nov 2024
Automating Robot Failure Recovery Using Vision-Language Models With Optimized Prompts Hongyi Chen Yunchao Yao Ruixuan Liu Changliu Liu Jeffrey Ichnowski 22 5 0 06 Sep 2024
FlowRetrieval: Flow-Guided Data Retrieval for Few-Shot Imitation Learning Li-Heng Lin Yuchen Cui Amber Xie Tianyu Hua Dorsa Sadigh 17 7 0 29 Aug 2024
Points2Plans: From Point Clouds to Long-Horizon Plans with Composable Relational Dynamics Yixuan Huang Christopher Agia Jimmy Wu Tucker Hermans Jeannette Bohg 3DPC 38 1 0 27 Aug 2024
Social Learning through Interactions with Other Agents: A Survey Dylan Hillier Cheston Tan Jing Jiang 37 0 0 31 Jul 2024
Words2Contact: Identifying Support Contacts from Verbal Instructions Using Foundation Models Dionis Totsila Quentin Rouxel Jean-Baptiste Mouret S. Ivaldi 33 1 0 19 Jul 2024
FLAIR: Feeding via Long-horizon AcquIsition of Realistic dishes Rajat Kumar Jenamani Priya Sundaresan Maram Sakr T. Bhattacharjee Dorsa Sadigh 20 9 0 10 Jul 2024
Safe MPC Alignment with Human Directional Feedback Zhixian Xie Wenlong Zhang Yi Ren Zhaoran Wang George J. Pappas Wanxin Jin 34 0 0 05 Jul 2024
Commonsense Reasoning for Legged Robot Adaptation with Vision-Language Models Annie S. Chen Alec M. Lessing Andy Tang Govind Chada Laura Smith Sergey Levine Chelsea Finn LM&Ro LRM 26 8 0 02 Jul 2024
VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought Gabriel H. Sarch Lawrence Jang Michael J. Tarr William W. Cohen Kenneth Marino Katerina Fragkiadaki LLMAG 31 0 0 20 Jun 2024
AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulation Chuyan Xiong Chengyu Shen Xiaoqi Li Kaichen Zhou Jiaming Liu Ruiping Wang Hao Dong LRM 30 10 0 17 Jun 2024
Action Contextualization: Adaptive Task Planning and Action Tuning using Large Language Models Sthithpragya Gupta Kunpeng Yao Loic Niederhauser A. Billard 21 1 0 19 Apr 2024
A Survey on Human-AI Teaming with Large Pre-Trained Models Vanshika Vats Marzia Binta Nizam Minghao Liu Ziyuan Wang Richard Ho ... Celeste Shen Rachel Shen Nafisa Hussain Kesav Ravichandran James Davis LM&MA 31 8 0 07 Mar 2024
Reliable, Adaptable, and Attributable Language Models with Retrieval Akari Asai Zexuan Zhong Danqi Chen Pang Wei Koh Luke Zettlemoyer Hanna Hajishirzi Wen-tau Yih KELM RALM 36 53 0 05 Mar 2024
RT-H: Action Hierarchies Using Language Suneel Belkhale Tianli Ding Ted Xiao P. Sermanet Quon Vuong Jonathan Tompson Yevgen Chebotar Debidatta Dwibedi Dorsa Sadigh LM&Ro 21 73 0 04 Mar 2024
Learning to Learn Faster from Human Feedback with Language Model Predictive Control Jacky Liang Fei Xia Wenhao Yu Andy Zeng Montse Gonzalez Arenas ... N. Heess Kanishka Rao Nik Stewart Jie Tan Carolina Parada LM&Ro 39 28 0 18 Feb 2024
"Task Success" is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviors L. Guan Yifan Zhou Denis Liu Yantian Zha H. B. Amor Subbarao Kambhampati LM&Ro 28 16 0 06 Feb 2024
RePLan: Robotic Replanning with Perception and Language Models Marta Skreta Zihan Zhou Jia Lin Yuan Kourosh Darvish Alán Aspuru-Guzik Animesh Garg LM&Ro LRM 27 26 0 08 Jan 2024
ExTraCT -- Explainable Trajectory Corrections from language inputs using Textual description of features J-Anne Yow N. P. Garg Manoj Ramanathan Wei Tech Ang 18 5 0 08 Jan 2024
Human Demonstrations are Generalizable Knowledge for Robots Te Cui Guangyan Chen Tianxing Zhou Zicai Peng Mengxiao Hu Haoyang Lu Haizhou Li Meiling Wang Yi Yang Yufeng Yue LM&Ro 14 6 0 05 Dec 2023
Incremental Learning of Humanoid Robot Behavior from Natural Interaction and Large Language Models Leonard Barmann Rainer Kartmann Fabian Peller-Konrad Jan Niehues Alexander H. Waibel Tamim Asfour LM&Ro 15 24 0 08 Sep 2023
Transformers generalize differently from information stored in context vs in weights Stephanie C. Y. Chan Ishita Dasgupta Junkyung Kim D. Kumaran Andrew Kyle Lampinen Felix Hill 95 45 0 11 Oct 2022
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 208 2,413 0 06 Oct 2022
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models Ishika Singh Valts Blukis Arsalan Mousavian Ankit Goyal Danfei Xu Jonathan Tremblay D. Fox Jesse Thomason Animesh Garg LM&Ro LLMAG 109 616 0 22 Sep 2022
Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation Mohit Shridhar Lucas Manuelli D. Fox LM&Ro 141 449 0 12 Sep 2022
Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners Zhenhailong Wang Manling Li Ruochen Xu Luowei Zhou Jie Lei ... Chenguang Zhu Derek Hoiem Shih-Fu Chang Mohit Bansal Heng Ji MLLM VLM 162 134 0 22 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022