Language-Driven Representation Learning for Robotics

24 February 2023

Dorsa Sadigh

Papers citing "Language-Driven Representation Learning for Robotics"

50 / 115 papers shown

Title
CIVIL: Causal and Intuitive Visual Imitation Learning Yinlong Dai Robert Ramirez Sanchez Ryan Jeronimus Shahabedin Sagheb Cara M. Nunez Heramb Nemlekar Dylan P. Losey 61 0 0 24 Apr 2025
RoboAct-CLIP: Video-Driven Pre-training of Atomic Action Understanding for Robotics Zhiyuan Zhang Yuxin He Yong Sun Junyu Shi Lijiang Liu Qiang Nie VLM 44 0 0 02 Apr 2025
Fine-Grained Evaluation of Large Vision-Language Models in Autonomous Driving Yue Li Meng Tian Zhenyu Lin Jiangtong Zhu Dechang Zhu Haiqiang Liu Zining Wang Yueyi Zhang Zhiwei Xiong Xinhai Zhao CoGe VLM 80 1 0 27 Mar 2025
Direct Post-Training Preference Alignment for Multi-Agent Motion Generation Models Using Implicit Feedback from Pre-training Demonstrations Ran Tian Kratarth Goel 41 0 0 25 Mar 2025
Dita: Scaling Diffusion Transformer for Generalist Vision-Language-Action Policy Zhi Hou Tianyi Zhang Yuwen Xiong Haonan Duan Hengjun Pu ... Chengyang Zhao X. Zhu Yu Qiao Jifeng Dai Y. Chen 59 1 0 25 Mar 2025
GR00T N1: An Open Foundation Model for Generalist Humanoid Robots Nvidia Johan Bjorck Fernando Castañeda Nikita Cherniadev Xingye Da ... Ao Zhang Hao Zhang Yizhou Zhao Ruijie Zheng Yuke Zhu VLM 59 15 0 18 Mar 2025
A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning Xin Wen Bingchen Zhao Yilun Chen Jiangmiao Pang Xiaojuan Qi LM&Ro 39 0 0 10 Mar 2025
CoinRobot: Generalized End-to-end Robotic Learning for Physical Intelligence Y. Zhao Huxian Liu Xiang Chen Jiankai Sun Jiahuan Yan Luhui Hu 59 0 0 07 Mar 2025
Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learning Baoqi Pei Y. Huang Jilan Xu Guo Chen Yuping He ... Yali Wang Weidi Xie Yu Qiao Fei Wu Limin Wang 41 0 0 02 Mar 2025
Physics-Driven Data Generation for Contact-Rich Manipulation via Trajectory Optimization Lujie Yang H. Suh Tong Zhao B. P. Graesdal Tarik Kelestemur Jiuguang Wang Tao Pang Russ Tedrake 74 2 0 27 Feb 2025
Point Policy: Unifying Observations and Actions with Key Points for Robot Manipulation Siddhant Haldar Lerrel Pinto 3DPC 58 2 0 27 Feb 2025
Temporal Representation Alignment: Successor Features Enable Emergent Compositionality in Robot Instruction Following Vivek Myers Bill Chunyuan Zheng Anca Dragan Kuan Fang Sergey Levine 60 0 0 08 Feb 2025
DriveLM: Driving with Graph Visual Question Answering Chonghao Sima Katrin Renz Kashyap Chitta L. Chen Hanxue Zhang Chengen Xie Jens Beißwenger Ping Luo Andreas Geiger Hongyang Li 84 160 0 17 Jan 2025
Motion Tracks: A Unified Representation for Human-Robot Transfer in Few-Shot Imitation Learning Juntao Ren Priya Sundaresan Dorsa Sadigh Sanjiban Choudhury Jeannette Bohg 37 13 0 13 Jan 2025
Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation Yang Tian Sizhe Yang Jia Zeng P. Wang Dahua Lin Hao Dong Jiangmiao Pang 76 14 0 19 Dec 2024
Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulation Yueru Jia Jiaming Liu Sixiang Chen Chenyang Gu Z. Wang ... Lily Lee Pengwei Wang Zhongyuan Wang Renrui Zhang Shanghang Zhang 87 11 0 27 Nov 2024
Vision Language Models are In-Context Value Learners Yecheng Jason Ma Joey Hejna Ayzaan Wahid Chuyuan Fu Dhruv Shah ... Dinesh Jayaraman Wenhao Yu Tingnan Zhang Dorsa Sadigh Fei Xia 49 4 0 07 Nov 2024
Pre-trained Visual Dynamics Representations for Efficient Policy Learning Hao Luo Bohan Zhou Zongqing Lu 30 0 0 05 Nov 2024
Sparsh: Self-supervised touch representations for vision-based tactile sensing Carolina Higuera Akash Sharma Chaithanya Krishna Bodduluri Taosha Fan Patrick E. Lancaster ... Michael Kaess Byron Boots Mike Lambeta Tingfan Wu Mustafa Mukadam 32 11 0 31 Oct 2024
Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Models Nils Blank Moritz Reuss Marcel Rühle Ömer Erdinç Yagmurlu Fabian Wenzel Oier Mees Rudolf Lioutikov LM&Ro OffRL 29 4 0 23 Oct 2024
Foundation Models for Rapid Autonomy Validation Alec Farid Peter Schleede Aaron Huang Christoffer Heckman 32 0 0 22 Oct 2024
Generative AI Agents in Autonomous Machines: A Safety Perspective Jason J. Jabbour Vijay Janapa Reddi AI4CE 38 3 0 20 Oct 2024
A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learning Shengjie Sun Runze Liu Jiafei Lyu J. Yang L. Zhang Xiu Li LRM 19 7 0 18 Oct 2024
CoPESD: A Multi-Level Surgical Motion Dataset for Training Large Vision-Language Models to Co-Pilot Endoscopic Submucosal Dissection Guankun Wang Han Xiao Huxin Gao Renrui Zhang Long Bai Xiaoxiao Yang Zhen Li Hongsheng Li Hongliang Ren 31 4 0 10 Oct 2024
SPA: 3D Spatial-Awareness Enables Effective Embodied Representation Haoyi Zhu Honghui Yang Yating Wang Jiange Yang Limin Wang Tong He 3DH 43 5 0 10 Oct 2024
GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation Chi-Lam Cheang Guangzeng Chen Ya Jing Tao Kong Hang Li ... Hongtao Wu Jiafeng Xu Yichu Yang Hanbo Zhang Minzhao Zhu VGen LM&Ro 61 51 0 08 Oct 2024
Robo-MUTUAL: Robotic Multimodal Task Specification via Unimodal Learning Jianxiong Li Zhihao Wang Jinliang Zheng Xiaoai Zhou Guanming Wang ... Yu Liu Jingjing Liu Ya-Qin Zhang Junzhi Yu Xianyuan Zhan 31 2 0 02 Oct 2024
Towards Generalizable Vision-Language Robotic Manipulation: A Benchmark and LLM-guided 3D Policy Ricardo Garcia Shizhe Chen Cordelia Schmid LM&Ro 31 5 0 02 Oct 2024
Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers Lirui Wang Xinlei Chen Jialiang Zhao Kaiming He 34 33 0 30 Sep 2024
Feature Extractor or Decision Maker: Rethinking the Role of Visual Encoders in Visuomotor Policies Ruiyu Wang Zheyu Zhuang Shutong Jin Nils Ingelhag Danica Kragic Florian T. Pokorny 20 0 0 30 Sep 2024
RoboNurse-VLA: Robotic Scrub Nurse System based on Vision-Language-Action Model Shunlei Li Jin Wang Rui Dai Wanyu Ma Wing Yin Ng Yingbai Hu Zheng Li 24 2 0 29 Sep 2024
Robotic-CLIP: Fine-tuning CLIP on Action Data for Robotic Applications Nghia Nguyen Minh Nhat Vu Tung D. Ta Baoru Huang T. Vo Ngan Le Anh Nguyen VLM CLIP 38 3 0 26 Sep 2024
Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation Homanga Bharadhwaj Debidatta Dwibedi Abhinav Gupta Shubham Tulsiani Carl Doersch Ted Xiao Dhruv Shah Fei Xia Dorsa Sadigh Sean Kirmani VGen LM&Ro 35 27 0 24 Sep 2024
Generalized Robot Learning Framework Jiahuan Yan Zhouyang Hong Yu Zhao Yu Tian Yunxin Liu Travis Davies Luhui Hu 31 0 0 18 Sep 2024
Robot Utility Models: General Policies for Zero-Shot Deployment in New Environments Haritheja Etukuru Norihito Naka Zijin Hu Seungjae Lee Julian Mehu Aaron Edsinger Chris Paxton Soumith Chintala Lerrel Pinto Nur Muhammad (Mahi) Shafiullah LM&Ro 31 23 0 09 Sep 2024
FlowRetrieval: Flow-Guided Data Retrieval for Few-Shot Imitation Learning Li-Heng Lin Yuchen Cui Amber Xie Tianyu Hua Dorsa Sadigh 19 8 0 29 Aug 2024
Actra: Optimized Transformer Architecture for Vision-Language-Action Models in Robot Learning Yueen Ma Dafeng Chi Shiguang Wu Yuecheng Liu Yuzheng Zhuang Jianye Hao Irwin King 34 5 0 02 Aug 2024
Pretrained Visual Representations in Reinforcement Learning Emlyn Williams Athanasios Polydoros SSL 20 1 0 24 Jul 2024
Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learning Zhecheng Yuan Tianming Wei Shuiqi Cheng Gu Zhang Yuanpei Chen Huazhe Xu 43 22 0 22 Jul 2024
MuTT: A Multimodal Trajectory Transformer for Robot Skills Claudius Kienle Benjamin Alt Onur Celik P. Becker Darko Katic Rainer Jäkel Gerhard Neumann 33 2 0 22 Jul 2024
Real-Time Anomaly Detection and Reactive Planning with Large Language Models Rohan Sinha Amine Elhafsi Christopher Agia Matthew Foutter Edward Schmerling Marco Pavone OffRL LRM 35 24 0 11 Jul 2024
FLAIR: Feeding via Long-horizon AcquIsition of Realistic dishes Rajat Kumar Jenamani Priya Sundaresan Maram Sakr T. Bhattacharjee Dorsa Sadigh 23 9 0 10 Jul 2024
Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals Moritz Reuss Ömer Erdinç Yagmurlu Fabian Wenzel Rudolf Lioutikov OffRL 28 41 0 08 Jul 2024
Object Segmentation from Open-Vocabulary Manipulation Instructions Based on Optimal Transport Polygon Matching with Multimodal Foundation Models Takayuki Nishimura Katsuyuki Kuyo Motonari Kambara Komei Sugiura DiffM 24 0 0 01 Jul 2024
RoboUniView: Visual-Language Model with Unified View Representation for Robotic Manipulaiton Fanfan Liu Feng Yan Liming Zheng Chengjian Feng Yiyang Huang Lin Ma LM&Ro 23 11 0 27 Jun 2024
Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulation Jiaming Zhou Teli Ma Kun-Yu Lin Ronghe Qiu Zifan Wang Junwei Liang 41 3 0 20 Jun 2024
OpenVLA: An Open-Source Vision-Language-Action Model Moo Jin Kim Karl Pertsch Siddharth Karamcheti Ted Xiao Ashwin Balakrishna ... Russ Tedrake Dorsa Sadigh Sergey Levine Percy Liang Chelsea Finn LM&Ro VLM 37 348 0 13 Jun 2024
Learning Manipulation by Predicting Interaction Jia Zeng Qingwen Bu Bangjun Wang Wenke Xia Li Chen ... Heming Cui Bin Zhao Xuelong Li Yu Qiao Hongyang Li 48 19 0 01 Jun 2024
Video-Language Critic: Transferable Reward Functions for Language-Conditioned Robotics Minttu Alakuijala Reginald McLean Isaac Woungang Nariman Farsad Samuel Kaski Pekka Marttinen Kai Yuan LM&Ro 29 0 0 30 May 2024
Interpretable Robotic Manipulation from Language Boyuan Zheng Jianlong Zhou Fang Chen LM&Ro 27 0 0 27 May 2024