Language-Driven Representation Learning for Robotics

24 February 2023

Dorsa Sadigh

Papers citing "Language-Driven Representation Learning for Robotics"

50 / 115 papers shown

Title
Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search Nicola Dainese Matteo Merler Minttu Alakuijala Pekka Marttinen LLMAG 36 7 0 24 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 67 41 0 23 May 2024
Meta-Control: Automatic Model-based Control Synthesis for Heterogeneous Robot Skills Tianhao Wei Liqian Ma Rui Chen Weiye Zhao Changliu Liu 37 2 0 18 May 2024
Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control Gunshi Gupta Karmesh Yadav Y. Gal Dhruv Batra Z. Kira Cong Lu Tim G. J. Rudner 39 7 0 09 May 2024
Learning Planning Abstractions from Language Weiyu Liu Geng Chen Joy Hsu Jiayuan Mao Jiajun Wu PINN 33 2 0 06 May 2024
What Foundation Models can Bring for Robot Learning in Manipulation : A Survey Dingzhe Li Yixiang Jin A. Yong Hongze Yu Jun Shi Xiaoshuai Hao Peng Hao Huaping Liu Fuchun Sun Bin Fang AI4CE LM&Ro 64 13 0 28 Apr 2024
SUGAR: Pre-training 3D Visual Representations for Robotics Shizhe Chen Ricardo Garcia Pinel Ivan Laptev Cordelia Schmid 37 14 0 01 Apr 2024
ViSaRL: Visual Reinforcement Learning Guided by Human Saliency Anthony Liang Jesse Thomason Erdem Biyik 24 7 0 16 Mar 2024
Spatiotemporal Predictive Pre-training for Robotic Motor Control Jiange Yang Bei Liu Jianlong Fu Bocheng Pan Gangshan Wu Limin Wang 34 10 0 08 Mar 2024
Efficient Data Collection for Robotic Manipulation via Compositional Generalization Jensen Gao Annie Xie Ted Xiao Chelsea Finn Dorsa Sadigh 29 19 0 08 Mar 2024
Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference Benjamin Eysenbach Vivek Myers Ruslan Salakhutdinov Sergey Levine AI4TS 39 8 0 06 Mar 2024
RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketches Priya Sundaresan Q. Vuong Jiayuan Gu Peng-Tao Xu Ted Xiao ... Ajinkya Jain Karol Hausman Dorsa Sadigh Jeannette Bohg S. Schaal VGen 24 24 0 05 Mar 2024
RT-H: Action Hierarchies Using Language Suneel Belkhale Tianli Ding Ted Xiao P. Sermanet Quon Vuong Jonathan Tompson Yevgen Chebotar Debidatta Dwibedi Dorsa Sadigh LM&Ro 29 74 0 04 Mar 2024
Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation Jonathan Yang Catherine Glossop Arjun Bhorkar Dhruv Shah Quan Vuong Chelsea Finn Dorsa Sadigh Sergey Levine 36 41 0 29 Feb 2024
MOSAIC: A Modular System for Assistive and Interactive Cooking Huaxiaoyue Wang K. Kedia Juntao Ren Rahma Abdullah Atiksh Bhardwaj ... Maximus Adrian Pace Yash Sharma Xiangwan Sun Neha Sunkara Sanjiban Choudhury 35 12 0 29 Feb 2024
Learning with Language-Guided State Abstractions Andi Peng Ilia Sucholutsky Belinda Z. Li T. Sumers Thomas L. Griffiths Jacob Andreas Julie A. Shah LM&Ro 43 13 0 28 Feb 2024
DecisionNCE: Embodied Multimodal Representations via Implicit Preference Learning Jianxiong Li Jinliang Zheng Yinan Zheng Liyuan Mao Xiaoming Hu ... Jihao Liu Yu Liu Jingjing Liu Ya-Qin Zhang Xianyuan Zhan LM&Ro OffRL 35 8 0 28 Feb 2024
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models Siddharth Karamcheti Suraj Nair Ashwin Balakrishna Percy Liang Thomas Kollar Dorsa Sadigh MLLM VLM 57 96 0 12 Feb 2024
Object-Centric Instruction Augmentation for Robotic Manipulation Junjie Wen Yichen Zhu Minjie Zhu Jinming Li Zhiyuan Xu ... Chaomin Shen Yaxin Peng Dong Liu Feifei Feng Jian Tang LM&Ro 62 16 0 05 Jan 2024
Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation Hongtao Wu Ya Jing Chi-Hou Cheang Guangzeng Chen Jiafeng Xu Xinghang Li Minghuan Liu Hang Li Tao Kong 27 94 0 20 Dec 2023
Value Explicit Pretraining for Learning Transferable Representations Kiran Lekkala Henghui Bao S. Sontakke Laurent Itti SSL 17 0 0 19 Dec 2023
Foundation Models in Robotics: Applications, Challenges, and the Future Roya Firoozi Johnathan Tucker Stephen Tian Anirudha Majumdar Jiankai Sun ... Brian Ichter Danny Driess Jiajun Wu Cewu Lu Mac Schwager LM&Ro AI4CE LRM VLM 35 137 0 13 Dec 2023
LLF-Bench: Benchmark for Interactive Learning from Language Feedback Ching-An Cheng Andrey Kolobov Dipendra Kumar Misra Allen Nie Adith Swaminathan 19 19 0 11 Dec 2023
FoMo Rewards: Can we cast foundation models as reward functions? Ekdeep Singh Lubana Johann Brehmer P. D. Haan Taco S. Cohen OffRL LRM 33 2 0 06 Dec 2023
Disentangled Interaction Representation for One-Stage Human-Object Interaction Detection Xubin Zhong Changxing Ding Yupeng Hu Dacheng Tao 19 0 0 04 Dec 2023
LanGWM: Language Grounded World Model Rudra P. K. Poudel Harit Pandya Chao Zhang Roberto Cipolla 17 5 0 29 Nov 2023
On Bringing Robots Home Nur Muhammad (Mahi) Shafiullah Anant Rai Haritheja Etukuru Yiqian Liu Ishan Misra Soumith Chintala Lerrel Pinto 27 76 0 27 Nov 2023
Generalizable Imitation Learning Through Pre-Trained Representations Wei-Di Chang F. Hogan D. Meger Gregory Dudek Gregory Dudek 33 1 0 15 Nov 2023
Robust and Communication-Efficient Federated Domain Adaptation via Random Features Zhanbo Feng Yuanjie Wang Jie Li Fan Yang Jiong Lou Tiebin Mi Robert C. Qiu Zhenyu Liao 16 2 0 08 Nov 2023
What Makes Pre-Trained Visual Representations Successful for Robust Manipulation? Kaylee Burns Zach Witzel Jubayer Ibn Hamid Tianhe Yu Chelsea Finn Karol Hausman OOD SSL 23 22 0 03 Nov 2023
Vision-Language Foundation Models as Effective Robot Imitators Xinghang Li Minghuan Liu Hanbo Zhang Cunjun Yu Jie Xu ... Ya Jing Weinan Zhang Huaping Liu Hang Li Tao Kong LM&Ro 24 134 0 02 Nov 2023
Eureka: Human-Level Reward Design via Coding Large Language Models Yecheng Jason Ma William Liang Guanzhi Wang De-An Huang Osbert Bastani Dinesh Jayaraman Yuke Zhu Linxi Fan A. Anandkumar 19 288 0 19 Oct 2023
Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Models Kevin Black Mitsuhiko Nakamoto P. Atreya Homer Walke Chelsea Finn Aviral Kumar Sergey Levine DiffM LM&Ro 24 132 0 16 Oct 2023
Open X-Embodiment: Robotic Learning Datasets and RT-X Models Open X-Embodiment Collaboration Abby OÑeill Abdul Rehman Abhinav Gupta Abhiram Maddukuri ... Zhuo Xu Zichen Jeff Cui Zichen Zhang Zipeng Fu Zipeng Lin LM&Ro 42 457 0 13 Oct 2023
Learning to Act from Actionless Videos through Dense Correspondences Po-Chen Ko Jiayuan Mao Yilun Du Shao-Hua Sun Josh Tenenbaum 22 74 0 12 Oct 2023
What Matters to You? Towards Visual Representation Alignment for Robot Learning Ran Tian Chenfeng Xu Masayoshi Tomizuka Jitendra Malik Andrea V. Bajcsy 19 9 0 11 Oct 2023
RoboHive: A Unified Framework for Robot Learning Vikash Kumar Rutav Shah Gaoyue Zhou Vincent Moens Vittorio Caggiano Jay Vakil Abhishek Gupta Aravind Rajeswaran 20 22 0 10 Oct 2023
Human-oriented Representation Learning for Robotic Manipulation Mingxiao Huo Mingyu Ding Chenfeng Xu Thomas Tian Xinghao Zhu Yao Mu Lingfeng Sun Masayoshi Tomizuka Wei Zhan SSL 28 12 0 04 Oct 2023
Pre-training with Synthetic Data Helps Offline Reinforcement Learning Zecheng Wang Che Wang Zixuan Dong Keith Ross OffRL 22 5 0 01 Oct 2023
Robotic Offline RL from Internet Videos via Value-Function Pre-Training Chethan Bhateja Derek Guo Dibya Ghosh Anika Singh Manan Tomar Q. Vuong Yevgen Chebotar Sergey Levine Aviral Kumar OffRL 34 20 0 22 Sep 2023
A Unified Masked Autoencoder with Patchified Skeletons for Motion Synthesis Esteve Valls Mascaro Hyemin Ahn Dongheui Lee CVBM 29 4 0 14 Aug 2023
MOMA-Force: Visual-Force Imitation for Real-World Mobile Manipulation Taozheng Yang Yaxing Jing Hongtao Wu Jiafeng Xu Kuankuan Sima Guangzeng Chen Qie Sima Tao Kong 20 17 0 07 Aug 2023
RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control Anthony Brohan Noah Brown Justice Carbajal Yevgen Chebotar Xi Chen ... Ted Xiao Peng-Tao Xu Sichun Xu Tianhe Yu Brianna Zitkovich LM&Ro LRM 22 1,086 0 28 Jul 2023
Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation Bokui (William) Shen Ge Yang Alan Yu J. Wong L. Kaelbling Phillip Isola VLM 16 102 0 27 Jul 2023
Large Language Models as General Pattern Machines Suvir Mirchandani F. Xia Peter R. Florence Brian Ichter Danny Driess Montse Gonzalez Arenas Kanishka Rao Dorsa Sadigh Andy Zeng LLMAG 44 183 0 10 Jul 2023
Polybot: Training One Policy Across Robots While Embracing Variability Jonathan Yang Dorsa Sadigh Chelsea Finn 21 34 0 07 Jul 2023
SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networks Xingyu Lin John So Sashwat Mahalingam Fangchen Liu Pieter Abbeel SSL 22 21 0 07 Jul 2023
HYDRA: Hybrid Robot Actions for Imitation Learning Suneel Belkhale Yuchen Cui Dorsa Sadigh 13 35 0 29 Jun 2023
A Universal Semantic-Geometric Representation for Robotic Manipulation Tong Zhang Yingdong Hu Hanchen Cui Hang Zhao Yang Gao 60 17 0 18 Jun 2023
Transferring Foundation Models for Generalizable Robotic Manipulation Jiange Yang Wenhui Tan Chuhao Jin Keling Yao Bei Liu Jianlong Fu Ruihua Song Gangshan Wu Limin Wang LM&Ro 47 6 0 09 Jun 2023