Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers

25 March 2022

Papers citing "Reshaping Robot Trajectories Using Natural Language Commands: A Study of Multi-Modal Data Alignment Using Transformers"

28 / 28 papers shown

Title
Trajectory Adaptation using Large Language Models Anurag Maurya Tashmoy Ghosh Ravi Prakash 19 0 0 17 Apr 2025
Skill Generalization with Verbs Rachel Ma Lyndon Lam Benjamin A. Spiegel Aditya Ganeshan Roma Patel Ben Abbatematteo D. Paulius Stefanie Tellex G. Konidaris LM&Ro 62 2 0 18 Oct 2024
Trajectory Improvement and Reward Learning from Comparative Language Feedback Zhaojing Yang Miru Jun J. Tien Stuart J. Russell Anca Dragan Erdem Bıyık 26 5 0 08 Oct 2024
Hey Robot! Personalizing Robot Navigation through Model Predictive Control with a Large Language Model Diego Martínez Baselga O. de Groot Luzia Knoedler Javier Alonso-Mora L. Riazuelo Luis Montano LM&Ro 37 0 0 20 Sep 2024
Lifelong Robot Library Learning: Bootstrapping Composable and Generalizable Skills for Embodied Control with Language Models Georgios Tziafas H. Kasaei KELM LM&Ro 20 8 0 26 Jun 2024
Robustness Testing of Multi-Modal Models in Varied Home Environments for Assistive Robots Lea Hirlimann Shengqiang Zhang Hinrich Schütze Philipp Wicke 19 0 0 18 Jun 2024
LLM-BT: Performing Robotic Adaptive Tasks based on Large Language Models and Behavior Trees Haotian Zhou Yunhan Lin Longwu Yan Jihong Zhu Huasong Min LM&Ro 17 9 0 08 Apr 2024
RT-H: Action Hierarchies Using Language Suneel Belkhale Tianli Ding Ted Xiao P. Sermanet Quon Vuong Jonathan Tompson Yevgen Chebotar Debidatta Dwibedi Dorsa Sadigh LM&Ro 23 73 0 04 Mar 2024
ExTraCT -- Explainable Trajectory Corrections from language inputs using Textual description of features J-Anne Yow N. P. Garg Manoj Ramanathan Wei Tech Ang 18 5 0 08 Jan 2024
Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections Lihan Zha Yuchen Cui Li-Heng Lin Minae Kwon Montse Gonzalez Arenas Andy Zeng Fei Xia Dorsa Sadigh 21 34 0 17 Nov 2023
GRID: A Platform for General Robot Intelligence Development Sai H. Vemprala Shuhang Chen Abhinav Shukla Dinesh Narayanan Ashish Kapoor 17 10 0 02 Oct 2023
LANCAR: Leveraging Language for Context-Aware Robot Locomotion in Unstructured Environments Chak Lam Shek Xiyang Wu Wesley A. Suttle Carl E. Busart Erin Zaroukian Dinesh Manocha Pratap Tokekar Amrit Singh Bedi LLMAG 35 8 0 30 Sep 2023
Grasp-Anything: Large-scale Grasp Dataset from Foundation Models An Vuong Minh Nhat Vu Hieu Le Baoru Huang B. Huynh T. Vo Andreas Kugi Anh Nguyen VLM 8 28 0 18 Sep 2023
A Comprehensive Review of Recent Research Trends on UAVs Khaled Telli O. Kraa Yassine Himeur A. Ouamane M. Boumehraz Shadi Atalla W. Mansoor 12 2 0 25 Jul 2023
Towards Language-Based Modulation of Assistive Robots through Multimodal Models Philipp Wicke Lufti Kerem cSenel Shengqiang Zhang Luis F. C. Figueredo Abdeldjallil Naceri Sami Haddadin Hinrich Schütze 11 2 0 26 Jun 2023
Language to Rewards for Robotic Skill Synthesis Wenhao Yu Nimrod Gileadi Chuyuan Fu Sean Kirmani Kuang-Huei Lee ... N. Heess Dorsa Sadigh Jie Tan Yuval Tassa F. Xia LM&Ro 16 265 0 14 Jun 2023
Knowledge-enhanced Agents for Interactive Text Games P. Chhikara Jiarui Zhang Filip Ilievski Jonathan M Francis Kaixin Ma LLMAG 26 8 0 08 May 2023
ChatGPT for Robotics: Design Principles and Model Abilities Sai H. Vemprala Rogerio Bonatti A. Bucker Ashish Kapoor LM&Ro 12 453 0 20 Feb 2023
"No, to the Right" -- Online Language Corrections for Robotic Manipulation via Shared Autonomy Yuchen Cui Siddharth Karamcheti Raj Palleti Nidhya Shivakumar Percy Liang Dorsa Sadigh LM&Ro 6 75 0 06 Jan 2023
On Transforming Reinforcement Learning by Transformer: The Development Trajectory Shengchao Hu Li Shen Ya-Qin Zhang Yixin Chen Dacheng Tao OffRL 23 23 0 29 Dec 2022
PACT: Perception-Action Causal Transformer for Autoregressive Robotics Pre-Training Rogerio Bonatti Sai H. Vemprala Shuang Ma Felipe Vieira Frujeri Shuhang Chen Ashish Kapoor 20 22 0 22 Sep 2022
Meta-Reinforcement Learning via Language Instructions Zhenshan Bing A. Koch Xiangtong Yao Kai-Qi Huang Alois C. Knoll LM&Ro 50 18 0 11 Sep 2022
Reconstructing Action-Conditioned Human-Object Interactions Using Commonsense Knowledge Priors Xi Wang Gengyan Li Yen-Ling Kuo Muhammed Kocabas Emre Aksan Otmar Hilliges 38 28 0 06 Sep 2022
LATTE: LAnguage Trajectory TransformEr A. Bucker Luis F. C. Figueredo Sami Haddadin Ashish Kapoor Shuang Ma Sai H. Vemprala Rogerio Bonatti LM&Ro 22 59 0 04 Aug 2022
Language-Conditioned Imitation Learning for Robot Manipulation Tasks Simon Stepputtis Joseph Campbell Mariano Phielipp Stefan Lee Chitta Baral H. B. Amor LM&Ro 111 192 0 22 Oct 2020
Transformer Networks for Trajectory Forecasting Francesco Giuliari Irtiza Hasan Marco Cristani Fabio Galasso 109 360 0 18 Mar 2020
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 250 922 0 24 Sep 2019
Help, Anna! Visual Navigation with Natural Multimodal Assistance via Retrospective Curiosity-Encouraging Imitation Learning Khanh Nguyen Hal Daumé LM&Ro EgoV 169 148 0 04 Sep 2019