Simple but Effective: CLIP Embeddings for Embodied AI

18 November 2021

Papers citing "Simple but Effective: CLIP Embeddings for Embodied AI"

50 / 173 papers shown

Title
BBSEA: An Exploration of Brain-Body Synchronization for Embodied Agents Sizhe Yang Qian Luo Anumpam Pani Yanchao Yang 22 2 0 13 Feb 2024
Towards Explainable, Safe Autonomous Driving with Language Embeddings for Novelty Identification and Active Learning: Framework and Experimental Analysis with Real-World Data Sets Ross Greer Mohan M. Trivedi 32 19 0 11 Feb 2024
Language-Based Augmentation to Address Shortcut Learning in Object Goal Navigation Dennis Hoftijzer Gertjan J. Burghouts Luuk J. Spreeuwers 13 1 0 07 Feb 2024
The Essential Role of Causality in Foundation World Models for Embodied AI Tarun Gupta Wenbo Gong Chao Ma Nick Pawlowski Agrin Hilmkil ... Jianfeng Gao Stefan Bauer Danica Kragic Bernhard Schölkopf Cheng Zhang 30 15 0 06 Feb 2024
Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learning Haoyi Zhu Yating Wang Di Huang Weicai Ye Wanli Ouyang Tong He SSL 3DPC 39 20 0 04 Feb 2024
True Knowledge Comes from Practice: Aligning LLMs with Embodied Environments via Reinforcement Learning Weihao Tan Wentao Zhang Shanqi Liu Longtao Zheng Xinrun Wang Bo An OffRL 36 16 0 25 Jan 2024
CLIP feature-based randomized control using images and text for multiple tasks and robots Kazuki Shibata Hideki Deguchi Shun Taguchi 21 1 0 18 Jan 2024
VoroNav: Voronoi-based Zero-shot Object Navigation with Large Language Model Pengying Wu Yao Mu Bingxian Wu Yi Hou Ji Ma Shanghang Zhang Chang-rui Liu LM&Ro 22 24 0 05 Jan 2024
Holodeck: Language Guided Generation of 3D Embodied AI Environments Yue Yang Fan-Yun Sun Luca Weihs Eli VanderBilt Alvaro Herrasti ... Lingjie Liu Chris Callison-Burch Mark Yatskar Aniruddha Kembhavi Christopher Clark LM&Ro 37 77 0 14 Dec 2023
Building Open-Ended Embodied Agent via Language-Policy Bidirectional Adaptation Shaopeng Zhai Jie Wang Tianyi Zhang Fuxian Huang Qi Zhang Ming Zhou Jing Hou Yu Qiao Yu Liu LLMAG LM&Ro 26 1 0 12 Dec 2023
Harmonic Mobile Manipulation Ruihan Yang Yejin Kim Aniruddha Kembhavi Xiaolong Wang Kiana Ehsani 23 13 0 11 Dec 2023
FoMo Rewards: Can we cast foundation models as reward functions? Ekdeep Singh Lubana Johann Brehmer P. D. Haan Taco S. Cohen OffRL LRM 38 2 0 06 Dec 2023
Understanding Representations Pretrained with Auxiliary Losses for Embodied Agent Planning Samrudhdhi B. Rangrej James J. Clark SSL 32 0 0 06 Dec 2023
Transfer Learning in Robotics: An Upcoming Breakthrough? A Review of Promises and Challenges Noémie Jaquier Michael C. Welle A. Gams Kunpeng Yao Bernardo Fichera A. Billard Aleš Ude Tamim Asfour Danica Kragic 25 14 0 29 Nov 2023
Active Open-Vocabulary Recognition: Let Intelligent Moving Mitigate CLIP Limitations Lei Fan Jianxiong Zhou Xiaoying Xing Ying Wu VLM 30 3 0 28 Nov 2023
Robot Learning in the Era of Foundation Models: A Survey Xuan Xiao Jiahang Liu Zhipeng Wang Yanmin Zhou Yong Qi Qian Cheng Bin He Shuo Jiang AI4CE LM&Ro 21 26 0 24 Nov 2023
Selective Visual Representations Improve Convergence and Generalization for Embodied AI Ainaz Eftekhar Kuo-Hao Zeng Jiafei Duan Ali Farhadi Aniruddha Kembhavi Ranjay Krishna 27 13 0 07 Nov 2023
Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI Yaoxian Song Penglei Sun Haoyu Liu Li Zhixu Wei Song Yanghua Xiao Xiaofang Zhou LM&Ro 51 13 0 07 Nov 2023
Exploitation-Guided Exploration for Semantic Embodied Navigation Justin Wasserman Girish Chowdhary Abhinav Gupta Unnat Jain 16 1 0 06 Nov 2023
Can Foundation Models Watch, Talk and Guide You Step by Step to Make a Cake? Yuwei Bao Keunwoo Peter Yu Yichi Zhang Shane Storks Itamar Bar-Yossef Alexander De La Iglesia Megan Su Xiao Lin Zheng Joyce Chai 44 8 0 01 Nov 2023
Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning Juan Rocamonde Victoriano Montesinos Elvis Nava Ethan Perez David Lindner VLM 31 74 0 19 Oct 2023
Zero-Shot Object Goal Visual Navigation With Class-Independent Relationship Network Xinting Li Shizhou Zhang Yue Lu Kerry Dan Lingyan Ran 34 1 0 15 Oct 2023
An Unbiased Look at Datasets for Visuo-Motor Pre-Training Sudeep Dasari M. K. Srirama Unnat Jain Abhinav Gupta SSL 32 34 0 13 Oct 2023
Universal Visual Decomposer: Long-Horizon Manipulation Made Easy Zichen Zhang Yunshuang Li Osbert Bastani Abhishek Gupta Dinesh Jayaraman Yecheng Jason Ma Luca Weihs 30 17 0 12 Oct 2023
GROOT: Learning to Follow Instructions by Watching Gameplay Videos Shaofei Cai Bowei Zhang Zihao Wang Xiaojian Ma Anji Liu Yitao Liang 83 26 0 12 Oct 2023
Co-NavGPT: Multi-Robot Cooperative Visual Semantic Navigation Using Vision Language Models Bangguo Yu Qihao Yuan Kailai Li H. Kasaei Ming Cao LM&Ro 38 28 0 11 Oct 2023
Human-oriented Representation Learning for Robotic Manipulation Mingxiao Huo Mingyu Ding Chenfeng Xu Thomas Tian Xinghao Zhu Yao Mu Lingfeng Sun Masayoshi Tomizuka Wei Zhan SSL 33 12 0 04 Oct 2023
What do we learn from a large-scale study of pre-trained visual representations in sim and real environments? Sneha Silwal Karmesh Yadav Tingfan Wu Jay Vakil Arjun Majumdar ... Dhruv Batra Aravind Rajeswaran Mrinal Kalakrishnan Franziska Meier Oleksandr Maksymets SSL LM&Ro 34 5 0 03 Oct 2023
Learning to Terminate in Object Navigation Yuhang Song Anh Nguyen Chun-Yi Lee 30 3 0 28 Sep 2023
An In-depth Survey of Large Language Model-based Artificial Intelligence Agents Pengyu Zhao Zijian Jin Ning Cheng LLMAG 30 20 0 23 Sep 2023
Bridging Zero-shot Object Navigation and Foundation Models through Pixel-Guided Navigation Skill Wenzhe Cai Siyuan Huang Guangran Cheng Yuxing Long Peng Gao Changyin Sun Hao Dong LM&Ro 19 41 0 19 Sep 2023
Find What You Want: Learning Demand-conditioned Object Attribute Space for Demand-driven Navigation Hongchen Wang Andy Guan Hong Chen Xiaoqi Li Mingdong Wu Hao Dong 16 14 0 15 Sep 2023
SayNav: Grounding Large Language Models for Dynamic Planning to Navigation in New Environments Abhinav Rajvanshi Karan Sikka Xiao Lin Bhoram Lee Han-Pang Chiu Alvaro Velasquez LM&Ro LRM LLMAG 6 50 0 08 Sep 2023
Object Goal Navigation with Recursive Implicit Maps Shizhe Chen Thomas Chabal Ivan Laptev Cordelia Schmid 22 19 0 10 Aug 2023
Robust Visual Sim-to-Real Transfer for Robotic Manipulation Ricardo Garcia Pinel Robin Strudel Shizhe Chen Etienne Arlaud Ivan Laptev Cordelia Schmid OffRL 21 4 0 28 Jul 2023
SCRAPS: Speech Contrastive Representations of Acoustic and Phonetic Spaces Iván Vallés-Pérez Grzegorz Beringer Piotr Bilinski G. Cook Roberto Barra-Chicote 11 1 0 23 Jul 2023
Learning Navigational Visual Representations with Semantic Map Supervision Yicong Hong Yang Zhou Ruiyi Zhang Franck Dernoncourt Trung Bui Stephen Gould Hao Tan SSL 30 21 0 23 Jul 2023
Is Imitation All You Need? Generalized Decision-Making with Dual-Phase Training Yao Wei Yanchao Sun Ruijie Zheng Sai H. Vemprala Rogerio Bonatti Shuhang Chen Ratnesh Madaan Zhongjie Ba Ashish Kapoor Shuang Ma OffRL 17 15 0 16 Jul 2023
Switching Head-Tail Funnel UNITER for Dual Referring Expression Comprehension with Fetch-and-Carry Tasks Ryosuke Korekata Motonari Kambara Yusuke Yoshida Shintaro Ishikawa Yosuke Kawasaki Masaki Takahashi K. Sugiura LM&Ro 28 5 0 14 Jul 2023
Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulation Annie Xie Lisa Lee Ted Xiao Chelsea Finn 21 54 0 07 Jul 2023
SpawnNet: Learning Generalizable Visuomotor Skills from Pre-trained Networks Xingyu Lin John So Sashwat Mahalingam Fangchen Liu Pieter Abbeel SSL 22 21 0 07 Jul 2023
DoReMi: Grounding Language Model by Detecting and Recovering from Plan-Execution Misalignment Yanjiang Guo Yen-Jen Wang Lihan Zha Zheyuan Jiang Jianyu Chen LM&Ro 19 39 0 01 Jul 2023
HabiCrowd: A High Performance Simulator for Crowd-Aware Visual Navigation Vuong Dinh An Toan Tien Nguyen Minh Nhat Vu Baoru Huang Dzung Nguyen H. Binh T. Vo Anh Nguyen 33 5 0 20 Jun 2023
Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation Mukul Khanna Yongsen Mao Hanxiao Jiang Sanjay Haresh Brennan Schacklett Dhruv Batra Alexander William Clegg Eric Undersander Angel X. Chang Manolis Savva 3DV 22 68 0 20 Jun 2023
A Universal Semantic-Geometric Representation for Robotic Manipulation Tong Zhang Yingdong Hu Hanchen Cui Hang Zhao Yang Gao 60 17 0 18 Jun 2023
ArtWhisperer: A Dataset for Characterizing Human-AI Interactions in Artistic Creations Kailas Vodrahalli James Y. Zou 25 5 0 13 Jun 2023
Embodied Executable Policy Learning with Language-based Scene Summarization Jielin Qiu Mengdi Xu William Jongwon Han Seungwhan Moon Ding Zhao LM&Ro 19 7 0 09 Jun 2023
CLIPGraphs: Multimodal Graph Networks to Infer Object-Room Affinities A. Agrawal Raghav Arora Ahana Datta Snehasis Banerjee Brojeshwar Bhowmick Krishna Murthy Jatavallabhula Mohan Sridharan Madhava Krishna 22 2 0 02 Jun 2023
LIV: Language-Image Representations and Rewards for Robotic Control Yecheng Jason Ma William Liang Vaidehi Som Vikash Kumar Amy Zhang Osbert Bastani Dinesh Jayaraman LM&Ro 26 120 0 01 Jun 2023
Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning Jialong Wu Haoyu Ma Chao Deng Mingsheng Long OffRL 24 24 0 29 May 2023