Affordances from Human Videos as a Versatile Representation for Robotics

17 April 2023

Papers citing "Affordances from Human Videos as a Versatile Representation for Robotics"

50 / 131 papers shown

Title
One-Shot Imitation under Mismatched Execution K. Kedia Prithwish Dan Sanjiban Choudhury Maximus Adrian Pace Sanjiban Choudhury 35 2 0 10 Sep 2024
ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation Wenlong Huang Chen Wang Y. Li Ruohan Zhang Li Fei-Fei 39 81 0 03 Sep 2024
Semantically Controllable Augmentations for Generalizable Robot Learning Zoey Chen Zhao Mandi Homanga Bharadhwaj Mohit Sharma Shuran Song Abhishek Gupta Vikash Kumar LM&Ro 29 5 0 02 Sep 2024
One-shot Video Imitation via Parameterized Symbolic Abstraction Graphs Jianren Wang Kangni Liu Dingkun Guo Xian Zhou Christopher G Atkeson 25 0 0 22 Aug 2024
Learning Precise Affordances from Egocentric Videos for Robotic Manipulation Gen Li Nikolaos Tsagkas Jifei Song Ruaridh Mon-Williams S. Vijayakumar Kun Shao Laura Sevilla-Lara 31 7 0 19 Aug 2024
HRP: Human Affordances for Robotic Pre-Training M. K. Srirama Sudeep Dasari Shikhar Bahl Abhinav Gupta 21 12 0 26 Jul 2024
Flow as the Cross-Domain Manipulation Interface Mengda Xu Zhenjia Xu Yinghao Xu Cheng Chi Gordon Wetzstein Manuela Veloso Shuran Song AI4CE 35 32 0 21 Jul 2024
Simultaneous Localization and Affordance Prediction for Tasks in Egocentric Video Zachary Chavis Hyun Soo Park Stephen J. Guy EgoV 25 0 0 18 Jul 2024
CookAR: Affordance Augmentations in Wearable AR to Support Kitchen Tool Interactions for People with Low Vision Jaewook Lee Andrew D. Tjahjadi Jiho Kim Junpu Yu Minji Park Jiawen Zhang Jon E. Froehlich Yapeng Tian Yuhang Zhao 20 6 0 18 Jul 2024
R+X: Retrieval and Execution from Everyday Human Videos Georgios Papagiannis Norman Di Palo Pietro Vitiello Edward Johns 48 15 0 17 Jul 2024
RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulation Yuxuan Kuang Junjie Ye Haoran Geng Jiageng Mao Congyue Deng Leonidas J. Guibas He Wang Yue Wang LM&Ro 38 20 0 05 Jul 2024
TieBot: Learning to Knot a Tie from Visual Demonstration through a Real-to-Sim-to-Real Approach Weikun Peng Jun Lv Yuwei Zeng Haonan Chen Siheng Zhao Jichen Sun Cewu Lu Lin Shao 26 1 0 03 Jul 2024
Mitigating the Human-Robot Domain Discrepancy in Visual Pre-training for Robotic Manipulation Jiaming Zhou Teli Ma Kun-Yu Lin Ronghe Qiu Zifan Wang Junwei Liang 41 3 0 20 Jun 2024
Learning Semantic Traversability with Egocentric Video and Automated Annotation Strategy Yunho Kim Jeong Hyun Lee Choongin Lee Juhyeok Mun D. Youm Jeongsoo Park Jemin Hwangbo 21 1 0 05 Jun 2024
Learning Manipulation by Predicting Interaction Jia Zeng Qingwen Bu Bangjun Wang Wenke Xia Li Chen ... Heming Cui Bin Zhao Xuelong Li Yu Qiao Hongyang Li 48 19 0 01 Jun 2024
SAM-E: Leveraging Visual Foundation Model with Sequence Imitation for Embodied Manipulation Junjie Zhang Chenjia Bai Haoran He Wenke Xia Zhigang Wang Bin Zhao Xiu Li Xuelong Li 35 12 0 30 May 2024
Empowering Embodied Manipulation: A Bimanual-Mobile Robot Manipulation Dataset for Household Tasks Tianle Zhang Dongjiang Li Yihang Li Zecui Zeng Lin Zhao ... Yue Chen Xuelong Wei Yibing Zhan Lusong Li Xiaodong He 22 6 0 29 May 2024
iVideoGPT: Interactive VideoGPTs are Scalable World Models Jialong Wu Shaofeng Yin Ningya Feng Xu He Dong Li Jianye Hao Mingsheng Long VGen 32 22 0 24 May 2024
Octo: An Open-Source Generalist Robot Policy Octo Model Team Dibya Ghosh Homer Walke Karl Pertsch Kevin Black ... Quan Vuong Ted Xiao Dorsa Sadigh Chelsea Finn Sergey Levine 55 333 0 20 May 2024
ManiFoundation Model for General-Purpose Robotic Manipulation of Contact Synthesis with Arbitrary Objects and Robots Zhixuan Xu Chongkai Gao Zixuan Liu Gang Yang Chenrui Tie ... Weikun Peng Debang Wang Tianyi Chen Zhouliang Yu Lin Shao 38 8 0 11 May 2024
Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splatting O. Shorinwa Johnathan Tucker Aliyah Smith Aiden Swann Timothy Chen Roya Firoozi Monroe Kennedy Mac Schwager 29 20 0 07 May 2024
Diff-IP2D: Diffusion-Based Hand-Object Interaction Prediction on Egocentric Videos Junyi Ma Jingyi Xu Xieyuanli Chen Hesheng Wang VGen 27 7 0 07 May 2024
Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks Murtaza Dalal Tarun Chiruvolu Devendra Singh Chaplot Ruslan Salakhutdinov LM&Ro 29 19 0 02 May 2024
Track2Act: Predicting Point Tracks from Internet Videos enables Diverse Zero-shot Robot Manipulation Homanga Bharadhwaj Roozbeh Mottaghi Abhinav Gupta Shubham Tulsiani 3DPC 41 15 0 02 May 2024
What Foundation Models can Bring for Robot Learning in Manipulation : A Survey Dingzhe Li Yixiang Jin A. Yong Hongze Yu Jun Shi Xiaoshuai Hao Peng Hao Huaping Liu Fuchun Sun Bin Fang AI4CE LM&Ro 64 12 0 28 Apr 2024
Retrieval-Augmented Embodied Agents Yichen Zhu Zhicai Ou Xiaofeng Mou Jian Tang 41 17 0 17 Apr 2024
QuasiSim: Parameterized Quasi-Physical Simulators for Dexterous Manipulations Transfer Xueyi Liu Kangbo Lyu Jieqiong Zhang Tao Du Li Yi 38 4 0 11 Apr 2024
Self-Explainable Affordance Learning with Embodied Caption Zhipeng Zhang Zhimin Wei Guolei Sun Peng Wang Luc Van Gool 40 2 0 08 Apr 2024
Text-driven Affordance Learning from Egocentric Vision Tomoya Yoshida Shuhei Kurita Taichi Nishimura Shinsuke Mori 35 5 0 03 Apr 2024
DITTO: Demonstration Imitation by Trajectory Transformation Nick Heppert Max Argus Tim Welschehold Thomas Brox Abhinav Valada 62 16 0 22 Mar 2024
Vid2Robot: End-to-end Video-conditioned Policy Learning with Cross-Attention Transformers Vidhi Jain Maria Attarian Nikhil J. Joshi Ayzaan Wahid Danny Driess ... Stefan Welker Christine Chan Igor Gilitschenski Yonatan Bisk Debidatta Dwibedi 68 27 0 19 Mar 2024
Spatiotemporal Predictive Pre-training for Robotic Motor Control Jiange Yang Bei Liu Jianlong Fu Bocheng Pan Gangshan Wu Limin Wang 29 10 0 08 Mar 2024
Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation Jonathan Yang Catherine Glossop Arjun Bhorkar Dhruv Shah Quan Vuong Chelsea Finn Dorsa Sadigh Sergey Levine 36 41 0 29 Feb 2024
Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots Cheng Chi Zhenjia Xu Chuer Pan Eric A. Cousineau Benjamin Burchfiel Siyuan Feng Russ Tedrake Shuran Song 17 184 0 15 Feb 2024
Learning by Watching: A Review of Video-based Learning Approaches for Robot Manipulation Chrisantus Eze Christopher Crick SSL 74 11 0 11 Feb 2024
Real-World Robot Applications of Foundation Models: A Review Kento Kawaharazuka T. Matsushima Andrew Gambardella Jiaxian Guo Chris Paxton Andy Zeng OffRL VLM LM&Ro 41 45 0 08 Feb 2024
InCoRo: In-Context Learning for Robotics Control with Feedback Loops Jiaqiang Ye Zhu Carla Gomez Cano David Vazquez Bermudez Michal Drozdzal LRM 16 7 0 07 Feb 2024
Zero-shot Imitation Policy via Search in Demonstration Dataset Federico Malato Florian Leopold Andrew Melnik Ville Hautamaki LM&Ro OffRL 13 6 0 29 Jan 2024
Zero-Shot Learning for the Primitives of 3D Affordance in General Objects Hyeonwoo Kim Sookwan Han Patrick Kwon Hanbyul Joo DiffM 33 13 0 23 Jan 2024
General Flow as Foundation Affordance for Scalable Robot Learning Chengbo Yuan Chuan Wen Tong Zhang Yang Gao AI4CE 18 31 0 21 Jan 2024
Seeing the Unseen: Visual Common Sense for Semantic Placement Ram Ramrakhya Aniruddha Kembhavi Dhruv Batra Z. Kira Kuo-Hao Zeng Luca Weihs VLM 33 4 0 15 Jan 2024
Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulation Yuanchen Ju Kaizhe Hu Guowei Zhang Gu Zhang Mingrun Jiang Huazhe Xu 23 38 0 15 Jan 2024
AffordanceLLM: Grounding Affordance from Vision Language Models Shengyi Qian Weifeng Chen Min Bai Xiong Zhou Zhuowen Tu Li Erran Li 8 20 0 12 Jan 2024
Any-point Trajectory Modeling for Policy Learning Chuan Wen Xingyu Lin John So Kai-xiang Chen Qi Dou Yang Gao Pieter Abbeel PINN VGen 33 79 0 28 Dec 2023
QUAR-VLA: Vision-Language-Action Model for Quadruped Robots Pengxiang Ding Han Zhao Wenxuan Song Zhitao Wang Zhenyu Wei Shangke Lyu Ningxi Yang Donglin Wang 25 19 0 22 Dec 2023
Shared Affordance-awareness via Augmented Reality for Proactive Assistance in Human-robot Collaboration Drake Moore Mark Zolotas T. Padır 11 1 0 20 Dec 2023
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis Yafei Hu Quanting Xie Vidhi Jain Jonathan M Francis Jay Patrikar ... Xiaolong Wang Sebastian A. Scherer Z. Kira Fei Xia Yonatan Bisk LM&Ro AI4CE 30 62 0 14 Dec 2023
Foundation Models in Robotics: Applications, Challenges, and the Future Roya Firoozi Johnathan Tucker Stephen Tian Anirudha Majumdar Jiankai Sun ... Brian Ichter Danny Driess Jiajun Wu Cewu Lu Mac Schwager LM&Ro AI4CE LRM VLM 35 136 0 13 Dec 2023
SKT-Hang: Hanging Everyday Objects via Object-Agnostic Semantic Keypoint Trajectory Generation Chia-Liang Kuo Yu-Wei Chao Yi-Ting Chen 23 1 0 08 Dec 2023
FoMo Rewards: Can we cast foundation models as reward functions? Ekdeep Singh Lubana Johann Brehmer P. D. Haan Taco S. Cohen OffRL LRM 33 2 0 06 Dec 2023