Affordances from Human Videos as a Versatile Representation for Robotics

17 April 2023

Papers citing "Affordances from Human Videos as a Versatile Representation for Robotics"

50 / 131 papers shown

Title
DexWild: Dexterous Human Interactions for In-the-Wild Robot Policies Tony Tao M. K. Srirama Jason Jingzhou Liu Kenneth Shaw Deepak Pathak 11 0 0 12 May 2025
UniDiffGrasp: A Unified Framework Integrating VLM Reasoning and VLM-Guided Part Diffusion for Open-Vocabulary Constrained Grasping with Dual Arms Xueyang Guo Hongwei Hu Chengye Song J. Chen Zilin Zhao Yu Fu Bowen Guan Zhenze Liu 16 0 0 11 May 2025
Web2Grasp: Learning Functional Grasps from Web Images of Hand-Object Interactions Hongyi Chen Yunchao Yao Yufei Ye Zhixuan Xu Homanga Bharadhwaj Jiashun Wang Shubham Tulsiani Zackory Erickson Jeffrey Ichnowski 24 0 0 07 May 2025
CrayonRobo: Object-Centric Prompt-Driven Vision-Language-Action Model for Robotic Manipulation Xiaoqi Li Lingyun Xu M. Zhang Jiaming Liu Yan Shen ... Jiahui Xu Liang Heng Siyuan Huang S. Zhang Hao Dong LM&Ro 36 0 0 04 May 2025
Interpretable Affordance Detection on 3D Point Clouds with Probabilistic Prototypes M. Li Korbinian Franz Rudolf Nils Blank Rudolf Lioutikov 3DPC 27 0 0 25 Apr 2025
AffordanceSAM: Segment Anything Once More in Affordance Grounding D. Jiang Mengmeng Wang Teli Ma H. Li Y. Liu Guang Dai L. Zhang 32 0 0 22 Apr 2025
Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Models Chen Wang Fei Xia Wenhao Yu Tingnan Zhang Ruohan Zhang Ce Liu Li Fei-Fei Jie Tan Jacky Liang 31 0 0 17 Apr 2025
RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins Yao Mu Tianxing Chen Z. Chen Shijia Peng Zhiqian Lan ... Mingkun Xu Lunkai Lin Zhiqiang Xie Mingyu Ding Ping Luo 24 1 0 17 Apr 2025
A0: An Affordance-Aware Hierarchical Model for General Robotic Manipulation Rongtao Xu J. Zhang Minghao Guo Youpeng Wen H. Yang ... Liqiong Wang Yuxuan Kuang Meng Cao Feng Zheng Xiaodan Liang 37 2 0 17 Apr 2025
How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions Aditya Prakash Benjamin Lundell Dmitry Andreychuk David Forsyth Saurabh Gupta H. Sawhney 26 0 0 16 Apr 2025
Novel Diffusion Models for Multimodal 3D Hand Trajectory Prediction Junyi Ma Wentao Bao Jingyi Xu Guanzhong Sun Xieyuanli Chen Hesheng Wang 30 0 0 10 Apr 2025
MAPLE: Encoding Dexterous Robotic Manipulation Priors Learned From Egocentric Videos Alexey Gavryushin Xi Wang Robert J. S. Malate Chenyu Yang X. Jia Shubh Goel Davide Liconti René Zurbrugg Robert K. Katzschmann Marc Pollefeys 31 0 0 08 Apr 2025
Tool-as-Interface: Learning Robot Policies from Human Tool Usage through Imitation Learning Haonan Chen Cheng Zhu Yunzhu Li Katherine Driggs-Campbell 21 0 0 06 Apr 2025
Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets Chuning Zhu Raymond Yu S. Feng Benjamin Burchfiel Paarth Shah Abhishek Gupta VGen 55 0 0 03 Apr 2025
Slot-Level Robotic Placement via Visual Imitation from Single Human Video Dandan Shan Kaichun Mo Wei Yang Yu-Wei Chao David Fouhey Dieter Fox Arsalan Mousavian 34 0 0 02 Apr 2025
ZeroMimic: Distilling Robotic Manipulation Skills from Web Videos Junyao Shi Zhuolun Zhao Tianyou Wang Ian Pedroza Amy Luo Jie Wang Jason Ma Dinesh Jayaraman LM&Ro 43 0 0 31 Mar 2025
2HandedAfforder: Learning Precise Actionable Bimanual Affordances from Human Videos Marvin Heidinger Snehal Jauhri V. Prasad Georgia Chalvatzaki 58 0 0 12 Mar 2025
FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction Dennis Rotondi Fabio Scaparro Hermann Blum Kai O. Arras 41 0 0 10 Mar 2025
VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation Hanzhi Chen Boyang Sun Anran Zhang Marc Pollefeys Stefan Leutenegger LM&Ro 59 0 0 10 Mar 2025
AirExo-2: Scaling up Generalizable Robotic Imitation Learning with Low-Cost Exoskeletons Hongjie Fang Chenxi Wang Yiming Wang J. Chen Shangning Xia ... Xinyu Zhan Lixin Yang Weiming Wang Cewu Lu Hao-Shu Fang 80 1 0 05 Mar 2025
AffordGrasp: In-Context Affordance Reasoning for Open-Vocabulary Task-Oriented Grasping in Clutter Yingbo Tang S. Zhang Xiaoshuai Hao Pengwei Wang Jianlong Wu Z. Wang Shanghang Zhang 56 4 0 02 Mar 2025
Phantom: Training Robots Without Robots Using Only Human Videos Marion Lepert Jiaying Fang Jeannette Bohg OffRL 40 5 0 02 Mar 2025
Subtask-Aware Visual Reward Learning from Segmented Demonstrations Changyeon Kim Minho Heo Doohyun Lee Jinwoo Shin Honglak Lee Joseph J. Lim Kimin Lee 32 0 0 28 Feb 2025
Multi-Keypoint Affordance Representation for Functional Dexterous Grasping Fan Yang DongSheng Luo Wenrui Chen Jiacheng Lin Junjie Cai Kailun Yang Z. Li Yaonan Wang 36 0 0 27 Feb 2025
Human2Robot: Learning Robot Actions from Paired Human-Robot Videos Sicheng Xie Haidong Cao Zejia Weng Zhen Xing Shiwei Shen Jiaqi Leng Xipeng Qiu Yanwei Fu Zuxuan Wu Yu Jiang 45 0 0 23 Feb 2025
RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation Kun Wu Chengkai Hou Jiaming Liu Zhengping Che Xiaozhu Ju ... Zhenyu Wang Pengju An Siyuan Qian S. Zhang Jian Tang LM&Ro 105 15 0 17 Feb 2025
Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation Yang Tian Sizhe Yang Jia Zeng P. Wang Dahua Lin Hao Dong Jiangmiao Pang 76 13 0 19 Dec 2024
Learning from Massive Human Videos for Universal Humanoid Pose Control Jiageng Mao Siheng Zhao Siqi Song Tianheng Shi Junjie Ye Mingtong Zhang Haoran Geng Jitendra Malik Vitor Campagnolo Guizilini Yue Wang 85 5 0 18 Dec 2024
HandsOnVLM: Vision-Language Models for Hand-Object Interaction Prediction Chen Bao Jiarui Xu Xiaolong Wang Abhinav Gupta Homanga Bharadhwaj 68 2 0 17 Dec 2024
Reinforcement Learning from Wild Animal Videos Elliot Chane-Sane Constant Roux O. Stasse Nicolas Mansard 77 0 0 05 Dec 2024
Leverage Task Context for Object Affordance Ranking Haojie Huang Hongchen Luo Wei-dong Zhai Yang Cao Zheng-jun Zha 59 0 0 25 Nov 2024
Tra-MoE: Learning Trajectory Prediction Model from Multiple Domains for Adaptive Policy Conditioning Jiange Yang Haoyi Zhu Y. Wang Gangshan Wu Tong He Limin Wang 83 2 0 21 Nov 2024
STEER: Flexible Robotic Manipulation via Dense Language Grounding Laura Smith A. Irpan Montserrat Gonzalez Arenas Sean Kirmani Dmitry Kalashnikov Dhruv Shah Ted Xiao LLMSV 32 1 0 05 Nov 2024
Learning Few-Shot Object Placement with Intra-Category Transfer Adrian Rofer Russell Buchanan Max Argus S. Vijayakumar Abhinav Valada 30 0 0 05 Nov 2024
Pre-trained Visual Dynamics Representations for Efficient Policy Learning Hao Luo Bohan Zhou Zongqing Lu 26 0 0 05 Nov 2024
On-Robot Reinforcement Learning with Goal-Contrastive Rewards Ondrej Biza Thomas Weng Lingfeng Sun Karl Schmeckpeper Tarik Kelestemur Yecheng Jason Ma Robert C. Platt Jan Willem van de Meent Lawson L. S. Wong OffRL 31 0 0 25 Oct 2024
Latent Action Pretraining from Videos Seonghyeon Ye Joel Jang Byeongguk Jeon Sejune Joo Jianwei Yang ... Kimin Lee Jianfeng Gao Luke Zettlemoyer Dieter Fox Minjoon Seo 30 19 0 15 Oct 2024
EgoOops: A Dataset for Mistake Action Detection from Egocentric Videos Referring to Procedural Texts Yuto Haneji Taichi Nishimura Hirotaka Kameko Keisuke Shirai Tomoya Yoshida Keiya Kajimura Koki Yamamoto Taiyu Cui Tomohiro Nishimoto Shinsuke Mori EgoV 44 0 0 07 Oct 2024
LeLaN: Learning A Language-Conditioned Navigation Policy from In-the-Wild Videos Noriaki Hirose Catherine Glossop A. Sridhar Dhruv Shah Oier Mees Sergey Levine LM&Ro 29 10 0 04 Oct 2024
Open-World Reinforcement Learning over Long Short-Term Imagination Jiajian Li Q. Wang Yunbo Wang Xin Jin Yang Li Wenjun Zeng Xiaokang Yang OCL VLM 47 1 0 04 Oct 2024
Learning Wheelchair Tennis Navigation from Broadcast Videos with Domain Knowledge Transfer and Diffusion Motion Planning Zixuan Wu Z. Zaidi Adithya Patil Qingyu Xiao Matthew C. Gombolay 64 0 0 29 Sep 2024
Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation Homanga Bharadhwaj Debidatta Dwibedi Abhinav Gupta Shubham Tulsiani Carl Doersch Ted Xiao Dhruv Shah Fei Xia Dorsa Sadigh Sean Kirmani VGen LM&Ro 35 21 0 24 Sep 2024
Skills Made to Order: Efficient Acquisition of Robot Cooking Skills Guided by Multiple Forms of Internet Data Mrinal Verghese C. Atkeson 29 0 0 23 Sep 2024
TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation Junjie Wen Y. X. Zhu Jinming Li Minjie Zhu Kun Wu ... Ran Cheng Chaomin Shen Yaxin Peng Feifei Feng Jian Tang LM&Ro 56 41 0 19 Sep 2024
Robot Manipulation in Salient Vision through Referring Image Segmentation and Geometric Constraints Chen Jiang Allie Luo Martin Jägersand 15 0 0 17 Sep 2024
Embodiment-Agnostic Action Planning via Object-Part Scene Flow Weiliang Tang Jia-Hui Pan Wei Zhan Jianshu Zhou Huaxiu Yao Yun-Hui Liu M. Tomizuka Mingyu Ding Chi-Wing Fu 41 0 0 16 Sep 2024
$DexSim2Real$^{2}$: Building Explicit World Model for Precise Articulated Object Dexterous Manipulation$ DexSim2Real $^{2}$ : Building Explicit World Model for Precise Articulated Object Dexterous Manipulation Taoran Jiang Liqian Ma Yixuan Guan Jiaojiao Meng Weihang Chen Zecui Zeng Lusong Li Dan Wu Jing Xu Rui Chen 18 0 0 13 Sep 2024
Hand-Object Interaction Pretraining from Videos Himanshu Gaurav Singh Antonio Loquercio Carmelo Sferrazza Jane Wu Haozhi Qi Pieter Abbeel Jitendra Malik 38 11 0 12 Sep 2024
HiRT: Enhancing Robotic Control with Hierarchical Robot Transformers Jianke Zhang Yanjiang Guo Xiaoyu Chen Yen-Jen Wang Yucheng Hu Chengming Shi Jianyu Chen 16 4 0 12 Sep 2024
INTRA: Interaction Relationship-aware Weakly Supervised Affordance Grounding Ji Ha Jang H. Seo Se Young Chun 29 2 0 10 Sep 2024