KETO: Learning Keypoint Representations for Tool Manipulation

26 October 2019

Zengyi Qin

Kuan Fang

Yuke Zhu

Li Fei-Fei

Silvio Savarese

3DPC

ArXiv PDF HTML

Papers citing "KETO: Learning Keypoint Representations for Tool Manipulation"

50 / 71 papers shown

Title
From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation Yifu Yuan Haiqin Cui Yibin Chen Zibin Dong Fei Ni Longxin Kou Jinyi Liu Pengyi Li Yan Zheng Jianye Hao 28 0 0 13 May 2025
Dynamic Robot Tool Use with Vision Language Models Noah Trupin Zixing Wang A. H. Qureshi 37 0 0 02 May 2025
Multi-Keypoint Affordance Representation for Functional Dexterous Grasping Fan Yang DongSheng Luo Wenrui Chen Jiacheng Lin Junjie Cai Kailun Yang Z. Li Yaonan Wang 46 0 0 27 Feb 2025
FUNCTO: Function-Centric One-Shot Imitation Learning for Tool Manipulation Chao Tang Anxing Xiao Yuhong Deng Tianrun Hu Wenlong Dong Hanbo Zhang David Hsu Hong Zhang 73 2 0 24 Feb 2025
Haptic-ACT: Bridging Human Intuition with Compliant Robotic Manipulation via Immersive VR Kelin Li Shubham M. Wagh Nitish Sharma Saksham Bhadani Wei Chen Chang Liu Petar Kormushev 32 2 0 18 Sep 2024
PLATO: Planning with LLMs and Affordances for Tool Manipulation Arvind Car Sai Sravan Yarlagadda Alison Bartsch Abraham George A. Farimani LM&Ro LLMAG 29 4 0 17 Sep 2024
ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation Wenlong Huang Chen Wang Y. Li Ruohan Zhang Li Fei-Fei 46 87 0 03 Sep 2024
RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics Wentao Yuan Jiafei Duan Valts Blukis Wilbert Pumacay Ranjay Krishna Adithyavairavan Murali Arsalan Mousavian Dieter Fox LM&Ro 42 45 0 15 Jun 2024
Composable Part-Based Manipulation Weiyu Liu Jiayuan Mao Joy Hsu Tucker Hermans Animesh Garg Jiajun Wu 43 11 0 09 May 2024
Learning Distributional Demonstration Spaces for Task-Specific Cross-Pose Estimation Jenny Wang Octavian A. Donca David Held 29 1 0 07 May 2024
WeightedPose: Generalizable Cross-Pose Estimation via Weighted SVD Xuxin Cheng Heng Yu Harry Zhang Wenxing Deng 36 0 0 03 May 2024
Track2Act: Predicting Point Tracks from Internet Videos enables Diverse Zero-shot Robot Manipulation Homanga Bharadhwaj Roozbeh Mottaghi Abhinav Gupta Shubham Tulsiani 3DPC 48 16 0 02 May 2024
Empowering Large Language Models on Robotic Manipulation with Affordance Prompting Guangran Cheng Chuheng Zhang Wenzhe Cai Li Zhao Changyin Sun Jiang Bian LM&Ro LLMAG 187 9 0 17 Apr 2024
FoundationGrasp: Generalizable Task-Oriented Grasping with Foundation Models Chao Tang Dehao Huang Wenlong Dong Ruinian Xu Hong Zhang 34 9 0 16 Apr 2024
PoCo: Policy Composition from and for Heterogeneous Robot Learning Lirui Wang Jialiang Zhao Yilun Du Edward H. Adelson Russ Tedrake 76 28 0 04 Feb 2024
Synthetic data enables faster annotation and robust segmentation for multi-object grasping in clutter Dongmyoung Lee Wei Chen Nicolás Rojas 18 2 0 24 Jan 2024
Any-point Trajectory Modeling for Policy Learning Chuan Wen Xingyu Lin John So Kai-xiang Chen Qi Dou Yang Gao Pieter Abbeel PINN VGen 33 79 0 28 Dec 2023
Towards Learning Geometric Eigen-Lengths Crucial for Fitting Tasks Yijia Weng Kaichun Mo Ruoxi Shi Yanchao Yang Leonidas J. Guibas 26 3 0 25 Dec 2023
Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis Yafei Hu Quanting Xie Vidhi Jain Jonathan M Francis Jay Patrikar ... Xiaolong Wang Sebastian A. Scherer Z. Kira Fei Xia Yonatan Bisk LM&Ro AI4CE 32 63 0 14 Dec 2023
SKT-Hang: Hanging Everyday Objects via Object-Agnostic Semantic Keypoint Trajectory Generation Chia-Liang Kuo Yu-Wei Chao Yi-Ting Chen 30 1 0 08 Dec 2023
Tracking Object Positions in Reinforcement Learning: A Metric for Keypoint Detection (extended version) Emma Cramer Jonas Reiher Sebastian Trimpe 19 0 0 01 Dec 2023
Learning to Design and Use Tools for Robotic Manipulation Ziang Liu Stephen Tian Michelle Guo C. Karen Liu Jiajun Wu 29 4 0 01 Nov 2023
Few-Shot In-Context Imitation Learning via Implicit Graph Alignment Vitalis Vosylius Edward Johns 21 9 0 18 Oct 2023
Learning Generalizable Tool-use Skills through Trajectory Generation Carl Qi Yilin Wu Sarthak Shetty Haoyue Liu Bowen Jiang Xingyu Lin David Held 36 3 0 29 Sep 2023
AnyOKP: One-Shot and Instance-Aware Object Keypoint Extraction with Pretrained ViT Fangbo Qin Taogang Hou Shan Lin Kaiyuan Wang Michael C. Yip Shan Yu 21 0 0 15 Sep 2023
3D Implicit Transporter for Temporally Consistent Keypoint Discovery Chengliang Zhong Yuhang Zheng Yupeng Zheng Hao Zhao Li Yi ... Pengfei Li Guyue Zhou Chao Yang Xinliang Zhang Jian Zhao 3DPC 27 15 0 10 Sep 2023
GraspGPT: Leveraging Semantic Knowledge from a Large Language Model for Task-Oriented Grasping Chao Tang Dehao Huang Wenqiang Ge Weiyu Liu Hong Zhang 24 66 0 25 Jul 2023
One-shot Imitation Learning via Interaction Warping Ondrej Biza Skye Thompson Kishore Reddy Pagidi Abhinav Kumar Elise van der Pol Robin G. Walters Thomas Kipf Jan Willem van de Meent Lawson L. S. Wong Robert W. Platt 30 13 0 21 Jun 2023
Learning Any-View 6DoF Robotic Grasping in Cluttered Scenes via Neural Surface Rendering Snehal Jauhri Ishikaa Lunawat Georgia Chalvatzaki 32 10 0 12 Jun 2023
Computational Design of Passive Grippers Milin Kodnongbua Ian Good J. I. Lipton Adriana Schulz 23 10 0 05 Jun 2023
Self-Supervised Learning of Action Affordances as Interaction Modes Liquang Wang Nikita Dvornik R. Dubeau Mayank Mittal Animesh Garg SSL 26 3 0 27 May 2023
DeRi-Bot: Learning to Collaboratively Manipulate Rigid Objects via Deformable Objects Zixing Wang A. H. Qureshi 19 2 0 22 May 2023
The Treachery of Images: Bayesian Scene Keypoints for Deep Policy Learning in Robotic Manipulation Jan Ole von Hartz Eugenio Chisari Tim Welschehold Wolfram Burgard Joschka Boedecker Abhinav Valada SSL 24 13 0 08 May 2023
Recent Advances of Deep Robotic Affordance Learning: A Reinforcement Learning Perspective Xintong Yang Ze Ji Jing Wu Yunyu Lai 33 12 0 09 Mar 2023
Tactile Tool Manipulation Yuki Shirai Devesh K. Jha A. Raghunathan Dennis W. Hong 24 18 0 17 Jan 2023
Where To Start? Transferring Simple Skills to Complex Environments Vitalis Vosylius Edward Johns 19 12 0 12 Dec 2022
TAX-Pose: Task-Specific Cross-Pose Estimation for Robot Manipulation Chuer Pan Brian Okorn Harry Zhang Ben Eisner David Held 22 57 0 17 Nov 2022
ToolFlowNet: Robotic Manipulation with Tools via Predicting Tool Flow from Point Clouds Daniel Seita Yufei Wang Sarthak J. Shetty Edward Li Zackory M. Erickson David Held 3DPC 28 49 0 16 Nov 2022
Learning Category-Level Manipulation Tasks from Point Clouds with Dynamic Graph CNNs Junchi Liang Abdeslam Boularias 3DPC 27 8 0 13 Sep 2022
K-VIL: Keypoints-based Visual Imitation Learning Jianfeng Gao Z. Tao Noémie Jaquier Tamim Asfour VGen SSL 44 23 0 07 Sep 2022
Transformers are Adaptable Task Planners Vidhi Jain Yixin Lin Eric Undersander Yonatan Bisk Akshara Rai 22 24 0 06 Jul 2022
Understanding Physical Effects for Effective Tool-use Zeyu Zhang Ziyuan Jiao Weiqi Wang Yixin Zhu Song-Chun Zhu Hangxin Liu 27 12 0 30 Jun 2022
Learning Keypoints from Synthetic Data for Robotic Cloth Folding Thomas Lips Victor-Louis De Gusseme Francis Wyffels 16 11 0 13 May 2022
Fixing Malfunctional Objects With Learned Physical Simulation and Functional Prediction Yining Hong Kaichun Mo L. Yi Leonidas J. Guibas Antonio Torralba J. Tenenbaum Chuang Gan 24 5 0 05 May 2022
Creative Problem Solving in Artificially Intelligent Agents: A Survey and Framework Evana Gizzi Lakshmi Nair Sonia Chernova Jivko Sinapov AI4CE 19 13 0 21 Apr 2022
Semantic keypoint-based pose estimation from single RGB frames Karl Schmeckpeper Philip R. Osteen Yufu Wang Georgios Pavlakos Kenneth Chaney Wyatt Jordan Xiaowei Zhou Konstantinos G. Derpanis Kostas Daniilidis 3DH 10 6 0 12 Apr 2022
DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Tools Xingyu Lin Zhiao Huang Yunzhu Li J. Tenenbaum David Held Chuang Gan 24 72 0 31 Mar 2022
Generalizable task representation learning from human demonstration videos: a geometric approach Jun Jin Martin Jägersand 11 5 0 28 Feb 2022
End-to-end Reinforcement Learning of Robotic Manipulation with Robust Keypoints Representation Tianying Wang En Yen Puang Marcus Lee Y. Wu Wei Jing SSL 23 5 0 12 Feb 2022
You Only Demonstrate Once: Category-Level Manipulation from Single Visual Demonstration Bowen Wen Wenzhao Lian Kostas Bekris S. Schaal 14 89 0 30 Jan 2022