Connecting Touch and Vision via Cross-Modal Prediction

14 June 2019

Yunzhu Li

Jun-Yan Zhu

Russ Tedrake

Antonio Torralba

ArXiv PDF HTML

Papers citing "Connecting Touch and Vision via Cross-Modal Prediction"

23 / 23 papers shown

Title
Sensor-Invariant Tactile Representation Harsh Gupta Yuchen Mo Shengmiao Jin Wenzhen Yuan 67 2 0 27 Feb 2025
AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors Ruoxuan Feng Jiangyu Hu Wenke Xia Tianci Gao Ao Shen Yuhao Sun Bin Fang Di Hu 52 5 0 15 Feb 2025
TextToucher: Fine-Grained Text-to-Touch Generation Jiahang Tu Hao Fu Fengyu Yang Hanbin Zhao Chao Zhang Hui Qian VLM DiffM 91 9 0 10 Jan 2025
Canonical Representation and Force-Based Pretraining of 3D Tactile for Dexterous Visuo-Tactile Policy Learning Tianhao Wu Jinzhou Li Jiyao Zhang Mingdong Wu Hao Dong SSL 44 6 0 26 Sep 2024
Multi-modal perception for soft robotic interactions using generative models Enrico Donato Egidio Falotico T. G. Thuruthel 39 2 0 05 Apr 2024
Multimodal Visual-Tactile Representation Learning through Self-Supervised Contrastive Pre-Training Vedant Dave Fotios Lygerakis Elmar Rueckert SSL 35 18 0 22 Jan 2024
Multi-Task Hypergraphs for Semi-supervised Learning using Earth Observations Mihai Cristian Pîrvu Alina Marcu A. Dobrescu N. Belbachir Marius Leordeanu 27 6 0 21 Aug 2023
Vis2Hap: Vision-based Haptic Rendering by Cross-modal Generation G. Cao Jiaqi Jiang N. Mao Danushka Bollegala Min Li Shan Luo 22 16 0 17 Jan 2023
See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation Hao Li Yizhi Zhang Junzhe Zhu Shaoxiong Wang Michelle A. Lee Huazhe Xu Edward H. Adelson Li Fei-Fei Ruohan Gao Jiajun Wu 32 60 0 07 Dec 2022
Learning Visuo-Haptic Skewering Strategies for Robot-Assisted Feeding Priya Sundaresan Suneel Belkhale Dorsa Sadigh 43 33 0 26 Nov 2022
Visuo-Tactile Transformers for Manipulation Yizhou Chen A. Sipos Mark Van der Merwe Nima Fazeli ViT 67 34 0 30 Sep 2022
Model Predictive Impedance Control with Gaussian Processes for Human and Environment Interaction Kevin Haninger C. Hegeler L. Peternel 32 7 0 15 Aug 2022
ARMANI: Part-level Garment-Text Alignment for Unified Cross-Modal Fashion Design Xujie Zhang Yuyang Sha Michael C. Kampffmeyer Zhenyu Xie Zequn Jie Chengwen Huang Jianqing Peng Xiaodan Liang 19 18 0 11 Aug 2022
Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learning Maximilian Du Olivia Y. Lee Suraj Nair Chelsea Finn OffRL 62 33 0 30 May 2022
Using Collocated Vision and Tactile Sensors for Visual Servoing and Localization A. N. Chaudhury Tim Man Wenzhen Yuan C. Atkeson 27 29 0 25 Apr 2022
Visual Pressure Estimation and Control for Soft Robotic Grippers Patrick Grady Jeremy Collins Samarth Brahmbhatt Christopher D. Twigg Chengcheng Tang James Hays Charles C. Kemp 24 8 0 14 Apr 2022
Multiscale Sensor Fusion and Continuous Control with Neural CDEs Sumeet Singh Francis McCann Ramirez Jacob Varley Andy Zeng Vikas Sindhwani 54 3 0 16 Mar 2022
Multi-Modal Fusion in Contact-Rich Precise Tasks via Hierarchical Policy Learning Piaopiao Jin Yi-Bin Lin Yanchao Tan Tiefeng Li Wei Yang 16 1 0 17 Feb 2022
Deep Tactile Experience: Estimating Tactile Sensor Output from Depth Sensor Data Karankumar Patel Soshi Iba Nawid Jamali 22 9 0 17 Oct 2021
Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent Projections Yashraj S. Narang Balakumar Sundaralingam Miles Macklin Arsalan Mousavian Dieter Fox 30 58 0 31 Mar 2021
Deep Learning and the Global Workspace Theory R. V. Rullen Ryota Kanai 45 66 0 04 Dec 2020
Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors Michelle A. Lee Matthew Tan Yuke Zhu Jeannette Bohg 49 25 0 01 Dec 2020
Making Sense of Vision and Touch: Learning Multimodal Representations for Contact-Rich Tasks Michelle A. Lee Yuke Zhu Peter Zachares Matthew Tan K. Srinivasan Silvio Savarese Fei-Fei Li Animesh Garg Jeannette Bohg SSL 23 208 0 28 Jul 2019