v1v2 (latest)

Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks

24 October 2018

Silvio Savarese

Li Fei-Fei

Papers citing "Making Sense of Vision and Touch: Self-Supervised Learning of Multimodal Representations for Contact-Rich Tasks"

50 / 219 papers shown

Title
Learning Haptic-based Object Pose Estimation for In-hand Manipulation Control with Underactuated Robotic HandsIEEE Transactions on Haptics (TOH), 2022 Osher Azulay Inbar Ben-David A. Sintov 156 14 0 06 Jul 2022
Imitation Learning for Nonprehensile Manipulation through Self-Supervised Learning Considering Motion SpeedIEEE Access (IEEE Access), 2022 Yuki Saigusa S. Sakaino T. Tsuji SSL 132 15 0 21 Jun 2022
Integrating High-Resolution Tactile Sensing into Grasp Stability PredictionCanadian Conference on Computer and Robot Vision (CRV), 2022 Lachlan Chumbley Morris Gu Rhys Newbury Jurgen Leitner Akansel Cosgun 78 2 0 12 Jun 2022
Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learning Maximilian Du Olivia Y. Lee Suraj Nair Chelsea Finn OffRL 158 38 0 30 May 2022
Learning Dense Reward with Temporal Variant Self-Supervision Yuning Wu Jieliang Luo Hui Li SSL 67 1 0 20 May 2022
Action Conditioned Tactile Prediction: case study on slip prediction Willow Mandil Kiyanoush Nazari E. AmirGhalamzan 130 19 0 19 May 2022
Learning A Simulation-based Visual Policy for Real-world Peg In Unseen HolesReview of Scientific Instruments (RSI), 2022 Liangru Xie Hongxiang Yu Kechun Xu Tong Yang Minhang Wang Haojian Lu R. Xiong Yue Wang 115 2 0 09 May 2022
Learning to Fill the Seam by Vision: Sub-millimeter Peg-in-hole on Unseen Shapes in Real WorldIEEE International Conference on Robotics and Automation (ICRA), 2022 Liang Xie Hongxiang Yu Yinghao Zhao Haodong Zhang Zhongxiang Zhou Minhang Wang Yue Wang R. Xiong 115 12 0 16 Apr 2022
Brainish: Formalizing A Multimodal Language for Intelligence and Consciousness Paul Pu Liang 206 7 0 14 Apr 2022
ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real TransferComputer Vision and Pattern Recognition (CVPR), 2022 Ruohan Gao Zilin Si Yen-Yu Chang Samuel Clarke Jeannette Bohg Li Fei-Fei Wenzhen Yuan Jiajun Wu 119 96 0 05 Apr 2022
Visuo-Haptic Object Perception for Robots: An OverviewAutonomous Robots (AR), 2022 Nicolás Navarro-Guerrero Sibel Toprak Josip Josifovski L. Jamone 151 51 0 22 Mar 2022
Multiscale Sensor Fusion and Continuous Control with Neural CDEsIEEE/RJS International Conference on Intelligent RObots and Systems (IROS), 2022 Sumeet Singh Francis McCann Ramirez Jacob Varley Andy Zeng Vikas Sindhwani 135 5 0 16 Mar 2022
Multi-View Dreaming: Multi-View World Model with Contrastive Learning Akira Kinose Masashi Okada Ryogo Okumura T. Taniguchi OffRL 90 11 0 15 Mar 2022
MIRROR: Differentiable Deep Social Projection for Assistive Human-Robot Communication Kaiqi Chen J. Fong Harold Soh 103 10 0 06 Mar 2022
High-Modality Multimodal Transformer: Quantifying Modality & Interaction Heterogeneity for High-Modality Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Jeffrey Tsaw Yudong Liu Shentong Mo Dani Yogatama Louis-Philippe Morency Ruslan Salakhutdinov 142 39 0 02 Mar 2022
InsertionNet 2.0: Minimal Contact Multi-Step Insertion Using Multimodal Multiview Sensory InputIEEE International Conference on Robotics and Automation (ICRA), 2022 Oren Spector Vladimir Tchuiev Dotan Di Castro 118 28 0 02 Mar 2022
COMPASS: Contrastive Multimodal Pretraining for Autonomous SystemsIEEE/RJS International Conference on Intelligent RObots and Systems (IROS), 2022 Shuang Ma Sai H. Vemprala Wenshan Wang Jayesh K. Gupta Yale Song Daniel J. McDuff Ashish Kapoor SSL 105 11 0 20 Feb 2022
Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic ManipulationIEEE Robotics and Automation Letters (RA-L), 2022 Rishabh Jangir Nicklas Hansen Sambaran Ghosal Mohit Jain Xiaolong Wang 264 78 0 19 Jan 2022
Contact-Rich Manipulation of a Flexible Object based on Deep Predictive Learning using Vision and Tactility Hideyuki Ichiwara Hiroshi Ito Kenjiro Yamamoto Hiroki Mori Tetsuya Ogata 122 25 0 13 Dec 2021
CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation TasksIEEE Robotics and Automation Letters (RA-L), 2021 Oier Mees Lukás Hermann Erick Rosete-Beas Wolfram Burgard LM&Ro 349 362 0 06 Dec 2021
Balancing Efficiency and Comfort in Robot-Assisted Bite TransferIEEE International Conference on Robotics and Automation (ICRA), 2021 Suneel Belkhale E. Gordon Yuxiao Chen S. Srinivasa Tapomayukh Bhattacharjee Dorsa Sadigh 151 23 0 22 Nov 2021
Learning Ultrasound Scanning Skills from Human DemonstrationsScience China Information Sciences (Sci China Inf Sci), 2021 Xutian Deng Ziwei Lei Yi Wang Miao Li 91 16 0 09 Nov 2021
Learning Robotic Ultrasound Scanning Skills via Human Demonstrations and Guided ExplorationsIEEE International Conference on Robotics and Biomimetics (ROBIO), 2021 Xutian Deng Yiting Chen Fei Chen Miao Li 100 29 0 02 Nov 2021
SEED: Series Elastic End Effectors in 6D for Visuotactile Tool UseIEEE/RJS International Conference on Intelligent RObots and Systems (IROS), 2021 H.J. Terry Suh Naveen Kuppuswamy Tao Pang P. Mitiguy A. Alspach Russ Tedrake 211 17 0 02 Nov 2021
Learning Robotic Manipulation Skills Using an Adaptive Force-Impedance Action Space Maximilian Ulmer Elie Aljalbout Sascha Schwarz Sami Haddadin 123 7 0 19 Oct 2021
Dual-Arm Adversarial Robot Learning Elie Aljalbout 86 2 0 15 Oct 2021
Seeking Visual Discomfort: Curiosity-driven Representations for Reinforcement Learning Elie Aljalbout Maximilian Ulmer Rudolph Triebel 104 2 0 02 Oct 2021
OSCAR: Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation J. Wong Viktor Makoviychuk Anima Anandkumar Yuke Zhu 127 15 0 02 Oct 2021
Making Curiosity Explicit in Vision-based RL Elie Aljalbout Maximilian Ulmer Rudolph Triebel OffRL 79 2 0 28 Sep 2021
Lifelong Robotic Reinforcement Learning by Retaining Experiences Annie Xie Chelsea Finn OffRL CLL 230 50 0 19 Sep 2021
ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations Ruohan Gao Yen-Yu Chang Shivani Mall Li Fei-Fei Jiajun Wu 228 91 0 16 Sep 2021
Dynamic Modeling of Hand-Object Interactions via Tactile SensingIEEE/RJS International Conference on Intelligent RObots and Systems (IROS), 2021 Qiang Zhang Yunzhu Li Yiyue Luo Wan Shou Michael Foshey Junchi Yan J. Tenenbaum Wojciech Matusik Antonio Torralba 102 22 0 09 Sep 2021
Exploring Data Aggregation and Transformations to Generalize across Visual Domains Antono DÍnnocente OOD 114 0 0 20 Aug 2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Zetian Wu Yun Cheng ... Peter Wu Michelle A. Lee Yuke Zhu Ruslan Salakhutdinov Louis-Philippe Morency VLM 174 212 0 15 Jul 2021
Multi-Modal Mutual Information (MuMMI) Training for Robust Self-Supervised Deep Reinforcement Learning Kaiqi Chen Yong Lee Harold Soh SSL 150 24 0 06 Jul 2021
Vision-driven Compliant Manipulation for Reliable, High-Precision Assembly Tasks A. S. Morgan Bowen Wen Junchi Liang Abdeslam Boularias A. Dollar Kostas Bekris 109 64 0 26 Jun 2021
panda-gym: Open-source goal-conditioned environments for robotic learning Quentin Gallouedec Nicolas Cazin Emmanuel Dellandrea Liming Chen OffRL 108 93 0 25 Jun 2021
What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments?International Conference on Machine Learning (ICML), 2021 Weijian Deng Stephen Gould Liang Zheng 151 69 0 10 Jun 2021
Planning Multimodal Exploratory Actions for Online Robot Attribute Learning Xiaohan Zhang Jivko Sinapov Shiqi Zhang OffRL 142 4 0 06 Jun 2021
Q-attention: Enabling Efficient Learning for Vision-based Robotic ManipulationIEEE Robotics and Automation Letters (RA-L), 2021 Stephen James Andrew J. Davison 170 135 0 31 May 2021
Composable Energy Policies for Reactive Motion Generation and Reinforcement Learning Julen Urain Anqi Li Puze Liu Carlo DÉramo Jan Peters 150 29 0 11 May 2021
InsertionNet -- A Scalable Solution for InsertionIEEE Robotics and Automation Letters (RA-L), 2021 Oren Spector Dotan Di Castro 97 60 0 29 Apr 2021
Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from DemonstrationsIEEE/RJS International Conference on Intelligent RObots and Systems (IROS), 2021 Trevor Ablett Yifan Zhai Jonathan Kelly SSL 154 6 0 28 Apr 2021
Tactile-RL for Insertion: Generalization to Objects of Unknown GeometryIEEE International Conference on Robotics and Automation (ICRA), 2021 Siyuan Dong Devesh K. Jha Diego Romeres Sangwoon Kim D. Nikovski Alberto Rodriguez 122 134 0 02 Apr 2021
Touch-based Curiosity for Sparse-Reward Tasks Sai Rajeswar Cyril Ibrahim Nitin Surya Florian Golemo David Vazquez Rameswar Panda Pedro H. O. Pinheiro 90 6 0 01 Apr 2021
Sim-to-Real for Robotic Tactile Sensing via Physics-Based Simulation and Learned Latent ProjectionsIEEE International Conference on Robotics and Automation (ICRA), 2021 Yashraj S. Narang Balakumar Sundaralingam Lukasz Wawrzyniak Arsalan Mousavian Dieter Fox 133 65 0 31 Mar 2021
Learning Deep Energy Shaping Policies for Stability-Guaranteed ManipulationIEEE Robotics and Automation Letters (RA-L), 2021 S. A. Khader Hang Yin Pietro Falco Danica Kragic 137 14 0 30 Mar 2021
Visionary: Vision architecture discovery for robot learningIEEE International Conference on Robotics and Automation (ICRA), 2021 Iretiayo Akinola A. Angelova Yao Lu Yevgen Chebotar Dmitry Kalashnikov Jacob Varley Julian Ibarz Michael S. Ryoo 120 10 0 26 Mar 2021
Preliminary Experimental Results of Context-Aware Teams of Multiple Autonomous Agents Operating under Constrained Communications J. Martinez-Lorenzo Jeffrey Hudack Yutao Jing Michael Shaham Zixuan Liang ... Juan Heredia Juesas Yuntao Ma Tristan Sweeney Nicolas Ares Ari Fox 136 0 0 25 Mar 2021
In-air Knotting of Rope using Dual-Arm Robot based on Deep LearningIEEE/RJS International Conference on Intelligent RObots and Systems (IROS), 2021 Kanata Suzuki Momomi Kanamura Y. Suga Hiroki Mori T. Ogata 182 24 0 17 Mar 2021