ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations

16 September 2021

Ruohan Gao

Yen-Yu Chang

Shivani Mall

Li Fei-Fei

Jiajun Wu

ArXiv PDF HTML

Papers citing "ObjectFolder: A Dataset of Objects with Implicit Visual, Auditory, and Tactile Representations"

50 / 55 papers shown

Title
Modality Selection and Skill Segmentation via Cross-Modality Attention Jiawei Jiang Kei Ota Devesh K. Jha Asako Kanezaki 22 0 0 20 Apr 2025
Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Dataset Z. Dong Ka Chen Zhaoyang Lv Hong-Xing Yu Yunzhi Zhang ... Xiaqing Pan Mingfei Yan Jiajun Wu Carl Ren Richard A. Newcombe 39 1 0 11 Apr 2025
X-Capture: An Open-Source Portable Device for Multi-Sensory Learning Samuel Clarke Suzannah Wistreich Yanjie Ze Jiajun Wu 31 0 0 03 Apr 2025
Visual Acoustic Fields Yuelei Li Hyunjin Kim Fangneng Zhan Ri-Zhao Qiu Mazeyu Ji Xiaojun Shan Xueyan Zou Paul Liang Hanspeter Pfister Xiaolong Wang 45 0 0 31 Mar 2025
AnyTouch: Learning Unified Static-Dynamic Representation across Multiple Visuo-tactile Sensors Ruoxuan Feng Jiangyu Hu Wenke Xia Tianci Gao Ao Shen Yuhao Sun Bin Fang Di Hu 42 3 0 15 Feb 2025
TextToucher: Fine-Grained Text-to-Touch Generation Jiahang Tu Hao Fu Fengyu Yang Hanbin Zhao Chao Zhang Hui Qian VLM DiffM 78 7 0 10 Jan 2025
Sparsh: Self-supervised touch representations for vision-based tactile sensing Carolina Higuera Akash Sharma Chaithanya Krishna Bodduluri Taosha Fan Patrick E. Lancaster ... Michael Kaess Byron Boots Mike Lambeta Tingfan Wu Mustafa Mukadam 32 11 0 31 Oct 2024
DiffSound: Differentiable Modal Sound Rendering and Inverse Rendering for Diverse Inference Tasks Xutong Jin Chenxi Xu Ruohan Gao Jiajun Wu Guoping Wang Sheng Li DiffM 33 2 0 20 Sep 2024
Play to the Score: Stage-Guided Dynamic Multi-Sensory Fusion for Robotic Manipulation Runze Yuan Tao Liu Wenke Ma Xuelong Li 32 7 0 02 Aug 2024
A Case Study on Visual-Audio-Tactile Cross-Modal Retrieval Jagoda Wojcik Jiaqi Jiang Jiacheng Wu Shan Luo 16 1 0 30 Jul 2024
Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI Y. Liu Weixing Chen Yongjie Bai Xiaodan Liang Guanbin Li Wen Gao Liang Lin LM&Ro SyDa AI4CE 48 47 0 09 Jul 2024
SonicSense: Object Perception from In-Hand Acoustic Vibration Jiaxun Liu Boyuan Chen 29 2 0 25 Jun 2024
NeST: Neural Stress Tensor Tomography by leveraging 3D Photoelasticity Akshat Dave Tianyi Zhang Aaron Young Ramesh Raskar Wolfgang Heidrich Ashok Veeraraghavan 26 2 0 14 Jun 2024
Touch100k: A Large-Scale Touch-Language-Vision Dataset for Touch-Centric Multimodal Representation Ning Cheng Changhao Guan Jing Gao Weihao Wang You Li Fandong Meng Jie Zhou Bin Fang Jinan Xu Wenjuan Han VLM 28 8 0 06 Jun 2024
Tactile-Augmented Radiance Fields Yiming Dou Fengyu Yang Yi Liu Antonio Loquercio Andrew Owens 26 18 0 07 May 2024
Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learning Entong Su Chengzhe Jia Yuzhe Qin Wenxuan Zhou Annabella Macaluso Binghao Huang Xiaolong Wang 28 4 0 18 Mar 2024
A survey of synthetic data augmentation methods in computer vision A. Mumuni F. Mumuni N. K. Gerrar 34 19 0 15 Mar 2024
Towards Comprehensive Multimodal Perception: Introducing the Touch-Language-Vision Dataset Ning Cheng You Li Jing Gao Bin Fang Jinan Xu Wenjuan Han 39 4 0 14 Mar 2024
A Touch, Vision, and Language Dataset for Multimodal Alignment Letian Fu Gaurav Datta Huang Huang Will Panitch Jaimyn Drake Joseph Ortiz Mustafa Mukadam Mike Lambeta Roberto Calandra Ken Goldberg VLM 25 32 0 20 Feb 2024
Binding Touch to Everything: Learning Unified Multimodal Tactile Representations Fengyu Yang Chao Feng Ziyang Chen Hyoungseob Park Daniel Wang ... Ziyao Zeng Xien Chen Rit Gangopadhyay Andrew Owens Alex Wong 36 53 0 31 Jan 2024
MultiPLY: A Multisensory Object-Centric Embodied Large Language Model in 3D World Yining Hong Zishuo Zheng Peihao Chen Yian Wang Junyan Li Chuang Gan 13 32 0 16 Jan 2024
How to Train Neural Field Representations: A Comprehensive Study and Benchmark Samuele Papa Riccardo Valperga David M. Knigge Miltiadis Kofinas Phillip Lippe J. Sonke E. Gavves 27 7 0 16 Dec 2023
Scene-Driven Multimodal Knowledge Graph Construction for Embodied AI Yaoxian Song Penglei Sun Haoyu Liu Li Zhixu Wei Song Yanghua Xiao Xiaofang Zhou LM&Ro 51 13 0 07 Nov 2023
Open X-Embodiment: Robotic Learning Datasets and RT-X Models Open X-Embodiment Collaboration Abby OÑeill Abdul Rehman Abhinav Gupta Abhiram Maddukuri ... Zhuo Xu Zichen Jeff Cui Zichen Zhang Zipeng Fu Zipeng Lin LM&Ro 42 457 0 13 Oct 2023
Generating Visual Scenes from Touch Fengyu Yang Jiacheng Zhang Andrew Owens DiffM 22 24 0 26 Sep 2023
PEANUT: A Human-AI Collaborative Tool for Annotating Audio-Visual Data Zheng Zhang Zheng Ning Chenliang Xu Yapeng Tian Toby Jia-Jun Li 56 6 0 27 Jul 2023
M3Act: Learning from Synthetic Human Group Activities Che-Jui Chang Danrui Li Aditya Bhat Parth Goel Honglu Zhou Seonghyeon Moon Samuel S. Sohn Sejong Yoon Vladimir Pavlovic Mubbasir Kapadia DiffM 10 5 0 29 Jun 2023
RealImpact: A Dataset of Impact Sound Fields for Real Objects Samuel Clarke Ruohan Gao Mason Wang M. Rau Julia Xu Jui-Hsien Wang Doug L. James Jiajun Wu 25 9 0 16 Jun 2023
The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objects Ruohan Gao Yiming Dou Hao Li Tanmay Agarwal Jeannette Bohg Yunzhu Li Li Fei-Fei Jiajun Wu 11 29 0 01 Jun 2023
Sonicverse: A Multisensory Simulation Platform for Embodied Household Agents that See and Hear Ruohan Gao Hao Li Gokul Dharan Zhuzhu Wang Chengshu Li Fei Xia Silvio Savarese Li Fei-Fei Jiajun Wu 26 11 0 01 Jun 2023
An Extensible Multimodal Multi-task Object Dataset with Materials Trevor Scott Standley Ruohan Gao Dawn Chen Jiajun Wu Silvio Savarese 14 1 0 29 Apr 2023
Learning Object-Centric Neural Scattering Functions for Free-Viewpoint Relighting and Scene Composition Hong-Xing Yu Michelle Guo Alireza Fathi Yen-Yu Chang E. R. Chan Ruohan Gao Thomas Funkhouser Jiajun Wu 28 20 0 10 Mar 2023
ROAD: Learning an Implicit Recursive Octree Auto-Decoder to Efficiently Encode 3D Shapes Sergey Zakharov Rares Ambrus Katherine Liu Adrien Gaidon 3DV 20 4 0 12 Dec 2022
MIRA: Mental Imagery for Robotic Affordances Lin Yen-Chen Pete Florence Andy Zeng Jonathan T. Barron Yilun Du Wei-Chiu Ma Anthony Simeonov Alberto Rodriguez Garcia Phillip Isola LM&Ro 21 33 0 12 Dec 2022
See, Hear, and Feel: Smart Sensory Fusion for Robotic Manipulation Hao Li Yizhi Zhang Junzhe Zhu Shaoxiong Wang Michelle A. Lee Huazhe Xu Edward H. Adelson Li Fei-Fei Ruohan Gao Jiajun Wu 13 58 0 07 Dec 2022
Canonical Fields: Self-Supervised Learning of Pose-Canonicalized Neural Fields Rohith Agaram Shaurya Dewan Rahul Sajnani A. Poulenard Madhava Krishna Srinath Sridhar 28 6 0 05 Dec 2022
Touch and Go: Learning from Human-Collected Vision and Touch Fengyu Yang Chenyang Ma Jiacheng Zhang Jing Zhu Wenzhen Yuan Andrew Owens 17 51 0 22 Nov 2022
SE(3)-Equivariant Relational Rearrangement with Neural Descriptor Fields Anthony Simeonov Yilun Du Lin Yen-Chen Alberto Rodriguez L. Kaelbling Tomas Lozano-Perez Pulkit Agrawal 21 52 0 17 Nov 2022
Physics-Informed Machine Learning: A Survey on Problems, Methods and Applications Zhongkai Hao Songming Liu Yichi Zhang Chengyang Ying Yao Feng Hang Su Jun Zhu PINN AI4CE 21 87 0 15 Nov 2022
Differentiable Physics Simulation of Dynamics-Augmented Neural Objects Simon Le Cleac'h Hong-Xing Yu Michelle Guo Taylor A. Howell Ruohan Gao Jiajun Wu Zachary Manchester Mac Schwager AI4CE 17 39 0 17 Oct 2022
Enhancing Generalizable 6D Pose Tracking of an In-Hand Object with Tactile Sensing Yun-Hai Liu Xiaomeng Xu Weihang Chen Hao Yuan He Wang Jing Xu Rui Chen Li Yi 58 10 0 08 Oct 2022
Impact Makes a Sound and Sound Makes an Impact: Sound Guides Representations and Explorations Xufeng Zhao C. Weber Muhammad Burhan Hafez S. Wermter 18 7 0 04 Aug 2022
Robust Change Detection Based on Neural Descriptor Fields Jiahui Fu Yilun Du Kurran Singh J. Tenenbaum J. Leonard 9 10 0 01 Aug 2022
Finding Fallen Objects Via Asynchronous Audio-Visual Integration Chuang Gan Yi Gu Siyuan Zhou Jeremy Schwartz S. Alter James Traer Dan Gutfreund J. Tenenbaum Josh H. McDermott Antonio Torralba 40 19 0 07 Jul 2022
HM3D-ABO: A Photo-realistic Dataset for Object-centric Multi-view 3D Reconstruction Zhenpei Yang Zaiwei Zhang Qi-Xing Huang 3DH 20 4 0 24 Jun 2022
Discretization Invariant Networks for Learning Maps between Neural Fields Clinton Jia Wang Polina Golland 17 0 0 02 Jun 2022
Play it by Ear: Learning Skills amidst Occlusion through Audio-Visual Imitation Learning Maximilian Du Olivia Y. Lee Suraj Nair Chelsea Finn OffRL 47 32 0 30 May 2022
Multimodal Conversational AI: A Survey of Datasets and Approaches Anirudh S. Sundar Larry Heck 30 29 0 13 May 2022
ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer Ruohan Gao Zilin Si Yen-Yu Chang Samuel Clarke Jeannette Bohg Li Fei-Fei Wenzhen Yuan Jiajun Wu 19 81 0 05 Apr 2022
PACS: A Dataset for Physical Audiovisual CommonSense Reasoning Samuel Yu Peter Wu Paul Pu Liang Ruslan Salakhutdinov Louis-Philippe Morency LRM 23 13 0 21 Mar 2022