Learning to Separate Object Sounds by Watching Unlabeled Video

5 April 2018

Papers citing "Learning to Separate Object Sounds by Watching Unlabeled Video"

50 / 78 papers shown

Title
Reading to Listen at the Cocktail Party: Multi-Modal Speech Separation Akam Rahimi Triantafyllos Afouras Andrew Zisserman 50 28 0 02 Jan 2025
SoundLoc3D: Invisible 3D Sound Source Localization and Classification Using a Multimodal RGB-D Acoustic Camera Yuhang He Sangyun Shin Anoop Cherian Niki Trigoni Andrew Markham 93 0 0 31 Dec 2024
Robust Active Speaker Detection in Noisy Environments Siva Sai Nagender Vasireddy Chenxu Zhang Xiaohu Guo Yapeng Tian 45 0 0 27 Mar 2024
A Unified Audio-Visual Learning Framework for Localization, Separation, and Recognition Shentong Mo Pedro Morgado 38 21 0 30 May 2023
CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos Hao-Wen Dong Naoya Takahashi Yuki Mitsufuji Julian McAuley Taylor Berg-Kirkpatrick VLM CLIP 31 25 0 14 Dec 2022
Using Multiple Instance Learning to Build Multimodal Representations Peiqi Wang W. Wells Seth Berkowitz Steven Horng Polina Golland SSL 26 6 0 11 Dec 2022
iQuery: Instruments as Queries for Audio-Visual Sound Separation Jiaben Chen Renrui Zhang Dongze Lian Jiaqi Yang Ziyao Zeng Jianbo Shi 34 27 0 07 Dec 2022
A Closer Look at Weakly-Supervised Audio-Visual Source Localization Shentong Mo Pedro Morgado 85 64 0 30 Aug 2022
Learning in Audio-visual Context: A Review, Analysis, and New Perspective Yake Wei Di Hu Yapeng Tian Xuelong Li 46 55 0 20 Aug 2022
AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separation Efthymios Tzinis Scott Wisdom Tal Remez J. Hershey 46 30 0 20 Jul 2022
Audio-Visual Segmentation Jinxing Zhou Jianyuan Wang Jingyang Zhang Weixuan Sun Jing Zhang Stan Birchfield Dan Guo Lingpeng Kong Meng Wang Yiran Zhong VOS 33 109 0 11 Jul 2022
Finding Fallen Objects Via Asynchronous Audio-Visual Integration Chuang Gan Yi Gu Siyuan Zhou Jeremy Schwartz S. Alter James Traer Dan Gutfreund J. Tenenbaum Josh H. McDermott Antonio Torralba 57 19 0 07 Jul 2022
Weakly-Supervised Action Detection Guided by Audio Narration Keren Ye Adriana Kovashka 43 0 0 12 May 2022
Sound Localization by Self-Supervised Time Delay Estimation Ziyang Chen David Fouhey Andrew Owens SSL 32 19 0 26 Apr 2022
The Sound of Bounding-Boxes Takashi Oya Shohei Iwase Shigeo Morishima 19 2 0 30 Mar 2022
Learning to Answer Questions in Dynamic Audio-Visual Scenarios Guangyao Li Yake Wei Yapeng Tian Chenliang Xu Ji-Rong Wen Di Hu 39 136 0 26 Mar 2022
Localizing Visual Sounds the Easy Way Shentong Mo Pedro Morgado 35 78 0 17 Mar 2022
Visually Supervised Speaker Detection and Localization via Microphone Array Davide Berghi A. Hilton Philip J. B. Jackson 27 11 0 07 Mar 2022
Listen to Interpret: Post-hoc Interpretability for Audio Networks with NMF Jayneel Parekh Sanjeel Parekh Pavlo Mozharovskyi Florence dÁlché-Buc G. Richard 24 22 0 23 Feb 2022
Active Audio-Visual Separation of Dynamic Sound Sources Sagnik Majumder Kristen Grauman 32 21 0 02 Feb 2022
Egocentric Deep Multi-Channel Audio-Visual Active Speaker Localization Hao Jiang Calvin Murdock V. Ithapu EgoV 36 41 0 06 Jan 2022
Sound and Visual Representation Learning with Multiple Pretraining Tasks A. Vasudevan Dengxin Dai Luc Van Gool SSL 38 6 0 04 Jan 2022
Soundify: Matching Sound Effects to Video David Chuan-En Lin Anastasis Germanidis Cristobal Valenzuela Yining Shi Nikolas Martelaro 30 16 0 17 Dec 2021
Geometry-Aware Multi-Task Learning for Binaural Audio Generation from Video Rishabh Garg Ruohan Gao Kristen Grauman 15 28 0 21 Nov 2021
Beyond Mono to Binaural: Generating Binaural Audio from Mono Audio with Depth and Cross Modal Attention Kranti K. Parida Siddharth Srivastava Gaurav Sharma MDE 38 20 0 15 Nov 2021
The Impact of Spatiotemporal Augmentations on Self-Supervised Audiovisual Representation Learning Haider Al-Tahan Y. Mohsenzadeh SSL AI4TS 39 0 0 13 Oct 2021
Ego4D: Around the World in 3,000 Hours of Egocentric Video Kristen Grauman Andrew Westbury Eugene Byrne Zachary Chavis Antonino Furnari ... Mike Zheng Shou Antonio Torralba Lorenzo Torresani Mingfei Yan Jitendra Malik EgoV 281 1,026 0 13 Oct 2021
Visual Scene Graphs for Audio Source Separation Moitreya Chatterjee Jonathan Le Roux Narendra Ahuja A. Cherian 26 36 0 24 Sep 2021
Temporal Knowledge Consistency for Unsupervised Visual Representation Learning Wei Feng Yuanjiang Wang Lihua Ma Ye Yuan Chi Zhang SSL 21 13 0 24 Aug 2021
The Right to Talk: An Audio-Visual Transformer Approach Thanh-Dat Truong C. Duong T. D. Vu H. Pham Bhiksha Raj Ngan Le Khoa Luu 63 36 0 06 Aug 2021
Visually Guided Sound Source Separation and Localization using Self-Supervised Motion Representations Lingyu Zhu Esa Rahtu 29 25 0 17 Apr 2021
Visually Informed Binaural Audio Generation without Binaural Audios Xudong Xu Hang Zhou Ziwei Liu Bo Dai Xiaogang Wang Dahua Lin DiffM 13 55 0 13 Apr 2021
Cyclic Co-Learning of Sounding Object Visual Grounding and Sound Separation Yapeng Tian Di Hu Chenliang Xu ObjD 21 88 0 05 Apr 2021
Can audio-visual integration strengthen robustness under multimodal attacks? Yapeng Tian Chenliang Xu AAML 38 37 0 05 Apr 2021
Unsupervised Sound Localization via Iterative Contrastive Learning Yan-Bo Lin Hung-Yu Tseng Hsin-Ying Lee Yen-Yu Lin Ming-Hsuan Yang SSL 32 34 0 01 Apr 2021
Robust Audio-Visual Instance Discrimination Pedro Morgado Ishan Misra Nuno Vasconcelos SSL 22 110 0 29 Mar 2021
Beyond Image to Depth: Improving Depth Prediction using Echoes Kranti K. Parida Siddharth Srivastava Gaurav Sharma MDE 50 37 0 15 Mar 2021
Learning Audio-Visual Correlations from Variational Cross-Modal Generation Ye Zhu Yu Wu Hugo Latapie Yi Yang Yan Yan SSL 44 20 0 05 Feb 2021
Music source separation conditioned on 3D point clouds Francesc Lluís V. Chatziioannou A. Hofmann 3DPC 24 5 0 03 Feb 2021
VisualVoice: Audio-Visual Speech Separation with Cross-Modal Consistency Ruohan Gao Kristen Grauman CVBM 196 199 0 08 Jan 2021
Semantic Audio-Visual Navigation Changan Chen Ziad Al-Halah Kristen Grauman 50 104 0 21 Dec 2020
Visual Speech Enhancement Without A Real Visual Stream Sindhu B. Hegde Prajwal K R Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar DiffM 20 17 0 20 Dec 2020
Learning Representations from Audio-Visual Spatial Alignment Pedro Morgado Yi Li Nuno Vasconcelos SSL 27 121 0 03 Nov 2020
Into the Wild with AudioScope: Unsupervised Audio-Visual Separation of On-Screen Sounds Efthymios Tzinis Scott Wisdom A. Jansen Shawn Hershey Tal Remez D. Ellis J. Hershey 39 69 0 02 Nov 2020
Audio-Visual Event Localization via Recursive Fusion by Joint Co-Attention Bin Duan Hao Tang Wei Wang Ziliang Zong Guowei Yang Yan Yan 33 59 0 14 Aug 2020
Self-Supervised Learning of Audio-Visual Objects from Video Triantafyllos Afouras Andrew Owens Joon Son Chung Andrew Zisserman SSL 19 253 0 10 Aug 2020
Unified Multisensory Perception: Weakly-Supervised Audio-Visual Video Parsing Yapeng Tian Dingzeyu Li Chenliang Xu 34 181 0 21 Jul 2020
Sep-Stereo: Visually Guided Stereophonic Audio Generation by Associating Source Separation Hang Zhou Xudong Xu Dahua Lin Xiaogang Wang Ziwei Liu DiffM 32 81 0 20 Jul 2020
Multiple Sound Sources Localization from Coarse to Fine Rui Qian Di Hu Heinrich Dinkel Mengyue Wu N. Xu Weiyao Lin 28 155 0 13 Jul 2020
AVLnet: Learning Audio-Visual Language Representations from Instructional Videos Andrew Rouditchenko Angie Boggust David Harwath Brian Chen D. Joshi ... Rogerio Feris Brian Kingsbury M. Picheny Antonio Torralba James R. Glass SSL 22 141 0 16 Jun 2020