Wav2CLIP: Learning Robust Audio Representations From CLIP

21 October 2021

Papers citing "Wav2CLIP: Learning Robust Audio Representations From CLIP"

39 / 189 papers shown

Title
BEATs: Audio Pre-Training with Acoustic Tokenizers Sanyuan Chen Yu-Huan Wu Chengyi Wang Shujie Liu Daniel C. Tompkins Zhuo Chen Furu Wei 30 253 0 18 Dec 2022
CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos Hao-Wen Dong Naoya Takahashi Yuki Mitsufuji Julian McAuley Taylor Berg-Kirkpatrick VLM CLIP 21 24 0 14 Dec 2022
LidarCLIP or: How I Learned to Talk to Point Clouds Georg Hess Adam Tonderski Christoffer Petersson Kalle AAstrom Lennart Svensson DiffM 19 22 0 13 Dec 2022
iQuery: Instruments as Queries for Audio-Visual Sound Separation Jiaben Chen Renrui Zhang Dongze Lian Jiaqi Yang Ziyao Zeng Jianbo Shi 16 26 0 07 Dec 2022
ClipCrop: Conditioned Cropping Driven by Vision-Language Model Zhihang Zhong Mingxi Cheng Zhirong Wu Yuhui Yuan Yinqiang Zheng Ji Li Han Hu Stephen Lin Yoichi Sato Imari Sato VLM CLIP 25 3 0 21 Nov 2022
TimbreCLIP: Connecting Timbre to Text and Images Nicolas Jonason Bob L. T. Sturm CLIP 22 4 0 21 Nov 2022
I Can't Believe There's No Images! Learning Visual Tasks Using only Language Supervision Sophia Gu Christopher Clark Aniruddha Kembhavi VLM 14 24 0 17 Nov 2022
Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation Yusong Wu K. Chen Tianyu Zhang Yuchen Hui Marianna Nezhurina Taylor Berg-Kirkpatrick Shlomo Dubnov CLIP 37 480 0 12 Nov 2022
I Hear Your True Colors: Image Guided Audio Generation Roy Sheffer Yossi Adi VLM 10 73 0 06 Nov 2022
On the Role of Visual Context in Enriching Music Representations Kleanthis Avramidis Shanti Stewart Shrikanth Narayanan 15 3 0 28 Oct 2022
Multimodal Contrastive Learning via Uni-Modal Coding and Cross-Modal Prediction for Multimodal Sentiment Analysis Ronghao Lin Haifeng Hu SSL 19 15 0 26 Oct 2022
IMU2CLIP: Multimodal Contrastive Learning for IMU Motion Sensors from Egocentric Videos and Text Seungwhan Moon Andrea Madotto Zhaojiang Lin Alireza Dirafzoon Aparajita Saraf Amy Bearman Babak Damavandi VLM 12 36 0 26 Oct 2022
SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model Yi-Jen Shih Hsuan-Fu Wang Heng-Jui Chang Layne Berry Hung-yi Lee David F. Harwath VLM CLIP 38 32 0 03 Oct 2022
Audio Retrieval with WavText5K and CLAP Training Soham Deshmukh Benjamin Elizalde Huaming Wang 3DV CLIP 113 50 0 28 Sep 2022
The Efficacy of Self-Supervised Speech Models for Audio Representations Tung-Yu Wu Chen An Li Tzu-Han Lin Tsung-Yuan Hsu Hung-yi Lee 24 5 0 26 Sep 2022
Robust Sound-Guided Image Manipulation Seung Hyun Lee Gyeongrok Oh Wonmin Byeon Sang Ho Yoon Jinkyu Kim Sangpil Kim DiffM 21 7 0 30 Aug 2022
Contrastive Audio-Language Learning for Music Ilaria Manco Emmanouil Benetos Elio Quinton Gyorgy Fazekas 25 44 0 25 Aug 2022
A Proposal for Foley Sound Synthesis Challenge Keunwoo Choi Sangshin Oh Minsung Kang Brian McFee 18 11 0 21 Jul 2022
Multimodal hierarchical Variational AutoEncoders with Factor Analysis latent space Alejandro Guerrero-López C. Sevilla-Salcedo Vanessa Gómez-Verdejo Pablo Martínez Olmos DRL 9 1 0 19 Jul 2022
Audio-guided Album Cover Art Generation with Genetic Algorithms James Marien Sam Leroux Bart Dhoedt Cedric De Boom 17 1 0 14 Jul 2022
OrdinalCLIP: Learning Rank Prompts for Language-Guided Ordinal Regression Wanhua Li Xiaoke Huang Zheng Hua Zhu Yansong Tang Xiu Li Jie Zhou Jiwen Lu 17 31 0 06 Jun 2022
Multimodal Knowledge Alignment with Reinforcement Learning Youngjae Yu Jiwan Chung Heeseung Yun Jack Hessel J. Park ... Prithviraj Ammanabrolu Rowan Zellers Ronan Le Bras Gunhee Kim Yejin Choi VLM 115 36 0 25 May 2022
Automated Audio Captioning: An Overview of Recent Progress and New Challenges Xinhao Mei Xubo Liu Mark D. Plumbley Wenwu Wang 21 37 0 12 May 2022
Learning Visual Styles from Audio-Visual Associations Tingle Li Yichen Liu Andrew Owens Hang Zhao DiffM 23 20 0 10 May 2022
Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representation Daisuke Niizumi Daiki Takeuchi Yasunori Ohishi N. Harada K. Kashino 24 65 0 26 Apr 2022
Sound-Guided Semantic Video Generation Seung Hyun Lee Gyeongrok Oh Wonmin Byeon Chanyoung Kim Wonjae Ryoo Sang Ho Yoon Hyunjun Cho Jihyun Bae Jinkyu Kim Sangpil Kim VGen 10 24 0 20 Apr 2022
VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidance Katherine Crowson Stella Biderman Daniel Kornis Dashiell Stander Eric Hallahan Louis Castricato Edward Raff CLIP 57 367 0 18 Apr 2022
BYOL for Audio: Exploring Pre-trained General-purpose Audio Representations Daisuke Niizumi Daiki Takeuchi Yasunori Ohishi N. Harada K. Kashino SSL 34 53 0 15 Apr 2022
How to Listen? Rethinking Visual Sound Localization Ho-Hsiang Wu Magdalena Fuentes Prem Seetharaman J. P. Bello ObjD 19 4 0 11 Apr 2022
Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language Andy Zeng Maria Attarian Brian Ichter K. Choromanski Adrian S. Wong ... Michael S. Ryoo Vikas Sindhwani Johnny Lee Vincent Vanhoucke Peter R. Florence ReLM LRM 8 569 0 01 Apr 2022
CMKD: CNN/Transformer-Based Cross-Model Knowledge Distillation for Audio Classification Yuan Gong Sameer Khurana Andrew Rouditchenko James R. Glass VLM 22 29 0 13 Mar 2022
HEAR: Holistic Evaluation of Audio Representations Joseph P. Turian Jordie Shier H. Khan Bhiksha Raj Björn W. Schuller ... P. Esling Pranay Manocha Shinji Watanabe Zeyu Jin Yonatan Bisk 31 99 0 06 Mar 2022
Music2Video: Automatic Generation of Music Video with fusion of audio and text Yoonjeon Kim Joel Jang Sumin Shin DiffM VGen 16 7 0 11 Jan 2022
Connecting the Dots between Audio and Text without Parallel Data through Visual Knowledge Transfer Yanpeng Zhao Jack Hessel Youngjae Yu Ximing Lu Rowan Zellers Yejin Choi 17 27 0 16 Dec 2021
Sound-Guided Semantic Image Manipulation Seung Hyun Lee Wonseok Roh Wonmin Byeon Sang Ho Yoon Chanyoung Kim Jinkyu Kim Sangpil Kim DiffM 16 43 0 30 Nov 2021
Creativity and Machine Learning: A Survey Giorgio Franceschelli Mirco Musolesi VLM AI4CE 19 38 0 06 Apr 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,764 0 24 Feb 2021
PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation Yuan Gong Yu-An Chung James R. Glass VLM 99 144 0 02 Feb 2021
Multi-task self-supervised learning for Robust Speech Recognition Mirco Ravanelli Jianyuan Zhong Santiago Pascual P. Swietojanski João Monteiro J. Trmal Yoshua Bengio SSL 171 288 0 25 Jan 2020