Wav2CLIP: Learning Robust Audio Representations From CLIP

21 October 2021

Papers citing "Wav2CLIP: Learning Robust Audio Representations From CLIP"

50 / 189 papers shown

Title
Hearing and Seeing Through CLIP: A Framework for Self-Supervised Sound Source Localization Sooyoung Park Arda Senocak Joon Son Chung VLM 43 0 0 08 May 2025
Synergy-CLIP: Extending CLIP with Multi-modal Integration for Robust Representation Learning Sangyeon Cho Jangyeong Jeon Mingi Kim Junyeong Kim CLIP VLM 76 0 0 30 Apr 2025
A Survey on Cross-Modal Interaction Between Music and Multimodal Data Sifei Li Mining Tan Feier Shen Minyan Luo Zijiao Yin Fan Tang W. Dong Changsheng Xu 57 0 0 17 Apr 2025
TARO: Timestep-Adaptive Representation Alignment with Onset-Aware Conditioning for Synchronized Video-to-Audio Synthesis Tri Ton Ji Woo Hong Chang D. Yoo VGen 24 0 0 08 Apr 2025
AVadCLIP: Audio-Visual Collaboration for Robust Video Anomaly Detection Peng Wu Wanshun Su Guansong Pang Yujia Sun Qingsen Yan Peng Wang Y. Zhang VLM 50 0 0 06 Apr 2025
SViQA: A Unified Speech-Vision Multimodal Model for Textless Visual Question Answering Bingxin Li 30 0 0 01 Apr 2025
A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives Shuyu Li Shulei Ji Zihao W. Wang Songruoyao Wu Jiaxing Yu K. Zhang MGen VGen 63 1 0 01 Apr 2025
Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMs Sanjoy Chowdhury Hanan Gani Nishit Anand Sayan Nag Ruohan Gao Mohamed Elhoseiny Salman Khan Dinesh Manocha LRM 40 0 0 29 Mar 2025
Continual Multimodal Contrastive Learning Xiaohao Liu Xiaobo Xia See-Kiong Ng Tat-Seng Chua CLL 57 0 0 19 Mar 2025
MACS: Multi-source Audio-to-image Generation with Contextual Significance and Semantic Alignment Hao Zhou Xiaobao Guo Yuzhe Zhu A. Kong DiffM 46 1 0 13 Mar 2025
Quality Over Quantity? LLM-Based Curation for a Data-Efficient Audio-Video Foundation Model Ali Vosoughi Dimitra Emmanouilidou H. Gamper 50 0 0 12 Mar 2025
FilmComposer: LLM-Driven Music Production for Silent Film Clips Zhifeng Xie Qile He Youjia Zhu Qiwei He Mengtian Li VGen 93 2 0 11 Mar 2025
ReelWave: A Multi-Agent Framework Toward Professional Movie Sound Generation Zixuan Wang Chi-Keung Tang Yu-Wing Tai DiffM VGen 58 0 0 10 Mar 2025
MM-StoryAgent: Immersive Narrated Storybook Video Generation with a Multi-Agent Paradigm across Text, Image and Audio Xuenan Xu Jiahao Mei Chenliang Li Yuning Wu M. Yan Shaopeng Lai J. Zhang Mengyue Wu VGen LLMAG 44 1 0 07 Mar 2025
Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities Sreyan Ghosh Zhifeng Kong Sonal Kumar S. Sakshi Jaehyeon Kim Wei Ping Rafael Valle Dinesh Manocha Bryan Catanzaro MLLM AuLLM LRM 49 5 0 06 Mar 2025
DGFM: Full Body Dance Generation Driven by Music Foundation Models Xinran Liu Zhenhua Feng Diptesh Kanojia Wenwu Wang DiffM 62 1 0 27 Feb 2025
GCDance: Genre-Controlled 3D Full Body Dance Generation Driven By Music Xinran Liu Xu Dong Diptesh Kanojia Wenwu Wang Zhenhua Feng DiffM 60 0 0 25 Feb 2025
Hyperdimensional Intelligent Sensing for Efficient Real-Time Audio Processing on Extreme Edge Sanggeon Yun Ryozo Masukawa Hanning Chen SungHeon Jeong Wenjun Huang Arghavan Rezvani Minhyoung Na Yoshiki Yamaguchi Mohsen Imani 47 1 0 15 Feb 2025
Audio-Language Datasets of Scenes and Events: A Survey Gijs Wijngaard Elia Formisano Michele Esposito M. Dumontier 79 2 0 10 Jan 2025
Smooth-Foley: Creating Continuous Sound for Video-to-Audio Generation Under Semantic Guidance Yaoyun Zhang Xuenan Xu Mengyue Wu VGen 26 0 0 24 Dec 2024
VinTAGe: Joint Video and Text Conditioning for Holistic Audio Generation Saksham Singh Kushwaha Yapeng Tian DiffM VGen 71 2 0 14 Dec 2024
A Decade of Deep Learning: A Survey on The Magnificent Seven Dilshod Azizov Muhammad Arslan Manzoor Velibor Bojkovic Yingxu Wang Z. Wang ... Liang Li Siwei Liu Yu Zhong Wei Liu Shangsong Liang OOD AI4TS MedIm 116 0 0 13 Dec 2024
Sound2Vision: Generating Diverse Visuals from Audio through Cross-Modal Latent Alignment Kim Sung-Bin Arda Senocak Hyunwoo Ha Tae-Hyun Oh DiffM 68 0 0 09 Dec 2024
Expanding Event Modality Applications through a Robust CLIP-Based Encoder SungHeon Jeong Hanning Chen Sanggeon Yun Suhyeon Cho Wenjun Huang Xiangjian Liu Mohsen Imani 98 1 0 04 Dec 2024
Gotta Hear Them All: Sound Source Aware Vision to Audio Generation Wei Guo Heng Wang Jianbo Ma Weidong Cai DiffM 85 3 0 23 Nov 2024
Robust AI-Synthesized Speech Detection Using Feature Decomposition Learning and Synthesizer Feature Augmentation Kuiyuan Zhang Zhongyun Hua Yushu Zhang Yifang Guo Tao Xiang 29 0 0 14 Nov 2024
Past, Present, and Future of Sensor-Based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task H. Haresamudram Chi Ian Tang Sungho Suh P. Lukowicz Thomas Ploetz 74 2 0 11 Nov 2024
ST-ITO: Controlling Audio Effects for Style Transfer with Inference-Time Optimization C. Steinmetz Shubhr Singh Marco Comunità Ilias Ibnyahya Shanxin Yuan Emmanouil Benetos Joshua Reiss 18 6 0 28 Oct 2024
PAT: Parameter-Free Audio-Text Aligner to Boost Zero-Shot Audio Classification Ashish Seth Ramaneswaran Selvakumar Sonal Kumar Sreyan Ghosh Dinesh Manocha VLM 35 0 0 19 Oct 2024
Enhancing Robustness in Deep Reinforcement Learning: A Lyapunov Exponent Approach Rory Young Nicolas Pugeault AAML 57 0 0 14 Oct 2024
Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition Zixuan Wang Chi-Keung Tang Chi-Keung Tang DiffM VGen LLMAG 41 4 0 04 Oct 2024
Bridging the Gap between Text, Audio, Image, and Any Sequence: A Novel Approach using Gloss-based Annotation Sen Fang Sizhou Chen Yalin Feng Xiaofeng Zhang T. Teoh 23 0 0 04 Oct 2024
Mamba Fusion: Learning Actions Through Questioning Zhikang Dong Apoorva Beedu Jason Sheinkopf Irfan Essa Mamba 60 2 0 17 Sep 2024
Augment, Drop & Swap: Improving Diversity in LLM Captions for Efficient Music-Text Representation Learning Ilaria Manco Justin Salamon Oriol Nieto 23 1 0 17 Sep 2024
Efficient Video to Audio Mapper with Visual Scene Detection Mingjing Yi Ming Li VGen 18 3 0 15 Sep 2024
ReCLAP: Improving Zero Shot Audio Classification by Describing Sounds Sreyan Ghosh Sonal Kumar Chandra Kiran Reddy Evuru Oriol Nieto R. Duraiswami Dinesh Manocha VLM 32 3 0 13 Sep 2024
Draw an Audio: Leveraging Multi-Instruction for Video-to-Audio Synthesis Qi Yang Binjie Mao Zili Wang Xing Nie Pengfei Gao Ying Guo Cheng Zhen Pengfei Yan Shiming Xiang VGen DiffM 30 5 0 10 Sep 2024
D&M: Enriching E-commerce Videos with Sound Effects by Key Moment Detection and SFX Matching Jingyu Liu Minquan Wang Ye Ma Bo Wang Aozhu Chen Quan Chen Peng Jiang Xirong Li 38 1 0 23 Aug 2024
Limitations in Employing Natural Language Supervision for Sensor-Based Human Activity Recognition -- And Ways to Overcome Them H. Haresamudram Apoorva Beedu Mashfiqui Rabbi Sankalita Saha Irfan Essa Thomas Ploetz 26 4 0 21 Aug 2024
BrewCLIP: A Bifurcated Representation Learning Framework for Audio-Visual Retrieval Zhenyu Lu Lakshay Sethi 45 0 0 19 Aug 2024
PSM: Learning Probabilistic Embeddings for Multi-scale Zero-Shot Soundscape Mapping Subash Khanal Eric Xing S. Sastry A. Dhakal Zhexiao Xiong Adeel Ahmad Nathan Jacobs 36 2 0 13 Aug 2024
VQ-CTAP: Cross-Modal Fine-Grained Sequence Representation Learning for Speech Processing Chunyu Qiang Wang Geng Yi Zhao Ruibo Fu Tao Wang ... Chen Zhang Hao Che Longbiao Wang Jianwu Dang Jianhua Tao AI4TS 36 0 0 11 Aug 2024
Towards Achieving Human Parity on End-to-end Simultaneous Speech Translation via LLM Agent Shanbo Cheng Zhichao Huang Tom Ko Hang Li Ningxin Peng Lu Xu Qini Zhang 48 3 0 31 Jul 2024
Aligning Sight and Sound: Advanced Sound Source Localization Through Audio-Visual Alignment Arda Senocak H. Ryu Junsik Kim Tae-Hyun Oh Hanspeter Pfister Joon Son Chung 24 3 0 18 Jul 2024
OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces Zehan Wang Ziang Zhang Hang Zhang Luping Liu Rongjie Huang Xize Cheng Hengshuang Zhao Zhou Zhao 30 9 0 16 Jul 2024
Unconstrained Open Vocabulary Image Classification: Zero-Shot Transfer from Text to Image via CLIP Inversion Philipp Allgeuer Kyra Ahrens Stefan Wermter CLIP VLM 27 3 0 15 Jul 2024
Masked Generative Video-to-Audio Transformers with Enhanced Synchronicity Santiago Pascual Chunghsin Yeh Ioannis Tsiamas Joan Serra DiffM VGen 28 15 0 15 Jul 2024
Meerkat: Audio-Visual Large Language Model for Grounding in Space and Time Sanjoy Chowdhury Sayan Nag Subhrajyoti Dasgupta Jun Chen Mohamed Elhoseiny Ruohan Gao Dinesh Manocha VLM MLLM 34 9 0 01 Jul 2024
FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Sounds Yiming Zhang Yicheng Gu Yanhong Zeng Zhening Xing Yuancheng Wang Zhizheng Wu Kai Chen VGen 23 35 0 01 Jul 2024
Bridging Language Gaps in Audio-Text Retrieval Zhiyong Yan Heinrich Dinkel Yongqing Wang Jizhong Liu Junbo Zhang Yujun Wang Bin Wang VLM 27 4 0 11 Jun 2024