UAVM: Towards Unifying Audio and Visual Models

UAVM: Towards Unifying Audio and Visual Models

29 July 2022

Alexander H. Liu

Andrew Rouditchenko

Papers citing "UAVM: Towards Unifying Audio and Visual Models"

14 / 14 papers shown

Title
A Survey of Recent Advances and Challenges in Deep Audio-Visual Correlation Learning Luis Vilaca Yi Yu Paula Vinan 70 0 0 24 Nov 2024
Generalized Multimodal Fusion via Poisson-Nernst-Planck Equation Jiayu Xiong Jing Wang Hengjing Xiang Jun Xue Chen Xu Zhouqiang Jiang 22 0 0 20 Oct 2024
Audio Mamba: Bidirectional State Space Model for Audio Representation Learning Mehmet Hamza Erol Arda Senocak Jiu Feng Joon Son Chung Mamba 62 18 0 05 Jun 2024
Detail-Enhanced Intra- and Inter-modal Interaction for Audio-Visual Emotion Recognition Tong Shi Xuri Ge Joemon M. Jose Nicolas Pugeault Paul Henderson 25 0 0 26 May 2024
Triple Disentangled Representation Learning for Multimodal Affective Analysis Ying Zhou Xuefeng Liang Han Chen Yin Zhao Xin Chen Lida Yu 43 3 0 29 Jan 2024
Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities A. Piergiovanni Isaac Noble Dahun Kim Michael S. Ryoo Victor Gomes A. Angelova 30 19 0 09 Nov 2023
OmniVec: Learning robust representations with cross modal sharing Siddharth Srivastava Gaurav Sharma SSL 16 64 0 07 Nov 2023
Multimodal Fish Feeding Intensity Assessment in Aquaculture Meng Cui Xubo Liu Haohe Liu Zhuangzhuang Du Tao Chen Guoping Lian Daoliang Li Wenwu Wang 18 5 0 10 Sep 2023
AudioFormer: Audio Transformer learns audio feature representations from discrete acoustic codes Zhaohui Li Haitao Wang Xinghua Jiang 24 1 0 14 Aug 2023
Looking Similar, Sounding Different: Leveraging Counterfactual Cross-Modal Pairs for Audiovisual Representation Learning Nikhil Singh Chih-Wei Wu Iroro Orife Mahdi M. Kalayeh 23 2 0 12 Apr 2023
Self-Supervised Multimodal Learning: A Survey Yongshuo Zong Oisin Mac Aodha Timothy M. Hospedales SSL 16 43 0 31 Mar 2023
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing Junyi Ao Rui Wang Long Zhou Chengyi Wang Shuo Ren ... Yu Zhang Zhihua Wei Yao Qian Jinyu Li Furu Wei 110 192 0 14 Oct 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Yin Cui Boqing Gong ViT 240 573 0 22 Apr 2021
PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation Yuan Gong Yu-An Chung James R. Glass VLM 99 144 0 02 Feb 2021