Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representation

26 April 2022

Papers citing "Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representation"

47 / 47 papers shown

Title
Myna: Masking-Based Contrastive Learning of Musical Representations Ori Yonay Tracy Hammond Tianbao Yang AAML 51 0 0 20 Feb 2025
Music Foundation Model as Generic Booster for Music Downstream Tasks Weihsiang Liao Yuhta Takida Yukara Ikemiya Zhi-Wei Zhong Chieh-Hsin Lai ... Stefan Uhlich Taketo Akama Woosung Choi Yuichiro Koyama Yuki Mitsufuji 43 0 0 02 Nov 2024
OpenMU: Your Swiss Army Knife for Music Understanding Mengjie Zhao Zhi-Wei Zhong Zhuoyuan Mao Shiqi Yang Wei-Hsiang Liao Shusuke Takahashi Hiromi Wakaki Yuki Mitsufuji OSLM 45 4 0 21 Oct 2024
Contrasting Deep Learning Models for Direct Respiratory Insufficiency Detection Versus Blood Oxygen Saturation Estimation M. Gauy Natalia Hitomi Koza Ricardo Mikio Morita Gabriel Rocha Stanzione Arnaldo Cândido Júnior L. Berti A. S. Levin E. Sabino F. Svartman Marcelo Finger 33 0 0 30 Jul 2024
SpecMaskGIT: Masked Generative Modeling of Audio Spectrograms for Efficient Audio Synthesis and Beyond Marco Comunità Zhi-Wei Zhong Akira Takahashi Shiqi Yang Mengjie Zhao Koichi Saito Yukara Ikemiya Takashi Shibuya Shusuke Takahashi Yuki Mitsufuji 47 2 0 25 Jun 2024
Scaling up masked audio encoder learning for general audio classification Heinrich Dinkel Zhiyong Yan Yongqing Wang Junbo Zhang Yujun Wang Bin Wang 22 2 0 11 Jun 2024
M2D-CLAP: Masked Modeling Duo Meets CLAP for Learning General-purpose Audio-Language Representation Daisuke Niizumi Daiki Takeuchi Yasunori Ohishi Noboru Harada Masahiro Yasuda Shunsuke Tsubaki Keisuke Imoto VLM 31 5 0 04 Jun 2024
Investigating Design Choices in Joint-Embedding Predictive Architectures for General Audio Representation Learning Alain Riou Stefan Lattner Gaëtan Hadjeres Geoffroy Peeters 21 2 0 14 May 2024
FairSSD: Understanding Bias in Synthetic Speech Detectors Amit Kumar Singh Yadav Kratika Bhagtani Davide Salvi Paolo Bestagini Edward J.Delp 24 5 0 17 Apr 2024
Guided Masked Self-Distillation Modeling for Distributed Multimedia Sensor Event Analysis Masahiro Yasuda Noboru Harada Yasunori Ohishi Shoichiro Saito Akira Nakayama Nobutaka Ono 29 3 0 12 Apr 2024
Masked Modeling Duo: Towards a Universal Audio Pre-training Framework Daisuke Niizumi Daiki Takeuchi Yasunori Ohishi Noboru Harada K. Kashino 29 10 0 09 Apr 2024
uaMix-MAE: Efficient Tuning of Pretrained Audio Transformers with Unsupervised Audio Mixtures Afrina Tabassum Dung N. Tran Trung D. Q. Dang Ismini Lourentzou K. Koishida 32 0 0 14 Mar 2024
Compression Robust Synthetic Speech Detection Using Patched Spectrogram Transformer Amit Kumar Singh Yadav Ziyue Xiang Kratika Bhagtani Paolo Bestagini Stefano Tubaro Edward J. Delp ViT 35 2 0 22 Feb 2024
Masked Audio Modeling with CLAP and Multi-Objective Learning Yifei Xin Xiulian Peng Yan Lu 42 8 0 29 Jan 2024
From Coarse to Fine: Efficient Training for Audio Spectrogram Transformers Jiu Feng Mehmet Hamza Erol Joon Son Chung Arda Senocak 16 1 0 16 Jan 2024
EAT: Self-Supervised Pre-Training with Efficient Audio Transformer Wenxi Chen Yuzhe Liang Ziyang Ma Zhisheng Zheng Xie Chen ViT 35 17 0 07 Jan 2024
SAIC: Integration of Speech Anonymization and Identity Classification Ming Cheng Xingjian Diao Shitong Cheng Wenjun Liu 34 6 0 23 Dec 2023
A-JEPA: Joint-Embedding Predictive Architecture Can Listen Zhengcong Fei Mingyuan Fan Junshi Huang 21 17 0 27 Nov 2023
Joint Music and Language Attention Models for Zero-shot Music Tagging Xingjian Du Zhesong Yu Jiaju Lin Bilei Zhu Qiuqiang Kong BDL VLM 33 8 0 16 Oct 2023
Test-Time Training for Speech Sri Harsha Dumpala Chandramouli Shama Sastry Sageev Oore 25 1 0 19 Sep 2023
EnCodecMAE: Leveraging neural codecs for universal audio representation learning L. Pepino Pablo Riera Luciana Ferrer 16 4 0 14 Sep 2023
Example-Based Framework for Perceptually Guided Audio Texture Generation Purnima Kamath Chitralekha Gupta L. Wyse Suranga Nanayakkara 11 4 0 23 Aug 2023
AudioFormer: Audio Transformer learns audio feature representations from discrete acoustic codes Zhaohui Li Haitao Wang Xinghua Jiang 24 1 0 14 Aug 2023
FlexiAST: Flexibility is What AST Needs Jiu Feng Mehmet Hamza Erol Joon Son Chung Arda Senocak 16 3 0 18 Jul 2023
On Frequency-Wise Normalizations for Better Recording Device Generalization in Audio Spectrogram Transformers Paul Primus Gerhard Widmer 14 0 0 20 Jun 2023
Speaker Embeddings as Individuality Proxy for Voice Stress Detection Zihan Wu Neil Scheidwasser Karl El Hajal Milos Cernak 24 3 0 09 Jun 2023
Self-supervised Audio Teacher-Student Transformer for Both Clip-level and Frame-level Tasks Xian Li Nian Shao Xiaofei Li ViT CLIP 10 25 0 07 Jun 2023
Masked Autoencoders with Multi-Window Local-Global Attention Are Better Audio Learners Sarthak Yadav Sergios Theodoridis Lars Kai Hansen Z. Tan 15 7 0 01 Jun 2023
Streaming Audio Transformers for Online Audio Tagging Heinrich Dinkel Zhiyong Yan Yongqing Wang Junbo Zhang Yujun Wang Bin Wang 19 4 0 29 May 2023
Masked Modeling Duo for Speech: Specializing General-Purpose Audio Representation to Speech using Denoising Distillation Daisuke Niizumi Daiki Takeuchi Yasunori Ohishi Noboru Harada K. Kashino 37 3 0 23 May 2023
Extending Audio Masked Autoencoders Toward Audio Restoration Zhi-Wei Zhong Hao Shi M. Hirano Kazuki Shimada Kazuya Tateishi Takashi Shibuya Shusuke Takahashi Yuki Mitsufuji 24 4 0 11 May 2023
DSVAE: Interpretable Disentangled Representation for Synthetic Speech Detection Amit Kumar Singh Yadav Kratika Bhagtani Ziyue Xiang Paolo Bestagini Stefano Tubaro Edward J. Delp DRL 21 6 0 06 Apr 2023
VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking Limin Wang Bingkun Huang Zhiyu Zhao Zhan Tong Yinan He Yi Wang Yali Wang Yu Qiao VGen 44 324 0 29 Mar 2023
Low-Complexity Audio Embedding Extractors Florian Schmid Khaled Koutini Gerhard Widmer 11 4 0 03 Mar 2023
Aerial Image Object Detection With Vision Transformer Detector (ViTDet) Liya Wang A. Tien 30 7 0 28 Jan 2023
BEATs: Audio Pre-Training with Acoustic Tokenizers Sanyuan Chen Yu-Huan Wu Chengyi Wang Shujie Liu Daniel C. Tompkins Zhuo Chen Furu Wei 22 253 0 18 Dec 2022
CLIPPO: Image-and-Language Understanding from Pixels Only Michael Tschannen Basil Mustafa N. Houlsby CLIP VLM 19 47 0 15 Dec 2022
Scaling Language-Image Pre-training via Masking Yanghao Li Haoqi Fan Ronghang Hu Christoph Feichtenhofer Kaiming He CLIP VLM 14 317 0 01 Dec 2022
XKD: Cross-modal Knowledge Distillation with Domain Alignment for Video Representation Learning Pritam Sarkar Ali Etemad 19 21 0 25 Nov 2022
Masked Modeling Duo: Learning Representations by Encouraging Both Networks to Model the Input Daisuke Niizumi Daiki Takeuchi Yasunori Ohishi Noboru Harada K. Kashino SSL 18 29 0 26 Oct 2022
Contrastive Audio-Visual Masked Autoencoder Yuan Gong Andrew Rouditchenko Alexander H. Liu David F. Harwath Leonid Karlinsky Hilde Kuehne James R. Glass 19 119 0 02 Oct 2022
A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond Chaoning Zhang Chenshuang Zhang Junha Song John Seon Keun Yi Kang Zhang In So Kweon SSL 42 70 0 30 Jul 2022
GAFX: A General Audio Feature eXtractor Zhaoyang Bu Han Zhang Xiaohu Zhu 15 0 0 19 Jul 2022
Masked Autoencoders that Listen Po-Yao (Bernie) Huang Hu Xu Juncheng Billy Li Alexei Baevski Michael Auli Wojciech Galuba Florian Metze Christoph Feichtenhofer 8 267 0 13 Jul 2022
Unleashing Vanilla Vision Transformer with Masked Image Modeling for Object Detection Yuxin Fang Shusheng Yang Shijie Wang Yixiao Ge Ying Shan Xinggang Wang 6 55 0 06 Apr 2022
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 258 7,412 0 11 Nov 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,764 0 24 Feb 2021