Performance-Efficiency Trade-offs in Unsupervised Pre-training for
Speech Recognition

Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition

14 September 2021

Kwangyoun Kim

Kilian Q. Weinberger

Papers citing "Performance-Efficiency Trade-offs in Unsupervised Pre-training for Speech Recognition"

19 / 19 papers shown

Title
Efficient Training of Self-Supervised Speech Foundation Models on a Compute Budget Andy T. Liu Yi-Cheng Lin Haibin Wu Stefan Winkler Hung-yi Lee 31 1 0 09 Sep 2024
Sustainable self-supervised learning for speech representations Luis Lugo Valentin Vielzeuf 29 2 0 11 Jun 2024
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 19 1 0 18 Dec 2023
When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants Anuj Diwan Eunsol Choi David F. Harwath 39 0 0 14 Jun 2023
Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization Hamza Kheddar Yassine Himeur S. Al-Maadeed Abbes Amira F. Bensaali 35 76 0 27 Apr 2023
Adapting Multilingual Speech Representation Model for a New, Underresourced Language through Multilingual Fine-tuning and Continued Pretraining Karol Nowakowski M. Ptaszynski Kyoko Murasaki Jagna Nieuwazny 11 23 0 18 Jan 2023
MelHuBERT: A simplified HuBERT on Mel spectrograms Tzu-Quan Lin Hung-yi Lee Hao Tang SSL 19 13 0 17 Nov 2022
Comparative layer-wise analysis of self-supervised speech models Ankita Pasad Bowen Shi Karen Livescu SSL 22 109 0 08 Nov 2022
Efficient Representation of Natural Image Patches Cheng-en Guo 14 0 0 24 Oct 2022
$RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise$ RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise Jinming Zhao Haomiao Yang Gholamreza Haffari Ehsan Shareghi VLM 11 2 0 16 Oct 2022
Wav2Seq: Pre-training Speech-to-Text Encoder-Decoder Models Using Pseudo Languages Felix Wu Kwangyoun Kim Shinji Watanabe Kyu Jeong Han Ryan T. McDonald Kilian Q. Weinberger Yoav Artzi SyDa 34 37 0 02 May 2022
On-demand compute reduction with stochastic wav2vec 2.0 Apoorv Vyas Wei-Ning Hsu Michael Auli Alexei Baevski 8 13 0 25 Apr 2022
Audio Self-supervised Learning: A Survey Shuo Liu Adria Mallol-Ragolta Emilia Parada-Cabeleiro Kun Qian Xingshuo Jing Alexander Kathan Bin Hu Bjoern W. Schuller SSL 22 106 0 02 Mar 2022
Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding Peter Sullivan Toshiko Shibano Muhammad Abdul-Mageed 34 11 0 10 Feb 2022
On the Use of External Data for Spoken Named Entity Recognition Ankita Pasad Felix Wu Suwon Shon Karen Livescu Kyu Jeong Han 32 16 0 14 Dec 2021
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing Sanyuan Chen Chengyi Wang Zhengyang Chen Yu-Huan Wu Shujie Liu ... Yao Qian Jian Wu Micheal Zeng Xiangzhan Yu Furu Wei SSL 73 1,694 0 26 Oct 2021
Multimodal Self-Supervised Learning of General Audio Representations Luyu Wang Pauline Luc Adrià Recasens Jean-Baptiste Alayrac Aaron van den Oord SSL 70 41 0 26 Apr 2021
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Yu Zhang James Qin Daniel S. Park Wei Han Chung-Cheng Chiu Ruoming Pang Quoc V. Le Yonghui Wu VLM SSL 136 307 0 20 Oct 2020
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020