LightHuBERT: Lightweight and Configurable Speech Representation Learning
with Once-for-All Hidden-Unit BERT

LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT

29 March 2022

Rui Wang

Haizhou Li

Papers citing "LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT"

11 / 11 papers shown

Title
StableQuant: Layer Adaptive Post-Training Quantization for Speech Foundation Models Yeona Hong Hyewon Han Woo-Jin Chung Hong-Goo Kang MQ 28 0 0 21 Apr 2025
A Large-Scale Evaluation of Speech Foundation Models Shu-Wen Yang Heng-Jui Chang Zili Huang Andy T. Liu Cheng-I Jeff Lai ... Kushal Lakhotia Shang-Wen Li Abdelrahman Mohamed Shinji Watanabe Hung-yi Lee 38 19 0 15 Apr 2024
Efficiency-oriented approaches for self-supervised speech representation learning Luis Lugo Valentin Vielzeuf SSL 19 1 0 18 Dec 2023
On-Device Constrained Self-Supervised Speech Representation Learning for Keyword Spotting via Knowledge Distillation Gene-Ping Yang Yue Gu Qingming Tang Dongsu Du Yuzong Liu 8 5 0 06 Jul 2023
Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation Kangwook Jang Sungnyun Kim Se-Young Yun Hoi-Rim Kim 19 5 0 19 May 2023
DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning Alexander H. Liu Heng-Jui Chang Michael Auli Wei-Ning Hsu James R. Glass 19 24 0 17 May 2023
I3D: Transformer architectures with input-dependent dynamic depth for speech recognition Yifan Peng Jaesong Lee Shinji Watanabe 22 16 0 14 Mar 2023
SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning Tzu-hsun Feng Annie Dong Ching-Feng Yeh Shu-Wen Yang Tzu-Quan Lin ... Xuankai Chang Shinji Watanabe Abdel-rahman Mohamed Shang-Wen Li Hung-yi Lee ELM SSL 19 33 0 16 Oct 2022
FitHuBERT: Going Thinner and Deeper for Knowledge Distillation of Speech Self-Supervised Learning Yeonghyeon Lee Kangwook Jang Jahyun Goo Youngmoon Jung Hoi-Rim Kim 10 28 0 01 Jul 2022
Oracle Teacher: Leveraging Target Information for Better Knowledge Distillation of CTC Models J. Yoon H. Kim Hyeon Seung Lee Sunghwan Ahn N. Kim 26 1 0 05 Nov 2021
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing Junyi Ao Rui Wang Long Zhou Chengyi Wang Shuo Ren ... Yu Zhang Zhihua Wei Yao Qian Jinyu Li Furu Wei 110 192 0 14 Oct 2021