DistilHuBERT: Speech Representation Learning by Layer-wise Distillation
of Hidden-unit BERT

DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT

5 October 2021

Papers citing "DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT"

16 / 116 papers shown

Title
Deep versus Wide: An Analysis of Student Architectures for Task-Agnostic Knowledge Distillation of Self-Supervised Speech Models Takanori Ashihara Takafumi Moriya Kohei Matsuura Tomohiro Tanaka 17 25 0 14 Jul 2022
FitHuBERT: Going Thinner and Deeper for Knowledge Distillation of Speech Self-Supervised Learning Yeonghyeon Lee Kangwook Jang Jahyun Goo Youngmoon Jung Hoi-Rim Kim 10 28 0 01 Jul 2022
A Systematic Comparison of Phonetic Aware Techniques for Speech Enhancement Or Tal Moshe Mandel Felix Kreuk Yossi Adi AAML 9 8 0 22 Jun 2022
BridgeTower: Building Bridges Between Encoders in Vision-Language Representation Learning Xiao Xu Chenfei Wu Shachar Rosenman Vasudev Lal Wanxiang Che Nan Duan 32 64 0 17 Jun 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 124 339 0 21 May 2022
Learning Lip-Based Audio-Visual Speaker Embeddings with AV-HuBERT Bowen Shi Abdel-rahman Mohamed Wei-Ning Hsu SSL 18 17 0 15 May 2022
Silence is Sweeter Than Speech: Self-Supervised Model Using Silence to Store Speaker Information Chiyu Feng Po-Chun Hsu Hung-yi Lee SSL 12 8 0 08 May 2022
HuBERT-EE: Early Exiting HuBERT for Efficient Speech Recognition J. Yoon Beom Jun Woo N. Kim 17 13 0 13 Apr 2022
Fast Real-time Personalized Speech Enhancement: End-to-End Enhancement Network (E3Net) and Knowledge Distillation Manthan Thakker Sefik Emre Eskimez Takuya Yoshioka Huaming Wang 14 28 0 02 Apr 2022
LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT Rui Wang Qibing Bai Junyi Ao Long Zhou Zhixiang Xiong Zhihua Wei Yu Zhang Tom Ko Haizhou Li 20 61 0 29 Mar 2022
Semi-FedSER: Semi-supervised Learning for Speech Emotion Recognition On Federated Learning using Multiview Pseudo-Labeling Tiantian Feng Shrikanth Narayanan 17 17 0 15 Mar 2022
SUPERB-SG: Enhanced Speech processing Universal PERformance Benchmark for Semantic and Generative Capabilities Hsiang-Sheng Tsai Heng-Jui Chang Wen-Chin Huang Zili Huang Kushal Lakhotia ... Hsuan-Jui Chen Shang-Wen Li Shinji Watanabe Abdel-rahman Mohamed Hung-yi Lee 10 107 0 14 Mar 2022
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language Alexei Baevski Wei-Ning Hsu Qiantong Xu Arun Babu Jiatao Gu Michael Auli SSL VLM ViT 24 823 0 07 Feb 2022
Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling Puyuan Peng David F. Harwath SSL 28 26 0 07 Feb 2022
Oracle Teacher: Leveraging Target Information for Better Knowledge Distillation of CTC Models J. Yoon H. Kim Hyeon Seung Lee Sunghwan Ahn N. Kim 26 1 0 05 Nov 2021
Multi-task self-supervised learning for Robust Speech Recognition Mirco Ravanelli Jianyuan Zhong Santiago Pascual P. Swietojanski João Monteiro J. Trmal Yoshua Bengio SSL 171 288 0 25 Jan 2020