Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo
Labelling

Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling

1 November 2023

Patrick von Platen

Alexander M. Rush

Papers citing "Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labelling"

10 / 10 papers shown

Title
Training and Inference Efficiency of Encoder-Decoder Speech Models Piotr .Zelasko Kunal Dhawan Daniel Galvez Krishna C. Puvvada Ankita Pasad Nithin Rao Koluguri Ke Hu Vitaly Lavrukhin Jagadeesh Balam Boris Ginsburg 41 0 0 07 Mar 2025
MT2KD: Towards A General-Purpose Encoder for Speech, Speaker, and Audio Events Xiaoyu Yang Qiujia Li Chao Zhang P. Woodland 18 0 0 25 Sep 2024
Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models Dominik Wagner Ilja Baumann K. Riedhammer Tobias Bocklet MQ 30 1 0 16 Jun 2024
Towards Effective and Efficient Non-autoregressive Decoding Using Block-based Attention Mask Tianzi Wang Xurong Xie Zhaoqing Li Shoukang Hu Zengrui Jin ... Shujie Hu Mengzhe Geng Guinan Li Helen Meng Xunying Liu 29 0 0 14 Jun 2024
Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages Yu Zhang Wei Han James Qin Yongqiang Wang Ankur Bapna ... Pedro J. Moreno Chung-Cheng Chiu J. Schalkwyk Franccoise Beaufays Yonghui Wu VLM 77 250 0 02 Mar 2023
FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech Alexis Conneau Min Ma Simran Khanuja Yu Zhang Vera Axelrod Siddharth Dalmia Jason Riesa Clara E. Rivera Ankur Bapna VLM 78 282 0 25 May 2022
Robustness Challenges in Model Distillation and Pruning for Natural Language Understanding Mengnan Du Subhabrata Mukherjee Yu Cheng Milad Shokouhi Xia Hu Ahmed Hassan Awadallah 44 13 0 16 Oct 2021
Earnings-21: A Practical Benchmark for ASR in the Wild Miguel Rio Natalie Delworth Ryan Westerman Michelle Huang Nishchal Bhandari Joseph Palakapilly Quinten McNamara Joshua Dong Piotr Żelasko Miguel Jetté 58 47 0 22 Apr 2021
Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognition Yu Zhang James Qin Daniel S. Park Wei Han Chung-Cheng Chiu Ruoming Pang Quoc V. Le Yonghui Wu VLM SSL 136 307 0 20 Oct 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018