One Teacher is Enough? Pre-trained Language Model Distillation from Multiple Teachers

2 June 2021

Papers citing "One Teacher is Enough? Pre-trained Language Model Distillation from Multiple Teachers"

8 / 8 papers shown

Title
MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU Yan Li So-Eon Kim Seong-Bae Park S. Han 21 0 0 15 Aug 2024
EBBS: An Ensemble with Bi-Level Beam Search for Zero-Shot Machine Translation Yuqiao Wen Behzad Shayegh Chenyang Huang Yanshuai Cao Lili Mou 48 4 0 29 Feb 2024
f-Divergence Minimization for Sequence-Level Knowledge Distillation Yuqiao Wen Zichao Li Wenyu Du Lili Mou 30 53 0 27 Jul 2023
Advances and Challenges in Meta-Learning: A Technical Review Anna Vettoruzzo Mohamed-Rafik Bouguelia Joaquin Vanschoren Thorsteinn Rögnvaldsson K. Santosh OffRL 19 70 0 10 Jul 2023
GKD: A General Knowledge Distillation Framework for Large-scale Pre-trained Language Model Shicheng Tan Weng Lam Tam Yuanchun Wang Wenwen Gong Yang Yang ... Jiahao Liu Jingang Wang Shuo Zhao Peng-Zhen Zhang Jie Tang ALM MoE 17 11 0 11 Jun 2023
Distillation from Heterogeneous Models for Top-K Recommendation SeongKu Kang Wonbin Kweon Dongha Lee Jianxun Lian Xing Xie Hwanjo Yu VLM 27 21 0 02 Mar 2023
Distilling the Knowledge of Romanian BERTs Using Multiple Teachers Andrei-Marius Avram Darius Catrina Dumitru-Clementin Cercel Mihai Dascualu Traian Rebedea Vasile Puaics Dan Tufics 22 12 0 23 Dec 2021
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 243 1,450 0 18 Mar 2020