Meta-KD: A Meta Knowledge Distillation Framework for Language Model
Compression across Domains

Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains

2 December 2020

Chengyu Wang

Yichang Zhang

Papers citing "Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains"

12 / 12 papers shown

Title
EvoP: Robust LLM Inference via Evolutionary Pruning Shangyu Wu Hongchao Du Ying Xiong Shuai Chen Tei-Wei Kuo Nan Guan Chun Jason Xue 34 1 0 19 Feb 2025
A Hybrid Cross-Stage Coordination Pre-ranking Model for Online Recommendation Systems Binglei Zhao Houying Qi Guang Xu Mian Ma Xiwei Zhao Feng Mei Sulong Xu Jinghe Hu 57 0 0 17 Feb 2025
MoDeGPT: Modular Decomposition for Large Language Model Compression Chi-Heng Lin Shangqian Gao James Seale Smith Abhishek Patel Shikhar Tuli Yilin Shen Hongxia Jin Yen-Chang Hsu 71 6 0 19 Aug 2024
MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU Yan Li So-Eon Kim Seong-Bae Park S. Han 21 0 0 15 Aug 2024
Position: Key Claims in LLM Research Have a Long Tail of Footnotes Anna Rogers A. Luccioni 45 19 0 14 Aug 2023
Few-Shot Learning of Compact Models via Task-Specific Meta Distillation Yong Wu Shekhor Chanda M. Hosseinzadeh Zhi Liu Yang Wang VLM 29 7 0 18 Oct 2022
EasyNLP: A Comprehensive and Easy-to-use Toolkit for Natural Language Processing Chengyu Wang Minghui Qiu Chen Shi Taolin Zhang Tingting Liu Lei Li J. Wang Ming Wang Jun Huang W. Lin 11 21 0 30 Apr 2022
HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression Chenhe Dong Yaliang Li Ying Shen Minghui Qiu VLM 30 7 0 16 Oct 2021
Learning to Augment for Data-Scarce Domain BERT Knowledge Distillation Lingyun Feng Minghui Qiu Yaliang Li Haitao Zheng Ying Shen 38 10 0 20 Jan 2021
Probabilistic Model-Agnostic Meta-Learning Chelsea Finn Kelvin Xu Sergey Levine BDL 165 666 0 07 Jun 2018
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks Chelsea Finn Pieter Abbeel Sergey Levine OOD 290 11,681 0 09 Mar 2017
Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results Antti Tarvainen Harri Valpola OOD MoMe 249 1,275 0 06 Mar 2017