Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2012.01266
Cited By
v1
v2 (latest)
Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains
Annual Meeting of the Association for Computational Linguistics (ACL), 2020
2 December 2020
Haojie Pan
Chengyu Wang
Minghui Qiu
Yichang Zhang
Yaliang Li
Yanjie Liang
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains"
27 / 27 papers shown
Experts are all you need: A Composable Framework for Large Language Model Inference
S. Sridharan
Sourjya Roy
A. Raghunathan
Kaushik Roy
MoE
227
0
0
28 Nov 2025
PrunedLoRA: Robust Gradient-Based structured pruning for Low-rank Adaptation in Fine-tuning
Xin Yu
Cong Xie
Ziyu Zhao
Tiantian Fan
Lingzhou Xue
Zhi-Li Zhang
307
0
0
30 Sep 2025
SPADE: Structured Pruning and Adaptive Distillation for Efficient LLM-TTS
T. Nguyen
Jaehun Kim
Ji-Hoon Kim
Shukjae Choi
Youshin Lim
Joon Son Chung
196
1
0
25 Sep 2025
DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models
Chengyu Wang
Junbing Yan
Yuanhao Yue
Yanjie Liang
359
6
0
21 Apr 2025
EvoP: Robust LLM Inference via Evolutionary Pruning
Shangyu Wu
Hongchao Du
Ying Xiong
Shuai Chen
Tei-Wei Kuo
Nan Guan
Chun Jason Xue
701
3
0
19 Feb 2025
A Hybrid Cross-Stage Coordination Pre-ranking Model for Online Recommendation Systems
The Web Conference (WWW), 2025
Binglei Zhao
Houying Qi
Guang Xu
Mian Ma
Xiwei Zhao
Feng Mei
Sulong Xu
Jinghe Hu
297
5
0
17 Feb 2025
MoDeGPT: Modular Decomposition for Large Language Model Compression
International Conference on Learning Representations (ICLR), 2024
Chi-Heng Lin
Shangqian Gao
James Seale Smith
Abhishek Patel
Shikhar Tuli
Yilin Shen
Hongxia Jin
Yen-Chang Hsu
894
35
0
19 Aug 2024
MIDAS: Multi-level Intent, Domain, And Slot Knowledge Distillation for Multi-turn NLU
North American Chapter of the Association for Computational Linguistics (NAACL), 2024
Yan Li
So-Eon Kim
Seong-Bae Park
S. Han
498
2
0
15 Aug 2024
Weight Copy and Low-Rank Adaptation for Few-Shot Distillation of Vision Transformers
Diana-Nicoleta Grigore
Mariana-Iuliana Georgescu
J. A. Justo
T. Johansen
Andreea-Iuliana Ionescu
Radu Tudor Ionescu
381
2
0
14 Apr 2024
CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models
Je-Yong Lee
Donghyun Lee
Genghan Zhang
Mo Tiwari
Azalia Mirhoseini
337
37
0
12 Apr 2024
Hierarchical Skip Decoding for Efficient Autoregressive Text Generation
Yunqi Zhu
Xuebing Yang
Yuanyuan Wu
Wensheng Zhang
400
6
0
22 Mar 2024
CLLMs: Consistency Large Language Models
Siqi Kou
Lanxiang Hu
Zhe He
Zhijie Deng
Hao Zhang
512
61
0
28 Feb 2024
Model Compression and Efficient Inference for Large Language Models: A Survey
Wenxiao Wang
Wei Chen
Yicong Luo
Yongliu Long
Zhengkai Lin
Liye Zhang
Binbin Lin
Deng Cai
Xiaofei He
MQ
362
93
0
15 Feb 2024
One-Shot Sensitivity-Aware Mixed Sparsity Pruning for Large Language Models
Hang Shao
Bei Liu
Bo Xiao
Ke Zeng
Guanglu Wan
Yanmin Qian
319
30
0
14 Oct 2023
Position: Key Claims in LLM Research Have a Long Tail of Footnotes
International Conference on Machine Learning (ICML), 2023
Anna Rogers
A. Luccioni
547
24
0
14 Aug 2023
Are Intermediate Layers and Labels Really Necessary? A General Language Model Distillation Method
Annual Meeting of the Association for Computational Linguistics (ACL), 2023
Shicheng Tan
Weng Lam Tam
Yuanchun Wang
Wenwen Gong
Shuo Zhao
Peng Zhang
Jie Tang
VLM
182
1
0
11 Jun 2023
Domain Private Transformers for Multi-Domain Dialog Systems
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023
Anmol Kabra
Ethan R. Elenberg
266
0
0
23 May 2023
LLM-Pruner: On the Structural Pruning of Large Language Models
Neural Information Processing Systems (NeurIPS), 2023
Xinyin Ma
Gongfan Fang
Xinchao Wang
880
766
0
19 May 2023
Few-Shot Learning of Compact Models via Task-Specific Meta Distillation
IEEE Workshop/Winter Conference on Applications of Computer Vision (WACV), 2022
Yong Wu
Shekhor Chanda
M. Hosseinzadeh
Zhi Liu
Yang Wang
VLM
317
10
0
18 Oct 2022
Meta Learning for Natural Language Processing: A Survey
North American Chapter of the Association for Computational Linguistics (NAACL), 2022
Hung-yi Lee
Shang-Wen Li
Ngoc Thang Vu
441
55
0
03 May 2022
EasyNLP: A Comprehensive and Easy-to-use Toolkit for Natural Language Processing
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022
Chengyu Wang
Minghui Qiu
Chen Shi
Taolin Zhang
Tingting Liu
Lei Li
Jiadong Wang
Ming Wang
Yanjie Liang
W. Lin
273
22
0
30 Apr 2022
DistilCSE: Effective Knowledge Distillation For Contrastive Sentence Embeddings
Chaochen Gao
Xing Wu
Peng Wang
Jue Wang
Liangjun Zang
Zhongyuan Wang
Songlin Hu
204
6
0
10 Dec 2021
Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI
IEEE Transactions on Knowledge and Data Engineering (TKDE), 2021
Jiangchao Yao
Shengyu Zhang
Yang Yao
Feng Wang
Jianxin Ma
...
Kun Kuang
Chao-Xiang Wu
Leilei Gan
Jingren Zhou
Hongxia Yang
441
155
0
11 Nov 2021
HRKD: Hierarchical Relational Knowledge Distillation for Cross-domain Language Model Compression
Chenhe Dong
Yaliang Li
Ying Shen
Minghui Qiu
VLM
371
8
0
16 Oct 2021
Learning to Teach with Student Feedback
Yitao Liu
Tianxiang Sun
Xipeng Qiu
Xuanjing Huang
VLM
192
6
0
10 Sep 2021
BERT Learns to Teach: Knowledge Distillation with Meta Learning
Annual Meeting of the Association for Computational Linguistics (ACL), 2021
Wangchunshu Zhou
Canwen Xu
Julian McAuley
378
110
0
08 Jun 2021
EasyTransfer -- A Simple and Scalable Deep Transfer Learning Platform for NLP Applications
International Conference on Information and Knowledge Management (CIKM), 2020
Minghui Qiu
Peng Li
Chengyu Wang
Hanjie Pan
Yaliang Li
...
Jun Yang
Yaliang Li
Yanjie Liang
Deng Cai
Jialin Li
VLM
SyDa
414
20
0
18 Nov 2020
1
Page 1 of 1
