LightPAFF: A Two-Stage Distillation Framework for Pre-training and
Fine-tuning

LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning

27 April 2020

Xu Tan

Papers citing "LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning"

6 / 6 papers shown

Title
ConCodeEval: Evaluating Large Language Models for Code Constraints in Domain-Specific Languages Mehant Kammakomati Sameer Pimparkhede Srikanth G. Tamilselvam Prince Kumar Pushpak Bhattacharyya ALM 40 0 0 03 Jul 2024
Direct Preference Knowledge Distillation for Large Language Models Yixing Li Yuxian Gu Li Dong Dequan Wang Yu Cheng Furu Wei 37 6 0 28 Jun 2024
Compression of Generative Pre-trained Language Models via Quantization Chaofan Tao Lu Hou Wei Zhang Lifeng Shang Xin Jiang Qun Liu Ping Luo Ngai Wong MQ 27 103 0 21 Mar 2022
Compacting Deep Neural Networks for Internet of Things: Methods and Applications Ke Zhang Hanbo Ying Hongning Dai Lin Li Yuangyuang Peng Keyi Guo Hongfang Yu 16 38 0 20 Mar 2021
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 192 473 0 12 Jun 2018
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 275 2,888 0 15 Sep 2016