Efficient Knowledge Distillation from Model Checkpoints

Efficient Knowledge Distillation from Model Checkpoints

12 October 2022

Gao Huang

Papers citing "Efficient Knowledge Distillation from Model Checkpoints"

10 / 10 papers shown

Title
Quantifying Knowledge Distillation Using Partial Information Decomposition Pasan Dissanayake Faisal Hamman Barproda Halder Ilia Sucholutsky Qiuyi Zhang Sanghamitra Dutta 36 0 0 12 Nov 2024
DεpS: Delayed ε-Shrinking for Faster Once-For-All Training Aditya Annavajjala Alind Khare Animesh Agrawal Igor Fedorov Hugo Latapie Myungjin Lee Alexey Tumanov CLL 42 0 0 08 Jul 2024
Gradual Divergence for Seamless Adaptation: A Novel Domain Incremental Learning Method Kishaan Jeeveswaran Elahe Arani Bahram Zonooz CLL 44 3 0 23 Jun 2024
DistilDoc: Knowledge Distillation for Visually-Rich Document Applications Jordy Van Landeghem Subhajit Maity Ayan Banerjee Matthew Blaschko Marie-Francine Moens Josep Lladós Sanket Biswas 50 2 0 12 Jun 2024
$$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning$ $\textit{Trans-LoRA}$ : towards data-free Transferable Parameter Efficient Finetuning Runqian Wang Soumya Ghosh David D. Cox Diego Antognini Aude Oliva Rogerio Feris Leonid Karlinsky 32 1 0 27 May 2024
Understanding the Effects of Projectors in Knowledge Distillation Yudong Chen Sen Wang Jiajun Liu Xuwei Xu Frank de Hoog Brano Kusy Zi Huang 26 0 0 26 Oct 2023
Tailoring Instructions to Student's Learning Levels Boosts Knowledge Distillation Yuxin Ren Zi-Qi Zhong Xingjian Shi Yi Zhu Chun Yuan Mu Li 21 7 0 16 May 2023
Learning Student-Friendly Teacher Networks for Knowledge Distillation D. Park Moonsu Cha C. Jeong Daesin Kim Bohyung Han 118 100 0 12 Feb 2021
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 192 473 0 12 Jun 2018
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications Andrew G. Howard Menglong Zhu Bo Chen Dmitry Kalenichenko Weijun Wang Tobias Weyand M. Andreetto Hartwig Adam 3DH 950 20,561 0 17 Apr 2017