How to Train the Teacher Model for Effective Knowledge Distillation

How to Train the Teacher Model for Effective Knowledge Distillation

25 July 2024

Shayan Mohajer Hamidi

Ahmed H. Salamah

Papers citing "How to Train the Teacher Model for Effective Knowledge Distillation"

10 / 10 papers shown

Title
Learning an Adaptive and View-Invariant Vision Transformer for Real-Time UAV Tracking You Wu Yongxin Li Mengyuan Liu Xucheng Wang Xiangyang Yang Hengzhou Ye Dan Zeng Qijun Zhao Shuiwang Li 85 0 0 28 Dec 2024
Distribution Alignment for Fully Test-Time Adaptation with Dynamic Online Data Streams Ziqiang Wang Zhixiang Chi Yanan Wu Li Gu Zhi Liu Konstantinos Plataniotis Yang Wang OOD TTA 36 2 0 16 Jul 2024
Adapting to Distribution Shift by Visual Domain Prompt Generation Zhixiang Chi Li Gu Tao Zhong Huan Liu Yuanhao Yu Konstantinos N Plataniotis Yang Wang VLM OOD 29 7 0 05 May 2024
Bayes Conditional Distribution Estimation for Knowledge Distillation Based on Conditional Mutual Information Linfeng Ye Shayan Mohajer Hamidi Renhao Tan En-Hui Yang VLM 27 12 0 16 Jan 2024
Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Experts Tao Zhong Zhixiang Chi Li Gu Yang Wang Yuanhao Yu Jingshan Tang OOD 64 29 0 08 Oct 2022
Distilling Knowledge via Knowledge Review Pengguang Chen Shu-Lin Liu Hengshuang Zhao Jiaya Jia 147 419 0 19 Apr 2021
All at Once: Temporally Adaptive Multi-Frame Interpolation with Advanced Motion Modeling Zhixiang Chi Rasoul Mohammadi Nasiri Zheng Liu Juwei Lu Jin Tang Konstantinos N Plataniotis 55 70 0 23 Jul 2020
Knowledge Distillation by On-the-Fly Native Ensemble Xu Lan Xiatian Zhu S. Gong 187 473 0 12 Jun 2018
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 267 404 0 09 Apr 2018
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 282 39,190 0 01 Sep 2014