Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matching

5 February 2021

Papers citing "Show, Attend and Distill:Knowledge Distillation via Attention-based Feature Matching"

12 / 12 papers shown

Title
Image Recognition with Online Lightweight Vision Transformer: A Survey Zherui Zhang Rongtao Xu Jie Zhou Changwei Wang Xingtian Pei ... Jiguang Zhang Li Guo Longxiang Gao W. Xu Shibiao Xu ViT 101 0 0 06 May 2025
Indirect Gradient Matching for Adversarial Robust Distillation Hongsin Lee Seungju Cho Changick Kim AAML FedML 48 2 0 06 Dec 2023
Understanding the Effects of Projectors in Knowledge Distillation Yudong Chen Sen Wang Jiajun Liu Xuwei Xu Frank de Hoog Brano Kusy Zi Huang 19 0 0 26 Oct 2023
Review helps learn better: Temporal Supervised Knowledge Distillation Dongwei Wang Zhi-Long Han Yanmei Wang Xi’ai Chen Baicheng Liu Yandong Tang 46 1 0 03 Jul 2023
Cross Architecture Distillation for Face Recognition Weisong Zhao Xiangyu Zhu Zhixiang He Xiaoyu Zhang Zhen Lei CVBM 13 6 0 26 Jun 2023
Semantic Scene Completion with Cleaner Self Fengyun Wang Dong Zhang Hanwang Zhang Jinhui Tang Qianru Sun 21 12 0 17 Mar 2023
Hilbert Distillation for Cross-Dimensionality Networks Dian Qin Haishuai Wang Zhe Liu Hongjia Xu Sheng Zhou Jiajun Bu 19 4 0 08 Nov 2022
Improved Feature Distillation via Projector Ensemble Yudong Chen Sen Wang Jiajun Liu Xuwei Xu Frank de Hoog Zi Huang 21 37 0 27 Oct 2022
Knowledge Distillation via the Target-aware Transformer Sihao Lin Hongwei Xie Bing Wang Kaicheng Yu Xiaojun Chang Xiaodan Liang G. Wang ViT 20 104 0 22 May 2022
Exploring Inter-Channel Correlation for Diversity-preserved KnowledgeDistillation Li Liu Qingle Huang Sihao Lin Hongwei Xie Bing Wang Xiaojun Chang Xiao-Xue Liang 20 100 0 08 Feb 2022
Oracle Teacher: Leveraging Target Information for Better Knowledge Distillation of CTC Models J. Yoon H. Kim Hyeon Seung Lee Sunghwan Ahn N. Kim 26 1 0 05 Nov 2021
Knowledge Distillation with Noisy Labels for Natural Language Understanding Shivendra Bhardwaj Abbas Ghaddar Ahmad Rashid Khalil Bibi Cheng-huan Li A. Ghodsi Philippe Langlais Mehdi Rezagholizadeh 19 1 0 21 Sep 2021