Heterogeneous Knowledge Distillation using Information Flow Modeling

2 May 2020

Nikolaos Passalis

Maria Tzelepi

Anastasios Tefas

Papers citing "Heterogeneous Knowledge Distillation using Information Flow Modeling"

30 / 30 papers shown

Title
Cross-Architecture Auxiliary Feature Space Translation for Efficient Few-Shot Personalized Object Detection F. Barbato Umberto Michieli J. Moon Pietro Zanuttigh Mete Ozay 42 2 0 01 Jul 2024
DistilDoc: Knowledge Distillation for Visually-Rich Document Applications Jordy Van Landeghem Subhajit Maity Ayan Banerjee Matthew Blaschko Marie-Francine Moens Josep Lladós Sanket Biswas 50 2 0 12 Jun 2024
Adversarial Sparse Teacher: Defense Against Distillation-Based Model Stealing Attacks Using Adversarial Examples Eda Yilmaz H. Keles AAML 21 2 0 08 Mar 2024
Choosing Wisely and Learning Deeply: Selective Cross-Modality Distillation via CLIP for Domain Generalization Jixuan Leng Yijiang Li Haohan Wang VLM 37 0 0 26 Nov 2023
Teacher-Student Architecture for Knowledge Distillation: A Survey Chengming Hu Xuan Li Danyang Liu Haolun Wu Xi Chen Ju Wang Xue Liu 21 16 0 08 Aug 2023
Towards a Smaller Student: Capacity Dynamic Distillation for Efficient Image Retrieval Yi Xie Huaidong Zhang Xuemiao Xu Jianqing Zhu Shengfeng He VLM 21 13 0 16 Mar 2023
Graph-based Knowledge Distillation: A survey and experimental evaluation Jing Liu Tongya Zheng Guanzheng Zhang Qinfen Hao 33 8 0 27 Feb 2023
OVO: One-shot Vision Transformer Search with Online distillation Zimian Wei H. Pan Xin-Yi Niu Dongsheng Li ViT 29 1 0 28 Dec 2022
Hilbert Distillation for Cross-Dimensionality Networks Dian Qin Haishuai Wang Zhe Liu Hongjia Xu Sheng Zhou Jiajun Bu 23 4 0 08 Nov 2022
Federated Learning with Privacy-Preserving Ensemble Attention Distillation Xuan Gong Liangchen Song Rishi Vedula Abhishek Sharma Meng Zheng ... Arun Innanje Terrence Chen Junsong Yuan David Doermann Ziyan Wu FedML 30 27 0 16 Oct 2022
MLink: Linking Black-Box Models from Multiple Domains for Collaborative Inference Mu Yuan Lan Zhang Zimu Zheng Yi-Nan Zhang Xiang-Yang Li 25 2 0 28 Sep 2022
Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition Chuanguang Yang Zhulin An Helong Zhou Fuzhen Zhuang Yongjun Xu Qian Zhang 43 50 0 23 Jul 2022
Locality Guidance for Improving Vision Transformers on Tiny Datasets Kehan Li Runyi Yu Zhennan Wang Li-ming Yuan Guoli Song Jie Chen ViT 32 44 0 20 Jul 2022
Fast Object Placement Assessment Li Niu Qingyang Liu Zhenchen Liu Jiangtong Li 21 14 0 28 May 2022
IDEAL: Query-Efficient Data-Free Learning from Black-box Models Jie M. Zhang Chen Chen Lingjuan Lyu 55 14 0 23 May 2022
Knowledge Distillation Meets Open-Set Semi-Supervised Learning Jing Yang Xiatian Zhu Adrian Bulat Brais Martínez Georgios Tzimiropoulos 33 8 0 13 May 2022
Generalized Knowledge Distillation via Relationship Matching Han-Jia Ye Su Lu De-Chuan Zhan FedML 22 20 0 04 May 2022
Knowledge Distillation as Efficient Pre-training: Faster Convergence, Higher Data-efficiency, and Better Transferability Ruifei He Shuyang Sun Jihan Yang Song Bai Xiaojuan Qi 34 36 0 10 Mar 2022
Iterative Self Knowledge Distillation -- From Pothole Classification to Fine-Grained and COVID Recognition Kuan-Chuan Peng 43 2 0 04 Feb 2022
Towards Streaming Egocentric Action Anticipation Antonino Furnari G. Farinella EgoV 33 6 0 11 Oct 2021
Partial to Whole Knowledge Distillation: Progressive Distilling Decomposed Knowledge Boosts Student Better Xuanyang Zhang Xinming Zhang Jian Sun 25 1 0 26 Sep 2021
Knowledge Distillation Using Hierarchical Self-Supervision Augmented Distribution Chuanguang Yang Zhulin An Linhang Cai Yongjun Xu 32 15 0 07 Sep 2021
Self-Regulation for Semantic Segmentation Zhangfu Dong Zhang Hanwang T. Jinhui Huang Xiansheng Sun Qianru 36 36 0 22 Aug 2021
Hierarchical Self-supervised Augmented Knowledge Distillation Chuanguang Yang Zhulin An Linhang Cai Yongjun Xu SSL 32 76 0 29 Jul 2021
Distilling a Powerful Student Model via Online Knowledge Distillation Shaojie Li Mingbao Lin Yan Wang Yongjian Wu Yonghong Tian Ling Shao Rongrong Ji FedML 27 47 0 26 Mar 2021
Student Network Learning via Evolutionary Knowledge Distillation Kangkai Zhang Chunhui Zhang Shikun Li Dan Zeng Shiming Ge 22 83 0 23 Mar 2021
Cross-Layer Distillation with Semantic Calibration Defang Chen Jian-Ping Mei Yuan Zhang Can Wang Yan Feng Chun-Yen Chen FedML 45 287 0 06 Dec 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,851 0 09 Jun 2020
Knowledge Distillation for Incremental Learning in Semantic Segmentation Umberto Michieli Pietro Zanuttigh CLL VLM 28 98 0 08 Nov 2019
Large scale distributed neural network training through online distillation Rohan Anil Gabriel Pereyra Alexandre Passos Róbert Ormándi George E. Dahl Geoffrey E. Hinton FedML 278 404 0 09 Apr 2018