Knowledge Distillation via Route Constrained Optimization

19 April 2019

Jiaheng Liu

Papers citing "Knowledge Distillation via Route Constrained Optimization"

29 / 29 papers shown

Title
Tailoring Instructions to Student's Learning Levels Boosts Knowledge Distillation Yuxin Ren Zi-Qi Zhong Xingjian Shi Yi Zhu Chun Yuan Mu Li 21 7 0 16 May 2023
Distillation from Heterogeneous Models for Top-K Recommendation SeongKu Kang Wonbin Kweon Dongha Lee Jianxun Lian Xing Xie Hwanjo Yu VLM 27 21 0 02 Mar 2023
HomoDistil: Homotopic Task-Agnostic Distillation of Pre-trained Transformers Chen Liang Haoming Jiang Zheng Li Xianfeng Tang Bin Yin Tuo Zhao VLM 24 24 0 19 Feb 2023
Supervision Complexity and its Role in Knowledge Distillation Hrayr Harutyunyan A. S. Rawat A. Menon Seungyeon Kim Surinder Kumar 22 12 0 28 Jan 2023
Learnable Heterogeneous Convolution: Learning both topology and strength Rongzhen Zhao Zhenzhi Wu Qikun Zhang 24 6 0 13 Jan 2023
BD-KD: Balancing the Divergences for Online Knowledge Distillation Ibtihel Amara N. Sepahvand B. Meyer W. Gross J. Clark 24 2 0 25 Dec 2022
3D Point Cloud Pre-training with Knowledge Distillation from 2D Images Yuan Yao Yuanhan Zhang Zhen-fei Yin Jiebo Luo Wanli Ouyang Xiaoshui Huang 3DPC 27 10 0 17 Dec 2022
Curriculum Temperature for Knowledge Distillation Zheng Li Xiang Li Lingfeng Yang Borui Zhao Renjie Song Lei Luo Jun Yu Li Jian Yang 22 132 0 29 Nov 2022
Respecting Transfer Gap in Knowledge Distillation Yulei Niu Long Chen Chan Zhou Hanwang Zhang 21 23 0 23 Oct 2022
Online Knowledge Distillation via Mutual Contrastive Learning for Visual Recognition Chuanguang Yang Zhulin An Helong Zhou Fuzhen Zhuang Yongjun Xu Qian Zhang 33 50 0 23 Jul 2022
Towards Efficient 3D Object Detection with Knowledge Distillation Jihan Yang Shaoshuai Shi Runyu Ding Zhe Wang Xiaojuan Qi 109 45 0 30 May 2022
Parameter-Efficient and Student-Friendly Knowledge Distillation Jun Rao Xv Meng Liang Ding Shuhan Qi Dacheng Tao 37 46 0 28 May 2022
A Closer Look at Self-Supervised Lightweight Vision Transformers Shaoru Wang Jin Gao Zeming Li Jian-jun Sun Weiming Hu ViT 67 41 0 28 May 2022
Localization Distillation for Object Detection Zhaohui Zheng Rongguang Ye Ping Wang Dongwei Ren Jun Wang W. Zuo Ming-Ming Cheng 21 64 0 12 Apr 2022
CoupleFace: Relation Matters for Face Recognition Distillation Jiaheng Liu Haoyu Qin Yichao Wu Jinyang Guo Ding Liang Ke Xu CVBM 21 19 0 12 Apr 2022
Knowledge Distillation as Efficient Pre-training: Faster Convergence, Higher Data-efficiency, and Better Transferability Ruifei He Shuyang Sun Jihan Yang Song Bai Xiaojuan Qi 24 36 0 10 Mar 2022
Iterative Self Knowledge Distillation -- From Pothole Classification to Fine-Grained and COVID Recognition Kuan-Chuan Peng 39 2 0 04 Feb 2022
ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation Shuohuan Wang Yu Sun Yang Xiang Zhihua Wu Siyu Ding ... Tian Wu Wei Zeng Ge Li Wen Gao Haifeng Wang ELM 33 79 0 23 Dec 2021
Improved Knowledge Distillation via Adversarial Collaboration Zhiqiang Liu Chengkai Huang Yanxia Liu 21 2 0 29 Nov 2021
Meta-Teacher For Face Anti-Spoofing Yunxiao Qin Zitong Yu Longbin Yan Zezheng Wang Chenxu Zhao Zhen Lei CVBM 25 61 0 12 Nov 2021
Pro-KD: Progressive Distillation by Following the Footsteps of the Teacher Mehdi Rezagholizadeh A. Jafari Puneeth Salad Pranav Sharma Ali Saheb Pasand A. Ghodsi 71 17 0 16 Oct 2021
Follow Your Path: a Progressive Method for Knowledge Distillation Wenxian Shi Yuxuan Song Hao Zhou Bohan Li Lei Li 17 14 0 20 Jul 2021
Distilling a Powerful Student Model via Online Knowledge Distillation Shaojie Li Mingbao Lin Yan Wang Yongjian Wu Yonghong Tian Ling Shao Rongrong Ji FedML 25 46 0 26 Mar 2021
Kernel Based Progressive Distillation for Adder Neural Networks Yixing Xu Chang Xu Xinghao Chen Wei Zhang Chunjing Xu Yunhe Wang 33 47 0 28 Sep 2020
Densely Guided Knowledge Distillation using Multiple Teacher Assistants Wonchul Son Jaemin Na Junyong Choi Wonjun Hwang 20 110 0 18 Sep 2020
Dynamic Group Convolution for Accelerating Convolutional Neural Networks Z. Su Linpu Fang Wenxiong Kang D. Hu M. Pietikäinen Li Liu 8 44 0 08 Jul 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,835 0 09 Jun 2020
Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation from a Blackbox Model Dongdong Wang Yandong Li Liqiang Wang Boqing Gong 16 48 0 31 Mar 2020
Search to Distill: Pearls are Everywhere but not the Eyes Yu Liu Xuhui Jia Mingxing Tan Raviteja Vemulapalli Yukun Zhu Bradley Green Xiaogang Wang 22 67 0 20 Nov 2019