NormKD: Normalized Logits for Knowledge Distillation

NormKD: Normalized Logits for Knowledge Distillation

1 August 2023

Papers citing "NormKD: Normalized Logits for Knowledge Distillation"

11 / 11 papers shown

Title
Image Recognition with Online Lightweight Vision Transformer: A Survey Zherui Zhang Rongtao Xu Jie Zhou Changwei Wang Xingtian Pei ... Jiguang Zhang Li Guo Longxiang Gao W. Xu Shibiao Xu ViT 48 0 0 06 May 2025
Feature Alignment and Representation Transfer in Knowledge Distillation for Large Language Models Junjie Yang Junhao Song Xudong Han Ziqian Bi Tianyang Wang ... Y. Zhang Qian Niu Benji Peng Keyu Chen Ming Liu VLM 40 0 0 18 Apr 2025
Cross-View Consistency Regularisation for Knowledge Distillation W. Zhang Dongnan Liu Weidong Cai Chao Ma 68 0 0 21 Dec 2024
Pre-training Distillation for Large Language Models: A Design Space Exploration Hao Peng Xin Lv Yushi Bai Zijun Yao J. Zhang Lei Hou Juanzi Li 23 3 0 21 Oct 2024
Kendall's $τ$ Coefficient for Logits Distillation Yuchen Guan Runxi Cheng Kang Liu Chun Yuan 21 0 0 26 Sep 2024
BiLD: Bi-directional Logits Difference Loss for Large Language Model Distillation Minchong Li Feng Zhou Xiaohui Song 29 2 0 19 Jun 2024
Dynamic Temperature Knowledge Distillation Yukang Wei Yu Bai 25 4 0 19 Apr 2024
GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creation Ayan Banerjee Sanket Biswas Josep Lladós Umapada Pal 30 1 0 17 Feb 2024
Distilling Knowledge via Knowledge Review Pengguang Chen Shu-Lin Liu Hengshuang Zhao Jiaya Jia 147 416 0 19 Apr 2021
Feature Pyramid Networks for Object Detection Tsung-Yi Lin Piotr Dollár Ross B. Girshick Kaiming He Bharath Hariharan Serge J. Belongie ObjD 166 21,643 0 09 Dec 2016
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 279 39,083 0 01 Sep 2014