v1v2 (latest)

Cross-Architecture Knowledge Distillation

Asian Conference on Computer Vision (ACCV), 2022

12 July 2022

Bing Li

ArXiv (abs)PDF HTML HuggingFace (1 upvotes)

Papers citing "Cross-Architecture Knowledge Distillation"

26 / 26 papers shown

Title
Logit-Based Losses Limit the Effectiveness of Feature Knowledge Distillation Nicholas Cooper Lijun Chen Sailesh Dwivedy Danna Gurari 100 0 0 18 Nov 2025
UHKD: A Unified Framework for Heterogeneous Knowledge Distillation via Frequency-Domain Representations Fengming Yu Haiwei Pan Kejia Zhang Jian Guan Haiying Jiang 137 0 0 28 Oct 2025
Data Efficient Any Transformer-to-Mamba Distillation via Attention Bridge Penghao Wang Yuhao Zhou Mengxuan Wu Panpan Zhang Zhangyang Wang Kai Wang Mamba 257 0 0 22 Oct 2025
MOCHA: Multi-modal Objects-aware Cross-arcHitecture Alignment Elena Camuffo F. Barbato Mete Ozay Simone Milani Umberto Michieli ObjD 291 1 0 17 Sep 2025
Efficient Learned Image Compression Through Knowledge Distillation Fabien Allemand Attilio Fiandrotti Sumanta Chaudhuri Alaa Mazouz 85 0 0 12 Sep 2025
Cross-Architecture Distillation Made Simple with Redundancy Suppression Weijia Zhang Yuehao Liu Wu Ran Chao Ma 162 2 0 29 Jul 2025
Clip4Retrofit: Enabling Real-Time Image Labeling on Edge Devices via Cross-Architecture CLIP Distillation Li Zhong Ahmed Ghazal Jun-Jun Wan Frederik Zilly Patrick Mackens Joachim E. Vollrath Bogdan Sorin Coseriu 351 1 0 23 May 2025
Hybrid Knowledge Transfer through Attention and Logit Distillation for On-Device Vision Systems in Agricultural IoT Stanley Mugisha Rashid Kisitu Florence Tushabe 157 3 0 21 Apr 2025
A Transformer-in-Transformer Network Utilizing Knowledge Distillation for Image Recognition Dewan Tauhid Rahman Yeahia Sarker Antar Mazumder Md. Shamim Anower ViT 173 0 0 24 Feb 2025
TransMamba: Fast Universal Architecture Adaption from Transformers to Mamba Xiuwei Chen Sihao Lin Xiao Dong Sihao Lin Meng Cao Jiawei Han Yina Zhuang J. N. Han Hang Xu Xiaodan Liang Mamba 292 4 0 21 Feb 2025
Multi-Level Decoupled Relational Distillation for Heterogeneous Architectures Yaoxin Yang Peng Ye Weihao Lin Kangcong Li Yan Wen Jia Hao Tao Chen 277 0 0 10 Feb 2025
HDKD: Hybrid Data-Efficient Knowledge Distillation Network for Medical Image Classification Omar S. El-Assiouti Ghada Hamed Dina Khattab H. M. Ebied 253 15 0 10 Jul 2024
Cross-Architecture Auxiliary Feature Space Translation for Efficient Few-Shot Personalized Object Detection F. Barbato Umberto Michieli J. Moon Pietro Zanuttigh Mete Ozay 209 4 0 01 Jul 2024
Performance of computer vision algorithms for fine-grained classification using crowdsourced insect imagesIET Computer Vision (ICV), 2024 Rita Pucci Vincent J. Kalkman Dan Stowell 166 4 0 04 Apr 2024
A Comprehensive Review of Knowledge Distillation in Computer Vision Sheikh Musa Kaleem Tufail Rouf Gousia Habib Tausifa Jan Saleem Brejesh Lall VLM 260 17 0 01 Apr 2024
CATFace: Cross-Attribute-Guided Transformer with Self-Attention Distillation for Low-Quality Face Recognition Niloufar Alipour Talemi Hossein Kashiani Nasser M. Nasrabadi ViT CVBM 203 8 0 05 Jan 2024
Dual Teacher Knowledge Distillation with Domain Alignment for Face Anti-spoofing Zhe Kong Wentian Zhang Tao Wang Kaihao Zhang Yuexiang Li Xiaoying Tang Tong Lu AAML CVBM 159 2 0 02 Jan 2024
Choosing Wisely and Learning Deeply: Selective Cross-Modality Distillation via CLIP for Domain Generalization Jixuan Leng Yijiang Li Haohan Wang VLM 421 1 0 26 Nov 2023
Distilling Efficient Vision Transformers from CNNs for Semantic SegmentationPattern Recognition (Pattern Recogn.), 2023 Xueye Zheng Yunhao Luo Pengyuan Zhou Lin Wang 197 28 0 11 Oct 2023
RingMo-lite: A Remote Sensing Multi-task Lightweight Network with CNN-Transformer Hybrid Framework Yuelei Wang Ting Zhang Liangjin Zhao Lin Hu Zhechao Wang ... Kaiqiang Chen Xuan Zeng Zhirui Wang Hongqi Wang Xian Sun 255 9 0 16 Sep 2023
Cross Architecture Distillation for Face RecognitionACM Multimedia (ACM MM), 2023 Weisong Zhao Xiangyu Zhu Zhixiang He Xiaoyu Zhang Zhen Lei CVBM 218 10 0 26 Jun 2023
Towards Efficient Task-Driven Model Reprogramming with Foundation Models Shoukai Xu Jiangchao Yao Ran Luo Shuhai Zhang Zihao Lian Zhuliang Yu Bo Han Yaowei Wang 211 6 0 05 Apr 2023
Knowledge Distillation in Vision Transformers: A Critical Review Gousia Habib Tausifa Jan Saleem Brejesh Lall 260 23 0 04 Feb 2023
SDR: Efficient Neural Re-ranking using Succinct Document Representation Nachshon Cohen Amit Portnoy B. Fetahu A. Ingber AI4TS 190 11 0 03 Oct 2021
Contrastive Representation DistillationInternational Conference on Learning Representations (ICLR), 2019 Yonglong Tian Dilip Krishnan Phillip Isola 1.2K 1,202 0 23 Oct 2019
Xception: Deep Learning with Depthwise Separable ConvolutionsComputer Vision and Pattern Recognition (CVPR), 2016 François Chollet MDE BDL PINN 2.8K 16,539 0 07 Oct 2016