Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation

17 May 2019

Papers citing "Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillation"

50 / 143 papers shown

Title
How to Train Your Metamorphic Deep Neural Network Thomas Sommariva Simone Calderara Angelo Porrello 28 0 0 07 May 2025
Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques Sanjay Surendranath Girija Shashank Kapoor Lakshit Arora Dipen Pradhan Aman Raj Ankit Shetgaonkar 57 0 0 05 May 2025
Erased but Not Forgotten: How Backdoors Compromise Concept Erasure Jonas Henry Grebe Tobias Braun Marcus Rohrbach Anna Rohrbach AAML 85 0 0 29 Apr 2025
Learning Critically: Selective Self Distillation in Federated Learning on Non-IID Data Yuting He Yiqiang Chen Xiaodong Yang H. Yu Yi-Hua Huang Yang Gu FedML 57 20 0 20 Apr 2025
Earth-Adapter: Bridge the Geospatial Domain Gaps with Mixture of Frequency Adaptation Xiaoxing Hu Ziyang Gong Yixuan Wang Yuru Jia Gen Luo Xue Yang 133 0 0 08 Apr 2025
NCAP: Scene Text Image Super-Resolution with Non-CAtegorical Prior Dongwoo Park Suk Pil Ko 153 0 0 01 Apr 2025
You Are Your Own Best Teacher: Achieving Centralized-level Performance in Federated Learning under Heterogeneous and Long-tailed Data Shanshan Yan Zexi Li Chao-Xiang Wu Meng Pang Yang Lu Yan Yan Hanzi Wang FedML 61 0 0 10 Mar 2025
Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models Xubin Wang Zhiqing Tang Jianxiong Guo Tianhui Meng Chenhao Wang Tian-sheng Wang Weijia Jia 52 1 0 08 Mar 2025
A Lightweight and Extensible Cell Segmentation and Classification Model for Whole Slide Images N. Shvetsov T. Kilvaer M. Tafavvoghi Anders Sildnes Kajsa Møllersen Lill-ToveRasmussen Busund L. A. Bongo VLM 71 1 0 26 Feb 2025
CR-CTC: Consistency regularization on CTC for improved speech recognition Zengwei Yao Wei Kang Xiaoyu Yang Fangjun Kuang Liyong Guo Han Zhu Zengrui Jin Zhaoqing Li Long Lin Daniel Povey 53 0 0 17 Feb 2025
Self-Improving Transformers Overcome Easy-to-Hard and Length Generalization Challenges Nayoung Lee Ziyang Cai Avi Schwarzschild Kangwook Lee Dimitris Papailiopoulos ReLM VLM LRM AI4CE 83 4 0 03 Feb 2025
QCS: Feature Refining from Quadruplet Cross Similarity for Facial Expression Recognition Cong Wang Li Chen Lili Wang Zhaofan Li Xuebin Lv 78 1 0 28 Jan 2025
sDREAMER: Self-distilled Mixture-of-Modality-Experts Transformer for Automatic Sleep Staging Jingyuan Chen Yuan Yao Mie Anderson Natalie Hauglund Celia Kjaerby Verena Untiet Maiken Nedergaard Jiebo Luo 43 1 0 28 Jan 2025
The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model Kaito Takanami Takashi Takahashi Ayaka Sakata 37 0 0 27 Jan 2025
Hybrid Attention Model Using Feature Decomposition and Knowledge Distillation for Glucose Forecasting Ebrahim Farahmand Shovito Barua Soumma Nooshin Taheri Chatrudi Hassan Ghasemzadeh 31 2 0 16 Nov 2024
A Lightweight Target-Driven Network of Stereo Matching for Inland Waterways Jing Su Yiqing Zhou Yu Zhang Chao Wang Yi Wei 3DV 28 0 0 10 Oct 2024
Harmonizing knowledge Transfer in Neural Network with Unified Distillation Yaomin Huang Zaomin Yan Chaomin Shen Faming Fang Guixu Zhang 31 0 0 27 Sep 2024
Network Fission Ensembles for Low-Cost Self-Ensembles Hojung Lee Jong-Seok Lee UQCV 58 0 0 05 Aug 2024
Continual Learning with Diffusion-based Generative Replay for Industrial Streaming Data Jiayi He Jiao Chen Qianmiao Liu Suyan Dai Jianhua Tang Dongpo Liu DiffM AI4CE 37 4 0 22 Jun 2024
DistilDoc: Knowledge Distillation for Visually-Rich Document Applications Jordy Van Landeghem Subhajit Maity Ayan Banerjee Matthew Blaschko Marie-Francine Moens Josep Lladós Sanket Biswas 50 2 0 12 Jun 2024
$$\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning$ $\textit{Trans-LoRA}$ : towards data-free Transferable Parameter Efficient Finetuning Runqian Wang Soumya Ghosh David D. Cox Diego Antognini Aude Oliva Rogerio Feris Leonid Karlinsky 32 1 0 27 May 2024
CrossMatch: Enhance Semi-Supervised Medical Image Segmentation with Perturbation Strategies and Knowledge Distillation Bin Zhao Chunshi Wang Shuxue Ding 43 2 0 01 May 2024
MergeNet: Knowledge Migration across Heterogeneous Models, Tasks, and Modalities Kunxi Li Tianyu Zhan Kairui Fu Shengyu Zhang Kun Kuang Jiwei Li Zhou Zhao Fei Wu MoMe 24 0 0 20 Apr 2024
Adversarial Sparse Teacher: Defense Against Distillation-Based Model Stealing Attacks Using Adversarial Examples Eda Yilmaz H. Keles AAML 16 2 0 08 Mar 2024
A Teacher-Free Graph Knowledge Distillation Framework with Dual Self-Distillation Lirong Wu Haitao Lin Zhangyang Gao Guojiang Zhao Stan Z. Li 40 8 0 06 Mar 2024
Continuous Sign Language Recognition Based on Motor attention mechanism and frame-level Self-distillation Qidan Zhu Jing Li Fei Yuan Quan Gan SLR 53 3 0 29 Feb 2024
Enhancing Multilingual Capabilities of Large Language Models through Self-Distillation from Resource-Rich Languages Yuan Zhang Yile Wang Zijun Liu Shuo Wang Xiaolong Wang Peng Li Maosong Sun Yang Liu LRM 27 11 0 19 Feb 2024
P2Seg: Pointly-supervised Segmentation via Mutual Distillation Zipeng Wang Xuehui Yu Xumeng Han Wenwen Yu Zhixun Huang Jianbin Jiao Zhenjun Han 31 0 0 18 Jan 2024
Adaptive Depth Networks with Skippable Sub-Paths Woochul Kang 33 1 0 27 Dec 2023
All but One: Surgical Concept Erasing with Model Preservation in Text-to-Image Diffusion Models Seunghoo Hong Juhun Lee Simon S. Woo 17 18 0 20 Dec 2023
Cooperative Learning for Cost-Adaptive Inference Xingli Fang Richard M. Bradford Jung-Eun Kim 32 1 0 13 Dec 2023
A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations Hui Ma Jian Wang Hongfei Lin Bo Zhang Yijia Zhang Bo Xu 23 40 0 31 Oct 2023
Towards Generalized Multi-stage Clustering: Multi-view Self-distillation Jiatai Wang Zhiwei Xu Xin Wang Tao Li 16 1 0 29 Oct 2023
Debiasing, calibrating, and improving Semi-supervised Learning performance via simple Ensemble Projector Khanh-Binh Nguyen 27 2 0 24 Oct 2023
Teacher-Student Architecture for Knowledge Distillation: A Survey Chengming Hu Xuan Li Danyang Liu Haolun Wu Xi Chen Ju Wang Xue Liu 21 16 0 08 Aug 2023
Sampling to Distill: Knowledge Transfer from Open-World Data Yuzheng Wang Zhaoyu Chen Jie M. Zhang Dingkang Yang Zuhao Ge Yang Liu Siao Liu Yunquan Sun Wenqiang Zhang Lizhe Qi 28 9 0 31 Jul 2023
Modify Training Directions in Function Space to Reduce Generalization Error Yi Yu Wenlian Lu Boyu Chen 27 0 0 25 Jul 2023
Expediting Building Footprint Extraction from High-resolution Remote Sensing Images via progressive lenient supervision Haonan Guo Bo Du Chen Wu Xin Su L. Zhang 18 0 0 23 Jul 2023
FedDefender: Client-Side Attack-Tolerant Federated Learning Sungwon Park Sungwon Han Fangzhao Wu Sundong Kim Bin Zhu Xing Xie Meeyoung Cha FedML AAML 25 20 0 18 Jul 2023
Intuitive Access to Smartphone Settings Using Relevance Model Trained by Contrastive Learning Joonyoung Kim Kangwook Lee Haebin Shin Hurnjoo Lee Sechun Kang Byunguk Choi Dong Shin Joohyung Lee 23 0 0 15 Jul 2023
Frameless Graph Knowledge Distillation Dai Shi Zhiqi Shao Yi Guo Junbin Gao 39 4 0 13 Jul 2023
Modality-Agnostic Learning for Medical Image Segmentation Using Multi-modality Self-distillation Qisheng He Nicholas Summerfield Ming Dong C. Glide-Hurst 19 1 0 06 Jun 2023
Is Synthetic Data From Diffusion Models Ready for Knowledge Distillation? Zheng Li Yuxuan Li Penghai Zhao Renjie Song Xiang Li Jian Yang 34 19 0 22 May 2023
Less Can Be More: Unsupervised Graph Pruning for Large-scale Dynamic Graphs Jintang Li Sheng Tian Ruofan Wu Liang Zhu Welong Zhao Changhua Meng Liang Chen Zibin Zheng Hongzhi Yin 34 10 0 18 May 2023
Learning Summary-Worthy Visual Representation for Abstractive Summarization in Video Zenan Xu Xiaojun Meng Yasheng Wang Qinliang Su Zexuan Qiu Xin Jiang Qun Liu 27 3 0 08 May 2023
Self-discipline on multiple channels Jiutian Zhao Liangchen Luo Hao Wang 26 0 0 27 Apr 2023
FSNet: Redesign Self-Supervised MonoDepth for Full-Scale Depth Prediction for Autonomous Driving Yuxuan Liu Zhenhua Xu Huaiyang Huang Lujia Wang Ming-Yu Liu MDE 46 3 0 21 Apr 2023
Distilling Token-Pruned Pose Transformer for 2D Human Pose Estimation Feixiang Ren ViT 19 2 0 12 Apr 2023
Self-Distillation for Gaussian Process Regression and Classification Kenneth Borup L. Andersen 11 2 0 05 Apr 2023
Improving Neural Topic Models with Wasserstein Knowledge Distillation Suman Adhya Debarshi Kumar Sanyal BDL 20 1 0 27 Mar 2023