Distilling the Knowledge in a Neural Network

9 March 2015

Papers citing "Distilling the Knowledge in a Neural Network"

50 / 327 papers shown

Title
Non-autoregressive Sequence-to-Sequence Vision-Language Models Kunyu Shi Qi Dong Luis Goncalves Zhuowen Tu Stefano Soatto VLM 72 3 0 04 Mar 2024
Rethinking Invariance Regularization in Adversarial Training to Improve Robustness-Accuracy Trade-off Futa Waseda Ching-Chun Chang Isao Echizen AAML 64 0 0 22 Feb 2024
On the Byzantine-Resilience of Distillation-Based Federated Learning Christophe Roux Max Zimmer Sebastian Pokutta AAML 80 1 0 19 Feb 2024
Large Language Models: A Survey Shervin Minaee Tomas Mikolov Narjes Nikzad M. Asgari-Chenaghlu R. Socher Xavier Amatriain Jianfeng Gao ALM LM&MA ELM 160 389 0 09 Feb 2024
Masked LoGoNet: Fast and Accurate 3D Image Analysis for Medical Domain Amin Karimi Monsefi Payam Karisani Mengxi Zhou Stacey S. Choi Nathan Doble Heng Ji Srinivasan Parthasarathy R. Ramnath 62 5 0 09 Feb 2024
Is Adversarial Training with Compressed Datasets Effective? Tong Chen Raghavendra Selvan AAML 97 0 0 08 Feb 2024
Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes Lucio Dery Steven Kolawole Jean-Francois Kagey Virginia Smith Graham Neubig Ameet Talwalkar 54 29 0 08 Feb 2024
EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 73 144 0 26 Jan 2024
Enhancing Consistency and Mitigating Bias: A Data Replay Approach for Incremental Learning Chenyang Wang Junjun Jiang Xingyu Hu Xianming Liu Xiangyang Ji 66 3 0 12 Jan 2024
Knowledge-enhanced Multi-perspective Video Representation Learning for Scene Recognition Xuzheng Yu Chen Jiang Wei Zhang Tian Gan Linlin Chao Jianan Zhao Yuan Cheng Qingpei Guo Wei Chu 53 0 0 09 Jan 2024
CBQ: Cross-Block Quantization for Large Language Models Xin Ding Xiaoyu Liu Zhijun Tu Yun-feng Zhang Wei Li ... Hanting Chen Yehui Tang Zhiwei Xiong Baoqun Yin Yunhe Wang MQ 56 14 0 13 Dec 2023
Unsupervised Video Domain Adaptation with Masked Pre-Training and Collaborative Self-Training Arun V. Reddy William Paul Corban Rivera Ketul Shah Celso M. de Melo Rama Chellappa 60 4 0 05 Dec 2023
Maximizing Discrimination Capability of Knowledge Distillation with Energy Function Seonghak Kim Gyeongdo Ham Suin Lee Donggon Jang Daeshik Kim 108 4 0 24 Nov 2023
Robustness-Reinforced Knowledge Distillation with Correlation Distance and Network Pruning Seonghak Kim Gyeongdo Ham Yucheol Cho Daeshik Kim 59 4 0 23 Nov 2023
Bridging Classical and Quantum Machine Learning: Knowledge Transfer From Classical to Quantum Neural Networks Using Knowledge Distillation Mohammad Junayed Hasan M.R.C. Mahdy 46 2 0 23 Nov 2023
Neural Lattice Reduction: A Self-Supervised Geometric Deep Learning Approach Giovanni Luca Marchetti Gabriele Cesa Kumar Pratik Arash Behboodi 85 2 0 14 Nov 2023
Audio-Visual Speaker Tracking: Progress, Challenges, and Future Directions Jinzheng Zhao Yong-mei Xu Xinyuan Qian Davide Berghi Peipei Wu Meng Cui Jianyuan Sun Philip J. B. Jackson Wenwu Wang BDL 70 7 0 23 Oct 2023
SpikeCLIP: A Contrastive Language-Image Pretrained Spiking Neural Network Changze Lv Tianlong Li Changze Lv Yufei Gu Jianhan Xu Cenyuan Zhang Muling Wu Xiaoqing Zheng Xuanjing Huang CLIP VLM 65 3 0 10 Oct 2023
DataDAM: Efficient Dataset Distillation with Attention Matching A. Sajedi Samir Khaki Ehsan Amjadian Lucy Z. Liu Y. Lawryshyn Konstantinos N. Plataniotis DD 82 64 0 29 Sep 2023
Diverse Target and Contribution Scheduling for Domain Generalization Shaocong Long Qianyu Zhou Soham Dan Lizhuang Ma Yuan Luo 81 8 0 28 Sep 2023
HPCR: Holistic Proxy-based Contrastive Replay for Online Continual Learning Huiwei Lin Shanshan Feng Baoquan Zhang Xutao Li Yew-Soon Ong CLL 64 0 0 26 Sep 2023
Interpretability-Aware Vision Transformer Yao Qiang Chengyin Li Prashant Khanduri D. Zhu ViT 118 7 0 14 Sep 2023
PILOT: A Pre-Trained Model-Based Continual Learning Toolbox Hai-Long Sun Da-Wei Zhou Han-Jia Ye De-Chuan Zhan CLL 168 29 0 13 Sep 2023
Training Acceleration of Low-Rank Decomposed Networks using Sequential Freezing and Rank Quantization Habib Hajimolahoseini Walid Ahmed Yang Liu OffRL MQ 37 7 0 07 Sep 2023
Linear Alignment of Vision-language Models for Image Captioning Fabian Paischer M. Hofmarcher Sepp Hochreiter Thomas Adler CLIP VLM 87 0 0 10 Jul 2023
Review helps learn better: Temporal Supervised Knowledge Distillation Dongwei Wang Zhi Han Yanmei Wang Xi’ai Chen Baichen Liu Yandong Tang 90 1 0 03 Jul 2023
Data-Free Quantization via Mixed-Precision Compensation without Fine-Tuning Jun Chen Shipeng Bai Tianxin Huang Mengmeng Wang Guanzhong Tian Y. Liu MQ 47 18 0 02 Jul 2023
Long-Tailed Continual Learning For Visual Food Recognition Jiangpeng He Luotao Lin Jack Ma H. Eicher-Miller Fengqing Zhu Fengqing M Zhu 76 14 0 01 Jul 2023
When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directions Weiming Zhuang Chen Chen Lingjuan Lyu Chong Chen Yaochu Jin Lingjuan Lyu AIFin AI4CE 112 93 0 27 Jun 2023
Learning without Forgetting for Vision-Language Models Da-Wei Zhou Yuanhan Zhang Jingyi Ning Jingyi Ning De-Chuan Zhan De-Chuan Zhan Ziwei Liu VLM CLL 100 41 0 30 May 2023
Learning to Learn from APIs: Black-Box Data-Free Meta-Learning Zixuan Hu Li Shen Zhenyi Wang Baoyuan Wu Chun Yuan Dacheng Tao 74 7 0 28 May 2023
Combining Multi-Objective Bayesian Optimization with Reinforcement Learning for TinyML M. Deutel G. Kontes Christopher Mutschler Jürgen Teich 122 0 0 23 May 2023
Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation Deyi Ji Haoran Wang Mingyuan Tao Jianqiang Huang Xiansheng Hua Hongtao Lu 97 61 0 06 May 2023
If At First You Don't Succeed: Test Time Re-ranking for Zero-shot, Cross-domain Retrieval Finlay G. C. Hudson W. Smith ViT 72 1 0 30 Mar 2023
ERSAM: Neural Architecture Search For Energy-Efficient and Real-Time Social Ambiance Measurement Chaojian Li Wenwan Chen Jiayi Yuan Yingyan Lin Ashutosh Sabharwal 40 0 0 19 Mar 2023
Deep Learning for Cross-Domain Few-Shot Visual Recognition: A Survey Huali Xu Shuaifeng Zhi Shuzhou Sun Vishal M. Patel Li Liu 76 13 0 15 Mar 2023
Oriented Object Detection in Optical Remote Sensing Images using Deep Learning: A Survey Kunlin Wang Zi Wang Zhang Li Ang Su Xichao Teng Minhao Liu Qifeng Yu Qifeng Yu ObjD 109 9 0 21 Feb 2023
Transformadores: Fundamentos teoricos y Aplicaciones J. D. L. Torre 93 0 0 18 Feb 2023
Learning a Consensus Sub-Network with Polarization Regularization and One Pass Training Xiaoying Zhi Varun Babbar P. Sun Fran Silavong Ruibo Shi Sean J. Moran Sean Moran 68 1 0 17 Feb 2023
FedICT: Federated Multi-task Distillation for Multi-access Edge Computing Zhiyuan Wu Sheng Sun Yuwei Wang Min Liu Quyang Pan Xue Jiang Bo Gao 67 32 0 01 Jan 2023
COLT: Cyclic Overlapping Lottery Tickets for Faster Pruning of Convolutional Neural Networks Md. Ismail Hossain Mohammed Rakib M. M. L. Elahi Nabeel Mohammed Shafin Rahman 60 1 0 24 Dec 2022
Exploiting Features and Logits in Heterogeneous Federated Learning Yun-Hin Chan Edith C.H. Ngai FedML 44 2 0 27 Oct 2022
MOVE: Effective and Harmless Ownership Verification via Embedded External Features Yiming Li Linghui Zhu Xiaojun Jia Yang Bai Yong Jiang Shutao Xia Xiaochun Cao Kui Ren AAML 66 13 0 04 Aug 2022
Reflective-Net: Learning from Explanations Johannes Schneider Michalis Vlachos FAtt OffRL LRM 66 18 0 27 Nov 2020
AdderSR: Towards Energy Efficient Image Super-Resolution Dehua Song Yunhe Wang Hanting Chen Chang Xu Chunjing Xu Dacheng Tao SupR 48 83 0 18 Sep 2020
Adversarial Examples on Object Recognition: A Comprehensive Survey A. Serban E. Poll Joost Visser AAML 55 73 0 07 Aug 2020
Labelling unlabelled videos from scratch with multi-modal self-supervision Yuki M. Asano Mandela Patrick Christian Rupprecht Andrea Vedaldi SSL 40 152 0 24 Jun 2020
Mix2FLD: Downlink Federated Learning After Uplink Federated Distillation With Two-Way Mixup Seungeun Oh Jihong Park Eunjeong Jeong Hyesung Kim M. Bennis Seong-Lyun Kim FedML 31 58 0 17 Jun 2020
Prior knowledge distillation based on financial time series Jie Fang Jianwu Lin AI4TS 15 6 0 16 Jun 2020
Multi-fidelity Neural Architecture Search with Knowledge Distillation I. Trofimov Nikita Klyuchnikov Mikhail Salnikov Alexander N. Filippov Evgeny Burnaev 44 15 0 15 Jun 2020