Distilling the Knowledge in a Neural Network

9 March 2015

Papers citing "Distilling the Knowledge in a Neural Network"

50 / 327 papers shown

Title
Unleashing the Potential of Pre-Trained Diffusion Models for Generalizable Person Re-Identification Jiachen Li Xiaojin Gong DiffM 113 0 0 10 Feb 2025
Who Taught You That? Tracing Teachers in Model Distillation Somin Wadhwa Chantal Shaib Silvio Amir Byron C. Wallace 134 2 0 10 Feb 2025
Provably Near-Optimal Federated Ensemble Distillation with Negligible Overhead Won-Jun Jang Hyeon-Seo Park Si-Hyeon Lee FedML 365 0 0 10 Feb 2025
Do we really have to filter out random noise in pre-training data for language models? Jinghan Ru Yuxin Xie Xianwei Zhuang Yuguo Yin Zhihui Guo Zhiming Liu Qianli Ren Yuexian Zou 115 4 0 10 Feb 2025
Efficient-vDiT: Efficient Video Diffusion Transformers With Attention Tile Hangliang Ding Dacheng Li Runlong Su Peiyuan Zhang Zhijie Deng Ion Stoica Hao Zhang VGen 80 6 0 10 Feb 2025
Compressing Model with Few Class-Imbalance Samples: An Out-of-Distribution Expedition Tian-Shuang Wu Shen-Huan Lyu Ning Chen Zhihao Qu Baoliu Ye OODD 65 0 0 09 Feb 2025
UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation Tao Zhang Jinyong Wen Zhen Chen Kun Ding Di Zhang Chunhong Pan 129 1 0 04 Feb 2025
Choose Your Model Size: Any Compression by a Single Gradient Descent Martin Genzel Patrick Putzky Pengfei Zhao Siyang Song Mattes Mollenhauer Robert Seidel Stefan Dietzel Thomas Wollmann 58 0 0 03 Feb 2025
Mini-ResEmoteNet: Leveraging Knowledge Distillation for Human-Centered Design Amna Murtada Omnia Abdelrhman Tahani Abdalla Attia 114 0 0 30 Jan 2025
Merino: Entropy-driven Design for Generative Language Models on IoT Devices Youpeng Zhao Ming Lin Huadong Tang Qiang Wu Jun Wang 96 0 0 28 Jan 2025
Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs Nicolas Boizard Kevin El Haddad C´eline Hudelot Pierre Colombo 100 15 0 28 Jan 2025
Variational Bayesian Adaptive Learning of Deep Latent Variables for Acoustic Knowledge Transfer Hu Hu Sabato Marco Siniscalchi Chao-Han Huck Yang Chin-Hui Lee 95 0 0 28 Jan 2025
Large Language Model Distilling Medication Recommendation Model Qidong Liu Xian Wu Xiangyu Zhao Yuanshao Zhu Zijian Zhang Feng Tian Yefeng Zheng LM&MA 116 18 0 28 Jan 2025
The Effect of Optimal Self-Distillation in Noisy Gaussian Mixture Model Kaito Takanami Takashi Takahashi Ayaka Sakata 65 1 0 27 Jan 2025
Towards Robust Unsupervised Attention Prediction in Autonomous Driving Mengshi Qi Xiaoyang Bi Pengfei Zhu Huadong Ma 104 0 0 25 Jan 2025
Unlearning Clients, Features and Samples in Vertical Federated Learning Ayush K. Varshney Konstantinos Vandikas V. Torra MU 49 1 0 23 Jan 2025
Accelerate High-Quality Diffusion Models with Inner Loop Feedback M. Gwilliam Han Cai Di Wu Abhinav Shrivastava Zhiyu Cheng 117 0 0 22 Jan 2025
YouLeQD: Decoding the Cognitive Complexity of Questions and Engagement in Online Educational Videos from Learners' Perspectives Nong Ming Sachin Sharma Jiho Noh AI4Ed 72 0 0 20 Jan 2025
Elucidating the Design Space of Dataset Condensation Shitong Shao Zikai Zhou Huanran Chen Zhiqiang Shen DD 93 9 0 20 Jan 2025
ACE: Anatomically Consistent Embeddings in Composition and Decomposition Ziyu Zhou Haozhe Luo M. Taher Jiaxuan Pang Xiaowei Ding Michael B. Gotway Jianming Liang MedIm 85 0 0 20 Jan 2025
A Comprehensive Survey of Foundation Models in Medicine Wasif Khan Seowung Leem Kyle B. See Joshua K. Wong Shaoting Zhang R. Fang AI4CE LM&MA VLM 160 21 0 17 Jan 2025
Revisiting Rogers' Paradox in the Context of Human-AI Interaction Katherine M. Collins Umang Bhatt Ilia Sucholutsky 101 1 0 16 Jan 2025
WhiSPA: Semantically and Psychologically Aligned Whisper with Self-Supervised Contrastive and Student-Teacher Learning Rajath Rao Adithya Ganesan Oscar Kjell Jonah Luby Akshay Raghavan ... B. Luft Camilo Ruggero Neville Ryant R. Kotov H. Andrew Schwartz 67 0 0 15 Jan 2025
Incrementally Learning Multiple Diverse Data Domains via Multi-Source Dynamic Expansion Model RunQing Wu Fei Ye QiHe Liu Guoxi Huang Jinyu Guo Rongyao Hu CLL 325 0 0 15 Jan 2025
Overcoming Language Priors for Visual Question Answering Based on Knowledge Distillation Daowan Peng Wei Wei 331 1 0 10 Jan 2025
TipSegNet: Fingertip Segmentation in Contactless Fingerprint Imaging L. Ruzicka Bernhard Kohn Clemens Heitzinger 104 0 0 10 Jan 2025
Merging Feed-Forward Sublayers for Compressed Transformers Neha Verma Kenton W. Murray Kevin Duh AI4CE 79 0 0 10 Jan 2025
FedSA: A Unified Representation Learning via Semantic Anchors for Prototype-based Federated Learning Yanbing Zhou Xiangmou Qu Chenlong You Jiyang Zhou Jingyue Tang Xin Zheng Chunmao Cai Yingbo Wu FedML 84 3 0 09 Jan 2025
iServe: An Intent-based Serving System for LLMs Dimitrios Liakopoulos Tianrui Hu Prasoon Sinha N. Yadwadkar VLM 378 0 0 08 Jan 2025
CURing Large Models: Compression via CUR Decomposition Sanghyeon Park Soo-Mook Moon 52 0 0 08 Jan 2025
InfiFusion: A Unified Framework for Enhanced Cross-Model Reasoning via LLM Fusion Zhaoyi Yan Zhijie Sang Yiming Zhang Yuhao Fu Baoyi He Qi Zhou Yining Di Chunlin Ji Shengyu Zhang Fei Wu MoMe LRM 78 2 0 06 Jan 2025
Quantization Meets Reasoning: Exploring LLM Low-Bit Quantization Degradation for Mathematical Reasoning Zhen Li Yupeng Su Runming Yang C. Xie Zehua Wang Zhongwei Xie Ngai Wong Hongxia Yang MQ LRM 82 4 0 06 Jan 2025
Registering Source Tokens to Target Language Spaces in Multilingual Neural Machine Translation Zhi Qu Yiran Wang Jiannan Mao Chenchen Ding Hideki Tanaka Masao Utiyama Taro Watanabe LRM 58 0 0 06 Jan 2025
Activity-aware Human Mobility Prediction with Hierarchical Graph Attention Recurrent Network Yihong Tang Junlin He Zhan Zhao HAI 98 6 0 03 Jan 2025
Self-Corrected Flow Distillation for Consistent One-Step and Few-Step Text-to-Image Generation Quan Dao Hao Phung T. Dao Dimitris Metaxas Anh Tran 119 1 0 22 Dec 2024
Spatial-Temporal Knowledge Distillation for Takeaway Recommendation Shuyuan Zhao Wei Chen Boyan Shi Liyong Zhou Shuohao Lin Huaiyu Wan 125 0 0 21 Dec 2024
Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Models Xiao Cui Mo Zhu Yulei Qin Liang Xie Wengang Zhou Haoyang Li 125 6 0 19 Dec 2024
Falcon: Faster and Parallel Inference of Large Language Models through Enhanced Semi-Autoregressive Drafting and Custom-Designed Decoding Tree Xiangxiang Gao Weisheng Xie Yiwei Xiang Feng Ji 117 6 0 17 Dec 2024
Wearable Accelerometer Foundation Models for Health via Knowledge Distillation Salar Abbaspourazad Anshuman Mishra Joseph D. Futoma Andrew C. Miller Ian Shapiro 123 0 0 15 Dec 2024
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs Akhiad Bercovich Tomer Ronen Talor Abramovich Nir Ailon Nave Assaf ... Ido Shahaf Oren Tropp Omer Ullman Argov Ran Zilberstein Ran El-Yaniv 130 3 0 28 Nov 2024
Fall Leaf Adversarial Attack on Traffic Sign Classification Anthony Etim Jakub Szefer AAML 106 3 0 27 Nov 2024
SoK: Decentralized AI (DeAI) Zhipeng Wang Rui Sun Elizabeth Lui Vatsal Shah Xihan Xiong Jiahao Sun Davide Crapis William Knottenbelt 136 1 0 26 Nov 2024
RECAST: Reparameterized, Compact weight Adaptation for Sequential Tasks Nazia Tasnim Bryan A. Plummer CLL OffRL 101 0 0 25 Nov 2024
Partial Knowledge Distillation for Alleviating the Inherent Inter-Class Discrepancy in Federated Learning Xiaoyu Gan Xizi Chen Jingyang Zhu Xiaomeng Wang Jingbo Jiang Chi-Ying Tsui FedML 116 0 0 23 Nov 2024
Anti-Forgetting Adaptation for Unsupervised Person Re-identification Hao Chen Francois Bremond Nicu Sebe Shiliang Zhang CLL 149 1 0 22 Nov 2024
Adversarial Prompt Distillation for Vision-Language Models Lin Luo Xin Wang Bojia Zi Shihao Zhao Xingjun Ma Yu-Gang Jiang AAML VLM 103 3 0 22 Nov 2024
FuseGPT: Learnable Layers Fusion of Generative Pre-trained Transformers Zehua Pei Hui-Ling Zhen Xianzhi Yu Sinno Jialin Pan Mingxuan Yuan Bei Yu AI4CE 139 3 0 21 Nov 2024
Label Distribution Shift-Aware Prediction Refinement for Test-Time Adaptation M-U Jang Hye Won Chung TTA 386 0 0 20 Nov 2024
Heuristic-Free Multi-Teacher Learning Huy Thong Nguyen En-Hung Chu Lenord Melvix Jazon Jiao Chunglin Wen Benjamin Louie 91 0 0 19 Nov 2024
Exploring Feature-based Knowledge Distillation for Recommender System: A Frequency Perspective Zhangchi Zhu Wei Zhang 65 0 0 16 Nov 2024