Distilling the Knowledge in a Neural Network

9 March 2015

Papers citing "Distilling the Knowledge in a Neural Network"

50 / 327 papers shown

Title
Query-based Knowledge Transfer for Heterogeneous Learning Environments Norah Alballa Wenxuan Zhang Ziquan Liu A. Abdelmoniem Mohamed Elhoseiny Marco Canini 90 0 0 12 Apr 2025
Pretraining Language Models for Diachronic Linguistic Change Discovery Elisabeth Fittschen Sabrina Li Tom Lippincott Leshem Choshen Craig Messner 78 0 0 07 Apr 2025
Corrected with the Latest Version: Make Robust Asynchronous Federated Learning Possible Chaoyi Lu Yiding Sun Pengbo Li Zhichuan Yang FedML 54 0 0 05 Apr 2025
RANa: Retrieval-Augmented Navigation G. Monaci Rafael Sampaio de Rezende Romain Deffayet G. Csurka G. Bono Hervé Déjean Stéphane Clinchant Christian Wolf 59 9 0 04 Apr 2025
Distillation and Refinement of Reasoning in Small Language Models for Document Re-ranking Chris Samarinas Hamed Zamani ALM LRM 103 2 0 04 Apr 2025
Catch Me if You Search: When Contextual Web Search Results Affect the Detection of Hallucinations Mahjabin Nahar Eun-Ju Lee Jin Won Park Dongwon Lee HILM 99 0 0 01 Apr 2025
Expanding-and-Shrinking Binary Neural Networks Xulong Shi Caiyi Sun Zhi Qi Liu Hao Xiaodong Yang MQ 117 0 0 31 Mar 2025
Pluggable Style Representation Learning for Multi-Style Transfer Hongda Liu Longguang Wang Weijun Guan Ye Zhang Yulan Guo 100 1 0 26 Mar 2025
Continual Learning With Quasi-Newton Methods Steven Vander Eeckt Hugo Van hamme CLL BDL 81 0 0 25 Mar 2025
Distilling Stereo Networks for Performant and Efficient Leaner Networks Rafia Rahim Samuel Woerz A. Zell 100 0 0 24 Mar 2025
Generative AI for Software Architecture. Applications, Trends, Challenges, and Future Directions Matteo Esposito Xiaozhou Li Sergio Moreschini Noman Ahmad T. Cerný Karthik Vaidhyanathan Valentina Lenarduzzi Davide Taibi AI4CE 64 0 0 17 Mar 2025
TikZero: Zero-Shot Text-Guided Graphics Program Synthesis Jonas Belouadi Eddy Ilg Margret Keuper Hideki Tanaka Masao Utiyama Raj Dabre Steffen Eger Simone Paolo Ponzetto 81 0 0 14 Mar 2025
Semantic-Supervised Spatial-Temporal Fusion for LiDAR-based 3D Object Detection Chaoqun Wang Xiaobin Hong Wenzhong Li Ruimao Zhang 3DPC 343 0 0 13 Mar 2025
EFC++: Elastic Feature Consolidation with Prototype Re-balancing for Cold Start Exemplar-free Incremental Learning Simone Magistri Tomaso Trinci Albin Soutif--Cormerais Joost van de Weijer Andrew D. Bagdanov 60 0 0 13 Mar 2025
Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Models Reza Shirkavand Peiran Yu Shangqian Gao Gowthami Somepalli Tom Goldstein Heng-Chiao Huang 146 2 0 13 Mar 2025
Robust Asymmetric Heterogeneous Federated Learning with Corrupted Clients Xiuwen Fang Mang Ye Di Lin FedML 103 1 0 12 Mar 2025
Training Plug-n-Play Knowledge Modules with Deep Context Distillation Lucas Caccia Alan Ansell Edoardo Ponti Ivan Vulić Alessandro Sordoni SyDa 394 0 0 11 Mar 2025
Training Domain Draft Models for Speculative Decoding: Best Practices and Insights Fenglu Hong Ravi Raju Jonathan Li Bo Li Urmish Thakker Avinash Ravichandran Swayambhoo Jain Changran Hu 57 0 0 10 Mar 2025
FEDS: Feature and Entropy-Based Distillation Strategy for Efficient Learned Image Compression H. Fu Jie Liang Zhenman Fang Jingning Han 78 0 0 09 Mar 2025
StructVPR++: Distill Structural and Semantic Knowledge with Weighting Samples for Visual Place Recognition Yanqing Shen Sanping Zhou Jingwen Fu Ke Xu Shitao Chen N. Zheng 103 0 0 09 Mar 2025
SplatPose: Geometry-Aware 6-DoF Pose Estimation from Single RGB Image via 3D Gaussian Splatting Linqi Yang Xiongwei Zhao Qihao Sun Ke Wang Ao Chen Peng Kang 3DGS 96 3 0 07 Mar 2025
Temporal Separation with Entropy Regularization for Knowledge Distillation in Spiking Neural Networks Kairong Yu Chengting Yu Tianqing Zhang Xiaochen Zhao Shu Yang Hongwei Wang Qiang Zhang Qi Xu 89 4 0 05 Mar 2025
Rapid Bone Scintigraphy Enhancement via Semantic Prior Distillation from Segment Anything Model Pengchen Liang Leijun Shi Huiping Yao Bin Pu Jianguo Chen ... Zheyu Chen Zhaozhao Xu Lite Xu Qing Chang Yiwei Li 91 0 0 04 Mar 2025
MAPS: Motivation-Aware Personalized Search via LLM-Driven Consultation Alignment Weicong Qin Yi Xu Weijie Yu Chenglei Shen Ming He Jianping Fan Xiao Zhang Jun Xu 140 0 0 03 Mar 2025
EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test Yuhui Li Fangyun Wei Chao Zhang Hongyang R. Zhang 147 11 0 03 Mar 2025
Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation Tiansheng Wen Yifei Wang Zequn Zeng Zhong Peng Yudi Su Xinyang Liu Bo Chen Hongwei Liu Stefanie Jegelka Chenyu You CLL 106 3 0 03 Mar 2025
Variations in Relevance Judgments and the Shelf Life of Test Collections Andrew Parry Maik Fröbe Harrisen Scells Ferdinand Schlatt Guglielmo Faggioli Saber Zerhoudi Sean MacAvaney Eugene Yang 59 2 0 28 Feb 2025
Investigating and Enhancing Vision-Audio Capability in Omnimodal Large Language Models Rui Hu Delai Qiu Shuyu Wei J.N. Zhang Yining Wang Shengping Liu Jitao Sang AuLLM VLM 76 0 0 27 Feb 2025
A Lightweight and Extensible Cell Segmentation and Classification Model for Whole Slide Images N. Shvetsov T. Kilvaer M. Tafavvoghi Anders Sildnes Kajsa Møllersen Lill-ToveRasmussen Busund L. A. Bongo VLM 83 1 0 26 Feb 2025
CLIMB-3D: Continual Learning for Imbalanced 3D Instance Segmentation Vishal Thengane Jean Lahoud Hisham Cholakkal Rao Muhammad Anwer L. Yin Xiatian Zhu Salman Khan CLL 360 0 0 24 Feb 2025
I2CKD : Intra- and Inter-Class Knowledge Distillation for Semantic Segmentation Ayoub Karine Thibault Napoléon M. Jridi VLM 173 0 0 24 Feb 2025
FLINT: Learning-based Flow Estimation and Temporal Interpolation for Scientific Ensemble Visualization Hamid Gadirov Jos B. T. M. Roerdink Steffen Frey AI4CE 96 1 0 24 Feb 2025
Improving the Transferability of Adversarial Examples by Inverse Knowledge Distillation Wenyuan Wu Zheng Liu Yong Chen Chao Su Dezhong Peng Xu Wang AAML 90 0 0 24 Feb 2025
GraphFM: Graph Factorization Machines for Feature Interaction Modeling Shu Wu Zekun Li Yunyue Su Zeyu Cui Xiaoyu Zhang Liang Wang 130 23 0 24 Feb 2025
Feature Aggregation with Latent Generative Replay for Federated Continual Learning of Socially Appropriate Robot Behaviours Nikhil Churamani Saksham Checker Fethiye Irmak Dogan Hao-Tien Lewis Chiang Hatice Gunes FedML 107 1 0 24 Feb 2025
Transfer Learning with Pre-trained Conditional Generative Models Shin'ya Yamaguchi Sekitoshi Kanai Atsutoshi Kumagai Daiki Chijiwa H. Kashima VLM CLL BDL DiffM 202 5 0 21 Feb 2025
A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks Thomas Schmied Thomas Adler Vihang Patil M. Beck Korbinian Poppel Johannes Brandstetter Günter Klambauer Razvan Pascanu Sepp Hochreiter 161 5 0 21 Feb 2025
External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation Mingfu Liang Xi Liu Rong Jin B. Liu Qiuling Suo ... Bo Long Wenlin Chen Rocky Liu Santanu Kolay Haoyang Li 56 2 0 20 Feb 2025
Scalable Model Merging with Progressive Layer-wise Distillation Jing Xu Jiazheng Li J.N. Zhang MoMe FedML 180 2 0 18 Feb 2025
CR-CTC: Consistency regularization on CTC for improved speech recognition Zengwei Yao Wei Kang Xiaoyu Yang Fangjun Kuang Liyong Guo Han Zhu Zengrui Jin Zhaoqing Li Long Lin Daniel Povey 67 2 0 17 Feb 2025
Mitigating Visual Knowledge Forgetting in MLLM Instruction-tuning via Modality-decoupled Gradient Descent Junda Wu Yuxin Xiong Xintong Li Yu Xia Ruoyu Wang ... Sungchul Kim Ryan Rossi Lina Yao Jingbo Shang Julian McAuley CLL VLM 81 0 0 17 Feb 2025
Forget the Data and Fine-Tuning! Just Fold the Network to Compress Dong Wang Haris Šikić Lothar Thiele O. Saukh 77 1 0 17 Feb 2025
Leave No One Behind: Enhancing Diversity While Maintaining Accuracy in Social Recommendation Lei Li Xiao Zhou 60 0 0 17 Feb 2025
Achieving Upper Bound Accuracy of Joint Training in Continual Learning Saleh Momeni Bing Liu CLL 114 1 0 17 Feb 2025
Small Models Struggle to Learn from Strong Reasoners Yuetai Li Xiang Yue Zhangchen Xu Fengqing Jiang Luyao Niu Bill Yuchen Lin Bhaskar Ramasubramanian Radha Poovendran LRM 59 22 0 17 Feb 2025
Smoothing Out Hallucinations: Mitigating LLM Hallucination with Smoothed Knowledge Distillation Hieu Nguyen Zihao He Shoumik Atul Gandre Ujjwal Pasupulety Sharanya Kumari Shivakumar Kristina Lerman HILM 82 1 0 16 Feb 2025
Shortcuts and Identifiability in Concept-based Models from a Neuro-Symbolic Lens Samuele Bortolotti Emanuele Marconato Paolo Morettin Andrea Passerini Stefano Teso 70 3 0 16 Feb 2025
Hybrid Offline-online Scheduling Method for Large Language Model Inference Optimization Bowen Pang Kai Li Ruifeng She Feifan Wang OffRL 67 2 0 14 Feb 2025
Bag of Tricks for Inference-time Computation of LLM Reasoning Fan Liu Wenshuo Chao Naiqiang Tan Hao Liu OffRL LRM 93 3 0 11 Feb 2025
LongReD: Mitigating Short-Text Degradation of Long-Context Large Language Models via Restoration Distillation Zican Dong Junyi Li Jinhao Jiang Mingyu Xu Wayne Xin Zhao Bin Wang Xin Wu VLM 252 4 0 11 Feb 2025