Distilling the Knowledge in a Neural Network

9 March 2015

Papers citing "Distilling the Knowledge in a Neural Network"

50 / 327 papers shown

Title
Autoencoding Random Forests Binh Duc Vu Jan Kapar Marvin N. Wright David S. Watson 44 0 0 27 May 2025
LLM-QFL: Distilling Large Language Model for Quantum Federated Learning Dev Gurung Shiva Raj Pokhrel FedML 83 0 0 24 May 2025
ALPS: Attention Localization and Pruning Strategy for Efficient Alignment of Large Language Models Hao Chen Haoze Li Zhiqing Xiao Lirong Gao Qi Zhang Xiaomeng Hu Ningtao Wang Xing Fu Junbo Zhao 58 0 0 24 May 2025
Task Specific Pruning with LLM-Sieve: How Many Parameters Does Your Task Really Need? Waleed Reda Abhinav Jangda Krishna Chintalapudi 34 0 0 23 May 2025
Distilling LLM Agent into Small Models with Retrieval and Code Tools Minki Kang Jongwon Jeong Seanie Lee Jaewoong Cho Sung Ju Hwang LRM 75 0 0 23 May 2025
Adaptive Semantic Token Communication for Transformer-based Edge Inference Alessio Devoto Jary Pomponi Mattia Merluzzi Paolo Di Lorenzo Simone Scardapane 75 0 0 23 May 2025
LookWhere? Efficient Visual Recognition by Learning Where to Look and What to See from Self-Supervision A. Fuller Yousef Yassin Junfeng Wen Daniel G. Kyrollos Tarek Ibrahim James R. Green Evan Shelhamer ViT 67 0 0 23 May 2025
L-MTP: Leap Multi-Token Prediction Beyond Adjacent Context for Large Language Models Xiaohao Liu Xiaobo Xia Weixiang Zhao Manyi Zhang Xianzhi Yu Xiu Su Shuo Yang See-Kiong Ng Tat-Seng Chua KELM LRM 37 0 0 23 May 2025
Clip4Retrofit: Enabling Real-Time Image Labeling on Edge Devices via Cross-Architecture CLIP Distillation Li Zhong Ahmed Ghazal Jun-Jun Wan Frederik Zilly Patrick Mackens Joachim E. Vollrath Bogdan Sorin Coseriu 80 0 0 23 May 2025
On Multilingual Encoder Language Model Compression for Low-Resource Languages Daniil Gurgurov Michal Gregor Josef van Genabith Simon Ostermann 75 0 0 22 May 2025
LLM-Based Emulation of the Radio Resource Control Layer: Towards AI-Native RAN Protocols Ziming Liu Bryan Liu Alvaro Valcarce Xiaoli Chu 100 0 0 22 May 2025
An Effective Training Framework for Light-Weight Automatic Speech Recognition Models Abdul Hannan Alessio Brutti Shah Nawaz Mubashir Noman 20 0 0 22 May 2025
MentalMAC: Enhancing Large Language Models for Detecting Mental Manipulation via Multi-Task Anti-Curriculum Distillation Yuansheng Gao Han Bao Tong Zhang Bin Li Zonghui Wang Wenzhi Chen 30 0 0 21 May 2025
VERDI: VLM-Embedded Reasoning for Autonomous Driving Bowen Feng Zhiting Mei Baiang Li Julian Ost Roger Girgis Anirudha Majumdar Felix Heide VLM LRM 102 0 0 21 May 2025
SEPS: A Separability Measure for Robust Unlearning in LLMs Wonje Jeung Sangyeon Yoon Albert No MU VLM 99 0 0 20 May 2025
Shadow-FT: Tuning Instruct via Base Taiqiang Wu Runming Yang Jiayi Li Pengfei Hu Ngai Wong Yujiu Yang 86 0 0 19 May 2025
ExpertSteer: Intervening in LLMs through Expert Knowledge Weixuan Wang Minghao Wu Barry Haddow Alexandra Birch LLMSV 98 0 0 18 May 2025
MoL for LLMs: Dual-Loss Optimization to Enhance Domain Expertise While Preserving General Capabilities Jingxue Chen Qingkun Tang Qianchun Lu Siyuan Fang 43 0 0 17 May 2025
Tracr-Injection: Distilling Algorithms into Pre-trained Language Models Tomás Vergara-Browne Álvaro Soto 71 0 0 15 May 2025
KDH-MLTC: Knowledge Distillation for Healthcare Multi-Label Text Classification Hajar Sakai Sarah Lam VLM 63 0 0 12 May 2025
A systematic review of challenges and proposed solutions in modeling multimodal data Maryam Farhadizadeh Maria Weymann Michael Blaß Johann Kraus Christopher Gundler Sebastian Walter Noah Hempen Harald Binde Nadine Binder 50 0 0 11 May 2025
AugMixCloak: A Defense against Membership Inference Attacks via Image Transformation Heqing Ren Chao Feng Alberto Huertas Burkhard Stiller 31 0 0 11 May 2025
Autoencoder-Based Hybrid Replay for Class-Incremental Learning Milad Khademi Nori Il-Min Kim Guanghui Wang CLL 69 0 0 09 May 2025
PIDiff: Image Customization for Personalized Identities with Diffusion Models Jinyu Gu Haipeng Liu M. Y. Wang Yijiao Wang 84 0 0 08 May 2025
ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via $α$ - $β$ -Divergence Guanghui Wang Zhiyong Yang Ziyi Wang Shi Wang Qianqian Xu Qingming Huang 116 0 0 07 May 2025
Image Recognition with Online Lightweight Vision Transformer: A Survey Zherui Zhang Rongtao Xu Jie Zhou Changwei Wang Xingtian Pei ... Jiguang Zhang Li Guo Longxiang Gao Wenyuan Xu Shibiao Xu ViT 350 0 0 06 May 2025
Rainbow Delay Compensation: A Multi-Agent Reinforcement Learning Framework for Mitigating Delayed Observation Songchen Fu Siang Chen Shaojing Zhao Letian Bai Ta Li Yonghong Yan 88 0 0 06 May 2025
Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques Sanjay Surendranath Girija Shashank Kapoor Lakshit Arora Dipen Pradhan Aman Raj Ankit Shetgaonkar 71 0 0 05 May 2025
FPGA-based Acceleration for Convolutional Neural Networks: A Comprehensive Review Junye Jiang Yaan Zhou Yuanhao Gong Haoxuan Yuan Shuanglong Liu 60 0 0 04 May 2025
Enhancing AI Face Realism: Cost-Efficient Quality Improvement in Distilled Diffusion Models with a Fully Synthetic Dataset Jakub Wąsala Bartłomiej Wrzalski Kornelia Noculak Yuliia Tarasenko Oliwer Krupa Jan Kocoń Grzegorz Chodak 59 0 0 04 May 2025
MISE: Meta-knowledge Inheritance for Social Media-Based Stressor Estimation Xin Wang Ling Feng Huijun Zhang Lei Cao Kaisheng Zeng Qi Li Yang Ding Yi Dai David A. Clifton 63 0 0 03 May 2025
Distilling Two-Timed Flow Models by Separately Matching Initial and Terminal Velocities Pramook Khungurn Pratch Piyawongwisal Sira Sriswadi Supasorn Suwajanakorn 58 0 0 02 May 2025
SmallPlan: Leverage Small Language Models for Sequential Path Planning with Simulation-Powered, LLM-Guided Distillation Quang P.M. Pham Khoi T.N. Nguyen Nhi H. Doan Cuong Pham Kentaro Inui Dezhen Song 132 0 0 01 May 2025
CognitionNet: A Collaborative Neural Network for Play Style Discovery in Online Skill Gaming Platform Rukma Talwadker Surajit Chakrabarty Aditya Pareek Tridib Mukherjee Deepak Saini 71 6 0 01 May 2025
A Framework for Elastic Adaptation of User Multiple Intents in Sequential Recommendation Zhikai Wang Yanyan Shen AI4TS 104 0 0 30 Apr 2025
Soft-Label Caching and Sharpening for Communication-Efficient Federated Distillation Kitsuya Azuma Takayuki Nishio Yuichi Kitagawa Wakako Nakano Takahito Tanimura FedML 94 0 0 28 Apr 2025
ConTextual: Improving Clinical Text Summarization in LLMs with Context-preserving Token Filtering and Knowledge Graphs Fahmida Liza Piya Rahmatollah Beheshti 181 0 0 23 Apr 2025
PointLoRA: Low-Rank Adaptation with Token Selection for Point Cloud Learning Song Wang Xiaolu Liu Lingdong Kong Jianyun Xu Chunyong Hu Gongfan Fang Wentong Li Jianke Zhu Xinchao Wang 59 0 0 22 Apr 2025
MonoTher-Depth: Enhancing Thermal Depth Estimation via Confidence-Aware Distillation Xingxing Zuo Nikhil Ranganathan Connor T. Lee Georgia Gkioxari Soon-Jo Chung VLM 78 1 0 21 Apr 2025
Learning Critically: Selective Self Distillation in Federated Learning on Non-IID Data Yuting He Yiqiang Chen Xiaodong Yang H. Yu Yi-Hua Huang Yang Gu FedML 105 21 0 20 Apr 2025
Empirical Evaluation of Knowledge Distillation from Transformers to Subquadratic Language Models Patrick Haller Jonas Golde Alan Akbik 53 0 0 19 Apr 2025
Teach Me How to Denoise: A Universal Framework for Denoising Multi-modal Recommender Systems via Guided Calibration Haoyang Li Hanwen Du You Li Junchen Fu Chunxiao Li Ziyi Zhuang Jiakang Li Yongxin Ni AI4TS 57 0 0 19 Apr 2025
LimitNet: Progressive, Content-Aware Image Offloading for Extremely Weak Devices & Networks A. Hojjat Janek Haberer Tayyaba Zainab Olaf Landsiedel 52 3 0 18 Apr 2025
From Large to Super-Tiny: End-to-End Optimization for Cost-Efficient LLMs Jiliang Ni Jiachen Pu Zhongyi Yang Kun Zhou Hui Wang Xiaoliang Xiao Dakui Wang Xin Li Jingfeng Luo Conggang Hu 56 0 0 18 Apr 2025
Perception Encoder: The best visual embeddings are not at the output of the network Daniel Bolya Po-Yao (Bernie) Huang Peize Sun Jang Hyun Cho Andrea Madotto ... Shiyu Dong Nikhila Ravi Daniel Li Piotr Dollár Christoph Feichtenhofer ObjD VOS 150 5 0 17 Apr 2025
Human Aligned Compression for Robust Models Samuel Räber Andreas Plesner Till Aczél Roger Wattenhofer AAML 78 0 0 16 Apr 2025
Teaching Large Language Models to Reason through Learning and Forgetting Tianwei Ni Allen Nie Sapana Chaudhary Yao Liu Huzefa Rangwala Rasool Fakoor ReLM CLL LRM 354 0 0 15 Apr 2025
HDC: Hierarchical Distillation for Multi-level Noisy Consistency in Semi-Supervised Fetal Ultrasound Segmentation Tran Quoc Khanh Le Nguyen Lan Vi Vu Ha-Hieu Pham Xuan-Loc Huynh T. Nguyen Minh Huu Nhat Le Quan Nguyen Hien Nguyen 50 0 0 14 Apr 2025
Shrinkage Initialization for Smooth Learning of Neural Networks Miao Cheng Feiyan Zhou Hongwei Zou Limin Wang AI4CE 42 0 0 12 Apr 2025
Probability Distribution Alignment and Low-Rank Weight Decomposition for Source-Free Domain Adaptive Brain Decoding Ganxi Xu Jinyi Long Hanrui Wu 73 0 0 12 Apr 2025