Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy

15 November 2017

Papers citing "Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracy"

50 / 59 papers shown

Title
Breaking the Memory Wall for Heterogeneous Federated Learning via Model Splitting Chunlin Tian Li Li Kahou Tam Yebo Wu Chengzhong Xu FedML 24 1 0 12 Oct 2024
Efficient Knowledge Distillation: Empowering Small Language Models with Teacher Model Insights Mohamad Ballout U. Krumnack Gunther Heidemann Kai-Uwe Kühnberger 33 2 0 19 Sep 2024
Relational Representation Distillation Nikolaos Giakoumoglou Tania Stathaki 32 0 0 16 Jul 2024
CMAT: A Multi-Agent Collaboration Tuning Framework for Enhancing Small Language Models Xuechen Liang Meiling Tao Yinghui Xia Yiting Xie Jun Wang JingSong Yang LLMAG 31 12 0 02 Apr 2024
Quantized Feature Distillation for Network Quantization Kevin Zhu Yin He Jianxin Wu MQ 24 9 0 20 Jul 2023
Self-Distilled Quantization: Achieving High Compression Rates in Transformer-Based Language Models James OÑeill Sourav Dutta VLM MQ 32 1 0 12 Jul 2023
MobileNMT: Enabling Translation in 15MB and 30ms Ye Lin Xiaohui Wang Zhexi Zhang Mingxuan Wang Tong Xiao Jingbo Zhu MQ 25 1 0 07 Jun 2023
RedBit: An End-to-End Flexible Framework for Evaluating the Accuracy of Quantized CNNs A. M. Ribeiro-dos-Santos João Dinis Ferreira O. Mutlu G. Falcão MQ 13 1 0 15 Jan 2023
Vertical Layering of Quantized Neural Networks for Heterogeneous Inference Hai Wu Ruifei He Hao Hao Tan Xiaojuan Qi Kaibin Huang MQ 19 2 0 10 Dec 2022
QFT: Post-training quantization via fast joint finetuning of all degrees of freedom Alexander Finkelstein Ella Fuchs Idan Tal Mark Grobman Niv Vosco Eldad Meller MQ 18 6 0 05 Dec 2022
BiViT: Extremely Compressed Binary Vision Transformer Yefei He Zhenyu Lou Luoming Zhang Jing Liu Weijia Wu Hong Zhou Bohan Zhuang ViT MQ 18 28 0 14 Nov 2022
AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks Louis Leconte S. Schechtman Eric Moulines 27 4 0 07 Nov 2022
Fast and Low-Memory Deep Neural Networks Using Binary Matrix Factorization Alireza Bordbar M. Kahaei MQ 23 0 0 24 Oct 2022
Designing and Training of Lightweight Neural Networks on Edge Devices using Early Halting in Knowledge Distillation Rahul Mishra Hari Prabhat Gupta 27 8 0 30 Sep 2022
Mixed-Precision Neural Networks: A Survey M. Rakka M. Fouda Pramod P. Khargonekar Fadi J. Kurdahi MQ 18 11 0 11 Aug 2022
Overlooked Poses Actually Make Sense: Distilling Privileged Knowledge for Human Motion Prediction Xiaoning Sun Qiongjie Cui Huaijiang Sun Bin Li Weiqing Li Jianfeng Lu 18 7 0 02 Aug 2022
BiTAT: Neural Network Binarization with Task-dependent Aggregated Transformation Geondo Park Jaehong Yoon H. Zhang Xingge Zhang S. Hwang Yonina C. Eldar MQ 28 1 0 04 Jul 2022
Knowledge Distillation via the Target-aware Transformer Sihao Lin Hongwei Xie Bing Wang Kaicheng Yu Xiaojun Chang Xiaodan Liang G. Wang ViT 20 104 0 22 May 2022
Neural Network Quantization for Efficient Inference: A Survey Olivia Weng MQ 17 22 0 08 Dec 2021
Arch-Net: Model Distillation for Architecture Agnostic Model Deployment Weixin Xu Zipeng Feng Shuangkang Fang Song Yuan Yi Yang Shuchang Zhou MQ 16 1 0 01 Nov 2021
Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation Sumanth Chennupati Mohammad Mahdi Kamani Zhongwei Cheng Lin Chen 19 4 0 19 Oct 2021
Secure Your Ride: Real-time Matching Success Rate Prediction for Passenger-Driver Pairs Yuandong Wang Hongzhi Yin Lian Wu Tong Chen Chunyang Liu 8 7 0 14 Sep 2021
Auto-Split: A General Framework of Collaborative Edge-Cloud AI Amin Banitalebi-Dehkordi Naveen Vedula J. Pei Fei Xia Lanjun Wang Yong Zhang 22 89 0 30 Aug 2021
Knowledge Distillation via Instance-level Sequence Learning Haoran Zhao Xin Sun Junyu Dong Zihe Dong Qiong Li 26 23 0 21 Jun 2021
Knowledge distillation: A good teacher is patient and consistent Lucas Beyer Xiaohua Zhai Amelie Royer L. Markeeva Rohan Anil Alexander Kolesnikov VLM 18 287 0 09 Jun 2021
Extremely Lightweight Quantization Robust Real-Time Single-Image Super Resolution for Mobile Devices Mustafa Ayazoglu 8 56 0 21 May 2021
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization Tianlong Chen Zhenyu (Allen) Zhang Xu Ouyang Zechun Liu Zhiqiang Shen Zhangyang Wang MQ 31 36 0 16 Apr 2021
Training Multi-bit Quantized and Binarized Networks with A Learnable Symmetric Quantizer Phuoc Pham J. Abraham Jaeyong Chung MQ 33 11 0 01 Apr 2021
VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference Steve Dai Rangharajan Venkatesan Haoxing Ren B. Zimmer W. Dally Brucek Khailany MQ 25 67 0 08 Feb 2021
Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression Cody Blakeney Xiaomin Li Yan Yan Ziliang Zong 32 39 0 05 Dec 2020
Ensemble Knowledge Distillation for CTR Prediction Jieming Zhu Jinyang Liu Weiqi Li Jincai Lai Xiuqiang He Liang Chen Zibin Zheng 31 55 0 08 Nov 2020
Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks Yoonho Boo Sungho Shin Jungwook Choi Wonyong Sung MQ 14 29 0 30 Sep 2020
Classification of Diabetic Retinopathy Using Unlabeled Data and Knowledge Distillation Sajjad Abbasi M. Hajabdollahi P. Khadivi N. Karimi Roshank Roshandel S. Shirani S. Samavi 14 18 0 01 Sep 2020
NASB: Neural Architecture Search for Binary Convolutional Neural Networks Baozhou Zhu Zaid Al-Ars P. Hofstee MQ 15 23 0 08 Aug 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,835 0 09 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 45 98 0 05 Jun 2020
Binary Neural Networks: A Survey Haotong Qin Ruihao Gong Xianglong Liu Xiao Bai Jingkuan Song N. Sebe MQ 36 457 0 31 Mar 2020
Evolutionary Bin Packing for Memory-Efficient Dataflow Inference Acceleration on FPGA M. Kroes L. Petrica S. Cotofana Michaela Blott 8 7 0 24 Mar 2020
Modeling Teacher-Student Techniques in Deep Neural Networks for Knowledge Distillation Sajjad Abbasi M. Hajabdollahi N. Karimi S. Samavi 10 28 0 31 Dec 2019
Towards Unified INT8 Training for Convolutional Neural Network Feng Zhu Ruihao Gong F. Yu Xianglong Liu Yanfei Wang Zhelong Li Xiuqi Yang Junjie Yan MQ 27 151 0 29 Dec 2019
Towards Efficient Training for Neural Network Quantization Qing Jin Linjie Yang Zhenyu A. Liao MQ 11 42 0 21 Dec 2019
Adaptive Loss-aware Quantization for Multi-bit Networks Zhongnan Qu Zimu Zhou Yun Cheng Lothar Thiele MQ 30 53 0 18 Dec 2019
QKD: Quantization-aware Knowledge Distillation Jangho Kim Yash Bhalgat Jinwon Lee Chirag I. Patel Nojun Kwak MQ 16 63 0 28 Nov 2019
Iteratively Training Look-Up Tables for Network Quantization Fabien Cardinaux Stefan Uhlich K. Yoshiyama Javier Alonso García Lukas Mauch Stephen Tiedemann Thomas Kemp Akira Nakamura MQ 27 16 0 12 Nov 2019
Structured Binary Neural Networks for Image Recognition Bohan Zhuang Chunhua Shen Mingkui Tan Peng Chen Lingqiao Liu Ian Reid MQ 20 17 0 22 Sep 2019
Differentiable Soft Quantization: Bridging Full-Precision and Low-Bit Neural Networks Ruihao Gong Xianglong Liu Shenghu Jiang Tian-Hao Li Peng Hu Jiazhen Lin F. Yu Junjie Yan MQ 21 445 0 14 Aug 2019
Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activations Bohan Zhuang Jing Liu Mingkui Tan Lingqiao Liu Ian Reid Chunhua Shen MQ 20 44 0 10 Aug 2019
GDRQ: Group-based Distribution Reshaping for Quantization Haibao Yu Tuopu Wen Guangliang Cheng Jiankai Sun Qi Han Jianping Shi MQ 25 3 0 05 Aug 2019
And the Bit Goes Down: Revisiting the Quantization of Neural Networks Pierre Stock Armand Joulin Rémi Gribonval Benjamin Graham Hervé Jégou MQ 29 149 0 12 Jul 2019
Divide and Conquer: Leveraging Intermediate Feature Representations for Quantized Training of Neural Networks Ahmed T. Elthakeb Prannoy Pilligundla Alex Cloninger H. Esmaeilzadeh MQ 18 8 0 14 Jun 2019