Model compression via distillation and quantization

15 February 2018

Dan Alistarh

Papers citing "Model compression via distillation and quantization"

50 / 133 papers shown

Title
Arch-Net: Model Distillation for Architecture Agnostic Model Deployment Weixin Xu Zipeng Feng Shuangkang Fang Song Yuan Yi Yang Shuchang Zhou MQ 24 1 0 01 Nov 2021
Adaptive Distillation: Aggregating Knowledge from Multiple Paths for Efficient Distillation Sumanth Chennupati Mohammad Mahdi Kamani Zhongwei Cheng Lin Chen 21 4 0 19 Oct 2021
ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Training Hui-Po Wang Sebastian U. Stich Yang He Mario Fritz FedML AI4CE 28 46 0 11 Oct 2021
Auto-Split: A General Framework of Collaborative Edge-Cloud AI Amin Banitalebi-Dehkordi Naveen Vedula J. Pei Fei Xia Lanjun Wang Yong Zhang 22 89 0 30 Aug 2021
Privacy-Preserving Machine Learning: Methods, Challenges and Directions Runhua Xu Nathalie Baracaldo J. Joshi 24 100 0 10 Aug 2021
QuPeD: Quantized Personalization via Distillation with Applications to Federated Learning Kaan Ozkara Navjot Singh Deepesh Data Suhas Diggavi FedML MQ 24 56 0 29 Jul 2021
Follow Your Path: a Progressive Method for Knowledge Distillation Wenxian Shi Yuxuan Song Hao Zhou Bohan Li Lei Li 17 14 0 20 Jul 2021
DANCE: DAta-Network Co-optimization for Efficient Segmentation Model Training and Inference Chaojian Li Wuyang Chen Yuchen Gu Tianlong Chen Yonggan Fu Zhangyang Wang Yingyan Lin 30 0 0 16 Jul 2021
Training Compact CNNs for Image Classification using Dynamic-coded Filter Fusion Mingbao Lin Bohong Chen Fei Chao Rongrong Ji VLM 25 23 0 14 Jul 2021
Improving the Efficiency of Transformers for Resource-Constrained Devices Hamid Tabani Ajay Balasubramaniam Shabbir Marzban Elahe Arani Bahram Zonooz 33 20 0 30 Jun 2021
Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Better Gaurav Menghani VLM MedIm 23 365 0 16 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 37 815 0 14 Jun 2021
BoolNet: Minimizing The Energy Consumption of Binary Neural Networks Nianhui Guo Joseph Bethge Haojin Yang Kai Zhong Xuefei Ning Christoph Meinel Yu Wang MQ 24 11 0 13 Jun 2021
Rethinking Transfer Learning for Medical Image Classification Le Peng Hengyue Liang Gaoxiang Luo Taihui Li Ju Sun VLM LM&MA 14 5 0 09 Jun 2021
RBNN: Memory-Efficient Reconfigurable Deep Binary Neural Network with IP Protection for Internet of Things Huming Qiu Hua Ma Zhi-Li Zhang Yifeng Zheng Anmin Fu Pan Zhou Yansong Gao Derek Abbott S. Al-Sarawi MQ 19 9 0 09 May 2021
Self-paced Resistance Learning against Overfitting on Noisy Labels Xiaoshuang Shi Zhenhua Guo Fuyong Xing Yun Liang Xiaofeng Zhu NoLa 21 20 0 07 May 2021
Differentiable Model Compression via Pseudo Quantization Noise Alexandre Défossez Yossi Adi Gabriel Synnaeve DiffM MQ 15 47 0 20 Apr 2021
Compact CNN Structure Learning by Knowledge Distillation Waqar Ahmed Andrea Zunino Pietro Morerio Vittorio Murino 27 5 0 19 Apr 2021
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization Tianlong Chen Zhenyu (Allen) Zhang Xu Ouyang Zechun Liu Zhiqiang Shen Zhangyang Wang MQ 37 36 0 16 Apr 2021
Training Multi-bit Quantized and Binarized Networks with A Learnable Symmetric Quantizer Phuoc Pham J. Abraham Jaeyong Chung MQ 33 11 0 01 Apr 2021
Student Network Learning via Evolutionary Knowledge Distillation Kangkai Zhang Chunhui Zhang Shikun Li Dan Zeng Shiming Ge 14 83 0 23 Mar 2021
Learnable Companding Quantization for Accurate Low-bit Neural Networks Kohei Yamamoto MQ 36 63 0 12 Mar 2021
Environmental Sound Classification on the Edge: A Pipeline for Deep Acoustic Networks on Extremely Resource-Constrained Devices Md Mohaimenuzzaman Christoph Bergmeir I. West B. Meyer 12 41 0 05 Mar 2021
Diversifying Sample Generation for Accurate Data-Free Quantization Xiangguo Zhang Haotong Qin Yifu Ding Ruihao Gong Qing Yan Renshuai Tao Yuhang Li F. Yu Xianglong Liu MQ 54 94 0 01 Mar 2021
An Information-Theoretic Justification for Model Pruning Berivan Isik Tsachy Weissman Albert No 86 35 0 16 Feb 2021
SEED: Self-supervised Distillation For Visual Representation Zhiyuan Fang Jianfeng Wang Lijuan Wang Lei Zhang Yezhou Yang Zicheng Liu SSL 236 190 0 12 Jan 2021
Auto-Agent-Distiller: Towards Efficient Deep Reinforcement Learning Agents via Neural Architecture Search Y. Fu Zhongzhi Yu Yongan Zhang Yingyan Lin 22 4 0 24 Dec 2020
Cross-Layer Distillation with Semantic Calibration Defang Chen Jian-Ping Mei Yuan Zhang Can Wang Yan Feng Chun-Yen Chen FedML 45 286 0 06 Dec 2020
Parallel Blockwise Knowledge Distillation for Deep Neural Network Compression Cody Blakeney Xiaomin Li Yan Yan Ziliang Zong 46 39 0 05 Dec 2020
Bringing AI To Edge: From Deep Learning's Perspective Di Liu Hao Kong Xiangzhong Luo Weichen Liu Ravi Subramaniam 52 116 0 25 Nov 2020
Auto Graph Encoder-Decoder for Neural Network Pruning Sixing Yu Arya Mazaheri Ali Jannesari GNN 19 38 0 25 Nov 2020
Neural Network Compression Via Sparse Optimization Tianyi Chen Bo Ji Yixin Shi Tianyu Ding Biyi Fang Sheng Yi Xiao Tu 36 15 0 10 Nov 2020
Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks Yoonho Boo Sungho Shin Jungwook Choi Wonyong Sung MQ 22 29 0 30 Sep 2020
TernaryBERT: Distillation-aware Ultra-low Bit BERT Wei Zhang Lu Hou Yichun Yin Lifeng Shang Xiao Chen Xin Jiang Qun Liu MQ 25 208 0 27 Sep 2020
Efficient Transformer-based Large Scale Language Representations using Hardware-friendly Block Structured Pruning Bingbing Li Zhenglun Kong Tianyun Zhang Ji Li Z. Li Hang Liu Caiwen Ding VLM 24 64 0 17 Sep 2020
Classification of Diabetic Retinopathy Using Unlabeled Data and Knowledge Distillation Sajjad Abbasi M. Hajabdollahi P. Khadivi N. Karimi Roshank Roshandel S. Shirani S. Samavi 14 18 0 01 Sep 2020
NASB: Neural Architecture Search for Binary Convolutional Neural Networks Baozhou Zhu Zaid Al-Ars P. Hofstee MQ 24 23 0 08 Aug 2020
Split Computing for Complex Object Detectors: Challenges and Preliminary Results Yoshitomo Matsubara Marco Levorato 38 24 0 27 Jul 2020
Tracking-by-Trackers with a Distilled and Reinforced Model Matteo Dunnhofer N. Martinel C. Micheloni VOT OffRL 27 4 0 08 Jul 2020
Operation-Aware Soft Channel Pruning using Differentiable Masks Minsoo Kang Bohyung Han AAML 33 138 0 08 Jul 2020
A Sequential Self Teaching Approach for Improving Generalization in Sound Event Recognition Anurag Kumar V. Ithapu 17 35 0 30 Jun 2020
Knowledge Distillation: A Survey Jianping Gou B. Yu Stephen J. Maybank Dacheng Tao VLM 19 2,837 0 09 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 45 98 0 05 Jun 2020
TIMELY: Pushing Data Movements and Interfaces in PIM Accelerators Towards Local and in Time Domain Weitao Li Pengfei Xu Yang Katie Zhao Haitong Li Yuan Xie Yingyan Lin 9 68 0 03 May 2020
Binary Neural Networks: A Survey Haotong Qin Ruihao Gong Xianglong Liu Xiao Bai Jingkuan Song N. Sebe MQ 50 457 0 31 Mar 2020
Introducing Pose Consistency and Warp-Alignment for Self-Supervised 6D Object Pose Estimation in Color Images Juil Sock Guillermo Garcia-Hernando Anil Armagan Tae-Kyun Kim 24 5 0 27 Mar 2020
GAN Compression: Efficient Architectures for Interactive Conditional GANs Muyang Li Ji Lin Yaoyao Ding Zhijian Liu Jun-Yan Zhu Song Han GAN 22 2 0 19 Mar 2020
Ternary Compression for Communication-Efficient Federated Learning Jinjin Xu W. Du Ran Cheng Wangli He Yaochu Jin MQ FedML 34 174 0 07 Mar 2020
MeliusNet: Can Binary Neural Networks Achieve MobileNet-level Accuracy? Joseph Bethge Christian Bartz Haojin Yang Ying Chen Christoph Meinel MQ 25 91 0 16 Jan 2020
Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation Chuteng Zhou Prad Kadambi Matthew Mattina P. Whatmough 11 35 0 14 Jan 2020