Scalable Methods for 8-bit Training of Neural Networks

25 May 2018

Papers citing "Scalable Methods for 8-bit Training of Neural Networks"

50 / 168 papers shown

Title
Training Transformers with 4-bit Integers Haocheng Xi Changhao Li Jianfei Chen Jun Zhu MQ 25 47 0 21 Jun 2023
SlimFit: Memory-Efficient Fine-Tuning of Transformer-based Models Using Training Dynamics A. Ardakani Altan Haan Shangyin Tan Doru-Thom Popovici Alvin Cheung Costin Iancu Koushik Sen 16 3 0 29 May 2023
Post-training Model Quantization Using GANs for Synthetic Data Generation Athanasios Masouris Mansi Sharma Adrian Boguszewski Alexander Kozlov Zhuo Wu Raymond Lo MQ 16 0 0 10 May 2023
CAMEL: Co-Designing AI Models and Embedded DRAMs for Efficient On-Device Learning Sai Qian Zhang Thierry Tambe Nestor Cuevas Gu-Yeon Wei David Brooks 18 4 0 04 May 2023
Diversifying the High-level Features for better Adversarial Transferability Zhiyuan Wang Zeliang Zhang Siyuan Liang Xiaosen Wang AAML 42 18 0 20 Apr 2023
Mathematical Challenges in Deep Learning V. Nia Guojun Zhang I. Kobyzev Michael R. Metel Xinlin Li ... S. Hemati M. Asgharian Linglong Kong Wulong Liu Boxing Chen AI4CE VLM 37 1 0 24 Mar 2023
MetaGrad: Adaptive Gradient Quantization with Hypernetworks Kaixin Xu Alina Hui Xiu Lee Ziyuan Zhao Zhe Wang Min-man Wu Weisi Lin MQ 17 1 0 04 Mar 2023
Data-Efficient Training of CNNs and Transformers with Coresets: A Stability Perspective Animesh Gupta Irtiza Hassan Dilip K. Prasad D. K. Gupta 21 2 0 03 Mar 2023
Virtualization of Tiny Embedded Systems with a robust real-time capable and extensible Stack Virtual Machine REXAVM supporting Material-integrated Intelligent Systems and Tiny Machine Learning S. Bosse Sarah Bornemann B. Lüssem 11 3 0 17 Feb 2023
Quantized Distributed Training of Large Models with Convergence Guarantees I. Markov Adrian Vladu Qi Guo Dan Alistarh MQ 26 11 0 05 Feb 2023
Training with Mixed-Precision Floating-Point Assignments Wonyeol Lee Rahul Sharma A. Aiken MQ 24 2 0 31 Jan 2023
Optimising Event-Driven Spiking Neural Network with Regularisation and Cutoff Dengyu Wu Gao Jin Han Yu Xinping Yi Xiaowei Huang 24 2 0 23 Jan 2023
Efficient On-device Training via Gradient Filtering Yuedong Yang Guihong Li R. Marculescu 31 18 0 01 Jan 2023
Randomized Quantization: A Generic Augmentation for Data Agnostic Self-supervised Learning Huimin Wu Chenyang Lei Xiao Sun Pengju Wang Qifeng Chen Kwang-Ting Cheng Stephen Lin Zhirong Wu MQ 30 5 0 19 Dec 2022
QFT: Post-training quantization via fast joint finetuning of all degrees of freedom Alexander Finkelstein Ella Fuchs Idan Tal Mark Grobman Niv Vosco Eldad Meller MQ 21 6 0 05 Dec 2022
Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Training Yunshan Zhong Gongrui Nan Yu-xin Zhang Fei Chao Rongrong Ji MQ 18 3 0 12 Nov 2022
NEON: Enabling Efficient Support for Nonlinear Operations in Resistive RAM-based Neural Network Accelerators Aditya Manglik Minesh Patel Haiyu Mao Behzad Salami Jisung Park Lois Orosa O. Mutlu 15 1 0 10 Nov 2022
AskewSGD : An Annealed interval-constrained Optimisation method to train Quantized Neural Networks Louis Leconte S. Schechtman Eric Moulines 29 4 0 07 Nov 2022
LightNorm: Area and Energy-Efficient Batch Normalization Hardware for On-Device DNN Training Seock-Hwan Noh Junsang Park Dahoon Park Jahyun Koo Jeik Choi Jaeha Kung 14 9 0 04 Nov 2022
Emergent Quantized Communication Boaz Carmeli Ron Meir Yonatan Belinkov MQ AI4CE 28 8 0 04 Nov 2022
Block-Wise Dynamic-Precision Neural Network Training Acceleration via Online Quantization Sensitivity Analytics Ruoyang Liu Chenhan Wei Yixiong Yang Wenxun Wang Huazhong Yang Yongpan Liu MQ 17 3 0 31 Oct 2022
NASA: Neural Architecture Search and Acceleration for Hardware Inspired Hybrid Networks Huihong Shi Haoran You Yang Katie Zhao Zhongfeng Wang Yingyan Lin 56 7 0 24 Oct 2022
Is Integer Arithmetic Enough for Deep Learning Training? Alireza Ghaffari Marzieh S. Tahaei Mohammadreza Tayaranian M. Asgharian V. Nia MQ 11 16 0 18 Jul 2022
Sub 8-Bit Quantization of Streaming Keyword Spotting Models for Embedded Chipsets Lu Zeng S. Parthasarathi Yuzong Liu Alex Escott S. Cheekatmalla N. Strom S. Vitaladevuni MQ 31 5 0 13 Jul 2022
Computational Complexity Evaluation of Neural Network Applications in Signal Processing Pedro J. Freire S. Srivallapanondh A. Napoli Jaroslaw E. Prilepsky S. Turitsyn 29 1 0 24 Jun 2022
GACT: Activation Compressed Training for Generic Network Architectures Xiaoxuan Liu Lianmin Zheng Dequan Wang Yukuo Cen Weize Chen ... Zhiyuan Liu Jie Tang Joey Gonzalez Michael W. Mahoney Alvin Cheung VLM GNN MQ 17 30 0 22 Jun 2022
Low-Precision Stochastic Gradient Langevin Dynamics Ruqi Zhang A. Wilson Chris De Sa BDL 18 14 0 20 Jun 2022
8-bit Numerical Formats for Deep Neural Networks Badreddine Noune Philip Jones Daniel Justus Dominic Masters Carlo Luschi MQ 15 33 0 06 Jun 2022
Extreme Compression for Pre-trained Transformers Made Simple and Efficient Xiaoxia Wu Z. Yao Minjia Zhang Conglong Li Yuxiong He MQ 19 31 0 04 Jun 2022
Wavelet Feature Maps Compression for Image-to-Image CNNs Shahaf E. Finder Yair Zohav Maor Ashkenazi Eran Treister 14 17 0 24 May 2022
ShiftAddNAS: Hardware-Inspired Search for More Accurate and Efficient Neural Networks Haoran You Baopu Li Huihong Shi Y. Fu Yingyan Lin 44 17 0 17 May 2022
Neural Architecture Search using Property Guided Synthesis Charles Jin P. Phothilimthana Sudip Roy 27 6 0 08 May 2022
Minimum Variance Unbiased N:M Sparsity for the Neural Gradients Brian Chmiel Itay Hubara Ron Banner Daniel Soudry 17 10 0 21 Mar 2022
LDP: Learnable Dynamic Precision for Efficient Deep Neural Network Training and Inference Zhongzhi Yu Y. Fu Shang Wu Mengquan Li Haoran You Yingyan Lin 24 1 0 15 Mar 2022
FlexBlock: A Flexible DNN Training Accelerator with Multi-Mode Block Floating Point Support Seock-Hwan Noh Jahyun Koo Seunghyun Lee Jongse Park Jaeha Kung AI4CE 29 17 0 13 Mar 2022
Practical Network Acceleration with Tiny Sets G. Wang Jianxin Wu 27 8 0 16 Feb 2022
PocketNN: Integer-only Training and Inference of Neural Networks via Direct Feedback Alignment and Pocket Activations in Pure C++ Jae-Su Song Fangzhen Lin MQ 7 7 0 08 Jan 2022
Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats Brian Chmiel Ron Banner Elad Hoffer Hilla Ben Yaacov Daniel Soudry MQ 25 22 0 19 Dec 2021
Low Precision Decentralized Distributed Training over IID and non-IID Data Sai Aparna Aketi Sangamesh Kodge Kaushik Roy MQ 17 9 0 17 Nov 2021
LVAC: Learned Volumetric Attribute Compression for Point Clouds using Coordinate Based Networks Berivan Isik P. Chou S. Hwang Nick Johnston G. Toderici 3DPC 26 28 0 17 Nov 2021
Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI Jiangchao Yao Shengyu Zhang Yang Yao Feng Wang Jianxin Ma ... Kun Kuang Chao-Xiang Wu Fei Wu Jingren Zhou Hongxia Yang 18 91 0 11 Nov 2021
Learned Image Compression for Machine Perception Felipe Codevilla Jean-Gabriel Simard Ross Goroshin C. Pal 22 20 0 03 Nov 2021
DAdaQuant: Doubly-adaptive quantization for communication-efficient Federated Learning Robert Hönig Yiren Zhao Robert D. Mullins FedML 104 53 0 31 Oct 2021
FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Rounding S. Zhang Bradley McDanel H. T. Kung MQ 10 64 0 28 Oct 2021
L2ight: Enabling On-Chip Learning for Optical Neural Networks via Efficient in-situ Subspace Optimization Jiaqi Gu Hanqing Zhu Chenghao Feng Zixuan Jiang Ray T. Chen D. Pan 18 28 0 27 Oct 2021
NeRV: Neural Representations for Videos Hao Chen Bo He Hanyu Wang Yixuan Ren Ser-Nam Lim Abhinav Shrivastava 23 241 0 26 Oct 2021
Shift-BNN: Highly-Efficient Probabilistic Bayesian Neural Network Training via Memory-Friendly Pattern Retrieving Qiyu Wan Haojun Xia Xingyao Zhang Lening Wang S. Song Xin Fu OOD 31 9 0 07 Oct 2021
Prune Your Model Before Distill It Jinhyuk Park Albert No VLM 38 27 0 30 Sep 2021
Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions Yang Wu Dingheng Wang Xiaotong Lu Fan Yang Guoqi Li W. Dong Jianbo Shi 29 18 0 30 Aug 2021
Taxonomy and Benchmarking of Precision-Scalable MAC Arrays Under Enhanced DNN Dataflow Representation Ehab M. Ibrahim L. Mei Marian Verhelst MQ 6 9 0 10 Aug 2021