Training Deep Neural Networks with 8-bit Floating Point Numbers

19 December 2018

Papers citing "Training Deep Neural Networks with 8-bit Floating Point Numbers"

50 / 212 papers shown

Title
A Statistical Framework for Low-bitwidth Training of Deep Neural NetworksNeural Information Processing Systems (NeurIPS), 2020 Jianfei Chen Yujie Gai Z. Yao Michael W. Mahoney Joseph E. Gonzalez MQ 109 68 0 27 Oct 2020
Not Half Bad: Exploring Half-Precision in Graph Convolutional Neural Networks John Brennan Stephen Bonner Amir Atapour-Abarghouei Philip T. G. Jackson B. Obara A. Mcgough GNN 126 3 0 23 Oct 2020
Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank DeterminationSIAM Journal on Mathematics of Data Science (SIMODS), 2020 Cole Hawkins Xing-er Liu Zheng Zhang BDL MQ 201 32 0 17 Oct 2020
FPRaker: A Processing Element For Accelerating Neural Network Training Omar Mohamed Awad Mostafa Mahmoud Isak Edo Vivancos Ali Hadi Zadeh Ciaran Bannon Anand Jayarajan Gennady Pekhimenko Andreas Moshovos 137 15 0 15 Oct 2020
Revisiting BFloat16 Training Pedram Zamirai Jian Zhang Christopher R. Aberger Christopher De Sa FedML MQ 124 21 0 13 Oct 2020
NITI: Training Integer Neural Networks Using Integer-only ArithmeticIEEE Transactions on Parallel and Distributed Systems (TPDS), 2020 Maolin Wang Seyedramin Rasoulinezhad Philip H. W. Leong Hayden Kwok-Hay So MQ 109 45 0 28 Sep 2020
An Analysis of Alternating Direction Method of Multipliers for Feed-forward Neural Networks Seyedeh Niusha Alavi Foumani Ce Guo Wayne Luk 34 1 0 06 Sep 2020
An FPGA Accelerated Method for Training Feed-forward Neural Networks Using Alternating Direction Method of Multipliers and LSMR Seyedeh Niusha Alavi Foumani Ce Guo Wayne Luk 66 3 0 06 Sep 2020
TensorDash: Exploiting Sparsity to Accelerate Deep Neural Network Training and InferenceMicro (MICRO), 2020 Mostafa Mahmoud Isak Edo Vivancos Ali Hadi Zadeh Omar Mohamed Awad Gennady Pekhimenko Jorge Albericio Andreas Moshovos MoE 164 61 0 01 Sep 2020
WrapNet: Neural Net Inference with Ultra-Low-Resolution Arithmetic Renkun Ni Hong-Min Chu Oscar Castañeda Ping Yeh-Chiang Christoph Studer Tom Goldstein MQ 110 15 0 26 Jul 2020
TinyTL: Reduce Activations, Not Trainable Parameters for Efficient On-Device Learning Han Cai Chuang Gan Ligeng Zhu Song Han 174 60 0 22 Jul 2020
Resource-Efficient Speech Mask Estimation for Multi-Channel Speech Enhancement Lukas Pfeifenberger Matthias Zöhrer Günther Schindler Wolfgang Roth Holger Fröning Franz Pernkopf 53 1 0 22 Jul 2020
AQD: Towards Accurate Fully-Quantized Object Detection Peng Chen Jing Liu Bohan Zhuang Zhuliang Yu Chunhua Shen MQ 212 9 0 14 Jul 2020
Learning compositional functions via multiplicative weight updates Jeremy Bernstein Jiawei Zhao M. Meister Xuan Li Anima Anandkumar Yisong Yue 170 30 0 25 Jun 2020
Multi-Precision Policy Enforced Training (MuPPET): A precision-switching strategy for quantised fixed-point training of CNNs A. Rajagopal D. A. Vink Stylianos I. Venieris C. Bouganis MQ 130 15 0 16 Jun 2020
Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors C. Coelho Aki Kuusela Shane Li Zhuang Hao T. Aarrestad Vladimir Loncar J. Ngadiuba M. Pierini Adrian Alan Pol S. Summers MQ 226 204 0 15 Jun 2020
Neural gradients are near-lognormal: improved quantized and sparse training Brian Chmiel Liad Ben-Uri Moran Shkolnik Elad Hoffer Ron Banner Daniel Soudry MQ 143 5 0 15 Jun 2020
O(1) Communication for Distributed SGD through Two-Level Gradient AveragingIEEE International Conference on Cluster Computing (Cluster), 2020 Subhadeep Bhattacharya Weikuan Yu Fahim Chowdhury FedML 84 3 0 12 Jun 2020
Power Consumption Variation over Activation Functions Leon Derczynski 93 8 0 12 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 261 109 0 05 Jun 2020
Exploring the Potential of Low-bit Training of Convolutional Neural Networks Kai Zhong Xuefei Ning Guohao Dai Zhenhua Zhu Tianchen Zhao Shulin Zeng Yu Wang Huazhong Yang MQ 227 12 0 04 Jun 2020
Improved stochastic rounding Lu Xia M. Anthonissen M. Hochstenbach B. Koren 31 5 0 31 May 2020
SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computation Yang Zhao Xiaohan Chen Yue Wang Chaojian Li Haoran You Y. Fu Yuan Xie Zinan Lin Yingyan Lin MQ 195 45 0 07 May 2020
SIPA: A Simple Framework for Efficient Networks Gihun Lee Sangmin Bae Jaehoon Oh Seyoung Yun 56 1 0 24 Apr 2020
EMPIR: Ensembles of Mixed Precision Deep Networks for Increased Robustness against Adversarial Attacks Sanchari Sen Balaraman Ravindran A. Raghunathan FedML AAML 129 66 0 21 Apr 2020
Entropy-Based Modeling for Estimating Soft Errors Impact on Binarized Neural Network Inference N. Khoshavi S. Sargolzaei A. Roohi Connor Broyles Yu Bi AAML 98 1 0 10 Apr 2020
COVID-MobileXpert: On-Device COVID-19 Patient Triage and Follow-up using Chest X-rays Xin Li Chengyin Li D. Zhu 149 81 0 06 Apr 2020
CNN2Gate: Toward Designing a General Framework for Implementation of Convolutional Neural Networks on FPGA Alireza Ghaffari Yvon Savaria 108 10 0 06 Apr 2020
A Survey of Methods for Low-Power Deep Learning and Computer VisionWorld Forum on Internet of Things (WF-IoT), 2020 Abhinav Goel Caleb Tung Yung-Hsiang Lu George K. Thiruvathukal VLM 107 100 0 24 Mar 2020
Trends and Advancements in Deep Neural Network Communication Felix Sattler Thomas Wiegand Wojciech Samek GNN 135 9 0 06 Mar 2020
Taurus: A Data Plane Architecture for Per-Packet MLInternational Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2020 Tushar Swamy Alexander Rucker M. Shahbaz Ishan Gaur K. Olukotun 111 98 0 12 Feb 2020
Low-Complexity LSTM Training and Inference with FloatSD8 Weight Representation Yu-Tung Liu T. Chiueh MQ 41 1 0 23 Jan 2020
Shifted and Squeezed 8-bit Floating Point format for Low-Precision Training of Deep Neural NetworksInternational Conference on Learning Representations (ICLR), 2020 Léopold Cambier Anahita Bhiwandiwalla Ting Gong M. Nekuii Oguz H. Elibol Hanlin Tang MQ 191 52 0 16 Jan 2020
Sparse Weight Activation TrainingNeural Information Processing Systems (NeurIPS), 2020 Md Aamir Raihan Tor M. Aamodt 246 77 0 07 Jan 2020
Towards Unified INT8 Training for Convolutional Neural NetworkComputer Vision and Pattern Recognition (CVPR), 2019 Feng Zhu Yazhe Niu F. Yu Xianglong Liu Yanfei Wang Zhelong Li Xiuqi Yang Junjie Yan MQ 185 166 0 29 Dec 2019
PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy-efficient ReRAMIEEE transactions on computers (IEEE Trans. Comput.), 2019 Aayush Ankit I. E. Hajj S. R. Chalamalasetti S. Agarwal M. Marinella M. Foltin J. Strachan D. Milojicic Wen-mei W. Hwu Kaushik Roy 109 75 0 24 Dec 2019
Band-limited Training and Inference for Convolutional Neural NetworksInternational Conference on Machine Learning (ICML), 2019 Adam Dziedzic John Paparrizos S. Krishnan Aaron J. Elmore Michael Franklin 116 62 0 21 Nov 2019
Auto-Precision Scaling for Distributed Deep LearningInformation Security Conference (IS), 2019 Ruobing Han J. Demmel Yang You 94 5 0 20 Nov 2019
On-Device Machine Learning: An Algorithms and Learning Theory Perspective Sauptik Dhar Junyao Guo Jiayi Liu S. Tripathi Unmesh Kurup Mohak Shah 299 162 0 02 Nov 2019
Adaptive Precision Training: Quantify Back Propagation in Neural Networks with Fixed-point Numbers Xishan Zhang Shaoli Liu Rui Zhang Yu Xie Di Huang ... Jiaming Guo Yu Kang Qi Guo Zidong Du Yunji Chen MQ 110 8 0 01 Nov 2019
E2-Train: Training State-of-the-art CNNs with Over 80% Energy SavingsNeural Information Processing Systems (NeurIPS), 2019 Yue Wang Ziyu Jiang Xiaohan Chen Pengfei Xu Yang Zhao Yingyan Lin Zinan Lin MQ 251 88 0 29 Oct 2019
Secure Evaluation of Quantized Neural NetworksIACR Cryptology ePrint Archive (IACR ePrint), 2019 Anders Dalskov Daniel E. Escudero Marcel Keller 203 146 0 28 Oct 2019
Adaptive Loss Scaling for Mixed Precision Training Ruizhe Zhao Brian K. Vogel Tanvir Ahmed 97 9 0 28 Oct 2019
QPyTorch: A Low-Precision Arithmetic Simulation Framework Tianyi Zhang Zhiqiu Lin Guandao Yang Christopher De Sa MQ 110 69 0 09 Oct 2019
QuaRL: Quantization for Fast and Environmentally Sustainable Reinforcement Learning Srivatsan Krishnan Maximilian Lam Sharad Chitlangia Zishen Wan Gabriel Barth-Maron Aleksandra Faust Vijay Janapa Reddi MQ 123 31 0 02 Oct 2019
Training Deep Neural Networks Using Posit Number SystemACM Symposium on Cloud Computing (SoCC), 2019 Jinming Lu Siyuan Lu Zhisheng Wang Chao Fang Jun Lin Zhongfeng Wang Li Du MQ 85 14 0 06 Sep 2019
Training High-Performance and Large-Scale Deep Neural Networks with Full 8-bit IntegersNeural Networks (NN), 2019 Yukuan Yang Shuang Wu Lei Deng Tianyi Yan Yuan Xie Guoqi Li MQ 260 117 0 05 Sep 2019
Cheetah: Mixed Low-Precision Hardware & Software Co-Design Framework for DNNs on the Edge H. F. Langroudi Zachariah Carmichael David Pastuch Dhireesha Kudithipudi 123 24 0 06 Aug 2019
Deep Learning Training on the Edge with Low-Precision Posits H. F. Langroudi Zachariah Carmichael Dhireesha Kudithipudi MQ 105 14 0 30 Jul 2019
DeepCABAC: A Universal Compression Algorithm for Deep Neural NetworksIEEE Journal on Selected Topics in Signal Processing (JSTSP), 2019 Simon Wiedemann H. Kirchhoffer Stefan Matlage Paul Haase Arturo Marbán ... Ahmed Osman D. Marpe H. Schwarz Thomas Wiegand Wojciech Samek 224 105 0 27 Jul 2019