Mixed Precision Training

10 October 2017

Paulius Micikevicius

Sharan Narang

Boris Ginsburg

Papers citing "Mixed Precision Training"

50 / 265 papers shown

Title
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Mohit Bansal MLLM 249 525 0 04 Feb 2021
EfficientQA : a RoBERTa Based Phrase-Indexed Question-Answering System Sofian Chaybouti Achraf Saghe A. Shabou RALM 37 8 0 06 Jan 2021
An Efficient Transformer Decoder with Compressed Sub-layers Yanyang Li Ye Lin Tong Xiao Jingbo Zhu 19 29 0 03 Jan 2021
Towards Zero-Shot Knowledge Distillation for Natural Language Processing Ahmad Rashid Vasileios Lioutas Abbas Ghaddar Mehdi Rezagholizadeh 13 27 0 31 Dec 2020
FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training Y. Fu Haoran You Yang Katie Zhao Yue Wang Chaojian Li K. Gopalakrishnan Zhangyang Wang Yingyan Lin MQ 30 32 0 24 Dec 2020
NeurST: Neural Speech Translation Toolkit Chengqi Zhao Mingxuan Wang Qianqian Dong Rong Ye Lei Li 14 32 0 18 Dec 2020
Meta Batch-Instance Normalization for Generalizable Person Re-Identification Seokeon Choi Taekyung Kim Minki Jeong Hyoungseob Park Changick Kim OOD 19 129 0 30 Nov 2020
Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformers Zhaoshuo Li Xingtong Liu Nathan G. Drenkow Andy S Ding Francis X. Creighton Russell H. Taylor Mathias Unberath MDE ViT 31 274 0 05 Nov 2020
Document-Level Relation Extraction with Adaptive Thresholding and Localized Context Pooling Wenxuan Zhou Kevin Huang Tengyu Ma Jing Huang 11 273 0 21 Oct 2020
FPRaker: A Processing Element For Accelerating Neural Network Training Omar Mohamed Awad Mostafa Mahmoud Isak Edo Vivancos Ali Hadi Zadeh Ciaran Bannon Anand Jayarajan Gennady Pekhimenko Andreas Moshovos 15 15 0 15 Oct 2020
PP-LinkNet: Improving Semantic Segmentation of High Resolution Satellite Imagery with Multi-stage Training An Tran Ali Zonoozi Jagannadan Varadarajan Hannes Kruppa SSeg 22 14 0 14 Oct 2020
Deep Volumetric Ambient Occlusion Dominik Engel Timo Ropinski 14 22 0 19 Aug 2020
Self-Supervised GAN Compression Chong Yu Jeff Pool 7 9 0 03 Jul 2020
LAMP: Large Deep Nets with Automated Model Parallelism for Image Segmentation Wentao Zhu Can Zhao Wenqi Li H. Roth Ziyue Xu Daguang Xu 3DV 24 18 0 22 Jun 2020
DS6, Deformation-aware Semi-supervised Learning: Application to Small Vessel Segmentation with Noisy Training Data S. Chatterjee Kartik Prabhu Mahantesh Pattadkal Gerda Bortsova Chompunuch Sarasaen Florian Dubost Hendrik Mattern Marleen de Bruijne Oliver Speck Andreas Nürnberger 4 18 0 18 Jun 2020
Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation Jungo Kasai Nikolaos Pappas Hao Peng James Cross Noah A. Smith 30 134 0 18 Jun 2020
Multi-Precision Policy Enforced Training (MuPPET): A precision-switching strategy for quantised fixed-point training of CNNs A. Rajagopal D. A. Vink Stylianos I. Venieris C. Bouganis MQ 16 14 0 16 Jun 2020
Automatic heterogeneous quantization of deep neural networks for low-latency inference on the edge for particle detectors C. Coelho Aki Kuusela Shane Li Zhuang Hao T. Aarrestad Vladimir Loncar J. Ngadiuba M. Pierini Adrian Alan Pol S. Summers MQ 13 175 0 15 Jun 2020
FastPitch: Parallel Text-to-speech with Pitch Prediction Adrian Lañcucki 6 332 0 11 Jun 2020
VirTex: Learning Visual Representations from Textual Annotations Karan Desai Justin Johnson SSL VLM 19 432 0 11 Jun 2020
Linformer: Self-Attention with Linear Complexity Sinong Wang Belinda Z. Li Madian Khabsa Han Fang Hao Ma 55 1,643 0 08 Jun 2020
An Overview of Neural Network Compression James OÑeill AI4CE 40 98 0 05 Jun 2020
High-Fidelity Audio Generation and Representation Learning with Guided Adversarial Autoencoder Kazi Nazmul Haque R. Rana Björn W Schuller DRL 24 12 0 01 Jun 2020
Optimizing Deep Learning Recommender Systems' Training On CPU Cluster Architectures Dhiraj D. Kalamkar E. Georganas S. Srinivasan Jianping Chen Mikhail Shiryaev A. Heinecke 43 47 0 10 May 2020
NTIRE 2020 Challenge on Spectral Reconstruction from an RGB Image Boaz Arad Radu Timofte Ohad Ben-Shahar Yi-Tun Lin G. Finlayson Shai Givati Mohamed H. Sedky 43 121 0 07 May 2020
FlexSA: Flexible Systolic Array Architecture for Efficient Pruned DNN Model Training Sangkug Lym M. Erez 11 25 0 27 Apr 2020
MXR-U-Nets for Real Time Hyperspectral Reconstruction Atmadeep Banerjee Akash Palrecha SupR 17 11 0 15 Apr 2020
Reducing Data Motion to Accelerate the Training of Deep Neural Networks Sicong Zhuang C. Malossi Marc Casas 17 0 0 05 Apr 2020
A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospects Zewen Li Wenjie Yang Shouheng Peng Fan Liu HAI 3DV 47 2,593 0 01 Apr 2020
Towards Rapid and Robust Adversarial Training with One-Step Attacks Leo Schwinn René Raab Björn Eskofier AAML 17 6 0 24 Feb 2020
Training Question Answering Models From Synthetic Data Raul Puri Ryan Spring M. Patwary M. Shoeybi Bryan Catanzaro ELM 24 158 0 22 Feb 2020
Stochastic Latent Residual Video Prediction Jean-Yves Franceschi E. Delasalles Mickaël Chen Sylvain Lamprier Patrick Gallinari VGen 26 159 0 21 Feb 2020
Compounding the Performance Improvements of Assembled Techniques in a Convolutional Neural Network Jungkyu Lee Taeryun Won Tae Kwan Lee Hyemin Lee Geonmo Gu K. Hong 26 57 0 17 Jan 2020
Fast is better than free: Revisiting adversarial training Eric Wong Leslie Rice J. Zico Kolter AAML OOD 20 1,158 0 12 Jan 2020
Towards Unified INT8 Training for Convolutional Neural Network Feng Zhu Ruihao Gong F. Yu Xianglong Liu Yanfei Wang Zhelong Li Xiuqi Yang Junjie Yan MQ 25 151 0 29 Dec 2019
PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy-efficient ReRAM Aayush Ankit I. E. Hajj S. R. Chalamalasetti S. Agarwal M. Marinella M. Foltin J. Strachan D. Milojicic Wen-mei W. Hwu Kaushik Roy 11 65 0 24 Dec 2019
MG-WFBP: Merging Gradients Wisely for Efficient Communication in Distributed Deep Learning S. Shi X. Chu Bo Li FedML 20 25 0 18 Dec 2019
Zero-shot Text Classification With Generative Language Models Raul Puri Bryan Catanzaro VLM 10 100 0 10 Dec 2019
JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus Makoto Morishita Jun Suzuki Masaaki Nagata LRM 30 64 0 25 Nov 2019
HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks Zhen Dong Z. Yao Yaohui Cai Daiyaan Arfeen A. Gholami Michael W. Mahoney Kurt Keutzer MQ 21 274 0 10 Nov 2019
Characterizing Deep Learning Training Workloads on Alibaba-PAI Mengdi Wang Chen Meng Guoping Long Chuan Wu Jun Yang Wei Lin Yangqing Jia 17 53 0 14 Oct 2019
MLPerf Training Benchmark Arya D. McCarthy Christine Cheng Cody Coleman Greg Diamos Paulius Micikevicius ... Carole-Jean Wu Lingjie Xu Masafumi Yamazaki C. Young Matei A. Zaharia 17 305 0 02 Oct 2019
NeMo: a toolkit for building AI applications using Neural Modules Oleksii Kuchaiev Jason Chun Lok Li Huyen Nguyen Oleksii Hrinchuk Ryan Leary ... Jack Cook P. Castonguay Mariya Popova Jocelyn Huang Jonathan M. Cohen 188 291 0 14 Sep 2019
Training Deep Neural Networks Using Posit Number System Jinming Lu Siyuan Lu Zhisheng Wang Chao Fang Jun Lin Zhongfeng Wang Li Du MQ 11 13 0 06 Sep 2019
Real-time Person Re-identification at the Edge: A Mixed Precision Approach Mohammadreza Baharani Shrey Mohan Hamed Tabkhi 19 10 0 19 Aug 2019
Deep Learning Training on the Edge with Low-Precision Posits H. F. Langroudi Zachariah Carmichael Dhireesha Kudithipudi MQ 11 14 0 30 Jul 2019
Sharing Attention Weights for Fast Transformer Tong Xiao Yinqiao Li Jingbo Zhu Zhengtao Yu Tongran Liu 17 50 0 26 Jun 2019
Mixed Precision Training With 8-bit Floating Point Naveen Mellempudi S. Srinivasan Dipankar Das Bharat Kaul MQ 8 68 0 29 May 2019
Which Tasks Should Be Learned Together in Multi-task Learning? Trevor Scott Standley Amir Zamir Dawn Chen Leonidas J. Guibas Jitendra Malik Silvio Savarese 13 502 0 18 May 2019
AI Enabling Technologies: A Survey V. Gadepally Justin A. Goodwin J. Kepner Albert Reuther Hayley Reynolds S. Samsi Jonathan Su David Martinez 17 24 0 08 May 2019