Mixed Precision Training

10 October 2017

Paulius Micikevicius

Sharan Narang

Boris Ginsburg

Papers citing "Mixed Precision Training"

50 / 265 papers shown

Title
Numerical Stability of DeepGOPlus Inference Inés Gonzalez Pepe Yohan Chatelain Gregory Kiar Tristan Glatard BDL 14 2 0 13 Dec 2022
CREPE: Open-Domain Question Answering with False Presuppositions Xinyan Velocity Yu Sewon Min Luke Zettlemoyer Hannaneh Hajishirzi 14 45 0 30 Nov 2022
RAMP: A Flat Nanosecond Optical Network and MPI Operations for Distributed Deep Learning Systems Alessandro Ottino Joshua L. Benjamin G. Zervas 17 7 0 28 Nov 2022
Tell Me What Happened: Unifying Text-guided Video Completion via Multimodal Masked Video Generation Tsu-jui Fu Licheng Yu Ning Zhang Cheng-Yang Fu Jong-Chyi Su William Yang Wang Sean Bell VGen 48 37 0 23 Nov 2022
Spikeformer: A Novel Architecture for Training High-Performance Low-Latency Spiking Neural Network Yudong Li Yunlin Lei Xu Yang 19 26 0 19 Nov 2022
GPS++: An Optimised Hybrid MPNN/Transformer for Molecular Property Prediction Dominic Masters Josef Dean Kerstin Klaser Zhiyi Li Sam Maddrell-Mander Adam Sanders Hatem Helal D. Beker Ladislav Rampášek Dominique Beaini 22 23 0 18 Nov 2022
MelHuBERT: A simplified HuBERT on Mel spectrograms Tzu-Quan Lin Hung-yi Lee Hao Tang SSL 24 13 0 17 Nov 2022
Massively Multilingual ASR on 70 Languages: Tokenization, Architecture, and Generalization Capabilities Andros Tjandra Nayan Singhal David C. Zhang Ozlem Kalinli Abdel-rahman Mohamed Duc Le M. Seltzer 19 12 0 10 Nov 2022
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model BigScience Workshop : Teven Le Scao Angela Fan Christopher Akiki ... Zhongli Xie Zifan Ye M. Bras Younes Belkada Thomas Wolf VLM 89 2,301 0 09 Nov 2022
MuMIC -- Multimodal Embedding for Multi-label Image Classification with Tempered Sigmoid Feng Wang Sarai Mizrachi Moran Beladev Guy Nadav Gil Amsalem Karen Lastmann Assaraf Hadas Harush Boker VLM 22 13 0 02 Nov 2022
Precision Machine Learning Eric J. Michaud Ziming Liu Max Tegmark 19 34 0 24 Oct 2022
Fast and Low-Memory Deep Neural Networks Using Binary Matrix Factorization Alireza Bordbar M. Kahaei MQ 17 0 0 24 Oct 2022
OLLA: Optimizing the Lifetime and Location of Arrays to Reduce the Memory Usage of Neural Networks Benoit Steiner Mostafa Elhoushi Jacob Kahn James Hegarty 29 8 0 24 Oct 2022
lo-fi: distributed fine-tuning without communication Mitchell Wortsman Suchin Gururangan Shen Li Ali Farhadi Ludwig Schmidt Michael G. Rabbat Ari S. Morcos 19 24 0 19 Oct 2022
Learning to Discover and Detect Objects V. Fomenko Ismail Elezi Deva Ramanan Laura Leal-Taixé Aljosa Osep ObjD 25 10 0 19 Oct 2022
Block Format Error Bounds and Optimal Block Size Selection I. Soloveychik I. Lyubomirsky Xin Eric Wang S. Bhoja MQ 27 4 0 11 Oct 2022
On Task-Adaptive Pretraining for Dialogue Response Selection Tzu-Hsiang Lin Ta-Chung Chi Anna Rumshisky 11 1 0 08 Oct 2022
An In-depth Study of Stochastic Backpropagation J. Fang Ming Xu Hao Chen Bing Shuai Z. Tu Joseph Tighe BDL 27 1 0 30 Sep 2022
From One to Many: Dynamic Cross Attention Networks for LiDAR and Camera Fusion Rui Wan Shuangjie Xu Wei Wu Xiaoyi Zou Tongyi Cao 3DPC 12 4 0 25 Sep 2022
Efficient Quantized Sparse Matrix Operations on Tensor Cores Shigang Li Kazuki Osawa Torsten Hoefler 72 31 0 14 Sep 2022
FP8 Formats for Deep Learning Paulius Micikevicius Dusan Stosic N. Burgess Marius Cornea Pradeep Dubey ... Naveen Mellempudi S. Oberman M. Shoeybi Michael Siu Hao Wu BDL VLM MQ 67 121 0 12 Sep 2022
Efficient Methods for Natural Language Processing: A Survey Marcos Vinícius Treviso Ji-Ung Lee Tianchu Ji Betty van Aken Qingqing Cao ... Emma Strubell Niranjan Balasubramanian Leon Derczynski Iryna Gurevych Roy Schwartz 28 109 0 31 Aug 2022
CLOWER: A Pre-trained Language Model with Contrastive Learning over Word and Character Representations Borun Chen Hongyin Tang Jiahao Bu Kai Zhang Jingang Wang Qifan Wang Haitao Zheng Wei Yu Wu Liqian Yu VLM 20 1 0 23 Aug 2022
A One-Shot Reparameterization Method for Reducing the Loss of Tile Pruning on DNNs Yancheng Li Qingzhong Ai Fumihiko Ino 25 0 0 29 Jul 2022
Liver Segmentation using Turbolift Learning for CT and Cone-beam C-arm Perfusion Imaging Hana Haseljić S. Chatterjee R. Frysch V. Kulvait V. Semshchikov ... Inga Brüsch T. Werncke Oliver Speck A. Nürnberger G. Rose OOD 22 2 0 20 Jul 2022
Dissecting Self-Supervised Learning Methods for Surgical Computer Vision Sanat Ramesh V. Srivastav Deepak Alapatt Tong Yu Aditya Murali ... Saurav Sharma A. Fleurentin Georgios Exarchakis Alexandros Karargyris N. Padoy 18 42 0 01 Jul 2022
e-CLIP: Large-Scale Vision-Language Representation Learning in E-commerce Wonyoung Shin Jonghun Park Taekang Woo Yongwoo Cho Kwangjin Oh Hwanjun Song VLM 24 16 0 01 Jul 2022
MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledge Linxi Fan Guanzhi Wang Yunfan Jiang Ajay Mandlekar Yuncong Yang Haoyi Zhu Andrew Tang De-An Huang Yuke Zhu Anima Anandkumar LM&Ro 42 347 0 17 Jun 2022
Recall Distortion in Neural Network Pruning and the Undecayed Pruning Algorithm Aidan Good Jia-Huei Lin Hannah Sieg Mikey Ferguson Xin Yu Shandian Zhe J. Wieczorek Thiago Serra 19 11 0 07 Jun 2022
MSR: Making Self-supervised learning Robust to Aggressive Augmentations Ying-Long Bai Erkun Yang Zhaoqing Wang Yuxuan Du Bo Han Cheng Deng Dadong Wang Tongliang Liu SSL 21 3 0 04 Jun 2022
Green Hierarchical Vision Transformer for Masked Image Modeling Lang Huang Shan You Mingkai Zheng Fei Wang Chao Qian T. Yamasaki 22 68 0 26 May 2022
ETAD: Training Action Detection End to End on a Laptop Shuming Liu Mengmeng Xu Chen Zhao Xu Zhao Bernard Ghanem 44 6 0 14 May 2022
RiCS: A 2D Self-Occlusion Map for Harmonizing Volumetric Objects Yunseok Jang Ruben Villegas Jimei Yang Duygu Ceylan Xin Sun Honglak Lee 3DH 14 0 0 14 May 2022
Adaptive Block Floating-Point for Analog Deep Learning Hardware Ayon Basumallik D. Bunandar Nicholas Dronen Nicholas Harris Ludmila Levkova Calvin McCarter Lakshmi Nair David Walter David Widemann 9 6 0 12 May 2022
PyramidCLIP: Hierarchical Feature Alignment for Vision-language Model Pretraining Yuting Gao Jinfeng Liu Zihan Xu Jinchao Zhang Ke Li Rongrong Ji Chunhua Shen VLM CLIP 25 100 0 29 Apr 2022
Vision-Language Pre-Training for Boosting Scene Text Detectors Sibo Song Jianqiang Wan Zhibo Yang Jun Tang Wenqing Cheng Xiang Bai Cong Yao VLM 34 24 0 29 Apr 2022
A DNN Framework for Learning Lagrangian Drift With Uncertainty J. Jenkins A. Paiement Y. Ourmières Julien Le Sommer J. Verron C. Ubelmann H. Glotin 24 5 0 12 Apr 2022
M $^2$ BEV: Multi-Camera Joint 3D Detection and Segmentation with Unified Birds-Eye View Representation Enze Xie Zhiding Yu Daquan Zhou Jonah Philion Anima Anandkumar Sanja Fidler Ping Luo J. Álvarez 28 179 0 11 Apr 2022
Robust Cross-Modal Representation Learning with Progressive Self-Distillation A. Andonian Shixing Chen Raffay Hamid VLM 19 55 0 10 Apr 2022
Federated Self-supervised Speech Representations: Are We There Yet? Yan Gao Javier Fernandez-Marques Titouan Parcollet Abhinav Mehrotra Nicholas D. Lane 22 13 0 06 Apr 2022
QuadraLib: A Performant Quadratic Neural Network Library for Architecture Optimization and Design Exploration Zirui Xu Fuxun Yu Jinjun Xiong Xiang Chen 25 23 0 01 Apr 2022
Automated Progressive Learning for Efficient Training of Vision Transformers Changlin Li Bohan Zhuang Guangrun Wang Xiaodan Liang Xiaojun Chang Yi Yang 16 46 0 28 Mar 2022
Large-scale Bilingual Language-Image Contrastive Learning ByungSoo Ko Geonmo Gu VLM 17 14 0 28 Mar 2022
A Survey of Robust Adversarial Training in Pattern Recognition: Fundamental, Theory, and Methodologies Zhuang Qian Kaizhu Huang Qiufeng Wang Xu-Yao Zhang OOD AAML ObjD 47 71 0 26 Mar 2022
FlexBlock: A Flexible DNN Training Accelerator with Multi-Mode Block Floating Point Support Seock-Hwan Noh Jahyun Koo Seunghyun Lee Jongse Park Jaeha Kung AI4CE 11 17 0 13 Mar 2022
A Mixed Quantization Network for Computationally Efficient Mobile Inverse Tone Mapping Juan Borrego-Carazo Mete Ozay Frederik Laboyrie Paul Wisbey MQ 19 0 0 12 Mar 2022
Tevatron: An Efficient and Flexible Toolkit for Dense Retrieval Luyu Gao Xueguang Ma Jimmy J. Lin Jamie Callan 22 75 0 11 Mar 2022
DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection Hao Zhang Feng Li Shilong Liu Lei Zhang Hang Su Jun Zhu L. Ni H. Shum ViT 51 1,367 0 07 Mar 2022
Carbon Footprint of Selecting and Training Deep Learning Models for Medical Image Analysis Raghavendra Selvan N. Bhagwat Lasse F. Wolff Anthony Benjamin Kanding Erik Dam 36 24 0 04 Mar 2022
Debiased Batch Normalization via Gaussian Process for Generalizable Person Re-Identification Jiawei Liu Zhipeng Huang Liang-Sheng Li Kecheng Zheng Zhengjun Zha 20 26 0 03 Mar 2022