Mixed Precision Training With 8-bit Floating Point

Mixed Precision Training With 8-bit Floating Point

29 May 2019

Naveen Mellempudi

Papers citing "Mixed Precision Training With 8-bit Floating Point"

11 / 11 papers shown

Title
$u-$\mu$P: The Unit-Scaled Maximal Update Parametrization$ u- $\mu$ P: The Unit-Scaled Maximal Update Parametrization Charlie Blake C. Eichenberg Josef Dean Lukas Balles Luke Y. Prince Bjorn Deiseroth Andres Felipe Cruz Salinas Carlo Luschi Samuel Weinbach Douglas Orr 53 9 0 24 Jul 2024
Stretching Each Dollar: Diffusion Training from Scratch on a Micro-Budget Vikash Sehwag Xianghao Kong Jingtao Li Michael Spranger Lingjuan Lyu DiffM 41 9 0 22 Jul 2024
FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Search Jordan Dotzel Gang Wu Andrew Li M. Umar Yun Ni ... Liqun Cheng Martin G. Dixon N. Jouppi Quoc V. Le Sheng R. Li MQ 25 3 0 07 Aug 2023
Numerical Stability of DeepGOPlus Inference Inés Gonzalez Pepe Yohan Chatelain Gregory Kiar Tristan Glatard BDL 19 2 0 13 Dec 2022
FP8 Formats for Deep Learning Paulius Micikevicius Dusan Stosic N. Burgess Marius Cornea Pradeep Dubey ... Naveen Mellempudi S. Oberman M. Shoeybi Michael Siu Hao Wu BDL VLM MQ 69 121 0 12 Sep 2022
8-bit Optimizers via Block-wise Quantization Tim Dettmers M. Lewis Sam Shleifer Luke Zettlemoyer MQ 17 268 0 06 Oct 2021
An Overview of Neural Network Compression James OÑeill AI4CE 45 98 0 05 Jun 2020
Deep Learning Training on the Edge with Low-Precision Posits H. F. Langroudi Zachariah Carmichael Dhireesha Kudithipudi MQ 16 14 0 30 Jul 2019
Neural Architecture Search with Reinforcement Learning Barret Zoph Quoc V. Le 264 5,326 0 05 Nov 2016
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Z. Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 716 6,743 0 26 Sep 2016
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 278 2,888 0 15 Sep 2016