Scalify: scale propagation for efficient low-precision LLM training

Scalify: scale propagation for efficient low-precision LLM training

24 July 2024

Paul Balança

Carlo Luschi

Andrew Fitzgibbon

Papers citing "Scalify: scale propagation for efficient low-precision LLM training"

3 / 3 papers shown

Title
ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks Xiaoxia Wu Haojun Xia Stephen Youn Zhen Zheng Shiyang Chen ... Reza Yazdani Aminabadi Yuxiong He Olatunji Ruwase Leon Song Zhewei Yao 66 8 0 14 Dec 2023
FP8-LM: Training FP8 Large Language Models Houwen Peng Kan Wu Yixuan Wei Guoshuai Zhao Yuxiang Yang ... Zheng-Wei Zhang Shuguang Liu Joe Chau Han Hu Peng Cheng MQ 59 37 0 27 Oct 2023
FP8 Formats for Deep Learning Paulius Micikevicius Dusan Stosic N. Burgess Marius Cornea Pradeep Dubey ... Naveen Mellempudi S. Oberman M. Shoeybi Michael Siu Hao Wu BDL VLM MQ 67 119 0 12 Sep 2022