Collage: Light-Weight Low-Precision Strategy for LLM Training

6 May 2024

Papers citing "Collage: Light-Weight Low-Precision Strategy for LLM Training"

5 / 5 papers shown

Title
Stochastic Rounding for LLM Training: Theory and Practice Kaan Ozkara Tao Yu Youngsuk Park 36 0 0 27 Feb 2025
ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation Z. Yao Xiaoxia Wu Cheng-rong Li Stephen Youn Yuxiong He MQ 63 57 0 15 Mar 2023
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,815 0 17 Sep 2019
NeMo: a toolkit for building AI applications using Neural Modules Oleksii Kuchaiev Jason Chun Lok Li Huyen Nguyen Oleksii Hrinchuk Ryan Leary ... Jack Cook P. Castonguay Mariya Popova Jocelyn Huang Jonathan M. Cohen 182 291 0 14 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018