1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training
with LAMB's Convergence Speed

1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed

13 April 2021

Samyam Rajbhandari

Yuxiong He

Papers citing "1-bit LAMB: Communication Efficient Large-Scale Large-Batch Training with LAMB's Convergence Speed"

5 / 5 papers shown

Title
Sketched Adaptive Federated Deep Learning: A Sharp Convergence Analysis Zhijie Chen Qiaobo Li A. Banerjee FedML 30 0 0 11 Nov 2024
SWARM Parallelism: Training Large Models Can Be Surprisingly Communication-Efficient Max Ryabinin Tim Dettmers Michael Diskin Alexander Borzunov MoE 22 31 0 27 Jan 2023
Large-Scale Deep Learning Optimizations: A Comprehensive Survey Xiaoxin He Fuzhao Xue Xiaozhe Ren Yang You 22 14 0 01 Nov 2021
Linearly Converging Error Compensated SGD Eduard A. Gorbunov D. Kovalev Dmitry Makarenko Peter Richtárik 163 77 0 23 Oct 2020
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,943 0 20 Apr 2018