ResearchTrend.AI
Model Parallelism on Distributed Infrastructure: A Literature Review from Theory to LLM Case-Studies

Model Parallelism on Distributed Infrastructure: A Literature Review from Theory to LLM Case-Studies

Felix Brakel
Uraz Odyurt
A. Varbanescu
    GNN
ArXivPDFHTML

Papers citing "Model Parallelism on Distributed Infrastructure: A Literature Review from Theory to LLM Case-Studies"

1 / 1 papers shown
Title
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
M. Shoeybi
M. Patwary
Raul Puri
P. LeGresley
Jared Casper
Bryan Catanzaro
MoE
243
1,791
0
17 Sep 2019