RAMP: A Flat Nanosecond Optical Network and MPI Operations for Distributed Deep Learning Systems

28 November 2022

Papers citing "RAMP: A Flat Nanosecond Optical Network and MPI Operations for Distributed Deep Learning Systems"

4 / 4 papers shown

Title
Partitioning Distributed Compute Jobs with Reinforcement Learning and Graph Neural Networks Christopher W. F. Parsonson Zacharaya Shabka Alessandro Ottino G. Zervas 21 0 0 31 Jan 2023
ZeRO-Offload: Democratizing Billion-Scale Model Training Jie Ren Samyam Rajbhandari Reza Yazdani Aminabadi Olatunji Ruwase Shuangyang Yang Minjia Zhang Dong Li Yuxiong He MoE 162 413 0 18 Jan 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 231 4,460 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,817 0 17 Sep 2019