Ravnest: Decentralized Asynchronous Training on Heterogeneous Devices

3 January 2024

Papers citing "Ravnest: Decentralized Asynchronous Training on Heterogeneous Devices"

6 / 6 papers shown

Title
Diffusion Models in Vision: A Survey Florinel-Alin Croitoru Vlad Hondru Radu Tudor Ionescu M. Shah DiffM VLM MedIm 188 1,098 0 10 Sep 2022
Diffusion Models: A Comprehensive Survey of Methods and Applications Ling Yang Zhilong Zhang Yingxia Shao Shenda Hong Runsheng Xu Yue Zhao Wentao Zhang Bin Cui Ming-Hsuan Yang DiffM MedIm 215 1,277 0 02 Sep 2022
Asynchronous Decentralized Distributed Training of Acoustic Models Xiaodong Cui Wei Zhang Abdullah Kayi Mingrui Liu Ulrich Finkler Brian Kingsbury G. Saon David S. Kung 14 3 0 21 Oct 2021
Distributed Hierarchical GPU Parameter Server for Massive Scale Deep Learning Ads Systems Weijie Zhao Deping Xie Ronglai Jia Yulei Qian Rui Ding Mingming Sun P. Li MoE 55 147 0 12 Mar 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,791 0 17 Sep 2019
Optimal Distributed Online Prediction using Mini-Batches O. Dekel Ran Gilad-Bachrach Ohad Shamir Lin Xiao 164 684 0 07 Dec 2010