Title
Towards Easy and Realistic Network Infrastructure Testing for Large-scale Machine Learning Jinsun Yoo ChonLam Lao Lianjie Cao Bob Lantz Minlan Yu Tushar Krishna Puneet Sharma 47 0 0 29 Apr 2025
Accelerating Mixture-of-Experts Training with Adaptive Expert Replication Athinagoras Skiadopoulos Mark Zhao Swapnil Gandhi Thomas Norrie Shrijeet Mukherjee Christos Kozyrakis MoE 85 0 0 28 Apr 2025
Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training Yijie Zheng Bangjun Xiao Lei Shi Xiaoyang Li Faming Wu Tianyu Li Xuefeng Xiao Y. Zhang Y. Wang Shouda Liu MLLM MoE 53 1 0 31 Mar 2025
Democratizing AI: Open-source Scalable LLM Training on GPU-based Supercomputers Siddharth Singh Prajwal Singhania Aditya K. Ranjan John Kirchenbauer Jonas Geiping ... Abhimanyu Hans Manli Shu Aditya Tomar Tom Goldstein A. Bhatele 89 2 0 12 Feb 2025
Revisiting Reliability in Large-Scale Machine Learning Research Clusters Apostolos Kokolis Michael Kuchnik John Hoffman Adithya Kumar Parth Malani Faye Ma Zachary DeVito S. Kalyan Saladi Carole-Jean Wu 55 7 0 29 Oct 2024
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 31 37 0 09 Jul 2024
The infrastructure powering IBM's Gen AI model development Talia Gershon Seetharami R. Seelam Brian M. Belgodere Milton Bonilla Lan Hoang ... Ruchir Puri Dakshi Agrawal Drew Thorstensen Joel Belog Brent Tang VLM 27 4 0 07 Jul 2024
A Survey on Failure Analysis and Fault Injection in AI Systems Guangba Yu Gou Tan Haojia Huang Zhenyu Zhang Pengfei Chen Roberto Natella Zibin Zheng 29 3 0 28 Jun 2024
Optimizing Large Model Training through Overlapped Activation Recomputation Ping Chen Wenjie Zhang Shuibing He Yingjie Gu Zhuwei Peng ... Yi Zheng Zhefeng Wang Yanlong Yin Gang Chen Gang Chen 33 5 0 13 Jun 2024
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 220 4,424 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,791 0 17 Sep 2019