Lazarus: Resilient and Elastic Training of Mixture-of-Experts Models with Adaptive Expert Placement

5 July 2024

Zhuohao Li

Danyang Zhuo

Papers citing "Lazarus: Resilient and Elastic Training of Mixture-of-Experts Models with Adaptive Expert Placement"

3 / 3 papers shown

Title
Shockwave: Fair and Efficient Cluster Scheduling for Dynamic Adaptation in Machine Learning Pengfei Zheng Rui Pan Tarannum Khan Shivaram Venkataraman Aditya Akella 24 20 0 30 Sep 2022
Tutel: Adaptive Mixture-of-Experts at Scale Changho Hwang Wei Cui Yifan Xiong Ziyue Yang Ze Liu ... Joe Chau Peng Cheng Fan Yang Mao Yang Y. Xiong MoE 92 108 0 07 Jun 2022
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 147 326 0 18 Feb 2022