TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training

TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training

20 February 2023

Dianhai Yu

ArXiv (abs)PDF HTML

Papers citing "TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training"

4 / 4 papers shown

Title
Speculative MoE: Communication Efficient Parallel MoE Inference with Speculative Token and Expert Pre-scheduling Yan Li Pengfei Zheng Shuang Chen Zewei Xu Yuanhao Lai Yunfei Du Zehao Wang MoE 648 1 0 06 Mar 2025
Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts Weilin Cai Juyong Jiang Le Qin Junwei Cui Sunghun Kim Jiayi Huang 278 19 0 07 Apr 2024
Task-Based MoE for Multitask Multilingual Machine Translation Hai Pham Young Jin Kim Subhabrata Mukherjee David P. Woodruff Barnabás Póczós Hany Awadalla MoE 116 6 0 30 Aug 2023
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference Ranggi Hwang Jianyu Wei Shijie Cao Changho Hwang Xiaohu Tang Ting Cao Mao Yang MoE 240 66 0 23 Aug 2023