ResearchTrend.AI
TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training

TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training

Chang-Qin Chen
Min Li
Zhihua Wu
Dianhai Yu
Chao Yang
    MoE
ArXiv (abs)PDFHTML

Papers citing "TA-MoE: Topology-Aware Large Scale Mixture-of-Expert Training"

4 / 4 papers shown
Title
Speculative MoE: Communication Efficient Parallel MoE Inference with Speculative Token and Expert Pre-scheduling
Speculative MoE: Communication Efficient Parallel MoE Inference with Speculative Token and Expert Pre-scheduling
Yan Li
Pengfei Zheng
Shuang Chen
Zewei Xu
Yuanhao Lai
Yunfei Du
Zehao Wang
MoE
648
1
0
06 Mar 2025
Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts
Shortcut-connected Expert Parallelism for Accelerating Mixture-of-Experts
Weilin Cai
Juyong Jiang
Le Qin
Junwei Cui
Sunghun Kim
Jiayi Huang
278
19
0
07 Apr 2024
Task-Based MoE for Multitask Multilingual Machine Translation
Task-Based MoE for Multitask Multilingual Machine Translation
Hai Pham
Young Jin Kim
Subhabrata Mukherjee
David P. Woodruff
Barnabás Póczós
Hany Awadalla
MoE
116
6
0
30 Aug 2023
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
Pre-gated MoE: An Algorithm-System Co-Design for Fast and Scalable Mixture-of-Expert Inference
Ranggi Hwang
Jianyu Wei
Shijie Cao
Changho Hwang
Xiaohu Tang
Ting Cao
Mao Yang
MoE
240
66
0
23 Aug 2023