Demystifying Softmax Gating Function in Gaussian Mixture of Experts

Demystifying Softmax Gating Function in Gaussian Mixture of Experts

5 May 2023

Huy Nguyen

TrungTin Nguyen

Papers citing "Demystifying Softmax Gating Function in Gaussian Mixture of Experts"

6 / 6 papers shown

Title
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications Siyuan Mu Sen Lin MoE 93 1 0 10 Mar 2025
Quadratic Gating Functions in Mixture of Experts: A Statistical Insight Pedram Akbarian Huy Le Nguyen Xing Han Nhat Ho MoE 32 0 0 15 Oct 2024
Revisiting Prefix-tuning: Statistical Benefits of Reparameterization among Prompts Minh Le Chau Nguyen Huy Nguyen Quyen Tran Trung Le Nhat Ho 30 3 0 03 Oct 2024
FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion Xing Han Huy Nguyen Carl Harris Nhat Ho S. Saria MoE 69 16 0 05 Feb 2024
A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts Huy Nguyen Pedram Akbarian TrungTin Nguyen Nhat Ho 16 10 0 22 Oct 2023
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 147 326 0 18 Feb 2022