A General Theory for Softmax Gating Multinomial Logistic Mixture of
Experts

A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts

22 October 2023

Huy Nguyen

Pedram Akbarian

TrungTin Nguyen

Papers citing "A General Theory for Softmax Gating Multinomial Logistic Mixture of Experts"

8 / 8 papers shown

Title
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications Siyuan Mu Sen Lin MoE 74 1 0 10 Mar 2025
Sigmoid Gating is More Sample Efficient than Softmax Gating in Mixture of Experts Huy Nguyen Nhat Ho Alessandro Rinaldo 37 3 0 22 May 2024
Generalization Error Analysis for Sparse Mixture-of-Experts: A Preliminary Study Jinze Zhao Peihao Wang Zhangyang Wang MoE 11 2 0 26 Mar 2024
On Least Square Estimation in Softmax Gating Mixture of Experts Huy Nguyen Nhat Ho Alessandro Rinaldo 36 13 0 05 Feb 2024
FuseMoE: Mixture-of-Experts Transformers for Fleximodal Fusion Xing Han Huy Nguyen Carl Harris Nhat Ho S. Saria MoE 47 16 0 05 Feb 2024
Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts? Huy Nguyen Pedram Akbarian Nhat Ho MoE 14 10 0 25 Jan 2024
Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Experts Huy Nguyen TrungTin Nguyen Khai Nguyen Nhat Ho MoE 30 12 0 12 May 2023
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 137 323 0 18 Feb 2022