MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts

18 October 2024

Papers citing "MomentumSMoE: Integrating Momentum into Sparse Mixture of Experts"

3 / 3 papers shown

Title
MoLEx: Mixture of Layer Experts for Finetuning with Sparse Upcycling R. Teo T. Nguyen MoE 53 2 0 14 Mar 2025
CAMEx: Curvature-aware Merging of Experts Dung V. Nguyen Minh H. Nguyen Luc Q. Nguyen R. Teo T. Nguyen Linh Duy Tran MoMe 70 2 0 26 Feb 2025
Tight Clusters Make Specialized Experts Stefan K. Nielsen R. Teo Laziz U. Abdullaev Tan M. Nguyen MoE 51 2 0 21 Feb 2025