XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection

v1v2 (latest)

XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection

27 February 2024

Chaozheng Wang

Cuiyun Gao

ArXiv (abs)PDF HTML Github (12★)

Papers citing "XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection"

8 / 8 papers shown

Title
AnyExperts: On-Demand Expert Allocation for Multimodal Language Models with Mixture of Expert Yuting Gao Wang Lan Hengyuan Zhao Linjiang Huang Si Liu Q. Guo MoE 140 0 0 23 Nov 2025
A Remarkably Efficient Paradigm to Multimodal Large Language Models for Sequential Recommendation Qiyong Zhong Jiajie Su Ming Yang Yunshan Ma Xiaolin Zheng Chaochao Chen 166 0 0 08 Nov 2025
TimeExpert: An Expert-Guided Video LLM for Video Temporal Grounding Zuhao Yang Yingchen Yu Yunqing Zhao Shijian Lu Song Bai 98 2 0 03 Aug 2025
UMoE: Unifying Attention and FFN with Shared Experts Yuanhang Yang Chaozheng Wang Jing Li MoE 219 0 0 12 May 2025
Union of Experts: Adapting Hierarchical Routing to Equivalently Decomposed Transformer Yujiao Yang Jing Lian Linhui Li MoE 329 0 0 04 Mar 2025
Rank Also Matters: Hierarchical Configuration for Mixture of Adapter Experts in LLM Fine-Tuning Peizhuang Cong Wenpu Liu Wenhan Yu Haochen Zhao Tong Yang ALM MoE 261 1 0 06 Feb 2025
CartesianMoE: Boosting Knowledge Sharing among Experts via Cartesian Product Routing in Mixture-of-ExpertsNorth American Chapter of the Association for Computational Linguistics (NAACL), 2024 Zhenpeng Su Xing Wu Zijia Lin Yizhe Xiong Minxuan Lv Guangyuan Ma Hui Chen Songlin Hu Guiguang Ding MoE 514 5 0 21 Oct 2024
Dynamic Mixture of Experts: An Auto-Tuning Approach for Efficient Transformer ModelsInternational Conference on Learning Representations (ICLR), 2024 Yongxin Guo Zhenglin Cheng Xiaoying Tang Tao Lin Tao Lin MoE 444 27 0 23 May 2024