Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient
for Convolutional Neural Networks

Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks

7 June 2023

Mohammed Nowaz Rabbani Chowdhury

Shuai Zhang

Papers citing "Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networks"

15 / 15 papers shown

Title
Learning Soft Sparse Shapes for Efficient Time-Series Classification Zhen Liu Yicheng Luo B. Li Emadeldeen Eldele Min-man Wu Qianli Ma AI4TS 29 0 0 11 May 2025
Backdoor Attacks Against Patch-based Mixture of Experts Cedric Chan Jona te Lintelo S. Picek AAML MoE 134 0 0 03 May 2025
SPMTrack: Spatio-Temporal Parameter-Efficient Fine-Tuning with Mixture of Experts for Scalable Visual Tracking Wenrui Cai Qingjie Liu Y. Wang MoE 60 0 0 24 Mar 2025
eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inference Suraiya Tairin Shohaib Mahmud Haiying Shen Anand Iyer MoE 138 0 0 10 Mar 2025
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications Siyuan Mu Sen Lin MoE 123 1 0 10 Mar 2025
Filtered not Mixed: Stochastic Filtering-Based Online Gating for Mixture of Large Language Models Raeid Saqur Anastasis Kratsios Florian Krach Yannick Limmer Jacob-Junqi Tian John Willes Blanka Horvath Frank Rudzicz MoE 45 0 0 24 Feb 2025
ViMoE: An Empirical Study of Designing Vision Mixture-of-Experts Xumeng Han Longhui Wei Zhiyang Dou Zipeng Wang Chenhui Qiang Xin He Yingfei Sun Zhenjun Han Qi Tian MoE 37 3 0 21 Oct 2024
What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding Hongkang Li Meng Wang Tengfei Ma Sijia Liu Zaixi Zhang Pin-Yu Chen MLT AI4CE 42 10 0 04 Jun 2024
A Provably Effective Method for Pruning Experts in Fine-tuned Sparse Mixture-of-Experts Mohammed Nowaz Rabbani Chowdhury Meng Wang K. E. Maghraoui Naigang Wang Pin-Yu Chen Christopher Carothers MoE 33 4 0 26 May 2024
SF-DQN: Provable Knowledge Transfer using Successor Feature for Deep Reinforcement Learning Shuai Zhang Heshan Devaka Fernando Miao Liu K. Murugesan Songtao Lu Pin-Yu Chen Tianyi Chen Meng Wang 38 1 0 24 May 2024
How Do Nonlinear Transformers Learn and Generalize in In-Context Learning? Hongkang Li Meng Wang Songtao Lu Xiaodong Cui Pin-Yu Chen MLT 40 14 0 23 Feb 2024
LocMoE: A Low-Overhead MoE for Large Language Model Training Jing Li Zhijie Sun Xuan He Li Zeng Yi Lin Entong Li Binfan Zheng Rongqian Zhao Xin Chen MoE 30 11 0 25 Jan 2024
From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscape Timothy R. McIntosh Teo Susnjak Tong Liu Paul Watters Malka N. Halgamuge 81 46 0 18 Dec 2023
On the Convergence and Sample Complexity Analysis of Deep Q-Networks with $ε$ -Greedy Exploration Shuai Zhang Hongkang Li Meng Wang Miao Liu Pin-Yu Chen Songtao Lu Sijia Liu K. Murugesan Subhajit Chaudhury 32 19 0 24 Oct 2023
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 160 327 0 18 Feb 2022