Adaptive Gating in Mixture-of-Experts based Language Models

11 October 2023

Papers citing "Adaptive Gating in Mixture-of-Experts based Language Models"

8 / 8 papers shown

Title
HybriMoE: Hybrid CPU-GPU Scheduling and Cache Management for Efficient MoE Inference Shuzhang Zhong Y. Sun Ling Liang Runsheng Wang R. Huang Meng Li MoE 59 0 0 08 Apr 2025
CoT-VLM4Tar: Chain-of-Thought Guided Vision-Language Models for Traffic Anomaly Resolution Tianchi Ren Haibo Hu Jiacheng Zuo Xinhong Chen Jianping Wang Chun Jason Xue Jen-Ming Wu Nan Guan 53 0 0 03 Mar 2025
HOBBIT: A Mixed Precision Expert Offloading System for Fast MoE Inference Peng Tang Jiacheng Liu X. Hou Yifei Pu Jing Wang Pheng-Ann Heng C. Li M. Guo MoE 59 7 0 03 Nov 2024
AdaMoLE: Fine-Tuning Large Language Models with Adaptive Mixture of Low-Rank Adaptation Experts Zefang Liu Jiahua Luo MoE KELM 33 11 0 01 May 2024
Adaptive Inference: Theoretical Limits and Unexplored Opportunities S. Hor Ying Qian Mert Pilanci Amin Arbabian 8 0 0 06 Feb 2024
Mixture-of-Experts with Expert Choice Routing Yan-Quan Zhou Tao Lei Han-Chu Liu Nan Du Yanping Huang Vincent Zhao Andrew M. Dai Zhifeng Chen Quoc V. Le James Laudon MoE 147 326 0 18 Feb 2022
Scalable and Efficient MoE Training for Multitask Multilingual Models Young Jin Kim A. A. Awan Alexandre Muzio Andres Felipe Cruz Salinas Liyang Lu Amr Hendy Samyam Rajbhandari Yuxiong He Hany Awadalla MoE 94 83 0 22 Sep 2021
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 170 3,508 0 10 Jun 2015