Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

23 January 2017

Papers citing "Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer"

50 / 495 papers shown

Title
Multi-modal Collaborative Optimization and Expansion Network for Event-assisted Single-eye Expression Recognition Runduo Han Xiuping Liu Shangxuan Yi Yi Zhang Hongchen Tan 0 0 0 17 May 2025
Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis Akarsh Kumar Jeff Clune Joel Lehman Kenneth O. Stanley OOD 4 0 0 16 May 2025
Extending Large Vision-Language Model for Diverse Interactive Tasks in Autonomous Driving Zongchuang Zhao Haoyu Fu Dingkang Liang Xin Zhou Dingyuan Zhang Hongwei Xie Bing Wang Xiang Bai MLLM VLM 49 0 0 13 May 2025
UMoE: Unifying Attention and FFN with Shared Experts Yuanhang Yang Chaozheng Wang Jing Li MoE 29 0 0 12 May 2025
Beyond CLIP Generalization: Against Forward&Backward Forgetting Adapter for Continual Learning of Vision-Language Models Songlin Dong Chenhao Ding Jiangyang Li Jizhou Han Qiang Wang Yuhang He Yihong Gong CLL VLM 40 0 0 12 May 2025
Making Small Language Models Efficient Reasoners: Intervention, Supervision, Reinforcement Xuechen Zhang Zijian Huang Chenshun Ni Ziyang Xiong J. Chen Samet Oymak ReLM LRM 45 0 0 12 May 2025
The power of fine-grained experts: Granularity boosts expressivity in Mixture of Experts Enric Boix Adserà Philippe Rigollet MoE 28 0 0 11 May 2025
Learning Soft Sparse Shapes for Efficient Time-Series Classification Zhen Liu Yicheng Luo Yangqiu Song Emadeldeen Eldele Min-man Wu Qianli Ma AI4TS 29 0 0 11 May 2025
QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration HamidReza Imani Jiaxin Peng Peiman Mohseni Abdolah Amirany Tarek A. El-Ghazawi MoE 31 0 0 10 May 2025
TREND: Tri-teaching for Robust Preference-based Reinforcement Learning with Demonstrations Shuaiyi Huang Mara Levy Anubhav Gupta Daniel Ekpo Ruijie Zheng Abhinav Shrivastava 28 0 0 09 May 2025
ORXE: Orchestrating Experts for Dynamically Configurable Efficiency Qingyuan Wang Guoxin Wang B. Cardiff Deepu John 38 0 0 07 May 2025
MoDE: Mixture of Diffusion Experts for Any Occluded Face Recognition Qiannan Fan Zhuoyang Li Jitong Li Chenyang Cao DiffM 40 0 0 07 May 2025
3D Gaussian Splatting Data Compression with Mixture of Priors Lei Liu Zhenghao Chen Dong Xu 3DGS 169 0 0 06 May 2025
Faster MoE LLM Inference for Extremely Large Models Haoqi Yang Luohe Shi Qiwei Li Zuchao Li Ping Wang Bo Du Mengjia Shen Hai Zhao MoE 63 0 0 06 May 2025
Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques Sanjay Surendranath Girija Shashank Kapoor Lakshit Arora Dipen Pradhan Aman Raj Ankit Shetgaonkar 57 0 0 05 May 2025
Learning Heterogeneous Mixture of Scene Experts for Large-scale Neural Radiance Fields Zhenxing Mi Ping Yin Xue Xiao Dan Xu MoE 49 0 0 04 May 2025
Backdoor Attacks Against Patch-based Mixture of Experts Cedric Chan Jona te Lintelo S. Picek AAML MoE 151 0 0 03 May 2025
Don't be lazy: CompleteP enables compute-efficient deep transformers Nolan Dey Bin Claire Zhang Lorenzo Noci Mufan Li Blake Bordelon Shane Bergsma Cengiz Pehlevan Boris Hanin Joel Hestness 44 0 0 02 May 2025
Mixture of Sparse Attention: Content-Based Learnable Sparse Attention via Expert-Choice Routing Piotr Piekos Róbert Csordás Jürgen Schmidhuber MoE VLM 99 1 0 01 May 2025
Improving Routing in Sparse Mixture of Experts with Graph of Tokens Tam Minh Nguyen Ngoc N. Tran Khai Nguyen Richard G. Baraniuk MoE 66 0 0 01 May 2025
Pushing the Limits of Low-Bit Optimizers: A Focus on EMA Dynamics Cong Xu Wenbin Liang Mo Yu Anan Liu Kaipeng Zhang Lizhuang Ma Yufei Guo Jun Wang Weixi Zhang MQ 57 0 0 01 May 2025
Memorization and Knowledge Injection in Gated LLMs Xu Pan Ely Hahami Zechen Zhang H. Sompolinsky KELM CLL RALM 104 1 0 30 Apr 2025
TT-LoRA MoE: Unifying Parameter-Efficient Fine-Tuning and Sparse Mixture-of-Experts Pradip Kunwar Minh Vu Maanak Gupta Mahmoud Abdelsalam Manish Bhattarai MoE MoMe 137 0 0 29 Apr 2025
Model Connectomes: A Generational Approach to Data-Efficient Language Models Klemen Kotar Greta Tuckute 51 0 0 29 Apr 2025
In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformer Zechuan Zhang Ji Xie Yu Lu Zongxin Yang Yi Yang DiffM 97 1 0 29 Apr 2025
MambaMoE: Mixture-of-Spectral-Spatial-Experts State Space Model for Hyperspectral Image Classification Yichu Xu Di Wang Hongzan Jiao Li Zhang L. Zhang Mamba 83 0 0 29 Apr 2025
X-Fusion: Introducing New Modality to Frozen Large Language Models Sicheng Mo Thao Nguyen Xun Huang Siddharth Srinivasan Iyer Yijun Li ... Eli Shechtman Krishna Kumar Singh Yong Jae Lee Bolei Zhou Yuheng Li 77 0 0 29 Apr 2025
MicarVLMoE: A Modern Gated Cross-Aligned Vision-Language Mixture of Experts Model for Medical Image Captioning and Report Generation Amaan Izhar Nurul Japar Norisma Idris Ting Dang MoE 73 0 0 29 Apr 2025
Accelerating Mixture-of-Experts Training with Adaptive Expert Replication Athinagoras Skiadopoulos Mark Zhao Swapnil Gandhi Thomas Norrie Shrijeet Mukherjee Christos Kozyrakis MoE 91 0 0 28 Apr 2025
Neural network task specialization via domain constraining Roman Malashin Daniil Ilyukhin 49 0 0 28 Apr 2025
Unveiling the Hidden: Movie Genre and User Bias in Spoiler Detection Haokai Zhang Shengtao Zhang Zijian Cai Heng Wang Ruixuan Zhu Zinan Zeng Minnan Luo 54 0 0 24 Apr 2025
Hexcute: A Tile-based Programming Language with Automatic Layout and Task-Mapping Synthesis Xiaotian Zhang Yaoyao Ding Yang Hu Gennady Pekhimenko 41 0 0 22 Apr 2025
MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core Dennis Liu Zijie Yan Xin Yao Tong Liu V. Korthikanti ... Jiajie Yao Chandler Zhou David Wu Xipeng Li J. Yang MoE 70 0 0 21 Apr 2025
Manipulating Multimodal Agents via Cross-Modal Prompt Injection Le Wang Zonghao Ying Tianyuan Zhang Siyuan Liang Shengshan Hu Mingchuan Zhang A. Liu Xianglong Liu AAML 33 1 0 19 Apr 2025
Dense Backpropagation Improves Training for Sparse Mixture-of-Experts Ashwinee Panda Vatsal Baherwani Zain Sarwar Benjamin Thérien Supriyo Chakraborty Tom Goldstein MoE 42 0 0 16 Apr 2025
Saliency-Motion Guided Trunk-Collateral Network for Unsupervised Video Object Segmentation Xiangyu Zheng Wanyun Li Songcheng He Jianping Fan Xiaoqiang Li We Zhang VOS 35 0 0 08 Apr 2025
Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Parallelism Design Mohan Zhang Pingzhi Li Jie Peng Mufan Qiu Tianlong Chen MoE 50 0 0 02 Apr 2025
Biologically Inspired Spiking Diffusion Model with Adaptive Lateral Selection Mechanism Linghao Feng Dongcheng Zhao Sicheng Shen Yi Zeng 67 0 0 31 Mar 2025
Mixture of Routers Jia-Chen Zhang Yu-Jie Xiong Xi-He Qiu Chun-Ming Xia Fei Dai MoE 76 0 0 30 Mar 2025
Exploiting Mixture-of-Experts Redundancy Unlocks Multimodal Generative Abilities Raman Dutt Harleen Hanspal Guoxuan Xia Petru-Daniel Tudosiu Alexander Black Yongxin Yang Steven G. McDonagh Sarah Parisot MoE 43 0 0 28 Mar 2025
MoLe-VLA: Dynamic Layer-skipping Vision Language Action Model via Mixture-of-Layers for Efficient Robot Manipulation Rongyu Zhang Menghang Dong Yuan Zhang Liang Heng Xiaowei Chi Gaole Dai Li Du Dan Wang Yuan Du MoE 84 0 0 26 Mar 2025
BiPrompt-SAM: Enhancing Image Segmentation via Explicit Selection between Point and Text Prompts Suzhe Xu Jialin Peng Chengyuan Zhang VLM 54 0 0 25 Mar 2025
Resilient Sensor Fusion under Adverse Sensor Failures via Multi-Modal Expert Fusion Konyul Park Yecheol Kim Daehun Kim Jun-Won Choi 41 0 0 25 Mar 2025
Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantization Minsu Kim Seongmin Hong RyeoWook Ko S. Choi Hunjong Lee Junsoo Kim Joo-Young Kim Jongse Park 57 0 0 24 Mar 2025
Core-Periphery Principle Guided State Space Model for Functional Connectome Classification Minheng Chen Xiaowei Yu Jing Zhang Tong Chen Chao-Yang Cao Yan Zhuang Yanjun Lyu Lu Zhang Tianming Liu D. Zhu Mamba 58 0 0 18 Mar 2025
BREEN: Bridge Data-Efficient Encoder-Free Multimodal Learning with Learnable Queries Tianle Li Yongming Rao Winston Hu Yu Cheng MLLM 66 0 0 16 Mar 2025
A Comprehensive Survey of Mixture-of-Experts: Algorithms, Theory, and Applications Siyuan Mu Sen Lin MoE 135 2 0 10 Mar 2025
eMoE: Task-aware Memory Efficient Mixture-of-Experts-Based (MoE) Model Inference Suraiya Tairin Shohaib Mahmud Haiying Shen Anand Iyer MoE 158 0 0 10 Mar 2025
MoFE: Mixture of Frozen Experts Architecture Jean Seo Jaeyoon Kim Hyopil Shin MoE 167 0 0 09 Mar 2025
Swift Hydra: Self-Reinforcing Generative Framework for Anomaly Detection with Multiple Mamba Models Nguyen H K. Do Truc Nguyen Malik Hassanaly Raed Alharbi Jung Taek Seo My T. Thai 54 0 0 09 Mar 2025