A Survey on Model MoErging: Recycling and Routing Among Specialized
Experts for Collaborative Learning

A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning

13 August 2024

Mohammed Muqeeth

Lucas Page-Caccia

Alessandro Sordoni

Papers citing "A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning"

12 / 12 papers shown

Title
FedMerge: Federated Personalization via Model Merging Shutong Chen Tianyi Zhou Guodong Long Jing Jiang Chengqi Zhang FedML MoMe 38 0 0 09 Apr 2025
Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging Zhenyi Lu Chenghao Fan Wei Wei Xiaoye Qu Dangyang Chen Yu Cheng MoMe 40 1 0 17 Jun 2024
Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters Jiazuo Yu Yunzhi Zhuge Lu Zhang Ping Hu Dong Wang Huchuan Lu You He VLM KELM CLL OODD 92 13 0 18 Mar 2024
Training Neural Networks from Scratch with Parallel Low-Rank Adapters Minyoung Huh Brian Cheung Jeremy Bernstein Phillip Isola Pulkit Agrawal 21 10 0 26 Feb 2024
Language Models are Multilingual Chain-of-Thought Reasoners Freda Shi Mirac Suzgun Markus Freitag Xuezhi Wang Suraj Srivats ... Yi Tay Sebastian Ruder Denny Zhou Dipanjan Das Jason W. Wei ReLM LRM 160 320 0 06 Oct 2022
ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts Akari Asai Mohammadreza Salehi Matthew E. Peters Hannaneh Hajishirzi 115 98 0 24 May 2022
Multitask Prompted Training Enables Zero-Shot Task Generalization Victor Sanh Albert Webson Colin Raffel Stephen H. Bach Lintang Sutawika ... T. Bers Stella Biderman Leo Gao Thomas Wolf Alexander M. Rush LRM 203 1,651 0 15 Oct 2021
SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer Tu Vu Brian Lester Noah Constant Rami Al-Rfou Daniel Matthew Cer VLM LRM 131 235 0 15 Oct 2021
Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference Sneha Kudugunta Yanping Huang Ankur Bapna M. Krikun Dmitry Lepikhin Minh-Thang Luong Orhan Firat MoE 102 87 0 24 Sep 2021
Robust Federated Learning by Mixture of Experts S. Parsaeefard Sayed Ehsan Etesami A. Leon-Garcia MoE FedML 18 4 0 23 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 272 3,784 0 18 Apr 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 217 3,054 0 23 Jan 2020