Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging

17 June 2024

Zhenyi Lu

Chenghao Fan

Wei Wei

Xiaoye Qu

Dangyang Chen

Yu Cheng

MoMe

ArXiv (abs)PDF HTML HuggingFace (2 upvotes)

Papers citing "Twin-Merging: Dynamic Integration of Modular Expertise in Model Merging"

43 / 43 papers shown

Title
Stay Unique, Stay Efficient: Preserving Model Personality in Multi-Task Merging Kuangpu Guo Yuhe Ding Jian Liang Zilei Wang Ran He MoMe 117 0 0 01 Dec 2025
A Systematic Study of Model Merging Techniques in Large Language Models Oğuz Kağan Hitit Leander Girrbach Zeynep Akata MoMe 289 0 0 26 Nov 2025
Defending Unauthorized Model Merging via Dual-Stage Weight Protection Wei-Jia Chen Min-Yen Tsai Cheng-Yi Lee Chia-Mu Yu MoMe AAML 397 0 0 14 Nov 2025
T3: Test-Time Model Merging in VLMs for Zero-Shot Medical Imaging Analysis Raza Imam Hu Wang Dwarikanath Mahapatra Mohammad Yaqub MoMe 284 0 0 31 Oct 2025
The Thinking Spectrum: An Empirical Study of Tunable Reasoning in LLMs through Model Merging Xiaochong Lan Yu Zheng Shiteng Cao Yong Li MoMe LRM 203 0 0 26 Sep 2025
Black-box Model Merging for Language-Model-as-a-Service with Massive Model Repositories Shilian Chen Jie Zhou Tianyu Huai Y. Lu Junsong Li ... Y. Yang Xin Li Qin Chen Hang Yan Liang He MoMe 189 0 0 16 Sep 2025
On Task Vectors and Gradients Luca Zhou Daniele Solombrino Donato Crisostomi Maria Sofia Bucarelli Giuseppe Alessio D’Inverno Fabrizio Silvestri Emanuele Rodolà MoMe 389 1 0 22 Aug 2025
Tensorized Clustered LoRA Merging for Multi-Task Interference Zhan Su Fengran Mo G. Liang Jinghan Zhang Bingbing Wen Prayag Tiwari Jian-Yun Nie MoMe 164 0 0 06 Aug 2025
Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Models Linan Yue Yichao Du Yizhi Wang W. Gao Fangzhou Yao ... Ye Liu Ziyu Xu Qi Liu Shimin Di Xiaoshi Zhong LRM 195 15 0 04 Aug 2025
STORM-BORN: A Challenging Mathematical Derivations Dataset Curated via a Human-in-the-Loop Multi-Agent FrameworkAnnual Meeting of the Association for Computational Linguistics (ACL), 2025 Wenhao Liu Zhenyi Lu Xinyu Hu Jierui Zhang Dailin Li ... Pei Zhang Chengbo Zhang Yuxiang Ren Xiaohong Huang Yan Ma OffRL 286 3 0 02 Jun 2025
Navigating the Accuracy-Size Trade-Off with Flexible Model Merging Akash Dhasade Divyansh Jhunjhunwala Milos Vujasinovic Gauri Joshi Anne-Marie Kermarrec MoMe 280 0 0 29 May 2025
Towards Minimizing Feature Drift in Model Merging: Layer-wise Task Vector Fusion for Adaptive Knowledge Integration Wenju Sun Qingyong Li Wen Wang Yang Liu Yangli-ao Geng Boyang Li MoMe 301 2 0 29 May 2025
Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model MergingAnnual Meeting of the Association for Computational Linguistics (ACL), 2025 Haobo Zhang Jiayu Zhou MoMe 256 1 0 28 May 2025
CarboFormer: A Lightweight Semantic Segmentation Architecture for Efficient Carbon Dioxide Detection Using Optical Gas Imaging Taminul Islam Toqi Tahamid Sarker M. Embaby Khaled R Ahmed A. AbuGhazaleh 152 0 0 23 May 2025
Activation-Guided Consensus Merging for Large Language Models Yuxuan Yao Shuqi Liu Zehua Liu Qintong Li Mingyang Liu Xiongwei Han Zhijiang Guo Han Wu Linqi Song MoMe 421 0 0 20 May 2025
CAT Merging: A Training-Free Approach for Resolving Conflicts in Model Merging Wenju Sun Qingyong Li Yangli-ao Geng Boyang Li MoMe 297 6 0 11 May 2025
FedMerge: Federated Personalization via Model Merging Shutong Chen Tianyi Zhou Guodong Long Jing Jiang Chengqi Zhang FedML MoMe 348 1 0 09 Apr 2025
MASS: MoErging through Adaptive Subspace Selection Donato Crisostomi Alessandro Zirilli Antonio Andrea Gargiulo Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Iacopo Masi Emanuele Rodolà MoMe 287 0 0 06 Apr 2025
AdaRank: Adaptive Rank Pruning for Enhanced Model Merging Chanhyuk Lee Jiho Choi Chanryeol Lee Donggyun Kim Seunghoon Hong MoMe 269 5 0 28 Mar 2025
Unlocking Efficient Long-to-Short LLM Reasoning with Model Merging Han Wu Yuxuan Yao Shuqi Liu Zehua Liu Mingwen Liu Xiongwei Han Xianrui Li Hui-Ling Zhen Tao Zhong Mingxuan Yuan MoMe LRM 403 35 0 26 Mar 2025
FW-Merging: Scaling Model Merging with Frank-Wolfe Optimization Hao Mark Chen S. Hu Wayne Luk Timothy M. Hospedales Hongxiang Fan MoMe 416 3 0 16 Mar 2025
From Task-Specific Models to Unified Systems: A Review of Model Merging Approaches Wei Ruan Tianze Yang Yimiao Zhou Tianming Liu Jin Lu MoMe 365 6 0 13 Mar 2025
Whoever Started the Interference Should End It: Guiding Data-Free Model Merging via Task Vectors Runxi Cheng Feng Xiong Yongxian Wei Wanyun Zhu Chun Yuan MoMe 355 12 0 11 Mar 2025
Task Vector Quantization for Memory-Efficient Model Merging Youngeun Kim Seunghwan Lee Aecheon Jung Bogon Ryu Sungeun Hong MQ MoMe 248 3 0 10 Mar 2025
Seeing Delta Parameters as JPEG Images: Data-Free Delta Compression with Discrete Cosine Transform Chenyu Huang Peng Ye Xinyu Wang Shenghe Zheng Biqing Qi Wenlong Zhang Wanli Ouyang Tao Chen 147 2 0 09 Mar 2025
Linear-MoE: Linear Sequence Modeling Meets Mixture-of-Experts Weigao Sun Disen Lan Tong Zhu Xiaoye Qu Yu Cheng MoE 497 6 0 07 Mar 2025
GNNMerge: Merging of GNN Models Without Accessing Training Data Vipul Garg Ishita Thakre Sayan Ranu MoMe 522 0 0 05 Mar 2025
CAMEx: Curvature-aware Merging of ExpertsInternational Conference on Learning Representations (ICLR), 2025 Dung V. Nguyen Minh H. Nguyen Luc Q. Nguyen R. Teo T. Nguyen Linh Duy Tran MoMe 351 6 0 26 Feb 2025
Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment Chenghao Fan Zhenyi Lu Sichen Liu Xiaoye Qu Xiaoye Qu Wei Wei Yu Cheng MoE 1.1K 9 0 24 Feb 2025
Scalable Model Merging with Progressive Layer-wise Distillation Jing Xu Jiazheng Li J.N. Zhang MoMe FedML 609 7 0 18 Feb 2025
1bit-Merging: Dynamic Quantized Merging for Large Language Models Shuqi Liu Yuxuan Yao Bowei He Zehua Liu Xiongwei Han Mingxuan Yuan Han Wu Linqi Song MoMe MQ 412 3 0 15 Feb 2025
LoRE-Merging: Exploring Low-Rank Estimation For Large Language Model Merging Zehua Liu Han Wu Yuxuan Yao Ruifeng She Xiongwei Han Tao Zhong Mingxuan Yuan MoMe 327 5 0 15 Feb 2025
Multi-Task Model Merging via Adaptive Weight Disentanglement Feng Xiong Runxi Cheng Wang Chen Zhanqiu Zhang Yiwen Guo Chun Yuan Ruifeng Xu MoMe 568 11 0 10 Jan 2025
PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2025 Mingyang Song Zhaochen Su Xiaoye Qu Jiawei Zhou Yu Cheng LRM 626 63 0 06 Jan 2025
Task Singular Vectors: Reducing Task Interference in Model MergingComputer Vision and Pattern Recognition (CVPR), 2024 Antonio Andrea Gargiulo Donato Crisostomi Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Emanuele Rodolà MoMe 588 53 0 26 Nov 2024
ATM: Improving Model Merging by Alternating Tuning and Merging Luca Zhou Daniele Solombrino Donato Crisostomi Maria Sofia Bucarelli Fabrizio Silvestri Emanuele Rodolà MoMe 467 6 0 05 Nov 2024
A Unified View of Delta Parameter Editing in Post-Trained Large-Scale Models Qiaoyu Tang Le Yu Bowen Yu Hongyu Lin Keming Lu Yaojie Lu Jia Zheng Le Sun MoMe 222 1 0 17 Oct 2024
Glider: Global and Local Instruction-Driven Expert Router Pingzhi Li Prateek Yadav Jaehong Yoon Jie Peng Yi-Lin Sung Joey Tianyi Zhou Tianlong Chen MoMe MoE 327 3 0 09 Oct 2024
DaWin: Training-free Dynamic Weight Interpolation for Robust AdaptationInternational Conference on Learning Representations (ICLR), 2024 Changdae Oh Yixuan Li Kyungwoo Song Sangdoo Yun Dongyoon Han OOD MoMe 460 15 0 03 Oct 2024
CLIP-MoE: Towards Building Mixture of Experts for CLIP with Diversified Multiplet Upcycling Jihai Zhang Xiaoye Qu Tong Zhu Yu Cheng 581 15 0 28 Sep 2024
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning Prateek Yadav Colin Raffel Mohammed Muqeeth Lucas Caccia Haokun Liu Tianlong Chen Joey Tianyi Zhou Leshem Choshen Alessandro Sordoni MoMe 416 28 0 13 Aug 2024
Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts Tong Zhu Daize Dong Xiaoye Qu Jiacheng Ruan Wenliang Chen Yu Cheng MoE 249 17 0 17 Jun 2024
On Giant's Shoulders: Effortless Weak to Strong by Dynamic Logits Fusion Chenghao Fan Zhenyi Lu Wei Wei Jie Tian Xiaoye Qu Dangyang Chen Yu Cheng MoMe 320 10 0 17 Jun 2024