Arcee's MergeKit: A Toolkit for Merging Large Language Models

20 March 2024

Papers citing "Arcee's MergeKit: A Toolkit for Merging Large Language Models"

50 / 59 papers shown

Title
Parameter-Efficient Checkpoint Merging via Metrics-Weighted Averaging Shi Jie Yu Sehyun Choi MoMe 37 0 0 23 Apr 2025
EasyEdit2: An Easy-to-use Steering Framework for Editing Large Language Models Ziwen Xu Shuxun Wang Kewei Xu Haoming Xu Mengru Wang Xinle Deng Yunzhi Yao Guozhou Zheng H. Chen Ningyu Zhang KELM LLMSV 50 0 0 21 Apr 2025
AdaMMS: Model Merging for Heterogeneous Multimodal Large Language Models with Unsupervised Coefficient Optimization Yiyang Du Xiaochen Wang C. Chen Jiabo Ye Yiru Wang ... J. Zhang Fei Huang Zhifang Sui Maosong Sun Y. Liu MoMe 44 0 0 31 Mar 2025
Model Assembly Learning with Heterogeneous Layer Weight Merging Yi-Kai Zhang Jin Wang Xu-Xiang Zhong De-Chuan Zhan Han-Jia Ye MoMe 37 0 0 27 Mar 2025
Command R7B Arabic: A Small, Enterprise Focused, Multilingual, and Culturally Aware Arabic LLM Yazeed Alnumay Alexandre Barbet Anna Bialas William Darling Shaan Desai ... Stephanie Howe Olivia Lasche Justin Lee Anirudh Shrinivason Jennifer Tracey 79 0 0 18 Mar 2025
PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing Cheng Deng Luoyang Sun Jiwen Jiang Yongcheng Zeng Xinjian Wu ... Haoyang Li Lei Chen Lionel M. Ni H. Zhang Jun Wang 56 0 0 15 Mar 2025
Ensemble Learning for Large Language Models in Text and Code Generation: A Survey Mari Ashiga Wei Jie Fan Wu Vardan K. Voskanyan Fateme Dinmohammadi P. Brookes Jingzhi Gong Zheng Wang 33 0 0 13 Mar 2025
SplatPose: Geometry-Aware 6-DoF Pose Estimation from Single RGB Image via 3D Gaussian Splatting Linqi Yang Xiongwei Zhao Qihao Sun Ke Wang Ao Chen Peng Kang 3DGS 49 0 0 07 Mar 2025
LEWIS (LayEr WIse Sparsity) -- A Training Free Guided Model Merging Approach Hetarth Chopra Vidhi Rambhia Vikram Adve MoMe 55 0 0 05 Mar 2025
Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge Yan-Lun Chen Yi-Ru Wei Chia-Yi Hsu Chia-Mu Yu Chun-ying Huang Ying-Dar Lin Yu-Sung Wu Wei-Bin Lee MoMe KELM 43 0 0 27 Feb 2025
MergePrint: Merge-Resistant Fingerprints for Robust Black-box Ownership Verification of Large Language Models Shojiro Yamabe Tsubasa Takahashi Futa Waseda Koki Wataoka MoMe 73 0 0 21 Feb 2025
Scalable Model Merging with Progressive Layer-wise Distillation Jing Xu Jiazheng Li J. Zhang MoMe FedML 72 0 0 18 Feb 2025
Propagation of Chaos for Mean-Field Langevin Dynamics and its Application to Model Ensemble Atsushi Nitanda Anzelle Lee Damian Tan Xing Kai Mizuki Sakaguchi Taiji Suzuki AI4CE 48 1 0 09 Feb 2025
Mol-MoE: Training Preference-Guided Routers for Molecule Generation Diego Calanzone P. DÓro Pierre-Luc Bacon 45 0 0 08 Feb 2025
Soup to go: mitigating forgetting during continual learning with model averaging Anat Kleiman Gintare Karolina Dziugaite Jonathan Frankle Sham Kakade Mansheej Paul MoMe CLL KELM 44 0 0 09 Jan 2025
A Novel Structure-Agnostic Multi-Objective Approach for Weight-Sharing Compression in Deep Neural Networks Rasa Khosrowshahli Shahryar Rahnamayan Beatrice Ombuki-Berman MQ 21 0 0 06 Jan 2025
Parameter-Efficient Interventions for Enhanced Model Merging Marcin Osial Daniel Marczak Bartosz Zieliñski MoMe 77 0 0 22 Dec 2024
How to Merge Your Multimodal Models Over Time? Sebastian Dziadzio Vishaal Udandarao Karsten Roth Ameya Prabhu Zeynep Akata Samuel Albanie Matthias Bethge MoMe 75 2 0 09 Dec 2024
Optimizing Large Language Models for Turkish: New Methodologies in Corpus Selection and Training Himmet Toprak Kesgin M. K. Yuce Eren Dogan M. E. Uzun Atahan Uz Elif Ince Yusuf Erdem Osama Shbib Ahmed Zeer M. Fatih Amasyali 59 0 0 03 Dec 2024
Neutralizing Backdoors through Information Conflicts for Large Language Models Chen Chen Yuchen Sun Xueluan Gong Jiaxin Gao K. Lam KELM AAML 59 0 0 27 Nov 2024
Fine-Tuning and Evaluating Open-Source Large Language Models for the Army Domain Daniel C. Ruiz John Sell 11 0 0 27 Oct 2024
Can Large Language Models Invent Algorithms to Improve Themselves? Yoichi Ishibashi Taro Yano Masafumi Oyamada AIFin LRM 22 1 0 21 Oct 2024
Exploring Model Kinship for Merging Large Language Models Yedi Hu Yunzhi Yao N. Zhang Shumin Deng H. Chen MoMe 24 0 0 16 Oct 2024
Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence Shangbin Feng Zifeng Wang Yike Wang Sayna Ebrahimi Hamid Palangi ... Nathalie Rauschmayr Yejin Choi Yulia Tsvetkov Chen-Yu Lee Tomas Pfister MoMe 23 3 0 15 Oct 2024
DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models Wenlong Deng Yize Zhao V. Vakilian Minghui Chen Xiaoxiao Li Christos Thrampoulidis 27 3 0 12 Oct 2024
Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation Thomas Gauthier-Caron Shamane Siriwardhana Elliot Stein Malikeh Ehghaghi Charles Goddard Mark McQuade Jacob Solawetz Maxime Labonne MoMe 18 0 0 10 Oct 2024
The Large Language Model GreekLegalRoBERTa Vasileios Saketos D. Pantazi Manolis Koubarakis AILaw 16 0 0 10 Oct 2024
Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild Xinyu Zhao Guoheng Sun Ruisi Cai Yukun Zhou Pingzhi Li ... Binhang Yuan Hongyi Wang Ang Li Zhangyang Wang Tianlong Chen MoMe ALM 15 0 0 07 Oct 2024
DaWin: Training-free Dynamic Weight Interpolation for Robust Adaptation Changdae Oh Yixuan Li Kyungwoo Song Sangdoo Yun Dongyoon Han OOD MoMe 26 4 0 03 Oct 2024
Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks Edan Kinderman Itay Hubara Haggai Maron Daniel Soudry MoMe 35 0 0 02 Oct 2024
HM3: Heterogeneous Multi-Class Model Merging Stefan Hackmann MoMe 20 0 0 27 Sep 2024
Eir: Thai Medical Large Language Models Yutthakorn Thiprak Rungtam Ngodngamthaweesuk Songtam Ngodngamtaweesuk LM&MA ELM 25 0 0 13 Sep 2024
Flexible and Effective Mixing of Large Language Models into a Mixture of Domain Experts Rhui Dih Lee L. Wynter R. Ganti MoE 26 1 0 30 Aug 2024
SQL-GEN: Bridging the Dialect Gap for Text-to-SQL Via Synthetic Data And Model Merging Mohammadreza Pourreza Ruoxi Sun Hailong Li Lesly Miculicich Tomas Pfister Sercan Ö. Arik MoMe 19 5 0 22 Aug 2024
MergeRepair: An Exploratory Study on Merging Task-Specific Adapters in Code LLMs for Automated Program Repair Meghdad Dehghan Jie JW Wu Fatemeh H. Fard Ali Ouni MoMe 27 1 0 18 Aug 2024
A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning Prateek Yadav Colin Raffel Mohammed Muqeeth Lucas Page-Caccia Haokun Liu Tianlong Chen Mohit Bansal Leshem Choshen Alessandro Sordoni MoMe 28 12 0 13 Aug 2024
Extend Model Merging from Fine-Tuned to Pre-Trained Large Language Models via Weight Disentanglement Le Yu Bowen Yu Haiyang Yu Fei Huang Yongbin Li MoMe 19 5 0 06 Aug 2024
Computer Audition: From Task-Specific Machine Learning to Foundation Models Andreas Triantafyllopoulos Iosif Tsangko Alexander Gebhard A. Mesaros Tuomas Virtanen Björn Schuller 26 1 0 22 Jul 2024
Knowledge Mechanisms in Large Language Models: A Survey and Perspective Meng Wang Yunzhi Yao Ziwen Xu Shuofei Qiao Shumin Deng ... Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen Ningyu Zhang 32 1 0 22 Jul 2024
It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization Bingdong Li Zixiang Di Yanting Yang Hong Qian Peng Yang Hao Hao Ke Tang Aimin Zhou MoMe 11 5 0 29 Jun 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 31 13 0 24 Jun 2024
LLM-based speaker diarization correction: A generalizable approach Georgios Efstathiadis Vijay Yadav Anzar Abbas 21 3 0 07 Jun 2024
FusionBench: A Comprehensive Benchmark of Deep Model Fusion A. Tang Li Shen Yong Luo Han Hu Bo Du Dacheng Tao ELM MoMe VLM 26 10 0 05 Jun 2024
Exploring the LLM Journey from Cognition to Expression with Linear Representations Yuzi Yan J. Li Yipin Zhang Dong Yan 28 1 0 27 May 2024
ConStat: Performance-Based Contamination Detection in Large Language Models Jasper Dekoninck Mark Niklas Muller Martin Vechev 27 0 0 25 May 2024
WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models Peng Wang Zexi Li Ningyu Zhang Ziwen Xu Yunzhi Yao Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen KELM CLL 32 14 0 23 May 2024
EMR-Merging: Tuning-Free High-Performance Model Merging Chenyu Huang Peng Ye Tao Chen Tong He Xiangyu Yue Wanli Ouyang MoMe 30 10 0 23 May 2024
Aloe: A Family of Fine-tuned Open Healthcare LLMs Ashwin Kumar Gururajan Enrique Lopez-Cuena Jordi Bayarri-Planas Adrián Tormos Daniel Hinjos ... Lucia Urcelay-Ganzabal Marta Gonzalez-Mallo Sergio Álvarez Napagao Eduard Ayguadé-Parra Ulises Cortés Dario Garcia-Gasulla ELM LM&MA 16 9 0 03 May 2024
Stepwise Alignment for Constrained Language Model Policy Optimization Akifumi Wachi Thien Q. Tran Rei Sato Takumi Tanabe Yohei Akimoto 26 2 0 17 Apr 2024
Have You Merged My Model? On The Robustness of Large Language Model IP Protection Methods Against Model Merging Tianshuo Cong Delong Ran Zesen Liu Xinlei He Jinyuan Liu Yichen Gong Qi Li Anyu Wang Xiaoyun Wang MoMe 25 4 0 08 Apr 2024