Evolutionary Optimization of Model Merging Recipes

28 January 2025

Papers citing "Evolutionary Optimization of Model Merging Recipes"

50 / 66 papers shown

Title
PatientDx: Merging Large Language Models for Protecting Data-Privacy in Healthcare José G. Moreno Jesus Lovon M'Rick Robin-Charlet Christine Damase-Michel L. Tamine MoMe LM&MA 48 0 0 24 Apr 2025
Metropolis-Hastings Captioning Game: Knowledge Fusion of Vision Language Models via Decentralized Bayesian Inference Yuta Matsui Ryosuke Yamaki Ryo Ueda Seitaro Shinagawa Tadahiro Taniguchi MLLM 28 1 0 13 Apr 2025
FuseRL: Dense Preference Optimization for Heterogeneous Model Fusion Longguang Zhong Fanqi Wan Ziyi Yang Guosheng Liang Tianyuan Shi Xiaojun Quan MoMe 53 0 0 09 Apr 2025
MASS: MoErging through Adaptive Subspace Selection Donato Crisostomi Alessandro Zirilli Antonio Andrea Gargiulo Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Iacopo Masi Emanuele Rodolà MoMe 40 0 0 06 Apr 2025
Challenges of Interaction in Optimizing Mixed Categorical-Continuous Variables Youhei Akimoto Xilin Gao Ze Kai Ng Daiki Morinaga 55 0 0 01 Apr 2025
Reinforced Model Merging J. N. Han Jingwen Ye Shunyu Liu Haofei Zhang Jie Song Zunlei Feng Mingli Song MoMe 49 0 0 27 Mar 2025
Ensemble Learning for Large Language Models in Text and Code Generation: A Survey Mari Ashiga Wei Jie Fan Wu Vardan K. Voskanyan Fateme Dinmohammadi P. Brookes Jingzhi Gong Zheng Wang 38 0 0 13 Mar 2025
RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs Zhongzhan Huang Guoming Ling Vincent S. Liang Yupei Lin Yandong Chen Shanshan Zhong Hefeng Wu Liang Lin LRM 50 1 0 08 Mar 2025
The Society of HiveMind: Multi-Agent Optimization of Foundation Model Swarms to Unlock the Potential of Collective Intelligence Noah Mamie Susie Xi Rao LLMAG AI4CE 51 0 0 07 Mar 2025
CABS: Conflict-Aware and Balanced Sparsification for Enhancing Model Merging Zongzhen Yang Binhang Qi Hailong Sun Wenrui Long Ruobing Zhao Xiang Gao MoMe 45 0 0 26 Feb 2025
LED-Merging: Mitigating Safety-Utility Conflicts in Model Merging with Location-Election-Disjoint Qianli Ma Dongrui Liu Qian Chen Linfeng Zhang Jing Shao MoMe 47 0 0 24 Feb 2025
Mixup Model Merge: Enhancing Model Merging Performance through Randomized Linear Interpolation Yue Zhou Yi-Ju Chang Yuan Wu MoMe 57 2 0 24 Feb 2025
Optimal Brain Iterative Merging: Mitigating Interference in LLM Merging Zhixiang Wang Zhenyu Mao Yixuan Qiao Yunfang Wu Biye Li MoMe 73 0 0 17 Feb 2025
Parameter-Efficient Interventions for Enhanced Model Merging Marcin Osial Daniel Marczak Bartosz Zieliñski MoMe 82 1 0 22 Dec 2024
SafetyDPO: Scalable Safety Alignment for Text-to-Image Generation Runtao Liu Chen I Chieh Jindong Gu Jipeng Zhang Renjie Pi Qifeng Chen Philip H. S. Torr Ashkan Khakzar Fabio Pizzati EGVM 94 0 0 13 Dec 2024
How to Merge Your Multimodal Models Over Time? Sebastian Dziadzio Vishaal Udandarao Karsten Roth Ameya Prabhu Zeynep Akata Samuel Albanie Matthias Bethge MoMe 82 2 0 09 Dec 2024
Constructing Multimodal Datasets from Scratch for Rapid Development of a Japanese Visual Language Model Keito Sasagawa Koki Maeda Issa Sugiura Shuhei Kurita Naoaki Okazaki Daisuke Kawahara VLM 22 0 0 30 Oct 2024
Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging Li Shen A. Tang Enneng Yang G. Guo Yong Luo Lefei Zhang Xiaochun Cao Bo Du Dacheng Tao MoMe 21 1 0 29 Oct 2024
JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation Shota Onohara Atsuyuki Miyai Yuki Imajuku Kazuki Egashira Jeonghun Baek Xiang Yue Graham Neubig Kiyoharu Aizawa OSLM 64 1 0 22 Oct 2024
SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgery Enneng Yang Li Shen Zhenyi Wang G. Guo Xingwei Wang Xiaocun Cao Jie Zhang Dacheng Tao MoMe 18 4 0 18 Oct 2024
Unconstrained Model Merging for Enhanced LLM Reasoning Yiming Zhang Baoyi He Shengyu Zhang Yuhao Fu Qi Zhou ... Guanghan Ning Linyi Li Chunlin Ji Fei Wu Hongxia Yang MoMe 27 0 0 17 Oct 2024
An Evolved Universal Transformer Memory Edoardo Cetin Qi Sun Tianyu Zhao Yujin Tang 30 0 0 17 Oct 2024
LoRA Soups: Merging LoRAs for Practical Skill Composition Tasks Akshara Prabhakar Yuanzhi Li Karthik Narasimhan Sham Kakade Eran Malach Samy Jelassi MoMe 15 1 0 16 Oct 2024
Exploring Model Kinship for Merging Large Language Models Yedi Hu Yunzhi Yao N. Zhang Shumin Deng H. Chen MoMe 29 1 0 16 Oct 2024
Agent Skill Acquisition for Large Language Models via CycleQD So Kuroki Taishi Nakamura Takuya Akiba Yujin Tang MoMe 24 0 0 16 Oct 2024
Model Swarms: Collaborative Search to Adapt LLM Experts via Swarm Intelligence Shangbin Feng Zifeng Wang Yike Wang Sayna Ebrahimi Hamid Palangi ... Nathalie Rauschmayr Yejin Choi Yulia Tsvetkov Chen-Yu Lee Tomas Pfister MoMe 25 3 0 15 Oct 2024
DARE the Extreme: Revisiting Delta-Parameter Pruning For Fine-Tuned Models Wenlong Deng Yize Zhao V. Vakilian Minghui Chen Xiaoxiao Li Christos Thrampoulidis 30 3 0 12 Oct 2024
Merging in a Bottle: Differentiable Adaptive Merging (DAM) and the Path from Averaging to Automation Thomas Gauthier-Caron Shamane Siriwardhana Elliot Stein Malikeh Ehghaghi Charles Goddard Mark McQuade Jacob Solawetz Maxime Labonne MoMe 23 0 0 10 Oct 2024
How Does Vision-Language Adaptation Impact the Safety of Vision Language Models? Seongyun Lee Geewook Kim Jiyeon Kim Hyunji Lee Hoyeon Chang Sue Hyun Park Minjoon Seo 31 0 0 10 Oct 2024
Model-GLUE: Democratized LLM Scaling for A Large Model Zoo in the Wild Xinyu Zhao Guoheng Sun Ruisi Cai Yukun Zhou Pingzhi Li ... Binhang Yuan Hongyi Wang Ang Li Zhangyang Wang Tianlong Chen MoMe ALM 21 0 0 07 Oct 2024
What Matters for Model Merging at Scale? Prateek Yadav Tu Vu Jonathan Lai Alexandra Chronopoulou Manaal Faruqui Mohit Bansal Tsendsuren Munkhdalai MoMe 39 12 0 04 Oct 2024
Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection T. Chen Zhentao Tan Tao Gong Yue Wu Qi Chu Bin Liu Jieping Ye Nenghai Yu KELM 38 2 0 03 Oct 2024
Determine-Then-Ensemble: Necessity of Top-k Union for Large Language Model Ensembling Yuxuan Yao Han Wu Mingyang Liu Sichun Luo Xiongwei Han Jie Liu Zhijiang Guo Linqi Song 34 4 0 03 Oct 2024
Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks Edan Kinderman Itay Hubara Haggai Maron Daniel Soudry MoMe 40 0 0 02 Oct 2024
The Construction of Instruction-tuned LLMs for Finance without Instruction Data Using Continual Pretraining and Model Merging Masanori Hirano Kentaro Imajo MoMe 21 0 0 30 Sep 2024
HM3: Hierarchical Multi-Objective Model Merging for Pretrained Models Yu Zhou Xingyu Wu Jibin Wu Liang Feng Kay Chen Tan MoMe 54 0 0 27 Sep 2024
Realistic Evaluation of Model Merging for Compositional Generalization Derek Tam Yash Kant Brian Lester Igor Gilitschenski Colin Raffel MoMe 16 4 0 26 Sep 2024
Development and bilingual evaluation of Japanese medical large language model within reasonably low computational resources Issey Sukeda ELM 21 1 0 18 Sep 2024
DataSculpt: Crafting Data Landscapes for Long-Context LLMs through Multi-Objective Partitioning Keer Lu Xiaonan Nie Zheng Liang Da Pan Shusen Zhang ... Weipeng Chen Zenan Zhou Guosheng Dong Bin Cui Wentao Zhang 24 0 0 02 Sep 2024
BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts Qizhen Zhang Nikolas Gritsch Dwaraknath Gnaneshwar Simon Guo David Cairuz ... Jakob N. Foerster Phil Blunsom Sebastian Ruder A. Ustun Acyr F. Locatelli MoMe MoE 34 3 0 15 Aug 2024
FuseChat: Knowledge Fusion of Chat Models Fanqi Wan Longguang Zhong Ziyi Yang Ruijun Chen Xiaojun Quan ALM KELM MoMe 23 23 0 15 Aug 2024
MangaUB: A Manga Understanding Benchmark for Large Multimodal Models Hikaru Ikuta Leslie Wöhler Kiyoharu Aizawa 18 2 0 26 Jul 2024
Knowledge Mechanisms in Large Language Models: A Survey and Perspective Meng Wang Yunzhi Yao Ziwen Xu Shuofei Qiao Shumin Deng ... Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen Ningyu Zhang 39 1 0 22 Jul 2024
Transformer Layers as Painters Qi Sun Marc Pickett Aakash Kumar Nain Llion Jones AI4CE 23 13 0 12 Jul 2024
SoupLM: Model Integration in Large Language and Multi-Modal Models Yue Bai Zichen Zhang Jiasen Lu Yun Fu MoMe 16 1 0 11 Jul 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 16 17 0 08 Jul 2024
LayerShuffle: Enhancing Robustness in Vision Transformers by Randomizing Layer Execution Order Matthias Anton Freiberger Peter Kun A. Løvlie Sebastian Risi 28 0 0 05 Jul 2024
Unlocking the Potential of Model Merging for Low-Resource Languages Mingxu Tao Chen Zhang Quzhe Huang Tianyao Ma Songfang Huang Dongyan Zhao Yansong Feng CLL MoMe 20 3 0 04 Jul 2024
PLeaS -- Merging Models with Permutations and Least Squares Anshul Nasery J. Hayase Pang Wei Koh Sewoong Oh MoMe 23 3 0 02 Jul 2024
DogeRM: Equipping Reward Models with Domain Knowledge through Model Merging Tzu-Han Lin Chen An Li Hung-yi Lee Yun-Nung Chen VLM ALM 26 1 0 01 Jul 2024