Model Merging by Uncertainty-Based Gradient Matching

19 October 2023

Mohammad Emtiyaz Khan

MoMe

FedML

ArXiv PDF HTML

Papers citing "Model Merging by Uncertainty-Based Gradient Matching"

38 / 38 papers shown

Title
Uncertainty Quantification for Machine Learning in Healthcare: A Survey L. J. L. Lopez Shaza Elsharief Dhiyaa Al Jorf Firas Darwish Congbo Ma Farah E. Shamout 23 0 0 04 May 2025
Dynamic Fisher-weighted Model Merging via Bayesian Optimization Sanwoo Lee Jiahao Liu Qifan Wang J. Wang Xunliang Cai Yunfang Wu MoMe 52 0 0 26 Apr 2025
Simplified Swarm Learning Framework for Robust and Scalable Diagnostic Services in Cancer Histopathology Yanjie Wu Yuhao Ji Saiho Lee Juniad Akram Ali Braytee Ali Anaissi 27 0 0 23 Apr 2025
Mitigating Parameter Interference in Model Merging via Sharpness-Aware Fine-Tuning Yeoreum Lee Jinwook Jung Sungyong Baik MoMe 40 0 0 20 Apr 2025
Leveraging Submodule Linearity Enhances Task Arithmetic Performance in LLMs Rui Dai Sile Hu Xu Shen Yonggang Zhang Xinmei Tian Jieping Ye MoMe 42 2 0 15 Apr 2025
MASS: MoErging through Adaptive Subspace Selection Donato Crisostomi Alessandro Zirilli Antonio Andrea Gargiulo Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Iacopo Masi Emanuele Rodolà MoMe 40 0 0 06 Apr 2025
RouterEval: A Comprehensive Benchmark for Routing LLMs to Explore Model-level Scaling Up in LLMs Zhongzhan Huang Guoming Ling Vincent S. Liang Yupei Lin Yandong Chen Shanshan Zhong Hefeng Wu Liang Lin LRM 52 1 0 08 Mar 2025
SplatPose: Geometry-Aware 6-DoF Pose Estimation from Single RGB Image via 3D Gaussian Splatting Linqi Yang Xiongwei Zhao Qihao Sun Ke Wang Ao Chen Peng Kang 3DGS 65 2 0 07 Mar 2025
GNNMerge: Merging of GNN Models Without Accessing Training Data Vipul Garg Ishita Thakre Sayan Ranu MoMe 72 0 0 05 Mar 2025
Portable Reward Tuning: Towards Reusable Fine-Tuning across Different Pretrained Models Daiki Chijiwa Taku Hasegawa Kyosuke Nishida Kuniko Saito Susumu Takeuchi 36 0 0 18 Feb 2025
Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion Binchi Zhang Zaiyi Zheng Zhengzhang Chen Jundong Li 52 0 0 01 Feb 2025
Evolutionary Optimization of Model Merging Recipes Takuya Akiba Makoto Shing Yujin Tang Qi Sun David Ha MoMe 98 96 0 28 Jan 2025
Federated Learning with Uncertainty and Personalization via Efficient Second-order Optimization Shivam Pal Aishwarya Gupta Saqib Sarwar Piyush Rai FedML 65 0 0 27 Nov 2024
Task Singular Vectors: Reducing Task Interference in Model Merging Antonio Andrea Gargiulo Donato Crisostomi Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Emanuele Rodolà MoMe 85 8 0 26 Nov 2024
Variational Low-Rank Adaptation Using IVON Bai Cong Nico Daheim Yuesong Shen Daniel Cremers Rio Yokota Mohammad Emtiyaz Khan Thomas Möllenhoff 26 2 0 07 Nov 2024
ATM: Improving Model Merging by Alternating Tuning and Merging Luca Zhou Daniele Solombrino Donato Crisostomi Maria Sofia Bucarelli Fabrizio Silvestri Emanuele Rodolà MoMe 34 4 0 05 Nov 2024
Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging Li Shen A. Tang Enneng Yang G. Guo Yong Luo Lefei Zhang Xiaochun Cao Bo Du Dacheng Tao MoMe 27 5 0 29 Oct 2024
Model merging with SVD to tie the Knots George Stoica Pratik Ramesh B. Ecsedi Leshem Choshen Judy Hoffman MoMe 18 8 0 25 Oct 2024
SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgery Enneng Yang Li Shen Zhenyi Wang G. Guo Xingwei Wang Xiaocun Cao Jie Zhang Dacheng Tao MoMe 24 4 0 18 Oct 2024
What Matters for Model Merging at Scale? Prateek Yadav Tu Vu Jonathan Lai Alexandra Chronopoulou Manaal Faruqui Mohit Bansal Tsendsuren Munkhdalai MoMe 44 12 0 04 Oct 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 27 17 0 08 Jul 2024
It's Morphing Time: Unleashing the Potential of Multiple LLMs via Multi-objective Optimization Bingdong Li Zixiang Di Yanting Yang Hong Qian Peng Yang Hao Hao Ke Tang Aimin Zhou MoMe 19 5 0 29 Jun 2024
Lottery Ticket Adaptation: Mitigating Destructive Interference in LLMs Ashwinee Panda Berivan Isik Xiangyu Qi Sanmi Koyejo Tsachy Weissman Prateek Mittal MoMe 42 12 0 24 Jun 2024
DEM: Distribution Edited Model for Training with Mixed Data Distributions Dhananjay Ram Aditya Rawal Momchil Hardalov Nikolaos Pappas Sheng Zha MoMe 22 1 0 21 Jun 2024
MAP: Low-compute Model Merging with Amortized Pareto Fronts via Quadratic Approximation Lu Li T. Zhang Zhiqi Bu Suyuchen Wang Huan He Jie Fu Yonghui Wu Jiang Bian Yong Chen Yoshua Bengio FedML MoMe 92 3 0 11 Jun 2024
Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reduction Tingchen Fu Deng Cai Lemao Liu Shuming Shi Rui Yan MoMe 42 13 0 22 May 2024
Towards Modular LLMs by Building and Reusing a Library of LoRAs O. Ostapenko Zhan Su E. Ponti Laurent Charlin Nicolas Le Roux Matheus Pereira Lucas Page-Caccia Alessandro Sordoni MoMe 27 30 0 18 May 2024
FissionFusion: Fast Geometric Generation and Hierarchical Souping for Medical Image Analysis Santosh Sanjeev Nuren Zhaksylyk Ibrahim Almakky Anees Ur Rehman Hashmi Mohammad Areeb Qazi Mohammad Yaqub 22 3 0 20 Mar 2024
Mastering Text, Code and Math Simultaneously via Fusing Highly Specialized Language Models Ning Ding Yulin Chen Ganqu Cui Xingtai Lv Weilin Zhao Ruobing Xie Bowen Zhou Zhiyuan Liu Maosong Sun ALM MoMe AI4CE 33 7 0 13 Mar 2024
Variational Learning is Effective for Large Deep Networks Yuesong Shen Nico Daheim Bai Cong Peter Nickl Gian Maria Marconi ... Rio Yokota Iryna Gurevych Daniel Cremers Mohammad Emtiyaz Khan Thomas Möllenhoff 24 21 0 27 Feb 2024
Fine-Grained Detoxification via Instance-Level Prefixes for Large Language Models Xin Yi Linlin Wang Xiaoling Wang Liang He MoMe 32 1 0 23 Feb 2024
Bayesian Uncertainty for Gradient Aggregation in Multi-Task Learning Idan Achituve I. Diamant Arnon Netzer Gal Chechik Ethan Fetaya UQCV 19 3 0 06 Feb 2024
Merging by Matching Models in Task Parameter Subspaces Derek Tam Mohit Bansal Colin Raffel MoMe 13 10 0 07 Dec 2023
Language and Task Arithmetic with Parameter-Efficient Layers for Zero-Shot Summarization Alexandra Chronopoulou Jonas Pfeiffer Joshua Maynez Xinyi Wang Sebastian Ruder Priyanka Agrawal MoMe 19 14 0 15 Nov 2023
BYOM: Building Your Own Multi-Task Model For Free Weisen Jiang Baijiong Lin Han Shi Yu Zhang Zhenguo Li James T. Kwok MoMe 22 5 0 03 Oct 2023
Elastic Weight Removal for Faithful and Abstractive Dialogue Generation Nico Daheim Nouha Dziri Mrinmaya Sachan Iryna Gurevych E. Ponti MoMe 21 30 0 30 Mar 2023
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 239 313 0 11 Sep 2022
Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam Mohammad Emtiyaz Khan Didrik Nielsen Voot Tangkaratt Wu Lin Y. Gal Akash Srivastava ODL 74 264 0 13 Jun 2018