Model Fusion via Optimal Transport

12 October 2019

Papers citing "Model Fusion via Optimal Transport"

49 / 49 papers shown

Title
QoS-Efficient Serving of Multiple Mixture-of-Expert LLMs Using Partial Runtime Reconfiguration HamidReza Imani Jiaxin Peng Peiman Mohseni Abdolah Amirany Tarek A. El-Ghazawi MoE 28 0 0 10 May 2025
Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry Mohammed Adnan Rohan Jain Ekansh Sharma Rahul Krishnan Yani Andrew Ioannou 56 0 0 08 May 2025
Task-conditioned Ensemble of Expert Models for Continuous Learning Renu Sharma Debasmita Pal Arun Ross OOD KELM 145 0 0 11 Apr 2025
FedBEns: One-Shot Federated Learning based on Bayesian Ensemble Jacopo Talpini Marco Savi Giovanni Neglia FedML Presented at ResearchTrend Connect \| FedML on 07 May 2025 76 0 0 19 Mar 2025
Geometric Knowledge-Guided Localized Global Distribution Alignment for Federated Learning Yanbiao Ma Wei-Ming Dai Wenke Huang Jiayi Chen 94 0 0 09 Mar 2025
Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translation Qiuming Zhao Guangzhi Sun Chao Zhang Mingxing Xu Thomas Fang Zheng MoMe VLM 154 0 0 24 Feb 2025
SuperMerge: An Approach For Gradient-Based Model Merging Haoyu Yang Zheng Zhang Saket Sathe MoMe 125 0 0 17 Feb 2025
Forget the Data and Fine-Tuning! Just Fold the Network to Compress Dong Wang Haris Šikić Lothar Thiele O. Saukh 53 0 0 17 Feb 2025
Training-free Heterogeneous Model Merging Zhengqi Xu Han Zheng Jie Song Li Sun Mingli Song MoMe 70 1 0 03 Jan 2025
ATM: Improving Model Merging by Alternating Tuning and Merging Luca Zhou Daniele Solombrino Donato Crisostomi Maria Sofia Bucarelli Fabrizio Silvestri Emanuele Rodolà MoMe 44 4 0 05 Nov 2024
A Unified Solution to Diverse Heterogeneities in One-shot Federated Learning Jun Bai Yiliao Song Di Wu Atul Sajjanhar Yong Xiang Wei Zhou Xiaohui Tao Yan Li Y. Li FedML 53 0 0 28 Oct 2024
Wolf2Pack: The AutoFusion Framework for Dynamic Parameter Fusion Bowen Tian Songning Lai Yutao Yue MoMe 30 0 0 08 Oct 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 38 18 0 08 Jul 2024
Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis Stefan Horoi Albert Manuel Orozco Camacho Eugene Belilovsky Guy Wolf FedML MoMe 29 9 0 07 Jul 2024
PLeaS -- Merging Models with Permutations and Least Squares Anshul Nasery J. Hayase Pang Wei Koh Sewoong Oh MoMe 45 3 0 02 Jul 2024
Stable Diffusion-based Data Augmentation for Federated Learning with Non-IID Data Mahdi Morafah M. Reisser Bill Lin Christos Louizos FedML 34 5 0 13 May 2024
Arcee's MergeKit: A Toolkit for Merging Large Language Models Charles Goddard Shamane Siriwardhana Malikeh Ehghaghi Luke Meyers Vladimir Karpukhin Brian Benedict Mark McQuade Jacob Solawetz MoMe KELM 84 77 0 20 Mar 2024
Merging Text Transformer Models from Different Initializations Neha Verma Maha Elbayad MoMe 56 7 0 01 Mar 2024
Towards Meta-Pruning via Optimal Transport Alexander Theus Olin Geimer Friedrich Wicke Thomas Hofmann Sotiris Anagnostidis Sidak Pal Singh MoMe 16 3 0 12 Feb 2024
Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods Akira Ito Masanori Yamada Atsutoshi Kumagai MoMe 61 5 0 06 Feb 2024
Transformer Fusion with Optimal Transport Moritz Imfeld Jacopo Graldi Marco Giordano Thomas Hofmann Sotiris Anagnostidis Sidak Pal Singh ViT MoMe 22 16 0 09 Oct 2023
Layer-wise Linear Mode Connectivity Linara Adilova Maksym Andriushchenko Michael Kamp Asja Fischer Martin Jaggi FedML FAtt MoMe 30 15 0 13 Jul 2023
Task Arithmetic in the Tangent Space: Improved Editing of Pre-Trained Models Guillermo Ortiz-Jiménez Alessandro Favero P. Frossard MoMe 42 106 0 22 May 2023
MGR: Multi-generator Based Rationalization Wei Liu Haozhao Wang Jun Wang Rui Li Xinyang Li Yuankai Zhang Yang Qiu 21 7 0 08 May 2023
Generalization Matters: Loss Minima Flattening via Parameter Hybridization for Efficient Online Knowledge Distillation Tianli Zhang Mengqi Xue Jiangtao Zhang Haofei Zhang Yu Wang Lechao Cheng Jie Song Mingli Song 28 5 0 26 Mar 2023
Architecture, Dataset and Model-Scale Agnostic Data-free Meta-Learning Zixuan Hu Li Shen Zhenyi Wang Tongliang Liu Chun Yuan Dacheng Tao 47 4 0 20 Mar 2023
Equivariant Architectures for Learning in Deep Weight Spaces Aviv Navon Aviv Shamsian Idan Achituve Ethan Fetaya Gal Chechik Haggai Maron 36 63 0 30 Jan 2023
Projected Subnetworks Scale Adaptation Siddhartha Datta N. Shadbolt VLM CLL 28 0 0 27 Jan 2023
Dataless Knowledge Fusion by Merging Weights of Language Models Xisen Jin Xiang Ren Daniel Preotiuc-Pietro Pengxiang Cheng FedML MoMe 21 213 0 19 Dec 2022
REPAIR: REnormalizing Permuted Activations for Interpolation Repair Keller Jordan Hanie Sedghi O. Saukh R. Entezari Behnam Neyshabur MoMe 46 94 0 15 Nov 2022
Bayesian Federated Neural Matching that Completes Full Information Peng Xiao Samuel Cheng FedML 24 2 0 15 Nov 2022
Federated Adaptive Prompt Tuning for Multi-Domain Collaborative Learning Shangchao Su Min Yang Bin Li Xiangyang Xue VLM FedML 30 18 0 15 Nov 2022
Local Model Reconstruction Attacks in Federated Learning and their Uses Ilias Driouich Chuan Xu Giovanni Neglia F. Giroire Eoin Thomas AAML FedML 29 2 0 28 Oct 2022
Federated Fuzzy Neural Network with Evolutionary Rule Learning Leijie Zhang Ye-ling Shi Yu-Cheng Chang Chin-Teng Lin FedML 21 15 0 26 Oct 2022
Deep Model Reassembly Xingyi Yang Zhou Daquan Songhua Liu Jingwen Ye Xinchao Wang MoMe 20 120 0 24 Oct 2022
Wasserstein Barycenter-based Model Fusion and Linear Mode Connectivity of Neural Networks A. K. Akash Sixu Li Nicolas García Trillos 26 12 0 13 Oct 2022
Deep Double Descent via Smooth Interpolation Matteo Gamba Erik Englesson Marten Bjorkman Hossein Azizpour 59 10 0 21 Sep 2022
Random initialisations performing above chance and how to find them Frederik Benzing Simon Schug Robert Meier J. Oswald Yassir Akram Nicolas Zucchet Laurence Aitchison Angelika Steger ODL 32 24 0 15 Sep 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 252 313 0 11 Sep 2022
An Optimal Transport Approach to Personalized Federated Learning Farzan Farnia Amirhossein Reisizadeh Ramtin Pedarsani Ali Jadbabaie OT OOD FedML 29 12 0 06 Jun 2022
Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning Lin Zhang Li Shen Liang Ding Dacheng Tao Ling-Yu Duan FedML 28 252 0 17 Mar 2022
Architecture Agnostic Federated Learning for Neural Networks Disha Makhija Xing Han Nhat Ho Joydeep Ghosh FedML 21 40 0 15 Feb 2022
Fed2: Feature-Aligned Federated Learning Fuxun Yu Weishan Zhang Zhuwei Qin Zirui Xu Di Wang Chenchen Liu Zhi Tian Xiang Chen FedML 28 74 0 28 Nov 2021
Towards Model Agnostic Federated Learning Using Knowledge Distillation A. Afonin Sai Praneeth Karimireddy FedML 30 44 0 28 Oct 2021
The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks R. Entezari Hanie Sedghi O. Saukh Behnam Neyshabur MoMe 37 215 0 12 Oct 2021
A Field Guide to Federated Optimization Jianyu Wang Zachary B. Charles Zheng Xu Gauri Joshi H. B. McMahan ... Mi Zhang Tong Zhang Chunxiang Zheng Chen Zhu Wennan Zhu FedML 184 411 0 14 Jul 2021
FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout Samuel Horváth Stefanos Laskaridis Mario Almeida Ilias Leondiadis Stylianos I. Venieris Nicholas D. Lane 178 267 0 26 Feb 2021
Emerging Trends in Federated Learning: From Model Fusion to Federated X Learning Shaoxiong Ji Yue Tan Teemu Saravirta Zhiqin Yang Yixin Liu Lauri Vasankari Shirui Pan Guodong Long A. Walid FedML 37 76 0 25 Feb 2021
Federated Learning with Matched Averaging Hongyi Wang Mikhail Yurochkin Yuekai Sun Dimitris Papailiopoulos Y. Khazaeni FedML 46 1,097 0 15 Feb 2020