REPAIR: REnormalizing Permuted Activations for Interpolation Repair

15 November 2022

Papers citing "REPAIR: REnormalizing Permuted Activations for Interpolation Repair"

50 / 78 papers shown

Title
Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry Mohammed Adnan Rohan Jain Ekansh Sharma Rahul Krishnan Yani Andrew Ioannou 49 0 0 08 May 2025
MASS: MoErging through Adaptive Subspace Selection Donato Crisostomi Alessandro Zirilli Antonio Andrea Gargiulo Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Iacopo Masi Emanuele Rodolà MoMe 40 0 0 06 Apr 2025
BECAME: BayEsian Continual Learning with Adaptive Model MErging Mei Li Yuxiang Lu Qinyan Dai Suizhi Huang Yue Ding Hongtao Lu CLL MoMe 44 0 0 03 Apr 2025
FedBEns: One-Shot Federated Learning based on Bayesian Ensemble Jacopo Talpini Marco Savi Giovanni Neglia FedML Presented at ResearchTrend Connect \| FedML on 07 May 2025 73 0 0 19 Mar 2025
Analyzing the Role of Permutation Invariance in Linear Mode Connectivity Keyao Zhan Puheng Li Lei Wu MoMe 77 0 0 13 Mar 2025
In-Model Merging for Enhancing the Robustness of Medical Imaging Classification Models Hu Wang Ibrahim Almakky Congbo Ma Numan Saeed Mohammad Yaqub MoMe 66 0 0 27 Feb 2025
MedForge: Building Medical Foundation Models Like Open Source Software Development Zheling Tan Kexin Ding Jin Gao Mu Zhou Dimitris N. Metaxas Shaoting Zhang Dequan Wang AI4CE 45 1 0 22 Feb 2025
Sens-Merging: Sensitivity-Guided Parameter Balancing for Merging Large Language Models Shuqi Liu Han Wu Bowei He Xiongwei Han M. Yuan Linqi Song MoMe 47 1 0 20 Feb 2025
Linear Mode Connectivity in Differentiable Tree Ensembles Ryuichi Kanoh M. Sugiyama 62 1 0 17 Feb 2025
Forget the Data and Fine-Tuning! Just Fold the Network to Compress Dong Wang Haris Šikić Lothar Thiele O. Saukh 44 0 0 17 Feb 2025
Multi-Task Model Merging via Adaptive Weight Disentanglement Feng Xiong Runxi Cheng Wang Chen Zhanqiu Zhang Yiwen Guo Chun Yuan Ruifeng Xu MoMe 92 4 0 10 Jan 2025
Training-free Heterogeneous Model Merging Zhengqi Xu Han Zheng Jie Song Li Sun Mingli Song MoMe 68 1 0 03 Jan 2025
Non-Uniform Parameter-Wise Model Merging Albert Manuel Orozco Camacho Stefan Horoi Guy Wolf Eugene Belilovsky MoMe FedML 73 0 0 20 Dec 2024
Task Singular Vectors: Reducing Task Interference in Model Merging Antonio Andrea Gargiulo Donato Crisostomi Maria Sofia Bucarelli Simone Scardapane Fabrizio Silvestri Emanuele Rodolà MoMe 87 8 0 26 Nov 2024
ATM: Improving Model Merging by Alternating Tuning and Merging Luca Zhou Daniele Solombrino Donato Crisostomi Maria Sofia Bucarelli Fabrizio Silvestri Emanuele Rodolà MoMe 40 4 0 05 Nov 2024
Model merging with SVD to tie the Knots George Stoica Pratik Ramesh B. Ecsedi Leshem Choshen Judy Hoffman MoMe 24 8 0 25 Oct 2024
SurgeryV2: Bridging the Gap Between Model Merging and Multi-Task Learning with Deep Representation Surgery Enneng Yang Li Shen Zhenyi Wang G. Guo Xingwei Wang Xiaocun Cao Jie Zhang Dacheng Tao MoMe 29 4 0 18 Oct 2024
The Non-Local Model Merging Problem: Permutation Symmetries and Variance Collapse Ekansh Sharma Daniel M. Roy Gintare Karolina Dziugaite MoMe 20 2 0 16 Oct 2024
What Matters for Model Merging at Scale? Prateek Yadav Tu Vu Jonathan Lai Alexandra Chronopoulou Manaal Faruqui Mohit Bansal Tsendsuren Munkhdalai MoMe 44 12 0 04 Oct 2024
Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks Edan Kinderman Itay Hubara Haggai Maron Daniel Soudry MoMe 45 0 0 02 Oct 2024
Realistic Evaluation of Model Merging for Compositional Generalization Derek Tam Yash Kant Brian Lester Igor Gilitschenski Colin Raffel MoMe 16 5 0 26 Sep 2024
Weight Scope Alignment: A Frustratingly Easy Method for Model Merging Yichu Xu Xin-Chun Li Le Gan De-Chuan Zhan MoMe 27 0 0 22 Aug 2024
Approaching Deep Learning through the Spectral Dynamics of Weights David Yunis Kumar Kshitij Patel Samuel Wheeler Pedro H. P. Savarese Gal Vardi Karen Livescu Michael Maire Matthew R. Walter 37 3 0 21 Aug 2024
Training-Free Model Merging for Multi-target Domain Adaptation Wenyi Li Huan-ang Gao Mingju Gao Beiwen Tian Rong Zhi Hao Zhao MoMe 43 5 0 18 Jul 2024
Merge, Ensemble, and Cooperate! A Survey on Collaborative Strategies in the Era of Large Language Models Jinliang Lu Ziliang Pang Min Xiao Yaochen Zhu Rui Xia Jiajun Zhang MoMe 29 17 0 08 Jul 2024
Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis Stefan Horoi Albert Manuel Orozco Camacho Eugene Belilovsky Guy Wolf FedML MoMe 19 9 0 07 Jul 2024
PLeaS -- Merging Models with Permutations and Least Squares Anshul Nasery J. Hayase Pang Wei Koh Sewoong Oh MoMe 38 3 0 02 Jul 2024
Neural Networks Trained by Weight Permutation are Universal Approximators Yongqiang Cai Gaohang Chen Zhonghua Qiao 59 1 0 01 Jul 2024
The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof Derek Lim Moe Putterman Robin Walters Haggai Maron Stefanie Jegelka 35 5 0 30 May 2024
$C^2M^3$ : Cycle-Consistent Multi-Model Merging Donato Crisostomi Marco Fumero Daniele Baieri F. Bernard Emanuele Rodolà MoMe 19 7 0 28 May 2024
The Platonic Representation Hypothesis Minyoung Huh Brian Cheung Tongzhou Wang Phillip Isola 72 107 0 13 May 2024
MaxFusion: Plug&Play Multi-Modal Generation in Text-to-Image Diffusion Models Nithin Gopalakrishnan Nair Jeya Maria Jose Valanarasu Vishal M. Patel MoMe 33 7 0 15 Apr 2024
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models Nastaran Saadati Minh Pham Nasla Saleem Joshua R. Waite Aditya Balu Zhanhong Jiang Chinmay Hegde Soumik Sarkar MoMe 35 1 0 11 Apr 2024
Simultaneous linear connectivity of neural networks modulo permutation Ekansh Sharma Devin Kwok Tom Denton Daniel M. Roy David Rolnick Gintare Karolina Dziugaite 171 7 0 09 Apr 2024
Continual Learning with Weight Interpolation Jkedrzej Kozal Jan Wasilewski Bartosz Krawczyk Michal Wo'zniak CLL MoMe 29 6 0 05 Apr 2024
Arcee's MergeKit: A Toolkit for Merging Large Language Models Charles Goddard Shamane Siriwardhana Malikeh Ehghaghi Luke Meyers Vladimir Karpukhin Brian Benedict Mark McQuade Jacob Solawetz MoMe KELM 75 76 0 20 Mar 2024
FedFisher: Leveraging Fisher Information for One-Shot Federated Learning Divyansh Jhunjhunwala Shiqiang Wang Gauri Joshi FedML 24 6 0 19 Mar 2024
MedMerge: Merging Models for Effective Transfer Learning to Medical Imaging Tasks Ibrahim Almakky Santosh Sanjeev Anees Ur Rehman Hashmi Mohammad Areeb Qazi Mohammad Yaqub Mohammad Yaqub FedML MoMe 67 3 0 18 Mar 2024
Training-Free Pretrained Model Merging Zhenxing Xu Ke Yuan Huiqiong Wang Yong Wang Mingli Song Jie Song MoMe 21 15 0 04 Mar 2024
Training Neural Networks from Scratch with Parallel Low-Rank Adapters Minyoung Huh Brian Cheung Jeremy Bernstein Phillip Isola Pulkit Agrawal 25 10 0 26 Feb 2024
Mind the Modality Gap: Towards a Remote Sensing Vision-Language Model via Cross-modal Alignment Angelos Zavras Dimitrios Michail Begum Demir Ioannis Papoutsis VLM 17 11 0 15 Feb 2024
Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods Akira Ito Masanori Yamada Atsutoshi Kumagai MoMe 41 5 0 06 Feb 2024
FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Models Feihong He Gang Li Mengyuan Zhang Leilei Yan Lingyu Si Fanzhang Li Li Shen DiffM 15 15 0 28 Jan 2024
Asynchronous Local-SGD Training for Language Modeling Bo Liu Rachita Chhaparia Arthur Douillard Satyen Kale Andrei A. Rusu Jiajun Shen Arthur Szlam MarcÁurelio Ranzato FedML 28 10 0 17 Jan 2024
Train ñ Trade: Foundations of Parameter Markets Tzu-Heng Huang Harit Vishwakarma Frederic Sala AIFin 13 2 0 07 Dec 2023
Merging by Matching Models in Task Parameter Subspaces Derek Tam Mohit Bansal Colin Raffel MoMe 19 10 0 07 Dec 2023
REDS: Resource-Efficient Deep Subnetworks for Dynamic Resource Constraints Francesco Corti Balz Maag Joachim Schauer U. Pferschy O. Saukh 21 2 0 22 Nov 2023
DiLoCo: Distributed Low-Communication Training of Language Models Arthur Douillard Qixuang Feng Andrei A. Rusu Rachita Chhaparia Yani Donchev A. Kuncoro MarcÁurelio Ranzato Arthur Szlam Jiajun Shen 56 31 0 14 Nov 2023
Fuse to Forget: Bias Reduction and Selective Memorization through Model Fusion Kerem Zaman Leshem Choshen Shashank Srivastava KELM MoMe 13 10 0 13 Nov 2023
Equivariant Deep Weight Space Alignment Aviv Navon Aviv Shamsian Ethan Fetaya Gal Chechik Nadav Dym Haggai Maron 16 21 0 20 Oct 2023