v1v2 (latest)

The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks

International Conference on Learning Representations (ICLR), 2021

12 October 2021

Papers citing "The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks"

50 / 212 papers shown

Title
Linear Mode Connectivity in Differentiable Tree EnsemblesInternational Conference on Learning Representations (ICLR), 2024 Ryuichi Kanoh M. Sugiyama 342 1 0 17 Feb 2025
Forget the Data and Fine-Tuning! Just Fold the Network to CompressInternational Conference on Learning Representations (ICLR), 2025 Dong Wang Haris Šikić Lothar Thiele O. Saukh 263 1 0 14 Feb 2025
Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion Binchi Zhang Zaiyi Zheng Zhengzhang Chen Wenlin Yao 466 5 0 01 Feb 2025
Merging Feed-Forward Sublayers for Compressed Transformers Neha Verma Kenton W. Murray Kevin Duh AI4CE 340 0 0 10 Jan 2025
Training-free Heterogeneous Model Merging Zhengqi Xu Han Zheng Jie Song Li Sun Weilong Dai MoMe 425 2 0 03 Jan 2025
Non-Uniform Parameter-Wise Model MergingBigData Congress [Services Society] (BSS), 2024 Albert Manuel Orozco Camacho Stefan Horoi Guy Wolf Eugene Belilovsky MoMe FedML 305 0 0 20 Dec 2024
Task Arithmetic Through The Lens Of One-Shot Federated Learning Zhixu Tao I. Mason Sanjeev R. Kulkarni Xavier Boix MoMe FedML 361 8 0 27 Nov 2024
ATM: Improving Model Merging by Alternating Tuning and Merging Luca Zhou Daniele Solombrino Donato Crisostomi Maria Sofia Bucarelli Fabrizio Silvestri Emanuele Rodolà MoMe 417 6 0 05 Nov 2024
Where Do Large Learning Rates Lead Us?Neural Information Processing Systems (NeurIPS), 2024 Ildus Sadrtdinov M. Kodryan Eduard Pokonechny E. Lobacheva Dmitry Vetrov AI4CE 280 5 0 29 Oct 2024
Efficient and Effective Weight-Ensembling Mixture of Experts for Multi-Task Model Merging Li Shen Anke Tang Enneng Yang G. Guo Yong Luo Lefei Zhang Xiaochun Cao Di Lin Dacheng Tao MoMe 157 16 0 29 Oct 2024
Model merging with SVD to tie the KnotsInternational Conference on Learning Representations (ICLR), 2024 George Stoica Pratik Ramesh B. Ecsedi Leshem Choshen Judy Hoffman MoMe 222 40 0 25 Oct 2024
In Search of the Successful Interpolation: On the Role of Sharpness in CLIP Generalization Alireza Abdollahpoorrostam 185 0 0 21 Oct 2024
Unconstrained Model Merging for Enhanced LLM Reasoning Yiming Zhang Baoyi He Shengyu Zhang Yuhao Fu Qi Zhou ... Guanghan Ning Linyi Li Chunlin Ji Leilei Gan Hongxia Yang MoMe 147 5 0 17 Oct 2024
The Non-Local Model Merging Problem: Permutation Symmetries and Variance Collapse Ekansh Sharma Daniel M. Roy Gintare Karolina Dziugaite MoMe 215 5 0 16 Oct 2024
Exploring Model Kinship for Merging Large Language Models Yedi Hu Yunzhi Yao Ningyu Zhang Shumin Deng Ningyu Zhang MoMe 369 1 0 16 Oct 2024
Deep Model Merging: The Sister of Neural Network Interpretability -- A Survey A. Khan Todd Nief Nathaniel Hudson Mansi Sakarvadia Daniel Grzenda Aswathy Ajith Jordan Pettyjohn Kyle Chard Ian Foster MoMe 103 0 0 16 Oct 2024
Sampling from Bayesian Neural Network Posteriors with Symmetric Minibatch Splitting Langevin DynamicsInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2024 Daniel Paulin Peter Whalley Neil K. Chada Benedict Leimkuhler BDL 319 6 0 14 Oct 2024
Wolf2Pack: The AutoFusion Framework for Dynamic Parameter Fusion Bowen Tian Songning Lai Yutao Yue MoMe 162 2 0 08 Oct 2024
Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models Theo Putterman Derek Lim Yoav Gelberg Stefanie Jegelka Haggai Maron AI4CE 232 11 0 05 Oct 2024
What Matters for Model Merging at Scale? Prateek Yadav Tu Vu Jonathan Lai Alexandra Chronopoulou Manaal Faruqui Joey Tianyi Zhou Tsendsuren Munkhdalai MoMe 206 39 0 04 Oct 2024
Foldable SuperNets: Scalable Merging of Transformers with Different Initializations and Tasks Edan Kinderman Itay Hubara Haggai Maron Daniel Soudry MoMe 272 3 0 02 Oct 2024
On the universality of neural encodings in CNNs Florentin Guth Brice Ménard SSL 222 7 0 28 Sep 2024
Realistic Evaluation of Model Merging for Compositional Generalization Derek Tam Yash Kant Brian Lester Igor Gilitschenski Colin Raffel MoMe 221 10 0 26 Sep 2024
Revisiting Deep Ensemble Uncertainty for Enhanced Medical Anomaly DetectionInternational Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), 2024 Yi Gu Yi Lin Kwang-Ting Cheng Hao Chen UQCV 185 5 0 26 Sep 2024
Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise ClusteringInternational Conference on Learning Representations (ICLR), 2024 Ziyu Zhao Tao Shen Didi Zhu Zexi Li Jing Su Xuwu Wang Kun Kuang Fei Wu MoMe 336 31 0 24 Sep 2024
Remove Symmetries to Control Model Expressivity and Improve OptimizationInternational Conference on Learning Representations (ICLR), 2024 Liu Ziyin Yizhou Xu Isaac Chuang AAML 445 4 0 28 Aug 2024
Weight Scope Alignment: A Frustratingly Easy Method for Model MergingEuropean Conference on Artificial Intelligence (ECAI), 2024 Yichu Xu Xin-Chun Li Le Gan De-Chuan Zhan MoMe 243 2 0 22 Aug 2024
Approaching Deep Learning through the Spectral Dynamics of Weights David Yunis Kumar Kshitij Patel Samuel Wheeler Pedro H. P. Savarese Gal Vardi Karen Livescu Michael Maire Matthew R. Walter 254 12 0 21 Aug 2024
SMILE: Zero-Shot Sparse Mixture of Low-Rank Experts Construction From Pre-Trained Foundation Models Anke Tang Li Shen Yong Luo Shuai Xie Han Hu Lefei Zhang Di Lin Dacheng Tao MoMe 216 9 0 19 Aug 2024
Variational Inference Failures Under Model Symmetries: Permutation Invariant Posteriors for Bayesian Neural Networks Yoav Gelberg Tycho F. A. van der Ouderaa Mark van der Wilk Y. Gal AAML 245 6 0 10 Aug 2024
The Ungrounded Alignment Problem Marc Pickett Aakash Kumar Nain Joseph Modayil Llion Jones 122 0 0 08 Aug 2024
Computer Audition: From Task-Specific Machine Learning to Foundation Models Andreas Triantafyllopoulos Iosif Tsangko Alexander Gebhard A. Mesaros Maria Sandsten B. Schuller 316 6 0 22 Jul 2024
Training-Free Model Merging for Multi-target Domain Adaptation Wenyi Li Huan-ang Gao Mingju Gao Beiwen Tian Rong Zhi Hao Zhao MoMe 196 10 0 18 Jul 2024
Harmony in Diversity: Merging Neural Networks with Canonical Correlation Analysis Stefan Horoi Albert Manuel Orozco Camacho Eugene Belilovsky Guy Wolf FedML MoMe 162 11 0 07 Jul 2024
Neural Networks Trained by Weight Permutation are Universal Approximators Yongqiang Cai Gaohang Chen Zhonghua Qiao 455 2 0 01 Jul 2024
WARP: On the Benefits of Weight Averaged Rewarded Policies Alexandre Ramé Johan Ferret Nino Vieillard Robert Dadashi Léonard Hussenot Pierre-Louis Cedoz Pier Giuseppe Sessa Sertan Girgin Arthur Douillard Olivier Bachem 239 31 0 24 Jun 2024
Landscaping Linear Mode Connectivity Sidak Pal Singh Linara Adilova Michael Kamp Asja Fischer Bernhard Scholkopf Thomas Hofmann 298 9 0 24 Jun 2024
PathoWAve: A Deep Learning-based Weight Averaging Method for Improving Domain Generalization in Histopathology Images Parastoo Sotoudeh Sharifi M. Omair Ahmad M. N. S. Swamy MoMe OOD 184 0 0 21 Jun 2024
Scale Equivariant Graph Metanetworks Ioannis Kalogeropoulos Giorgos Bouritsas Yannis Panagakis 288 15 0 15 Jun 2024
Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion Anke Tang Li Shen Yong Luo Shiwei Liu Han Hu Di Lin MoMe 155 12 0 14 Jun 2024
The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof Derek Lim Moe Putterman Robin Walters Haggai Maron Stefanie Jegelka 384 14 0 30 May 2024
Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment Keming Lu Bowen Yu Fei Huang Yang Fan Runji Lin Chang Zhou MoMe 153 26 0 28 May 2024
$C^2M^3$ : Cycle-Consistent Multi-Model Merging Donato Crisostomi Marco Fumero Daniele Baieri F. Bernard Emanuele Rodolà MoMe 223 13 0 28 May 2024
Structured Partial Stochasticity in Bayesian Neural Networks Tommy Rochussen 208 0 0 27 May 2024
FedCal: Achieving Local and Global Calibration in Federated Learning via Aggregated Parameterized Scaler Hongyi Peng Han Yu Xiaoli Tang Xiaoxiao Li 203 7 0 24 May 2024
Manifold Metric: A Loss Landscape Approach for Predicting Model Performance Pranshu Malviya Jerry Huang A. Baratin Quentin Fournier Sarath Chandar 209 0 0 24 May 2024
WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language ModelsNeural Information Processing Systems (NeurIPS), 2024 Peng Wang Zexi Li Ningyu Zhang Ziwen Xu Yunzhi Yao Yong Jiang Pengjun Xie Fei Huang Huajun Chen KELM CLL 246 55 0 23 May 2024
MiniCache: KV Cache Compression in Depth Dimension for Large Language ModelsNeural Information Processing Systems (NeurIPS), 2024 Akide Liu Jing Liu Zizheng Pan Yefei He Gholamreza Haffari Bohan Zhuang MQ 187 64 0 23 May 2024
Text-to-Model: Text-Conditioned Neural Network Diffusion for Train-Once-for-All Personalization Zexi Li Lingzhi Gao Chao Wu AI4CE DiffM 303 6 0 23 May 2024
Exploring and Exploiting the Asymmetric Valley of Deep Neural Networks Xin-Chun Li Jinli Tang Bo Zhang Lan Li De-Chuan Zhan 199 2 0 21 May 2024

All Papers

The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks

Papers citing "The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks"