v1v2 (latest)

The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks

International Conference on Learning Representations (ICLR), 2021

12 October 2021

Papers citing "The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks"

50 / 212 papers shown

Title
Visualizing, Rethinking, and Mining the Loss Landscape of Deep Neural Networks Yichu Xu Xin-Chun Li Lan Li De-Chuan Zhan 250 2 0 21 May 2024
Model Extrapolation Expedites Alignment Chujie Zheng Ziqi Wang Mengyue Yang Shiyu Huang Nanyun Peng MoMe 313 37 0 25 Apr 2024
Stepwise Alignment for Constrained Language Model Policy Optimization Akifumi Wachi Thien Q. Tran Rei Sato Takumi Tanabe Yohei Akimoto 167 16 0 17 Apr 2024
DIMAT: Decentralized Iterative Merging-And-Training for Deep Learning Models Nastaran Saadati Minh Pham Nasla Saleem Joshua R. Waite Aditya Balu Zhanhong Jiang Chinmay Hegde Soumik Sarkar MoMe 175 4 0 11 Apr 2024
Simultaneous linear connectivity of neural networks modulo permutation Ekansh Sharma Devin Kwok Tom Denton Daniel M. Roy David Rolnick Gintare Karolina Dziugaite 392 9 0 09 Apr 2024
Exploring Neural Network Landscapes: Star-Shaped and Geodesic Connectivity Zhanran Lin Puheng Li Lei Wu 395 9 0 09 Apr 2024
Continual Learning with Weight Interpolation Jkedrzej Kozal Jan Wasilewski Bartosz Krawczyk Michal Wo'zniak CLL MoMe 377 9 0 05 Apr 2024
Arcee's MergeKit: A Toolkit for Merging Large Language Models Charles Goddard Shamane Siriwardhana Malikeh Ehghaghi Luke Meyers Vladimir Karpukhin Brian Benedict Mark McQuade Jacob Solawetz MoMe KELM 599 158 0 20 Mar 2024
FedFisher: Leveraging Fisher Information for One-Shot Federated LearningInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2024 Divyansh Jhunjhunwala Shiqiang Wang Gauri Joshi FedML 173 18 0 19 Mar 2024
MedMerge: Merging Models for Effective Transfer Learning to Medical Imaging Tasks Ibrahim Almakky Santosh Sanjeev Anees Ur Rehman Hashmi Mohammad Areeb Qazi Mohammad Yaqub Mohammad Yaqub FedML MoMe 340 6 0 18 Mar 2024
Fisher Mask Nodes for Language Model MergingInternational Conference on Language Resources and Evaluation (LREC), 2024 Thennal D K Ganesh Nathan Suchithra M S MoMe AI4CE 297 7 0 14 Mar 2024
Do Deep Neural Network Solutions Form a Star Domain?International Conference on Learning Representations (ICLR), 2024 Ankit Sonthalia Alexander Rubinstein Ehsan Abbasnejad Seong Joon Oh MoMe 1.2K 4 5 12 Mar 2024
Training-Free Pretrained Model Merging Zhenxing Xu Ke Yuan Huiqiong Wang Yong Wang Weilong Dai Mingli Song MoMe 314 21 0 04 Mar 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 487 33 0 04 Mar 2024
$Uniform $\mathcal{C}^k$ Approximation of $G$-Invariant and Antisymmetric Functions, Embedding Dimensions, and Polynomial Representations$ Uniform $\mathcal{C}^k$ Approximation of $G$ -Invariant and Antisymmetric Functions, Embedding Dimensions, and Polynomial Representations Soumya Ganguly Khoa Tran Rahul Sarkar 183 0 0 02 Mar 2024
Merging Text Transformer Models from Different Initializations Neha Verma Maha Elbayad MoMe 272 11 0 01 Mar 2024
Analyzing and Reducing Catastrophic Forgetting in Parameter Efficient Tuning Weijieying Ren Xinlong Li Lei Wang Tianxiang Zhao Wei Qin CLL KELM 277 55 0 29 Feb 2024
Training Neural Networks from Scratch with Parallel Low-Rank Adapters Minyoung Huh Brian Cheung Jeremy Bernstein Phillip Isola Pulkit Agrawal 200 14 0 26 Feb 2024
Improved Generalization of Weight Space Networks via Augmentations Aviv Shamsian Aviv Navon David W. Zhang Yan Zhang Ethan Fetaya Gal Chechik Haggai Maron 239 16 0 06 Feb 2024
On the Emergence of Cross-Task Linearity in the Pretraining-Finetuning ParadigmInternational Conference on Machine Learning (ICML), 2024 Zhanpeng Zhou Zijun Chen Yilan Chen Bo Zhang Junchi Yan MoMe 289 21 0 06 Feb 2024
Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search MethodsInternational Conference on Learning Representations (ICLR), 2024 Akira Ito Masanori Yamada Atsutoshi Kumagai MoMe 620 11 0 06 Feb 2024
How Good is a Single Basin?International Conference on Artificial Intelligence and Statistics (AISTATS), 2024 Kai Lion Lorenzo Noci Thomas Hofmann Gregor Bachmann UQCV 125 3 0 05 Feb 2024
Unification of Symmetries Inside Neural Networks: Transformer, Feedforward and Neural ODE Koji Hashimoto Yuji Hirono Akiyoshi Sannai AI4CE 182 12 0 04 Feb 2024
Improving Model Fusion by Training-time Neuron Alignment with Fixed Neuron AnchorsIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024 Zexi Li Zhiqi Li Jie Lin Zhenyuan Zhang Tao Lin Chao Wu Tao Lin Chao Wu 298 5 0 02 Feb 2024
Merging Multi-Task Models via Weight-Ensembling Mixture of Experts Anke Tang Li Shen Yong Luo Nan Yin Lefei Zhang Dacheng Tao MoMe 212 79 0 01 Feb 2024
A Compact Representation for Bayesian Neural Networks By Removing Permutation Symmetry Tim Z. Xiao Weiyang Liu Kushagra Pandey 142 6 0 31 Dec 2023
Deep Learning on Object-centric 3D Neural Fields Pierluigi Zama Ramirez Luca de Luigi Daniele Sirocchi Adriano Cardace Riccardo Spezialetti Francesco Ballerini Samuele Salti Luigi Di Stefano 274 6 0 20 Dec 2023
Disentangling Linear Mode-Connectivity Gul Sena Altintas Gregor Bachmann Lorenzo Noci Thomas Hofmann 280 9 0 15 Dec 2023
Weighted Ensemble Models Are Strong Continual LearnersEuropean Conference on Computer Vision (ECCV), 2023 Imad Eddine Marouf Subhankar Roy Enzo Tartaglione Stéphane Lathuilière CLL 261 34 0 14 Dec 2023
Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion Anke Tang Li Shen Yong Luo Liang Ding Han Hu Bo Du Dacheng Tao MoMe 206 29 0 11 Dec 2023
Train ñ Trade: Foundations of Parameter MarketsNeural Information Processing Systems (NeurIPS), 2023 Tzu-Heng Huang Harit Vishwakarma Frederic Sala AIFin 151 3 0 07 Dec 2023
Graph Metanetworks for Processing Diverse Neural Architectures Derek Lim Haggai Maron Marc T. Law Jonathan Lorraine James Lucas AI4CE 303 43 0 07 Dec 2023
Merging by Matching Models in Task Parameter Subspaces Derek Tam Mohit Bansal Colin Raffel MoMe 261 20 0 07 Dec 2023
REDS: Resource-Efficient Deep Subnetworks for Dynamic Resource ConstraintsIEEE Transactions on Mobile Computing (IEEE TMC), 2023 Francesco Corti Balz Maag Joachim Schauer U. Pferschy O. Saukh 277 3 0 22 Nov 2023
Large Learning Rates Improve Generalization: But How Large Are We Talking About? E. Lobacheva Eduard Pockonechnyy M. Kodryan Dmitry Vetrov AI4CE 105 1 0 19 Nov 2023
Data Augmentations in Deep Weight Spaces Aviv Shamsian David W. Zhang Aviv Navon Yan Zhang Miltiadis Kofinas ... E. Gavves Cees G. M. Snoek Ethan Fetaya Gal Chechik Haggai Maron 257 2 0 15 Nov 2023
Proving Linear Mode Connectivity of Neural Networks via Optimal TransportInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2023 Damien Ferbach Baptiste Goujaud Gauthier Gidel Hadrien Hendrikx MoMe 341 17 0 29 Oct 2023
Understanding the Inner Workings of Language Models Through Representation DissimilarityConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Davis Brown Charles Godfrey Nicholas Konz Jonathan Tu Henry Kvinge 157 12 0 23 Oct 2023
Equivariant Deep Weight Space Alignment Aviv Navon Aviv Shamsian Ethan Fetaya Gal Chechik Nadav Dym Haggai Maron 255 29 0 20 Oct 2023
On permutation symmetries in Bayesian neural network posteriors: a variational perspectiveNeural Information Processing Systems (NeurIPS), 2023 Simone Rossi Ankit Singh T. Hannagan 187 3 0 16 Oct 2023
A Symmetry-Aware Exploration of Bayesian Neural Network PosteriorsInternational Conference on Learning Representations (ICLR), 2023 Olivier Laurent Emanuel Aldea Gianni Franchi BDL UQCV 227 9 0 12 Oct 2023
Going Beyond Neural Network Feature Similarity: The Network Feature Complexity and Its Interpretation Using Category TheoryInternational Conference on Learning Representations (ICLR), 2023 Yiting Chen Zhanpeng Zhou Junchi Yan 212 10 0 10 Oct 2023
Transformer Fusion with Optimal TransportInternational Conference on Learning Representations (ICLR), 2023 Moritz Imfeld Jacopo Graldi Marco Giordano Thomas Hofmann Sotiris Anagnostidis Sidak Pal Singh ViT MoMe 390 28 0 09 Oct 2023
Robot Fleet Learning via Policy MergingInternational Conference on Learning Representations (ICLR), 2023 Lirui Wang Kaiqing Zhang Allan Zhou Max Simchowitz Russ Tedrake 370 5 0 02 Oct 2023
Neural Processing of Tri-Plane Hybrid Neural FieldsInternational Conference on Learning Representations (ICLR), 2023 Adriano Cardace Pierluigi Zama Ramirez Francesco Ballerini Allan Zhou Samuele Salti Luigi Di Stefano AI4CE 337 8 0 02 Oct 2023
Symmetry Induces Structure and Constraint of LearningInternational Conference on Machine Learning (ICML), 2023 Liu Ziyin 341 18 0 29 Sep 2023
Mode Connectivity and Data Heterogeneity of Federated Learning Tailin Zhou Jun Zhang Danny H. K. Tsang FedML 202 4 0 29 Sep 2023
Deep Model Fusion: A Survey Weishi Li Yong Peng Miao Zhang Liang Ding Han Hu Li Shen FedML MoMe 229 84 0 27 Sep 2023
Geodesic Mode Connectivity Charlie Tan Theodore Long Sarah Zhao Rudolf Laine 139 3 0 24 Aug 2023
Mode Combinability: Exploring Convex Combinations of Permutation Aligned ModelsNeural Networks (Neural Netw.), 2023 Adrián Csiszárik M. Kiss Péter Korösi-Szabó Márton Muntag Gergely Papp D. Varga MoMe 133 1 0 22 Aug 2023