v1v2 (latest)

The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks

International Conference on Learning Representations (ICLR), 2021

12 October 2021

Papers citing "The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks"

50 / 212 papers shown

Title
UnIVAL: Unified Model for Image, Video, Audio and Language Tasks Mustafa Shukor Corentin Dancette Alexandre Ramé Matthieu Cord MoMe MLLM 243 53 0 30 Jul 2023
A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiency Jiawei Shao Zijian Li Wenqiang Sun Tailin Zhou Yuchang Sun Lumin Liu Zehong Lin Yuyi Mao Jun Zhang FedML 234 39 0 20 Jul 2023
Going Beyond Linear Mode Connectivity: The Layerwise Linear Feature ConnectivityNeural Information Processing Systems (NeurIPS), 2023 Zhanpeng Zhou Yongyi Yang Xiaojiang Yang Junchi Yan Wei Hu 226 45 0 17 Jul 2023
Layer-wise Linear Mode ConnectivityInternational Conference on Learning Representations (ICLR), 2023 Linara Adilova Maksym Andriushchenko Michael Kamp Asja Fischer Martin Jaggi FedML FAtt MoMe 373 19 0 13 Jul 2023
Sparse Model Soups: A Recipe for Improved Pruning via Model AveragingInternational Conference on Learning Representations (ICLR), 2023 Max Zimmer Christoph Spiegel Sebastian Pokutta MoMe 375 18 0 29 Jun 2023
Lookaround Optimizer: $k$ steps around, 1 step averageNeural Information Processing Systems (NeurIPS), 2023 Jiangtao Zhang Shunyu Liu Mingli Song Tongtian Zhu Zhenxing Xu Weilong Dai MoMe 330 8 0 13 Jun 2023
Hidden symmetries of ReLU networksInternational Conference on Machine Learning (ICML), 2023 J. E. Grigsby Kathryn A. Lindsey David Rolnick 170 26 0 09 Jun 2023
Investigating the Effect of Misalignment on Membership Privacy in the White-box SettingProceedings on Privacy Enhancing Technologies (PoPETs), 2023 Ana-Maria Cretu Daniel Jones Yves-Alexandre de Montjoye Shruti Tople AAML 154 8 0 08 Jun 2023
Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardsNeural Information Processing Systems (NeurIPS), 2023 Alexandre Ramé Guillaume Couairon Mustafa Shukor Corentin Dancette Jean-Baptiste Gaya Laure Soulier Matthieu Cord MoMe 271 196 0 07 Jun 2023
Soft Merging of Experts with Adaptive Routing Mohammed Muqeeth Haokun Liu Colin Raffel MoMe MoE 226 74 0 06 Jun 2023
Input-gradient space particle inference for neural network ensemblesInternational Conference on Learning Representations (ICLR), 2023 Trung Trinh Markus Heinonen Luigi Acerbi Samuel Kaski UQCV 170 4 0 05 Jun 2023
TIES-Merging: Resolving Interference When Merging ModelsNeural Information Processing Systems (NeurIPS), 2023 Prateek Yadav Derek Tam Leshem Choshen Colin Raffel Joey Tianyi Zhou MoMe 300 494 0 02 Jun 2023
A Rainbow in Deep Network Black Boxes Florentin Guth Brice Ménard G. Rochette S. Mallat 267 19 0 29 May 2023
Investigating how ReLU-networks encode symmetriesNeural Information Processing Systems (NeurIPS), 2023 Georg Bökman Fredrik Kahl 163 7 0 26 May 2023
Transferring Learning Trajectories of Neural NetworksInternational Conference on Learning Representations (ICLR), 2023 Daiki Chijiwa 183 4 0 23 May 2023
Neural Functional TransformersNeural Information Processing Systems (NeurIPS), 2023 Allan Zhou Kaien Yang Yiding Jiang Kaylee Burns Winnie Xu Samuel Sokota J. Zico Kolter Chelsea Finn 160 42 0 22 May 2023
Subspace-Configurable Networks Dong Wang O. Saukh Xiaoxi He Lothar Thiele OOD 231 0 0 22 May 2023
Improving Convergence and Generalization Using Parameter SymmetriesInternational Conference on Learning Representations (ICLR), 2023 Bo Zhao Robert Mansel Gower Robin Walters Rose Yu MoMe 318 21 0 22 May 2023
Exploring the Complexity of Deep Neural Networks through Functional EquivalenceInternational Conference on Machine Learning (ICML), 2023 Guohao Shen 282 6 0 19 May 2023
ZipIt! Merging Models from Different Tasks without TrainingInternational Conference on Learning Representations (ICLR), 2023 George Stoica Daniel Bolya J. Bjorner Pratik Ramesh Taylor N. Hearn Judy Hoffman VLM MoMe 346 158 0 04 May 2023
An Empirical Study of Multimodal Model MergingConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Yi-Lin Sung Linjie Li Kevin Qinghong Lin Zhe Gan Joey Tianyi Zhou Lijuan Wang MoMe 237 52 0 28 Apr 2023
Typical and atypical solutions in non-convex neural networks with discrete and continuous weightsPhysical Review E (PRE), 2023 Carlo Baldassi Enrico M. Malatesta Gabriele Perugini R. Zecchina MQ 213 20 0 26 Apr 2023
Expand-and-Cluster: Parameter Recovery of Neural NetworksInternational Conference on Machine Learning (ICML), 2023 Flavio Martinelli Berfin Simsek W. Gerstner Johanni Brea 416 12 0 25 Apr 2023
PopulAtion Parameter Averaging (PAPA) Alexia Jolicoeur-Martineau Emy Gervais Kilian Fatras Yan Zhang Damien Scieur MoMe 391 25 0 06 Apr 2023
On the Variance of Neural Network Training with respect to Test Sets and DistributionsInternational Conference on Learning Representations (ICLR), 2023 Keller Jordan OOD 204 17 0 04 Apr 2023
Type-II Saddles and Probabilistic Stability of Stochastic Gradient Descent Liu Ziyin Botao Li Tomer Galanti Masakuni Ueda 198 8 0 23 Mar 2023
Merging Decision Transformers: Weight Averaging for Forming Multi-Task PoliciesIEEE International Conference on Robotics and Automation (ICRA), 2023 Daniel Lawson A. H. Qureshi MoMe OffRL 302 14 0 14 Mar 2023
To Stay or Not to Stay in the Pre-train Basin: Insights on Ensembling in Transfer LearningNeural Information Processing Systems (NeurIPS), 2023 Ildus Sadrtdinov Dmitrii Pozdeev Dmitry Vetrov E. Lobacheva 188 7 0 06 Mar 2023
DART: Diversify-Aggregate-Repeat Training Improves Generalization of Neural NetworksComputer Vision and Pattern Recognition (CVPR), 2023 Samyak Jain Sravanti Addepalli P. Sahu Priyam Dey R. Venkatesh Babu MoMe OOD 260 27 0 28 Feb 2023
Permutation Equivariant Neural FunctionalsNeural Information Processing Systems (NeurIPS), 2023 Allan Zhou Kaien Yang Kaylee Burns Adriano Cardace Yiding Jiang Samuel Sokota J. Zico Kolter Chelsea Finn 246 64 0 27 Feb 2023
Identifying Equivalent Training DynamicsNeural Information Processing Systems (NeurIPS), 2023 William T. Redman J. M. Bello-Rivas M. Fonoberova Ryan Mohr Ioannis G. Kevrekidis Igor Mezić 244 8 0 17 Feb 2023
Revisiting Weighted Aggregation in Federated Learning with Neural NetworksInternational Conference on Machine Learning (ICML), 2023 Zexi Li Tao Lin Xinyi Shang Chao-Xiang Wu FedML 263 96 0 14 Feb 2023
Deep Learning on Implicit Neural Representations of ShapesInternational Conference on Learning Representations (ICLR), 2023 Luca de Luigi Adriano Cardace Riccardo Spezialetti Pierluigi Zama Ramirez Samuele Salti Luigi Di Stefano 161 57 0 10 Feb 2023
Knowledge is a Region in Weight Space for Fine-tuned Language ModelsConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Almog Gueta Elad Venezian Colin Raffel Noam Slonim Yoav Katz Leshem Choshen 236 56 0 09 Feb 2023
Efficient displacement convex optimization with particle gradient descentInternational Conference on Machine Learning (ICML), 2023 Hadi Daneshmand Jason D. Lee Chi Jin 218 6 0 09 Feb 2023
Equivariant Architectures for Learning in Deep Weight SpacesInternational Conference on Machine Learning (ICML), 2023 Aviv Navon Aviv Shamsian Idan Achituve Ethan Fetaya Gal Chechik Haggai Maron 254 84 0 30 Jan 2023
Re-basin via implicit Sinkhorn differentiationComputer Vision and Pattern Recognition (CVPR), 2022 F. Guerrero-Peña H. R. Medeiros Thomas Dubail Masih Aminbeidokhti Mohammadhadi Shateri M. Pedersoli MoMe 262 57 0 22 Dec 2022
Model Ratatouille: Recycling Diverse Models for Out-of-Distribution GeneralizationInternational Conference on Machine Learning (ICML), 2022 Alexandre Ramé Kartik Ahuja Jianyu Zhang Matthieu Cord Léon Bottou David Lopez-Paz MoMe OODD 365 100 0 20 Dec 2022
Transformers learn in-context by gradient descentInternational Conference on Machine Learning (ICML), 2022 J. Oswald Eyvind Niklasson E. Randazzo João Sacramento A. Mordvintsev A. Zhmoginov Max Vladymyrov MLT 386 620 0 15 Dec 2022
Editing Models with Task ArithmeticInternational Conference on Learning Representations (ICLR), 2022 Gabriel Ilharco Marco Tulio Ribeiro Mitchell Wortsman Suchin Gururangan Ludwig Schmidt Hannaneh Hajishirzi Ali Farhadi KELM MoMe MU 935 707 0 08 Dec 2022
Linear Interpolation In Parameter Space is Good Enough for Fine-Tuned Language Models Mark Rofin Nikita Balagansky Daniil Gavrilov MoMe KELM 139 7 0 22 Nov 2022
Mechanistic Mode ConnectivityInternational Conference on Machine Learning (ICML), 2022 Ekdeep Singh Lubana Eric J. Bigelow Robert P. Dick David M. Krueger Hidenori Tanaka 222 55 0 15 Nov 2022
REPAIR: REnormalizing Permuted Activations for Interpolation RepairInternational Conference on Learning Representations (ICLR), 2022 Keller Jordan Hanie Sedghi O. Saukh R. Entezari Behnam Neyshabur MoMe 326 115 0 15 Nov 2022
Towards Better Out-of-Distribution Generalization of Neural Algorithmic Reasoning Tasks Sadegh Mahdavi Kevin Swersky Thomas Kipf Milad Hashemi Christos Thrampoulidis Renjie Liao LRM OOD NAI 192 32 0 01 Nov 2022
Symmetries, flat minima, and the conserved quantities of gradient flowInternational Conference on Learning Representations (ICLR), 2022 Bo Zhao I. Ganev Robin Walters Rose Yu Nima Dehmamy 296 26 0 31 Oct 2022
lo-fi: distributed fine-tuning without communication Mitchell Wortsman Suchin Gururangan Shen Li Ali Farhadi Ludwig Schmidt Michael G. Rabbat Ari S. Morcos 278 24 0 19 Oct 2022
Mean-field analysis for heavy ball methods: Dropout-stability, connectivity, and global convergence Diyuan Wu Vyacheslav Kungurtsev Marco Mondelli 144 3 0 13 Oct 2022
Wasserstein Barycenter-based Model Fusion and Linear Mode Connectivity of Neural Networks A. K. Akash Sixu Li Nicolas García Trillos 174 15 0 13 Oct 2022
Stochastic optimization on matrices and a graphon McKean-Vlasov limit Zaïd Harchaoui Sewoong Oh Soumik Pal Raghav Somani Raghavendra Tripathi 258 3 0 02 Oct 2022
Random initialisations performing above chance and how to find them Frederik Benzing Simon Schug Robert Meier J. Oswald Yassir Akram Nicolas Zucchet Laurence Aitchison Angelika Steger ODL 341 28 0 15 Sep 2022

All Papers

The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks

Papers citing "The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks"