The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof

The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof

30 May 2024

Stefanie Jegelka

Papers citing "The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof"

14 / 14 papers shown

Title
The Empirical Impact of Reducing Symmetries on the Performance of Deep Ensembles and MoE Andrei Chernov Oleg Novitskij 41 0 0 24 Feb 2025
Parameter Symmetry Breaking and Restoration Determines the Hierarchical Learning in AI Systems Liu Ziyin Yizhou Xu T. Poggio Isaac Chuang 48 4 0 07 Feb 2025
Learning on LoRAs: GL-Equivariant Processing of Low-Rank Weight Spaces for Large Finetuned Models Theo Putterman Derek Lim Yoav Gelberg Stefanie Jegelka Haggai Maron AI4CE 43 5 0 05 Oct 2024
Remove Symmetries to Control Model Expressivity and Improve Optimization Liu Ziyin Yizhou Xu Isaac Chuang AAML 38 1 0 28 Aug 2024
Liquid Fourier Latent Dynamics Networks for fast GPU-based numerical simulations in computational cardiology Matteo Salvador Alison L. Marsden AI4CE 27 1 0 19 Aug 2024
Vanishing Feature: Diagnosing Model Merging and Beyond Xingyu Qu Samuel Horvath MoMe 48 2 0 05 Feb 2024
Plateau in Monotonic Linear Interpolation -- A "Biased" View of Loss Landscape for Deep Networks Xiang Wang Annie Wang Mo Zhou Rong Ge MoMe 158 10 0 03 Oct 2022
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 239 313 0 11 Sep 2022
Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry Fabrizio Pittorino Antonio Ferraro Gabriele Perugini Christoph Feinauer Carlo Baldassi R. Zecchina 193 24 0 07 Feb 2022
Analyzing Monotonic Linear Interpolation in Neural Network Loss Landscapes James Lucas Juhan Bae Michael Ruogu Zhang Stanislav Fort R. Zemel Roger C. Grosse MoMe 146 28 0 22 Apr 2021
A Practical Method for Constructing Equivariant Multilayer Perceptrons for Arbitrary Matrix Groups Marc Finzi Max Welling A. Wilson 71 185 0 19 Apr 2021
Optimizing Mode Connectivity via Neuron Alignment N. Joseph Tatro Pin-Yu Chen Payel Das Igor Melnyk P. Sattigeri Rongjie Lai MoMe 223 80 0 05 Sep 2020
Understanding symmetries in deep networks Vijay Badrinarayanan Bamdev Mishra R. Cipolla 211 42 0 03 Nov 2015
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 111 577 0 27 Feb 2015