Symmetries, flat minima, and the conserved quantities of gradient flow

Symmetries, flat minima, and the conserved quantities of gradient flow

31 October 2022

Robin G. Walters

Papers citing "Symmetries, flat minima, and the conserved quantities of gradient flow"

14 / 14 papers shown

Title
Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion Binchi Zhang Zaiyi Zheng Zhengzhang Chen Jundong Li 52 0 0 01 Feb 2025
The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof Derek Lim Moe Putterman Robin Walters Haggai Maron Stefanie Jegelka 32 5 0 30 May 2024
Keep the Momentum: Conservation Laws beyond Euclidean Gradient Flows Sibylle Marcotte Rémi Gribonval Gabriel Peyré 16 0 0 21 May 2024
Level Set Teleportation: An Optimization Perspective Aaron Mishkin A. Bietti Robert Mansel Gower 23 1 0 05 Mar 2024
Disentangling Linear Mode-Connectivity Gul Sena Altintas Gregor Bachmann Lorenzo Noci Thomas Hofmann 13 6 0 15 Dec 2023
A Symmetry-Aware Exploration of Bayesian Neural Network Posteriors Olivier Laurent Emanuel Aldea Gianni Franchi BDL UQCV 10 5 0 12 Oct 2023
$${\rm E}(3)$-Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning$ ${\rm E}(3)$ -Equivariant Actor-Critic Methods for Cooperative Multi-Agent Reinforcement Learning Dingyang Chen Qi Zhang 17 2 0 23 Aug 2023
A Novel Convolutional Neural Network Architecture with a Continuous Symmetry Y. Liu Han-Juan Shao Bing Bai AI4CE 11 1 0 03 Aug 2023
Improving Convergence and Generalization Using Parameter Symmetries Bo-Lu Zhao Robert Mansel Gower Robin G. Walters Rose Yu MoMe 17 13 0 22 May 2023
Git Re-Basin: Merging Models modulo Permutation Symmetries Samuel K. Ainsworth J. Hayase S. Srinivasa MoMe 239 313 0 11 Sep 2022
Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry Fabrizio Pittorino Antonio Ferraro Gabriele Perugini Christoph Feinauer Carlo Baldassi R. Zecchina 193 24 0 07 Feb 2022
Continuous vs. Discrete Optimization of Deep Neural Networks Omer Elkabetz Nadav Cohen 58 44 0 14 Jul 2021
Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics D. Kunin Javier Sagastuy-Breña Surya Ganguli Daniel L. K. Yamins Hidenori Tanaka 99 77 0 08 Dec 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,878 0 15 Sep 2016