On the Periodic Behavior of Neural Network Training with Batch
Normalization and Weight Decay

On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay

29 June 2021

Nadezhda Chirkova

Dmitry Vetrov

Papers citing "On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay"

6 / 6 papers shown

Title
Normalization and effective learning rates in reinforcement learning Clare Lyle Zeyu Zheng Khimya Khetarpal James Martens H. V. Hasselt Razvan Pascanu Will Dabney 19 7 0 01 Jul 2024
From Stability to Chaos: Analyzing Gradient Descent Dynamics in Quadratic Regression Xuxing Chen Krishnakumar Balasubramanian Promit Ghosal Bhavya Agrawalla 28 7 0 02 Oct 2023
Scale-invariant Bayesian Neural Networks with Connectivity Tangent Kernel Sungyub Kim Si-hun Park Kyungsu Kim Eunho Yang BDL 26 4 0 30 Sep 2022
Adapting the Linearised Laplace Model Evidence for Modern Deep Learning Javier Antorán David Janz J. Allingham Erik A. Daxberger Riccardo Barbano Eric T. Nalisnick José Miguel Hernández-Lobato UQCV BDL 27 28 0 17 Jun 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 37 69 0 14 Jun 2022
Robust Training of Neural Networks Using Scale Invariant Architectures Zhiyuan Li Srinadh Bhojanapalli Manzil Zaheer Sashank J. Reddi Surinder Kumar 19 27 0 02 Feb 2022