Demystifying Batch Normalization in ReLU Networks: Equivalent Convex
Optimization Models and Implicit Regularization

Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization

2 March 2021

Batu Mehmet Ozturkler

Morteza Mardani

Papers citing "Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization"

12 / 12 papers shown

Title
Linguistic Collapse: Neural Collapse in (Large) Language Models Robert Wu Vardan Papyan 53 13 0 28 May 2024
The Convex Landscape of Neural Networks: Characterizing Global Optima and Stationary Points via Lasso Models Tolga Ergen Mert Pilanci 21 2 0 19 Dec 2023
When Deep Learning Meets Polyhedral Theory: A Survey Joey Huchette Gonzalo Muñoz Thiago Serra Calvin Tsay AI4CE 99 33 0 29 Apr 2023
Neural Collapse: A Review on Modelling Principles and Generalization Vignesh Kothapalli 35 76 0 08 Jun 2022
Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers Arda Sahiner Tolga Ergen Batu Mehmet Ozturkler John M. Pauly Morteza Mardani Mert Pilanci 43 33 0 17 May 2022
Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions Aaron Mishkin Arda Sahiner Mert Pilanci OffRL 77 30 0 02 Feb 2022
Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks Tolga Ergen Mert Pilanci 34 16 0 18 Oct 2021
The Convex Geometry of Backpropagation: Neural Network Gradient Flows Converge to Extreme Points of the Dual Convex Program Yifei Wang Mert Pilanci MLT MDE 55 11 0 13 Oct 2021
Parallel Deep Neural Networks Have Zero Duality Gap Yifei Wang Tolga Ergen Mert Pilanci 79 10 0 13 Oct 2021
Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs Tolga Ergen Mert Pilanci OffRL MLT 42 33 0 11 Oct 2021
Scaled ReLU Matters for Training Vision Transformers Pichao Wang Xue Wang Haowen Luo Jingkai Zhou Zhipeng Zhou Fan Wang Hao Li Rong Jin 23 41 0 08 Sep 2021
Convex Geometry and Duality of Over-parameterized Neural Networks Tolga Ergen Mert Pilanci MLT 47 54 0 25 Feb 2020