v1v2 (latest)

Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification

24 September 2017

Boris Ginsburg

Papers citing "Comparison of Batch Normalization and Weight Normalization Algorithms for the Large-scale Image Classification"

33 / 33 papers shown

Title
NoMorelization: Building Normalizer-Free Models from a Sample's Perspective Yu Xie Yuwen Yang Yue Ding Hongtao Lu 208 2 0 13 Oct 2022
Breaking Time Invariance: Assorted-Time Normalization for RNNsNeural Processing Letters (NPL), 2022 Cole Pospisil Vasily Zadorozhnyy Qiang Ye 63 1 0 28 Sep 2022
Batch Layer Normalization, A new normalization layer for CNNs and RNNInternational Conference on Advances in Artificial Intelligence (ICAAI), 2022 A. Ziaee Erion cCano 154 21 0 19 Sep 2022
Can CNNs Be More Robust Than Transformers?International Conference on Learning Representations (ICLR), 2022 Zeyu Wang Yutong Bai Yuyin Zhou Cihang Xie UQCV OOD 209 52 0 07 Jun 2022
Understanding AdamW through Proximal Methods and Scale-Freeness Zhenxun Zhuang Mingrui Liu Ashok Cutkosky Francesco Orabona 229 99 0 31 Jan 2022
Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch DependenceNeural Information Processing Systems (NeurIPS), 2021 A. Labatie Dominic Masters Zach Eaton-Rosen Carlo Luschi 269 21 0 07 Jun 2021
"BNN - BN = ?": Training Binary Neural Networks without Batch Normalization Tianlong Chen Zhenyu Zhang Xu Ouyang Zechun Liu Zhiqiang Shen Zinan Lin MQ 183 40 0 16 Apr 2021
ECG-TCN: Wearable Cardiac Arrhythmia Detection with a Temporal Convolutional NetworkInternational Conference on Artificial Intelligence Circuits and Systems (ICAICS), 2021 T. Ingolfsson Xiaying Wang Michael Hersche Luca Bompani Lukas Cavigelli Luca Benini 160 52 0 25 Mar 2021
High-Performance Large-Scale Image Recognition Without NormalizationInternational Conference on Machine Learning (ICML), 2021 Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 453 579 0 11 Feb 2021
Weight Rescaling: Effective and Robust Regularization for Deep Neural Networks with Batch Normalization Ziquan Liu Yufei Cui Jia Wan Yushun Mao Antoni B. Chan 209 2 0 06 Feb 2021
Advances in Electron Microscopy with Deep Learning Jeffrey M. Ede 572 3 0 04 Jan 2021
Normalization Techniques in Training DNNs: Methodology, Analysis and ApplicationIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020 Lei Huang Jie Qin Yi Zhou Fan Zhu Li Liu Ling Shao AI4CE 304 364 0 27 Sep 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 792 88 0 17 Sep 2020
Optimization Theory for ReLU Neural Networks Trained with Normalization LayersInternational Conference on Machine Learning (ICML), 2020 Yonatan Dukler Quanquan Gu Guido Montúfar 185 30 0 11 Jun 2020
EEG-TCNet: An Accurate Temporal Convolutional Network for Embedded Motor-Imagery Brain-Machine InterfacesIEEE International Conference on Systems, Man and Cybernetics (SMC), 2020 T. Ingolfsson Michael Hersche Xiaying Wang Nobuaki Kobayashi Lukas Cavigelli Luca Benini 162 266 0 31 May 2020
A Simple Convergence Proof of Adam and Adagrad Alexandre Défossez Léon Bottou Francis R. Bach Nicolas Usunier 375 196 0 05 Mar 2020
A Comprehensive and Modularized Statistical Framework for Gradient Norm Equality in Deep Neural NetworksIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2020 Zhaodong Chen Lei Deng Bangyan Wang Guoqi Li Yuan Xie 236 30 0 01 Jan 2020
Mean Shift Rejection: Training Deep Neural Networks Without Minibatch Statistics or Normalization B. Ruff Taylor Beck Joscha Bach 108 3 0 29 Nov 2019
SparseTrain:Leveraging Dynamic Sparsity in Training DNNs on General-Purpose SIMD Processors Zhangxiaowen Gong Houxiang Ji Christopher W. Fletcher C. Hughes Josep Torrellas 128 5 0 22 Nov 2019
Understanding the Disharmony between Weight Normalization Family and Weight Decay: $ε-$ shifted $L_2$ Regularizer Li Xiang Chen Shuo Xia Yan Yang Jian 122 3 0 14 Nov 2019
Mean Spectral Normalization of Deep Neural Networks for Embedded Automation Anand Subramanian N. Chong 72 2 0 09 Jul 2019
How to Initialize your Network? Robust Initialization for WeightNorm & ResNetsNeural Information Processing Systems (NeurIPS), 2019 Devansh Arpit Victor Campos Yoshua Bengio 202 61 0 05 Jun 2019
Online Normalization for Training Neural NetworksNeural Information Processing Systems (NeurIPS), 2019 Vitaliy Chiley I. Sharapov Atli Kosson Urs Koster R. Reece S. D. L. Fuente Vishal Subbiah Michael James OnRL 190 59 0 15 May 2019
Theory III: Dynamics and Generalization in Deep Networks Andrzej Banburski Q. Liao Alycia Lee Lorenzo Rosasco Fernanda De La Torre Jack Hidary T. Poggio AI4CE 234 3 0 12 Mar 2019
Accelerating Training of Deep Neural Networks with a Standardization Loss Jasmine Collins Johannes Ballé Jonathon Shlens 134 3 0 03 Mar 2019
Equi-normalization of Neural NetworksInternational Conference on Learning Representations (ICLR), 2019 Pierre Stock Benjamin Graham Rémi Gribonval Edouard Grave ODL 104 19 0 27 Feb 2019
A Mean Field Theory of Batch Normalization Greg Yang Jeffrey Pennington Vinay Rao Jascha Narain Sohl-Dickstein S. Schoenholz 186 184 0 21 Feb 2019
Deep learning for pedestrians: backpropagation in CNNs L. Boué 3DV PINN 113 6 0 29 Nov 2018
Stochastic Normalizations as Bayesian Learning Alexander Shekhovtsov B. Flach UQCV BDL 128 16 0 01 Nov 2018
Towards Understanding Regularization in Batch Normalization Ping Luo Xinjiang Wang Wenqi Shao Zhanglin Peng MLT AI4CE 284 187 0 04 Sep 2018
Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimization Jonas Köhler Hadi Daneshmand Aurelien Lucchi M. Zhou K. Neymeyr Thomas Hofmann 240 95 0 27 May 2018
Normalization of Neural Networks using Analytic Variance Propagation Alexander Shekhovtsov B. Flach 125 6 0 28 Mar 2018
Norm matters: efficient and accurate normalization schemes in deep networks Elad Hoffer Ron Banner Itay Golan Daniel Soudry OffRL 228 185 0 05 Mar 2018