How Does Batch Normalization Help Optimization?

29 May 2018

Papers citing "How Does Batch Normalization Help Optimization?"

38 / 138 papers shown

Title
Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks Soham De Samuel L. Smith ODL 11 20 0 24 Feb 2020
The Early Phase of Neural Network Training Jonathan Frankle D. Schwab Ari S. Morcos 11 170 0 24 Feb 2020
RatLesNetv2: A Fully Convolutional Network for Rodent Brain Lesion Segmentation Juan Miguel Valverde Artem Shatillo Riccardo De Feo O. Gröhn Alejandra Sierra Jussi Tohka MedIm 18 21 0 24 Jan 2020
Empirical Studies on the Properties of Linear Regions in Deep Neural Networks Xiao Zhang Dongrui Wu 6 38 0 04 Jan 2020
SM-NAS: Structural-to-Modular Neural Architecture Search for Object Detection Lewei Yao Hang Xu Wei Zhang Xiaodan Liang Zhenguo Li 13 79 0 22 Nov 2019
Understanding and Improving Layer Normalization Jingjing Xu Xu Sun Zhiyuan Zhang Guangxiang Zhao Junyang Lin FAtt 18 338 0 16 Nov 2019
Streaming convolutional neural networks for end-to-end learning with multi-megapixel images H. Pinckaers Bram van Ginneken G. Litjens MedIm 11 93 0 11 Nov 2019
Turbo Autoencoder: Deep learning based channel codes for point-to-point communication channels Yihan Jiang Hyeji Kim Himanshu Asnani Sreeram Kannan Sewoong Oh Pramod Viswanath 22 134 0 08 Nov 2019
Transformers without Tears: Improving the Normalization of Self-Attention Toan Q. Nguyen Julian Salazar 36 224 0 14 Oct 2019
The Non-IID Data Quagmire of Decentralized Machine Learning Kevin Hsieh Amar Phanishayee O. Mutlu Phillip B. Gibbons 6 555 0 01 Oct 2019
Towards Understanding the Transferability of Deep Representations Hong Liu Mingsheng Long Jianmin Wang Michael I. Jordan 16 25 0 26 Sep 2019
Instance Enhancement Batch Normalization: an Adaptive Regulator of Batch Noise Senwei Liang Zhongzhan Huang Mingfu Liang Haizhao Yang 25 57 0 12 Aug 2019
Attentive Normalization Xilai Li Wei Sun Tianfu Wu OOD ViT 20 31 0 04 Aug 2019
Switchable Normalization for Learning-to-Normalize Deep Representation Ping Luo Ruimao Zhang Jiamin Ren Zhanglin Peng Jingyu Li 28 73 0 22 Jul 2019
Learning to Forget for Meta-Learning Sungyong Baik Seokil Hong Kyoung Mu Lee CLL KELM 11 87 0 13 Jun 2019
Principled Training of Neural Networks with Direct Feedback Alignment Julien Launay Iacopo Poli Florent Krzakala 16 35 0 11 Jun 2019
The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks Ryo Karakida S. Akaho S. Amari 19 39 0 07 Jun 2019
An Empirical Study on Hyperparameters and their Interdependence for RL Generalization Xingyou Song Yilun Du Jacob Jackson AI4CE 19 8 0 02 Jun 2019
Learning to learn via Self-Critique Antreas Antoniou Amos Storkey SSL 21 17 0 24 May 2019
Deep Neural Networks for Marine Debris Detection in Sonar Images Matias Valdenegro-Toro 27 25 0 13 May 2019
Nested Variational Autoencoder for Topic Modeling on Microtexts with Word Vectors Trung Trinh Tho Quan Trung Mai BDL 9 2 0 01 May 2019
Deep Representation with ReLU Neural Networks Andreas Heinecke W. Hwang 18 0 0 29 Mar 2019
Micro-Batch Training with Batch-Channel Normalization and Weight Standardization Siyuan Qiao Huiyu Wang Chenxi Liu Wei Shen Alan Yuille MQ 17 144 0 25 Mar 2019
Mean-field Analysis of Batch Normalization Ming-Bo Wei J. Stokes D. Schwab MLT 17 8 0 06 Mar 2019
Towards Robust ResNet: A Small Step but A Giant Leap Jingfeng Zhang Bo Han L. Wynter K. H. Low Mohan S. Kankanhalli 16 41 0 28 Feb 2019
U-NetPlus: A Modified Encoder-Decoder U-Net Architecture for Semantic and Instance Segmentation of Surgical Instrument S. Hasan Cristian A. Linte MedIm 17 92 0 24 Feb 2019
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise Yeming Wen Kevin Luk Maxime Gazeau Guodong Zhang Harris Chan Jimmy Ba ODL 15 22 0 21 Feb 2019
Mode Normalization Lucas Deecke Iain Murray Hakan Bilen OOD 29 33 0 12 Oct 2018
Context-Aware Systems for Sequential Item Recommendation Moin Nadeem D. Stansbury Shane Mooney AI4Ed 14 2 0 21 Sep 2018
A Domain Agnostic Normalization Layer for Unsupervised Adversarial Domain Adaptation Rob Romijnders Panagiotis Meletis Gijs Dubbelman AI4CE 16 27 0 14 Sep 2018
Convolutional Neural Networks for the segmentation of microcalcification in Mammography Imaging Gabriele Valvano Gianmarco Santini N. Martini A. Ripoli C. Iacconi D. Chiappino D. Latta 11 51 0 11 Sep 2018
Towards Understanding Regularization in Batch Normalization Ping Luo Xinjiang Wang Wenqi Shao Zhanglin Peng MLT AI4CE 8 179 0 04 Sep 2018
Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks Nikola B. Kovachki Andrew M. Stuart BDL 38 136 0 10 Aug 2018
Troubling Trends in Machine Learning Scholarship Zachary Chase Lipton Jacob Steinhardt 15 288 0 09 Jul 2018
Restructuring Batch Normalization to Accelerate CNN Training Wonkyung Jung Daejin Jung and Byeongho Kim Sunjung Lee Wonjong Rhee Jung Ho Ahn 18 62 0 04 Jul 2018
Data augmentation instead of explicit regularization Alex Hernández-García Peter König 30 141 0 11 Jun 2018
Whitening and Coloring batch transform for GANs Aliaksandr Siarohin E. Sangineto N. Sebe 17 49 0 01 Jun 2018
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 278 2,888 0 15 Sep 2016