Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks

24 February 2020

Papers citing "Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks"

7 / 7 papers shown

Title
Speeding up Deep Model Training by Sharing Weights and Then Unsharing Shuo Yang Le Hou Xiaodan Song Qiang Liu Denny Zhou 110 9 0 08 Oct 2021
The Traveling Observer Model: Multi-task Learning Through Spatial Variable Embeddings Elliot Meyerson Risto Miikkulainen 11 12 0 05 Oct 2020
Superkernel Neural Architecture Search for Image Denoising Marcin Mo.zejko Tomasz Latkowski Lukasz Treszczotko Michal Szafraniuk K. Trojanowski SupR 20 16 0 19 Apr 2020
Evolving Normalization-Activation Layers Hanxiao Liu Andrew Brock Karen Simonyan Quoc V. Le 12 79 0 06 Apr 2020
WaveCRN: An Efficient Convolutional Recurrent Neural Network for End-to-end Speech Enhancement Tsun-An Hsieh Hsin-Min Wang Xugang Lu Yu Tsao 40 60 0 06 Apr 2020
Pipelined Backpropagation at Scale: Training Large Models without Batches Atli Kosson Vitaliy Chiley Abhinav Venigalla Joel Hestness Urs Koster 30 33 0 25 Mar 2020
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 220 348 0 14 Jun 2018