Improving the Trainability of Deep Neural Networks through Layerwise Batch-Entropy Regularization

1 August 2022

Papers citing "Improving the Trainability of Deep Neural Networks through Layerwise Batch-Entropy Regularization"

3 / 3 papers shown

Title
ReLU's Revival: On the Entropic Overload in Normalization-Free Large Language Models N. Jha Brandon Reagen OffRL AI4CE 33 0 0 12 Oct 2024
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping James Martens Andy Ballard Guillaume Desjardins G. Swirszcz Valentin Dalibard Jascha Narain Sohl-Dickstein S. Schoenholz 88 43 0 05 Oct 2021
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks Lechao Xiao Yasaman Bahri Jascha Narain Sohl-Dickstein S. Schoenholz Jeffrey Pennington 220 348 0 14 Jun 2018