Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergence

30 May 2019

Papers citing "Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergence"

16 / 16 papers shown

Title
Better Schedules for Low Precision Training of Deep Neural Networks Cameron R. Wolfe Anastasios Kyrillidis 45 1 0 04 Mar 2024
Critical Learning Periods Emerge Even in Deep Linear Networks Michael Kleinman Alessandro Achille Stefano Soatto 34 3 0 23 Aug 2023
FUN with Fisher: Improving Generalization of Adapter-Based Cross-lingual Transfer with Scheduled Unfreezing Chen Cecilia Liu Jonas Pfeiffer Ivan Vulić Iryna Gurevych CLL 24 9 0 13 Jan 2023
A Comprehensive Survey of Data Augmentation in Visual Reinforcement Learning Guozheng Ma Zhen Wang Zhecheng Yuan Xueqian Wang Bo Yuan Dacheng Tao OffRL 35 26 0 10 Oct 2022
High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation Jimmy Ba Murat A. Erdogdu Taiji Suzuki Zhichao Wang Denny Wu Greg Yang MLT 31 121 0 03 May 2022
Cyclical Focal Loss L. Smith 30 14 0 16 Feb 2022
PRUNIX: Non-Ideality Aware Convolutional Neural Network Pruning for Memristive Accelerators Ali Alshaarawy A. Amirsoleimani R. Genov 11 1 0 03 Feb 2022
Consensus Control for Decentralized Deep Learning Lingjing Kong Tao R. Lin Anastasia Koloskova Martin Jaggi Sebastian U. Stich 19 75 0 09 Feb 2021
Advances in Electron Microscopy with Deep Learning Jeffrey M. Ede 29 2 0 04 Jan 2021
Usable Information and Evolution of Optimal Representations During Training Michael Kleinman Alessandro Achille Daksh Idnani J. Kao 21 13 0 06 Oct 2020
Effective Regularization Through Loss-Function Metalearning Santiago Gonzalez Risto Miikkulainen 24 5 0 02 Oct 2020
Review: Deep Learning in Electron Microscopy Jeffrey M. Ede 29 79 0 17 Sep 2020
The Early Phase of Neural Network Training Jonathan Frankle D. Schwab Ari S. Morcos 16 170 0 24 Feb 2020
The Break-Even Point on Optimization Trajectories of Deep Neural Networks Stanislaw Jastrzebski Maciej Szymczak Stanislav Fort Devansh Arpit Jacek Tabor Kyunghyun Cho Krzysztof J. Geras 42 154 0 21 Feb 2020
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 281 2,888 0 15 Sep 2016
The Loss Surfaces of Multilayer Networks A. Choromańska Mikael Henaff Michaël Mathieu Gerard Ben Arous Yann LeCun ODL 179 1,185 0 30 Nov 2014