Geometry of Optimization and Implicit Regularization in Deep Learning

8 May 2017

Papers citing "Geometry of Optimization and Implicit Regularization in Deep Learning"

32 / 32 papers shown

Title
Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning Subhojyoti Mukherjee Josiah P. Hanna Qiaomin Xie Robert Nowak 89 2 0 07 Jun 2024
PRIOR: Personalized Prior for Reactivating the Information Overlooked in Federated Learning Mingjia Shi Yuhao Zhou Kai Wang Huaizheng Zhang Shudong Huang Qing Ye Jiangcheng Lv 34 10 0 13 Oct 2023
Robust Implicit Regularization via Weight Normalization H. Chou Holger Rauhut Rachel A. Ward 42 7 0 09 May 2023
Convex Dual Theory Analysis of Two-Layer Convolutional Neural Networks with Soft-Thresholding Chunyan Xiong Meng Lu Xiaotong Yu JIAN-PENG Cao Zhong Chen D. Guo X. Qu MLT 43 0 0 14 Apr 2023
Iterative regularization in classification via hinge loss diagonal descent Vassilis Apidopoulos T. Poggio Lorenzo Rosasco S. Villa 34 2 0 24 Dec 2022
How Much Data Are Augmentations Worth? An Investigation into Scaling Laws, Invariance, and Implicit Regularization Jonas Geiping Micah Goldblum Gowthami Somepalli Ravid Shwartz-Ziv Tom Goldstein A. Wilson 31 35 0 12 Oct 2022
Fast Convex Optimization for Two-Layer ReLU Networks: Equivalent Model Classes and Cone Decompositions Aaron Mishkin Arda Sahiner Mert Pilanci OffRL 77 30 0 02 Feb 2022
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks Benjamin Bowman Guido Montúfar 28 11 0 12 Jan 2022
What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory Rahul Parhi Robert D. Nowak MLT 38 70 0 07 May 2021
The Discovery of Dynamics via Linear Multistep Methods and Deep Learning: Error Estimation Q. Du Yiqi Gu Haizhao Yang Chao Zhou 26 20 0 21 Mar 2021
Reproducing Activation Function for Deep Learning Senwei Liang Liyao Lyu Chunmei Wang Haizhao Yang 36 21 0 13 Jan 2021
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks Xiangyu Chang Yingcong Li Samet Oymak Christos Thrampoulidis 37 50 0 16 Dec 2020
NeurIPS 2020 Competition: Predicting Generalization in Deep Learning Yiding Jiang Pierre Foret Scott Yak Daniel M. Roy H. Mobahi Gintare Karolina Dziugaite Samy Bengio Suriya Gunasekar Isabelle M Guyon Behnam Neyshabur Google Research OOD 24 55 0 14 Dec 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 52 259 0 18 Nov 2020
Effective Regularization Through Loss-Function Metalearning Santiago Gonzalez Risto Miikkulainen 34 5 0 02 Oct 2020
Geometric compression of invariant manifolds in neural nets J. Paccolat Leonardo Petrini Mario Geiger Kevin Tyloo Matthieu Wyart MLT 60 34 0 22 Jul 2020
A theory of independent mechanisms for extrapolation in generative models M. Besserve Rémy Sun Dominik Janzing Bernhard Schölkopf 28 25 0 01 Apr 2020
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime Stéphane dÁscoli Maria Refinetti Giulio Biroli Florent Krzakala 98 152 0 02 Mar 2020
The Local Elasticity of Neural Networks Hangfeng He Weijie J. Su 45 44 0 15 Oct 2019
Does Learning Require Memorization? A Short Tale about a Long Tail Vitaly Feldman TDI 61 483 0 12 Jun 2019
The Implicit Bias of AdaGrad on Separable Data Qian Qian Xiaoyuan Qian 37 23 0 09 Jun 2019
Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models Mor Shpigel Nacson Suriya Gunasekar Jason D. Lee Nathan Srebro Daniel Soudry 33 92 0 17 May 2019
From Variational to Deterministic Autoencoders Partha Ghosh Mehdi S. M. Sajjadi Antonio Vergari Michael J. Black Bernhard Schölkopf DRL 37 269 0 29 Mar 2019
Copying Machine Learning Classifiers Irene Unceta Jordi Nin O. Pujol 14 18 0 05 Mar 2019
An Empirical Study of Large-Batch Stochastic Gradient Descent with Structured Covariance Noise Yeming Wen Kevin Luk Maxime Gazeau Guodong Zhang Harris Chan Jimmy Ba ODL 25 22 0 21 Feb 2019
Scaling description of generalization with number of parameters in deep learning Mario Geiger Arthur Jacot S. Spigler Franck Gabriel Levent Sagun Stéphane dÁscoli Giulio Biroli Clément Hongler Matthieu Wyart 52 195 0 06 Jan 2019
Approximation and Estimation for High-Dimensional Deep Learning Networks Andrew R. Barron Jason M. Klusowski 27 59 0 10 Sep 2018
Stochastic Gradient/Mirror Descent: Minimax Optimality and Implicit Regularization Navid Azizan B. Hassibi 24 61 0 04 Jun 2018
Characterizing Implicit Bias in Terms of Optimization Geometry Suriya Gunasekar Jason D. Lee Daniel Soudry Nathan Srebro AI4CE 46 399 0 22 Feb 2018
Implicit Regularization in Matrix Factorization Suriya Gunasekar Blake E. Woodworth Srinadh Bhojanapalli Behnam Neyshabur Nathan Srebro 36 487 0 25 May 2017
Deep Reinforcement Learning: An Overview Yuxi Li OffRL VLM 106 1,505 0 25 Jan 2017
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 127 577 0 27 Feb 2015