Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks

24 January 2019

Papers citing "Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks"

50 / 192 papers shown

Title
Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives Zida Cheng Chuanwei Ruan Siheng Chen Sushant Kumar Ya-Qin Zhang 14 16 0 23 Oct 2021
Provable Regret Bounds for Deep Online Learning and Control Xinyi Chen Edgar Minasyan Jason D. Lee Elad Hazan 21 6 0 15 Oct 2021
Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations Jiayao Zhang Hua Wang Weijie J. Su 27 7 0 11 Oct 2021
Does Preprocessing Help Training Over-parameterized Neural Networks? Zhao-quan Song Shuo Yang Ruizhe Zhang 27 49 0 09 Oct 2021
On the Impact of Stable Ranks in Deep Nets B. Georgiev L. Franken Mayukh Mukherjee Georgios Arvanitidis 13 3 0 05 Oct 2021
Assessing the Quality of the Datasets by Identifying Mislabeled Samples Vaibhav Pulastya Gaurav Nuti Yash Kumar Atri Tanmoy Chakraborty NoLa 25 5 0 10 Sep 2021
NASI: Label- and Data-agnostic Neural Architecture Search at Initialization Yao Shu Shaofeng Cai Zhongxiang Dai Beng Chin Ooi K. H. Low 14 43 0 02 Sep 2021
Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization Difan Zou Yuan Cao Yuanzhi Li Quanquan Gu MLT AI4CE 41 37 0 25 Aug 2021
A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective? Hiroaki Mikami Kenji Fukumizu Shogo Murai Shuji Suzuki Yuta Kikuchi Taiji Suzuki S. Maeda Kohei Hayashi 38 12 0 25 Aug 2021
A Theoretical Analysis of Fine-tuning with Linear Teachers Gal Shachaf Alon Brutzkus Amir Globerson 26 17 0 04 Jul 2021
The Values Encoded in Machine Learning Research Abeba Birhane Pratyusha Kalluri Dallas Card William Agnew Ravit Dotan Michelle Bao 25 273 0 29 Jun 2021
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction Dominik Stöger Mahdi Soltanolkotabi ODL 31 74 0 28 Jun 2021
Precise characterization of the prior predictive distribution of deep ReLU networks Lorenzo Noci Gregor Bachmann Kevin Roth Sebastian Nowozin Thomas Hofmann BDL UQCV 16 32 0 11 Jun 2021
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective Geoff Pleiss John P. Cunningham 26 24 0 11 Jun 2021
FL-NTK: A Neural Tangent Kernel-based Framework for Federated Learning Convergence Analysis Baihe Huang Xiaoxiao Li Zhao-quan Song Xin Yang FedML 23 16 0 11 May 2021
RATT: Leveraging Unlabeled Data to Guarantee Generalization Saurabh Garg Sivaraman Balakrishnan J. Zico Kolter Zachary Chase Lipton 25 29 0 01 May 2021
Generalization Guarantees for Neural Architecture Search with Train-Validation Split Samet Oymak Mingchen Li Mahdi Soltanolkotabi AI4CE OOD 31 13 0 29 Apr 2021
A Neural Pre-Conditioning Active Learning Algorithm to Reduce Label Complexity Seo Taek Kong Soomin Jeon Dongbin Na Jaewon Lee Honglak Lee Kyu-Hwan Jung 13 6 0 08 Apr 2021
LocalDrop: A Hybrid Regularization for Deep Neural Networks Ziqing Lu Chang Xu Bo Du Takashi Ishida L. Zhang Masashi Sugiyama 19 14 0 01 Mar 2021
Experiments with Rich Regime Training for Deep Learning Xinyan Li A. Banerjee 21 2 0 26 Feb 2021
Learning with invariances in random features and kernel models Song Mei Theodor Misiakiewicz Andrea Montanari OOD 46 89 0 25 Feb 2021
On the Validity of Modeling SGD with Stochastic Differential Equations (SDEs) Zhiyuan Li Sadhika Malladi Sanjeev Arora 33 78 0 24 Feb 2021
A linearized framework and a new benchmark for model selection for fine-tuning Aditya Deshpande Alessandro Achille Avinash Ravichandran Hao Li L. Zancato Charless C. Fowlkes Rahul Bhotika Stefano Soatto Pietro Perona ALM 107 46 0 29 Jan 2021
Reproducing Activation Function for Deep Learning Senwei Liang Liyao Lyu Chunmei Wang Haizhao Yang 28 21 0 13 Jan 2021
Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning Zeyuan Allen-Zhu Yuanzhi Li FedML 16 354 0 17 Dec 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 45 257 0 18 Nov 2020
On Function Approximation in Reinforcement Learning: Optimism in the Face of Large State Spaces Zhuoran Yang Chi Jin Zhaoran Wang Mengdi Wang Michael I. Jordan 18 18 0 09 Nov 2020
A Bayesian Perspective on Training Speed and Model Selection Clare Lyle Lisa Schut Binxin Ru Y. Gal Mark van der Wilk 31 23 0 27 Oct 2020
Continual Learning in Low-rank Orthogonal Subspaces Arslan Chaudhry Naeemullah Khan P. Dokania Philip H. S. Torr CLL 33 113 0 22 Oct 2020
Deep Learning is Singular, and That's Good Daniel Murfet Susan Wei Mingming Gong Hui Li Jesse Gell-Redman T. Quella UQCV 16 26 0 22 Oct 2020
Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teacher Guangda Ji Zhanxing Zhu 51 42 0 20 Oct 2020
For self-supervised learning, Rationality implies generalization, provably Yamini Bansal Gal Kaplun Boaz Barak OOD SSL 52 22 0 16 Oct 2020
Regularizing Neural Networks via Adversarial Model Perturbation Yaowei Zheng Richong Zhang Yongyi Mao AAML 22 95 0 10 Oct 2020
On the linearity of large non-linear models: when and why the tangent kernel is constant Chaoyue Liu Libin Zhu M. Belkin 14 138 0 02 Oct 2020
Deep Equals Shallow for ReLU Networks in Kernel Regimes A. Bietti Francis R. Bach 12 86 0 30 Sep 2020
Sanity-Checking Pruning Methods: Random Tickets can Win the Jackpot Jingtong Su Yihang Chen Tianle Cai Tianhao Wu Ruiqi Gao Liwei Wang J. Lee 6 85 0 22 Sep 2020
Deep Neural Tangent Kernel and Laplace Kernel Have the Same RKHS Lin Chen Sheng Xu 19 93 0 22 Sep 2020
Generalized Leverage Score Sampling for Neural Networks J. Lee Ruoqi Shen Zhao-quan Song Mengdi Wang Zheng Yu 13 42 0 21 Sep 2020
Predicting Training Time Without Training L. Zancato Alessandro Achille Avinash Ravichandran Rahul Bhotika Stefano Soatto 18 24 0 28 Aug 2020
Deep Networks and the Multiple Manifold Problem Sam Buchanan D. Gilboa John N. Wright 166 39 0 25 Aug 2020
Multiple Descent: Design Your Own Generalization Curve Lin Chen Yifei Min M. Belkin Amin Karbasi DRL 18 61 0 03 Aug 2020
Single-Timescale Actor-Critic Provably Finds Globally Optimal Policy Zuyue Fu Zhuoran Yang Zhaoran Wang 15 42 0 02 Aug 2020
The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training Andrea Montanari Yiqiao Zhong 36 95 0 25 Jul 2020
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy E. Moroshko Suriya Gunasekar Blake E. Woodworth J. Lee Nathan Srebro Daniel Soudry 27 85 0 13 Jul 2020
Provably Efficient Neural Estimation of Structural Equation Model: An Adversarial Approach Luofeng Liao You-Lin Chen Zhuoran Yang Bo Dai Zhaoran Wang Mladen Kolar 22 32 0 02 Jul 2020
Generalisation Guarantees for Continual Learning with Orthogonal Gradient Descent Mehdi Abbana Bennani Thang Doan Masashi Sugiyama CLL 42 61 0 21 Jun 2020
Breaking the Curse of Many Agents: Provable Mean Embedding Q-Iteration for Mean-Field Reinforcement Learning Lingxiao Wang Zhuoran Yang Zhaoran Wang 16 26 0 21 Jun 2020
Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains Matthew Tancik Pratul P. Srinivasan B. Mildenhall Sara Fridovich-Keil N. Raghavan Utkarsh Singhal R. Ramamoorthi Jonathan T. Barron Ren Ng 60 2,335 0 18 Jun 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 28 32 0 18 Jun 2020
Shape Matters: Understanding the Implicit Bias of the Noise Covariance Jeff Z. HaoChen Colin Wei J. Lee Tengyu Ma 18 93 0 15 Jun 2020