A Convergence Theory for Deep Learning via Over-Parameterization

9 November 2018

Papers citing "A Convergence Theory for Deep Learning via Over-Parameterization"

50 / 370 papers shown

Title
Deep Architecture Connectivity Matters for Its Convergence: A Fine-Grained Analysis Wuyang Chen Wei Huang Xinyu Gong Boris Hanin Zhangyang Wang 35 7 0 11 May 2022
Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD Konstantinos E. Nikolakakis Farzin Haddadpour Amin Karbasi Dionysios S. Kalogerias 43 17 0 26 Apr 2022
Beyond the Quadratic Approximation: the Multiscale Structure of Neural Network Loss Landscapes Chao Ma D. Kunin Lei Wu Lexing Ying 25 27 0 24 Apr 2022
On Feature Learning in Neural Networks with Global Convergence Guarantees Zhengdao Chen Eric Vanden-Eijnden Joan Bruna MLT 36 13 0 22 Apr 2022
On Convergence Lemma and Convergence Stability for Piecewise Analytic Functions Xiaotie Deng Hanyu Li Ningyuan Li 15 0 0 04 Apr 2022
$Training Fully Connected Neural Networks is $\exists\mathbb{R}$-Complete$ Training Fully Connected Neural Networks is $\exists\mathbb{R}$ -Complete Daniel Bertschinger Christoph Hertrich Paul Jungeblut Tillmann Miltzow Simon Weber OffRL 64 30 0 04 Apr 2022
Convergence of gradient descent for deep neural networks S. Chatterjee ODL 21 20 0 30 Mar 2022
Random matrix analysis of deep neural network weight matrices M. Thamm Max Staats B. Rosenow 37 12 0 28 Mar 2022
On the (Non-)Robustness of Two-Layer Neural Networks in Different Learning Regimes Elvis Dohmatob A. Bietti AAML 39 13 0 22 Mar 2022
TVConv: Efficient Translation Variant Convolution for Layout-aware Visual Processing Jie Chen Tianlang He Weipeng Zhuo Li Ma Sangtae Ha Shueng-Han Gary Chan CVBM 21 24 0 20 Mar 2022
On the Spectral Bias of Convolutional Neural Tangent and Gaussian Process Kernels Amnon Geifman Meirav Galun David Jacobs Ronen Basri 30 13 0 17 Mar 2022
Global Convergence of MAML and Theory-Inspired Neural Architecture Search for Few-Shot Learning Haoxiang Wang Yite Wang Ruoyu Sun Bo-wen Li 33 27 0 17 Mar 2022
Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning Weixin Liang Yuhui Zhang Yongchan Kwon Serena Yeung James Zou VLM 52 394 0 03 Mar 2022
The Spectral Bias of Polynomial Neural Networks Moulik Choraria L. Dadi Grigorios G. Chrysos Julien Mairal V. Cevher 24 18 0 27 Feb 2022
Sparse Neural Additive Model: Interpretable Deep Learning with Feature Selection via Group Sparsity Shiyun Xu Zhiqi Bu Pratik Chaudhari Ian Barnett 27 21 0 25 Feb 2022
Semi-Implicit Hybrid Gradient Methods with Application to Adversarial Robustness Beomsu Kim Junghoon Seo AAML 22 0 0 21 Feb 2022
Random Feature Amplification: Feature Learning and Generalization in Neural Networks Spencer Frei Niladri S. Chatterji Peter L. Bartlett MLT 30 29 0 15 Feb 2022
Benchmark Assessment for DeepSpeed Optimization Library G. Liang I. Alsmadi 34 3 0 12 Feb 2022
Noise Regularizes Over-parameterized Rank One Matrix Recovery, Provably Tianyi Liu Yan Li Enlu Zhou Tuo Zhao 38 1 0 07 Feb 2022
Demystify Optimization and Generalization of Over-parameterized PAC-Bayesian Learning Wei Huang Chunrui Liu Yilan Chen Tianyu Liu R. Xu BDL MLT 19 2 0 04 Feb 2022
Neural Collaborative Filtering Bandits via Meta Learning Yikun Ban Yunzhe Qi Tianxin Wei Jingrui He OffRL 33 9 0 31 Jan 2022
Faster Convergence of Local SGD for Over-Parameterized Models Tiancheng Qin S. Rasoul Etesami César A. Uribe FedML 38 6 0 30 Jan 2022
Improved Overparametrization Bounds for Global Convergence of Stochastic Gradient Descent for Shallow Neural Networks Bartlomiej Polaczyk J. Cyranka ODL 33 3 0 28 Jan 2022
Learning Proximal Operators to Discover Multiple Optima Lingxiao Li Noam Aigerman Vladimir G. Kim Jiajin Li Kristjan Greenewald Mikhail Yurochkin Justin Solomon 47 1 0 28 Jan 2022
A Kernel-Expanded Stochastic Neural Network Y. Sun F. Liang 23 5 0 14 Jan 2022
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks Benjamin Bowman Guido Montúfar 28 11 0 12 Jan 2022
In Defense of the Unitary Scalarization for Deep Multi-Task Learning Vitaly Kurin Alessandro De Palma Ilya Kostrikov Shimon Whiteson M. P. Kumar 39 74 0 11 Jan 2022
Stochastic Weight Averaging Revisited Hao Guo Jiyong Jin B. Liu 35 29 0 03 Jan 2022
Separation of Scales and a Thermodynamic Description of Feature Learning in Some CNNs Inbar Seroussi Gadi Naveh Zohar Ringel 35 51 0 31 Dec 2021
Over-Parametrized Matrix Factorization in the Presence of Spurious Stationary Points Armin Eftekhari 24 1 0 25 Dec 2021
On the Convergence and Robustness of Adversarial Training Yisen Wang Xingjun Ma James Bailey Jinfeng Yi Bowen Zhou Quanquan Gu AAML 212 345 0 15 Dec 2021
Rethinking Influence Functions of Neural Networks in the Over-parameterized Regime Rui Zhang Shihua Zhang TDI 27 21 0 15 Dec 2021
Training Multi-Layer Over-Parametrized Neural Network in Subquadratic Time Zhao Song Licheng Zhang Ruizhe Zhang 32 64 0 14 Dec 2021
Convergence proof for stochastic gradient descent in the training of deep neural networks with ReLU activation for constant target functions Martin Hutzenthaler Arnulf Jentzen Katharina Pohl Adrian Riekert Luca Scarpa MLT 34 6 0 13 Dec 2021
A New Measure of Model Redundancy for Compressed Convolutional Neural Networks Feiqing Huang Yuefeng Si Yao Zheng Guodong Li 39 1 0 09 Dec 2021
On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons Fangshuo Liao Anastasios Kyrillidis 49 16 0 05 Dec 2021
Fast Graph Neural Tangent Kernel via Kronecker Sketching Shunhua Jiang Yunze Man Zhao Song Zheng Yu Danyang Zhuo 29 6 0 04 Dec 2021
Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models Tri Dao Beidi Chen Kaizhao Liang Jiaming Yang Zhao Song Atri Rudra Christopher Ré 33 75 0 30 Nov 2021
Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization Thanh Nguyen-Tang Sunil R. Gupta A. Nguyen Svetha Venkatesh OffRL 29 29 0 27 Nov 2021
On the Equivalence between Neural Network and Support Vector Machine Yilan Chen Wei Huang Lam M. Nguyen Tsui-Wei Weng AAML 25 18 0 11 Nov 2021
SGD Through the Lens of Kolmogorov Complexity Gregory Schwartzman 35 1 0 10 Nov 2021
Subquadratic Overparameterization for Shallow Neural Networks Chaehwan Song Ali Ramezani-Kebrya Thomas Pethick Armin Eftekhari V. Cevher 30 31 0 02 Nov 2021
Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives Zida Cheng Chuanwei Ruan Siheng Chen Sushant Kumar Ya Zhang 27 16 0 23 Oct 2021
Quantifying Epistemic Uncertainty in Deep Learning Ziyi Huang H. Lam Haofeng Zhang UQCV BDL UD PER 24 12 0 23 Oct 2021
S-Cyc: A Learning Rate Schedule for Iterative Pruning of ReLU-based Networks Shiyu Liu Chong Min John Tan Mehul Motani CLL 29 4 0 17 Oct 2021
Provable Regret Bounds for Deep Online Learning and Control Xinyi Chen Edgar Minasyan Jason D. Lee Elad Hazan 36 6 0 15 Oct 2021
Why Lottery Ticket Wins? A Theoretical Perspective of Sample Complexity on Pruned Neural Networks Shuai Zhang Meng Wang Sijia Liu Pin-Yu Chen Jinjun Xiong UQCV MLT 31 13 0 12 Oct 2021
A global convergence theory for deep ReLU implicit networks via over-parameterization Tianxiang Gao Hailiang Liu Jia Liu Hridesh Rajan Hongyang Gao MLT 31 16 0 11 Oct 2021
Imitating Deep Learning Dynamics via Locally Elastic Stochastic Differential Equations Jiayao Zhang Hua Wang Weijie J. Su 35 8 0 11 Oct 2021
Does Preprocessing Help Training Over-parameterized Neural Networks? Zhao Song Shuo Yang Ruizhe Zhang 38 49 0 09 Oct 2021