Implicit Regularization in Matrix Factorization

25 May 2017

Srinadh Bhojanapalli

Papers citing "Implicit Regularization in Matrix Factorization"

50 / 133 papers shown

Title
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent Zhiyuan Li Tianhao Wang Jason D. Lee Sanjeev Arora 45 28 0 08 Jul 2022
Orthogonal Matrix Retrieval with Spatial Consensus for 3D Unknown-View Tomography Shuai Huang Mona Zehni Ivan Dokmanić Zhizhen Zhao 35 12 0 06 Jul 2022
Tensor-on-Tensor Regression: Riemannian Optimization, Over-parameterization, Statistical-computational Gap, and Their Interplay Yuetian Luo Anru R. Zhang 33 19 0 17 Jun 2022
Benefits of Additive Noise in Composing Classes with Bounded Capacity A. F. Pour H. Ashtiani 35 3 0 14 Jun 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction Kaifeng Lyu Zhiyuan Li Sanjeev Arora FAtt 52 71 0 14 Jun 2022
Efficient Scheduling of Data Augmentation for Deep Reinforcement Learning Byungchan Ko Jungseul Ok OnRL 27 5 0 01 Jun 2022
Unraveling Attention via Convex Duality: Analysis and Interpretations of Vision Transformers Arda Sahiner Tolga Ergen Batu Mehmet Ozturkler John M. Pauly Morteza Mardani Mert Pilanci 43 33 0 17 May 2022
projUNN: efficient method for training deep networks with unitary matrices B. Kiani Randall Balestriero Yann LeCun S. Lloyd 54 32 0 10 Mar 2022
Robust Training under Label Noise by Over-parameterization Sheng Liu Zhihui Zhu Qing Qu Chong You NoLa OOD 34 106 0 28 Feb 2022
Thinking Outside the Ball: Optimal Learning with Gradient Descent for Generalized Linear Stochastic Convex Optimization I Zaghloul Amir Roi Livni Nathan Srebro 32 6 0 27 Feb 2022
Implicit Regularization in Hierarchical Tensor Factorization and Deep Convolutional Neural Networks Noam Razin Asaf Maman Nadav Cohen 49 29 0 27 Jan 2022
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks Benjamin Bowman Guido Montúfar 28 11 0 12 Jan 2022
On generalization bounds for deep networks based on loss surface implicit regularization Masaaki Imaizumi Johannes Schmidt-Hieber ODL 34 3 0 12 Jan 2022
Benign Overfitting in Adversarially Robust Linear Classification Jinghui Chen Yuan Cao Quanquan Gu AAML SILM 34 10 0 31 Dec 2021
Over-Parametrized Matrix Factorization in the Presence of Spurious Stationary Points Armin Eftekhari 24 1 0 25 Dec 2021
Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias Frederic Koehler Viraj Mehta Chenghui Zhou Andrej Risteski DRL 41 12 0 13 Dec 2021
DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization Aviral Kumar Rishabh Agarwal Tengyu Ma Aaron Courville George Tucker Sergey Levine OffRL 31 65 0 09 Dec 2021
Depth Without the Magic: Inductive Bias of Natural Gradient Descent A. Kerekes Anna Mészáros Ferenc Huszár ODL 37 4 0 22 Nov 2021
Rethinking Neural vs. Matrix-Factorization Collaborative Filtering: the Theoretical Perspectives Zida Cheng Chuanwei Ruan Siheng Chen Sushant Kumar Ya Zhang 27 16 0 23 Oct 2021
Understanding Dimensional Collapse in Contrastive Self-supervised Learning Li Jing Pascal Vincent Yann LeCun Yuandong Tian SSL 25 339 0 18 Oct 2021
AIR-Net: Adaptive and Implicit Regularization Neural Network for Matrix Completion Zhemin Li Tao Sun Hongxia Wang Bao Wang 50 6 0 12 Oct 2021
Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect Yuqing Wang Minshuo Chen T. Zhao Molei Tao AI4CE 64 40 0 07 Oct 2021
On Margin Maximization in Linear and ReLU Networks Gal Vardi Ohad Shamir Nathan Srebro 52 28 0 06 Oct 2021
Powerpropagation: A sparsity inducing weight reparameterisation Jonathan Richard Schwarz Siddhant M. Jayakumar Razvan Pascanu P. Latham Yee Whye Teh 98 54 0 01 Oct 2021
Rank Overspecified Robust Matrix Recovery: Subgradient Method and Exact Recovery Lijun Ding Liwei Jiang Yudong Chen Qing Qu Zhihui Zhu 31 29 0 23 Sep 2021
Simple, Fast, and Flexible Framework for Matrix Completion with Infinite Width Neural Networks Adityanarayanan Radhakrishnan George Stefanakis M. Belkin Caroline Uhler 35 25 0 31 Jul 2021
SGD: The Role of Implicit Regularization, Batch-size and Multiple-epochs Satyen Kale Ayush Sekhari Karthik Sridharan 196 29 0 11 Jul 2021
Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction Dominik Stöger Mahdi Soltanolkotabi ODL 42 75 0 28 Jun 2021
RATT: Leveraging Unlabeled Data to Guarantee Generalization Saurabh Garg Sivaraman Balakrishnan J. Zico Kolter Zachary Chase Lipton 32 30 0 01 May 2021
Risk Bounds for Over-parameterized Maximum Margin Classification on Sub-Gaussian Mixtures Yuan Cao Quanquan Gu M. Belkin 11 51 0 28 Apr 2021
The Low-Rank Simplicity Bias in Deep Networks Minyoung Huh H. Mobahi Richard Y. Zhang Brian Cheung Pulkit Agrawal Phillip Isola 35 110 0 18 Mar 2021
Experiments with Rich Regime Training for Deep Learning Xinyan Li A. Banerjee 32 2 0 26 Feb 2021
On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent Shahar Azulay E. Moroshko Mor Shpigel Nacson Blake E. Woodworth Nathan Srebro Amir Globerson Daniel Soudry AI4CE 38 73 0 19 Feb 2021
Convergence of stochastic gradient descent schemes for Lojasiewicz-landscapes Steffen Dereich Sebastian Kassing 34 27 0 16 Feb 2021
SGD Generalizes Better Than GD (And Regularization Doesn't Help) I Zaghloul Amir Tomer Koren Roi Livni 29 46 0 01 Feb 2021
Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning Rishabh Agarwal Marlos C. Machado Pablo Samuel Castro Marc G. Bellemare OffRL 29 164 0 13 Jan 2021
Rank-One Measurements of Low-Rank PSD Matrices Have Small Feasible Sets T. Roddenberry Santiago Segarra Anastasios Kyrillidis 24 0 0 17 Dec 2020
Provable Benefits of Overparameterization in Model Compression: From Double Descent to Pruning Neural Networks Xiangyu Chang Yingcong Li Samet Oymak Christos Thrampoulidis 35 50 0 16 Dec 2020
NeurIPS 2020 Competition: Predicting Generalization in Deep Learning Yiding Jiang Pierre Foret Scott Yak Daniel M. Roy H. Mobahi Gintare Karolina Dziugaite Samy Bengio Suriya Gunasekar Isabelle M Guyon Behnam Neyshabur Google Research OOD 24 55 0 14 Dec 2020
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 52 259 0 18 Nov 2020
Direction Matters: On the Implicit Bias of Stochastic Gradient Descent with Moderate Learning Rate Jingfeng Wu Difan Zou Vladimir Braverman Quanquan Gu 18 18 0 04 Nov 2020
A case where a spindly two-layer linear network whips any neural network with a fully connected input layer Manfred K. Warmuth W. Kotłowski Ehsan Amid MLT 36 1 0 16 Oct 2020
A Unifying View on Implicit Bias in Training Linear Neural Networks Chulhee Yun Shankar Krishnan H. Mobahi MLT 26 80 0 06 Oct 2020
Quickly Finding a Benign Region via Heavy Ball Momentum in Non-Convex Optimization Jun-Kun Wang Jacob D. Abernethy 24 7 0 04 Oct 2020
Deep matrix factorizations Pierre De Handschutter Nicolas Gillis Xavier Siebert BDL 32 40 0 01 Oct 2020
A Scalable, Adaptive and Sound Nonconvex Regularizer for Low-rank Matrix Completion Yaqing Wang Quanming Yao James T. Kwok 24 0 0 14 Aug 2020
Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy E. Moroshko Suriya Gunasekar Blake E. Woodworth Jason D. Lee Nathan Srebro Daniel Soudry 35 85 0 13 Jul 2020
Principal Component Networks: Parameter Reduction Early in Training R. Waleffe Theodoros Rekatsinas 3DPC 19 9 0 23 Jun 2020
Exploring Weight Importance and Hessian Bias in Model Pruning Mingchen Li Yahya Sattar Christos Thrampoulidis Samet Oymak 30 3 0 19 Jun 2020
When Does Preconditioning Help or Hurt Generalization? S. Amari Jimmy Ba Roger C. Grosse Xuechen Li Atsushi Nitanda Taiji Suzuki Denny Wu Ji Xu 36 32 0 18 Jun 2020