Implicit Regularization in Deep Matrix Factorization

31 May 2019

Papers citing "Implicit Regularization in Deep Matrix Factorization"

45 / 45 papers shown

Title
Gradient Descent Robustly Learns the Intrinsic Dimension of Data in Training Convolutional Neural Networks Chenyang Zhang Peifeng Gao Difan Zou Yuan Cao OOD MLT 57 0 0 11 Apr 2025
Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries Chris Kolb T. Weber Bernd Bischl David Rügamer 97 0 0 04 Feb 2025
Optimization Insights into Deep Diagonal Linear Networks Hippolyte Labarrière C. Molinari Lorenzo Rosasco S. Villa Cristian Vega 66 0 0 21 Dec 2024
From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks Clémentine Dominé Nicolas Anguita A. Proca Lukas Braun D. Kunin P. Mediano Andrew M. Saxe 27 3 0 22 Sep 2024
Multiple Rotation Averaging with Constrained Reweighting Deep Matrix Factorization Shiqi Li Jihua Zhu Yifan Xie Naiwen Hu Mingchen Zhu Zhongyu Li Di Wang 41 0 0 15 Sep 2024
How Neural Networks Learn the Support is an Implicit Regularization Effect of SGD Pierfrancesco Beneventano Andrea Pinto Tomaso A. Poggio MLT 24 1 0 17 Jun 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation Can Yaras Peng Wang Laura Balzano Qing Qu AI4CE 24 12 0 06 Jun 2024
Pretraining with Random Noise for Fast and Robust Learning without Weight Transport Jeonghwan Cheon Sang Wan Lee Se-Bum Paik OOD 43 1 0 27 May 2024
Connectivity Shapes Implicit Regularization in Matrix Factorization Models for Matrix Completion Zhiwei Bai Jiajie Zhao Yaoyu Zhang AI4CE 23 0 0 22 May 2024
How Do Recommendation Models Amplify Popularity Bias? An Analysis from the Spectral Perspective Siyi Lin Chongming Gao Jiawei Chen Sheng Zhou Binbin Hu Yan Feng Chun-Yen Chen Can Wang 16 5 0 18 Apr 2024
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations Akshay Kumar Jarvis D. Haupt ODL 40 3 0 12 Mar 2024
Neural Redshift: Random Networks are not Random Functions Damien Teney A. Nicolicioiu Valentin Hartmann Ehsan Abbasnejad 86 18 0 04 Mar 2024
The Expected Loss of Preconditioned Langevin Dynamics Reveals the Hessian Rank Amitay Bar Rotem Mulayoff T. Michaeli Ronen Talmon 35 0 0 21 Feb 2024
Implicit Bias and Fast Convergence Rates for Self-attention Bhavya Vasudeva Puneesh Deora Christos Thrampoulidis 21 13 0 08 Feb 2024
Enhancing Cross-Category Learning in Recommendation Systems with Multi-Layer Embedding Training Selim F. Yilmaz Benjamin Ghaemmaghami A. Singh Benjamin Cho Leo Orshansky Lei Deng Michael Orshansky AI4TS 11 0 0 27 Sep 2023
Critical Learning Periods Emerge Even in Deep Linear Networks Michael Kleinman Alessandro Achille Stefano Soatto 26 3 0 23 Aug 2023
Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization Hancheng Min Enrique Mallada René Vidal MLT 17 19 0 24 Jul 2023
The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks Yuan Cao Difan Zou Yuan-Fang Li Quanquan Gu MLT 19 5 0 20 Jun 2023
Robust Sparse Mean Estimation via Incremental Learning Jianhao Ma Ruidi Chen Yinghui He S. Fattahi Wei Hu 11 0 0 24 May 2023
ReLU Neural Networks with Linear Layers are Biased Towards Single- and Multi-Index Models Suzanna Parkinson Greg Ongie Rebecca Willett 48 6 0 24 May 2023
Convergence of Alternating Gradient Descent for Matrix Factorization R. Ward T. Kolda 19 6 0 11 May 2023
Robust Implicit Regularization via Weight Normalization H. Chou Holger Rauhut Rachel A. Ward 15 7 0 09 May 2023
Saddle-to-Saddle Dynamics in Diagonal Linear Networks Scott Pesme Nicolas Flammarion 17 35 0 02 Apr 2023
Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss Pierre Bréchet Katerina Papagiannouli Jing An Guido Montúfar 10 3 0 06 Mar 2023
Generalization on the Unseen, Logic Reasoning and Degree Curriculum Emmanuel Abbe Samy Bengio Aryo Lotfi Kevin Rizk LRM 10 47 0 30 Jan 2023
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing Jikai Jin Zhiyuan Li Kaifeng Lyu S. Du Jason D. Lee MLT 22 34 0 27 Jan 2023
Task Discovery: Finding the Tasks that Neural Networks Generalize on Andrei Atanov Andrei Filatov Teresa Yeo Ajay Sohmshetty Amir Zamir OOD 20 10 0 01 Dec 2022
Deep Linear Networks can Benignly Overfit when Shallow Ones Do Niladri S. Chatterji Philip M. Long 8 8 0 19 Sep 2022
On the Implicit Bias in Deep-Learning Algorithms Gal Vardi FedML AI4CE 19 72 0 26 Aug 2022
Blessing of Nonconvexity in Deep Linear Models: Depth Flattens the Optimization Landscape Around the True Solution Jianhao Ma S. Fattahi 17 5 0 15 Jul 2022
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent Zhiyuan Li Tianhao Wang Jason D. Lee Sanjeev Arora 19 27 0 08 Jul 2022
Reconstructing Training Data from Trained Neural Networks Niv Haim Gal Vardi Gilad Yehudai Ohad Shamir Michal Irani 11 130 0 15 Jun 2022
Non-convex online learning via algorithmic equivalence Udaya Ghai Zhou Lu Elad Hazan 8 8 0 30 May 2022
Do Residual Neural Networks discretize Neural Ordinary Differential Equations? Michael E. Sander Pierre Ablin Gabriel Peyré 8 25 0 29 May 2022
Gaussian Pre-Activations in Neural Networks: Myth or Reality? Pierre Wolinski Julyan Arbel AI4CE 65 8 0 24 May 2022
The Mechanism of Prediction Head in Non-contrastive Self-supervised Learning Zixin Wen Yuanzhi Li SSL 16 34 0 12 May 2022
Machine Learning and Deep Learning -- A review for Ecologists Maximilian Pichler F. Hartig 8 119 0 11 Apr 2022
Side Effects of Learning from Low-dimensional Data Embedded in a Euclidean Space Juncai He R. Tsai Rachel A. Ward 20 8 0 01 Mar 2022
Thinking Outside the Ball: Optimal Learning with Gradient Descent for Generalized Linear Stochastic Convex Optimization I Zaghloul Amir Roi Livni Nathan Srebro 6 6 0 27 Feb 2022
Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias Frederic Koehler Viraj Mehta Chenghui Zhou Andrej Risteski DRL 13 12 0 13 Dec 2021
Learning Rich Nearest Neighbor Representations from Self-supervised Ensembles Bram Wallace Devansh Arpit Huan Wang Caiming Xiong SSL OOD 10 0 0 19 Oct 2021
Parallel Deep Neural Networks Have Zero Duality Gap Yifei Wang Tolga Ergen Mert Pilanci 62 10 0 13 Oct 2021
Gradient Starvation: A Learning Proclivity in Neural Networks Mohammad Pezeshki Sekouba Kaba Yoshua Bengio Aaron Courville Doina Precup Guillaume Lajoie MLT 29 257 0 18 Nov 2020
Machine learning for neural decoding Joshua I. Glaser Ari S. Benjamin Raeed H. Chowdhury M. Perich L. Miller Konrad Paul Kording 19 234 0 02 Aug 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,696 0 15 Sep 2016