ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2106.15013
  4. Cited By
Small random initialization is akin to spectral learning: Optimization
  and generalization guarantees for overparameterized low-rank matrix
  reconstruction

Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction

28 June 2021
Dominik Stöger
Mahdi Soltanolkotabi
    ODL
ArXivPDFHTML

Papers citing "Small random initialization is akin to spectral learning: Optimization and generalization guarantees for overparameterized low-rank matrix reconstruction"

50 / 53 papers shown
Title
Understanding the Learning Dynamics of LoRA: A Gradient Flow Perspective on Low-Rank Adaptation in Matrix Factorization
Ziqing Xu
Hancheng Min
Lachlan Ewen MacDonald
Jinqi Luo
Salma Tarmoun
Enrique Mallada
René Vidal
AI4CE
51
0
0
10 Mar 2025
$k$-SVD with Gradient Descent
kkk-SVD with Gradient Descent
Emily Gan
Yassir Jedra
Devavrat Shah
61
0
0
01 Feb 2025
Stability properties of gradient flow dynamics for the symmetric
  low-rank matrix factorization problem
Stability properties of gradient flow dynamics for the symmetric low-rank matrix factorization problem
Hesameddin Mohammadi
Mohammad Tinati
Stephen Tu
Mahdi Soltanolkotabi
M. Jovanović
73
0
0
24 Nov 2024
BLAST: Block-Level Adaptive Structured Matrices for Efficient Deep
  Neural Network Inference
BLAST: Block-Level Adaptive Structured Matrices for Efficient Deep Neural Network Inference
Changwoo Lee
Soo Min Kwon
Qing Qu
Hun-Seok Kim
25
0
0
28 Oct 2024
Convergence Guarantees for the DeepWalk Embedding on Block Models
Convergence Guarantees for the DeepWalk Embedding on Block Models
Christopher Harker
Aditya Bhaskara
13
0
0
26 Oct 2024
On the Crucial Role of Initialization for Matrix Factorization
On the Crucial Role of Initialization for Matrix Factorization
Bingcong Li
Liang Zhang
Aryan Mokhtari
Niao He
28
1
0
24 Oct 2024
Guarantees of a Preconditioned Subgradient Algorithm for
  Overparameterized Asymmetric Low-rank Matrix Recovery
Guarantees of a Preconditioned Subgradient Algorithm for Overparameterized Asymmetric Low-rank Matrix Recovery
Paris Giampouras
HanQin Cai
René Vidal
35
3
0
22 Oct 2024
Robust Low-rank Tensor Train Recovery
Robust Low-rank Tensor Train Recovery
Zhen Qin
Zhihui Zhu
41
1
0
19 Oct 2024
Provable Acceleration of Nesterov's Accelerated Gradient for Rectangular
  Matrix Factorization and Linear Neural Networks
Provable Acceleration of Nesterov's Accelerated Gradient for Rectangular Matrix Factorization and Linear Neural Networks
Zhenghao Xu
Yuqing Wang
T. Zhao
Rachel Ward
Molei Tao
24
0
0
12 Oct 2024
Non-convex matrix sensing: Breaking the quadratic rank barrier in the
  sample complexity
Non-convex matrix sensing: Breaking the quadratic rank barrier in the sample complexity
Dominik Stoger
Yizhe Zhu
28
2
0
20 Aug 2024
Compressible Dynamics in Deep Overparameterized Low-Rank Learning &
  Adaptation
Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptation
Can Yaras
Peng Wang
Laura Balzano
Qing Qu
AI4CE
29
12
0
06 Jun 2024
Can Implicit Bias Imply Adversarial Robustness?
Can Implicit Bias Imply Adversarial Robustness?
Hancheng Min
René Vidal
34
3
0
24 May 2024
Connectivity Shapes Implicit Regularization in Matrix Factorization Models for Matrix Completion
Connectivity Shapes Implicit Regularization in Matrix Factorization Models for Matrix Completion
Zhiwei Bai
Jiajie Zhao
Yaoyu Zhang
AI4CE
30
0
0
22 May 2024
Implicit Regularization of Gradient Flow on One-Layer Softmax Attention
Implicit Regularization of Gradient Flow on One-Layer Softmax Attention
Heejune Sheen
Siyu Chen
Tianhao Wang
Harrison H. Zhou
MLT
33
10
0
13 Mar 2024
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations
Early Directional Convergence in Deep Homogeneous Neural Networks for Small Initializations
Akshay Kumar
Jarvis D. Haupt
ODL
44
3
0
12 Mar 2024
Absence of spurious solutions far from ground truth: A low-rank analysis
  with high-order losses
Absence of spurious solutions far from ground truth: A low-rank analysis with high-order losses
Ziye Ma
Ying Chen
Javad Lavaei
Somayeh Sojoudi
23
1
0
10 Mar 2024
The Implicit Bias of Heterogeneity towards Invariance: A Study of Multi-Environment Matrix Sensing
The Implicit Bias of Heterogeneity towards Invariance: A Study of Multi-Environment Matrix Sensing
Yang Xu
Yihong Gu
Cong Fang
43
0
0
03 Mar 2024
VEC-SBM: Optimal Community Detection with Vectorial Edges Covariates
VEC-SBM: Optimal Community Detection with Vectorial Edges Covariates
Guillaume Braun
Masashi Sugiyama
35
0
0
29 Feb 2024
Low-Tubal-Rank Tensor Recovery via Factorized Gradient Descent
Low-Tubal-Rank Tensor Recovery via Factorized Gradient Descent
Zhiyu Liu
Zhi-Long Han
Yandong Tang
Xi-Le Zhao
Yao Wang
40
1
0
22 Jan 2024
Guaranteed Nonconvex Factorization Approach for Tensor Train Recovery
Guaranteed Nonconvex Factorization Approach for Tensor Train Recovery
Zhen Qin
M. Wakin
Zhihui Zhu
40
5
0
05 Jan 2024
Efficient Compression of Overparameterized Deep Models through
  Low-Dimensional Learning Dynamics
Efficient Compression of Overparameterized Deep Models through Low-Dimensional Learning Dynamics
Soo Min Kwon
Zekai Zhang
Dogyoon Song
Laura Balzano
Qing Qu
35
2
0
08 Nov 2023
Algorithmic Regularization in Tensor Optimization: Towards a Lifted
  Approach in Matrix Sensing
Algorithmic Regularization in Tensor Optimization: Towards a Lifted Approach in Matrix Sensing
Ziye Ma
Javad Lavaei
Somayeh Sojoudi
28
2
0
24 Oct 2023
A Quadratic Synchronization Rule for Distributed Deep Learning
A Quadratic Synchronization Rule for Distributed Deep Learning
Xinran Gu
Kaifeng Lyu
Sanjeev Arora
Jingzhao Zhang
Longbo Huang
46
1
0
22 Oct 2023
Provably Accelerating Ill-Conditioned Low-rank Estimation via Scaled
  Gradient Descent, Even with Overparameterization
Provably Accelerating Ill-Conditioned Low-rank Estimation via Scaled Gradient Descent, Even with Overparameterization
Cong Ma
Xingyu Xu
Tian Tong
Yuejie Chi
16
9
0
09 Oct 2023
How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing:
  The Curses of Symmetry and Initialization
How Over-Parameterization Slows Down Gradient Descent in Matrix Sensing: The Curses of Symmetry and Initialization
Nuoya Xiong
Lijun Ding
Simon S. Du
26
11
0
03 Oct 2023
Asymmetric matrix sensing by gradient descent with small random
  initialization
Asymmetric matrix sensing by gradient descent with small random initialization
J. S. Wind
38
1
0
04 Sep 2023
Transformers as Support Vector Machines
Transformers as Support Vector Machines
Davoud Ataee Tarzanagh
Yingcong Li
Christos Thrampoulidis
Samet Oymak
37
43
0
31 Aug 2023
Early Neuron Alignment in Two-layer ReLU Networks with Small
  Initialization
Early Neuron Alignment in Two-layer ReLU Networks with Small Initialization
Hancheng Min
Enrique Mallada
René Vidal
MLT
32
19
0
24 Jul 2023
Implicit regularization in AI meets generalized hardness of
  approximation in optimization -- Sharp results for diagonal linear networks
Implicit regularization in AI meets generalized hardness of approximation in optimization -- Sharp results for diagonal linear networks
J. S. Wind
Vegard Antun
A. Hansen
17
4
0
13 Jul 2023
The Inductive Bias of Flatness Regularization for Deep Matrix
  Factorization
The Inductive Bias of Flatness Regularization for Deep Matrix Factorization
Khashayar Gatmiry
Zhiyuan Li
Ching-Yao Chuang
Sashank J. Reddi
Tengyu Ma
Stefanie Jegelka
ODL
17
11
0
22 Jun 2023
Transformers learn through gradual rank increase
Transformers learn through gradual rank increase
Enric Boix-Adserà
Etai Littwin
Emmanuel Abbe
Samy Bengio
J. Susskind
36
33
0
12 Jun 2023
Gradient descent in matrix factorization: Understanding large
  initialization
Gradient descent in matrix factorization: Understanding large initialization
Hengchao Chen
Xin Chen
Mohamad Elmasri
Qiang Sun
AI4CE
23
1
0
30 May 2023
Fast and Accurate Estimation of Low-Rank Matrices from Noisy
  Measurements via Preconditioned Non-Convex Gradient Descent
Fast and Accurate Estimation of Low-Rank Matrices from Noisy Measurements via Preconditioned Non-Convex Gradient Descent
Jialun Zhang
Hong-Ming Chiu
Richard Y. Zhang
29
5
0
26 May 2023
Robust Implicit Regularization via Weight Normalization
Robust Implicit Regularization via Weight Normalization
H. Chou
Holger Rauhut
Rachel A. Ward
28
7
0
09 May 2023
The Power of Preconditioning in Overparameterized Low-Rank Matrix
  Sensing
The Power of Preconditioning in Overparameterized Low-Rank Matrix Sensing
Xingyu Xu
Yandi Shen
Yuejie Chi
Cong Ma
40
34
0
02 Feb 2023
Understanding Incremental Learning of Gradient Descent: A Fine-grained
  Analysis of Matrix Sensing
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing
Jikai Jin
Zhiyuan Li
Kaifeng Lyu
S. Du
Jason D. Lee
MLT
46
34
0
27 Jan 2023
Eigenvalue initialisation and regularisation for Koopman autoencoders
Eigenvalue initialisation and regularisation for Koopman autoencoders
Jack W. Miller
Charles OÑeill
N. Constantinou
Omri Azencot
12
2
0
23 Dec 2022
Rank-1 Matrix Completion with Gradient Descent and Small Random
  Initialization
Rank-1 Matrix Completion with Gradient Descent and Small Random Initialization
Daesung Kim
Hye Won Chung
22
2
0
19 Dec 2022
Behind the Scenes of Gradient Descent: A Trajectory Analysis via Basis
  Function Decomposition
Behind the Scenes of Gradient Descent: A Trajectory Analysis via Basis Function Decomposition
Jianhao Ma
Li-Zhen Guo
S. Fattahi
34
4
0
01 Oct 2022
A Validation Approach to Over-parameterized Matrix and Image Recovery
A Validation Approach to Over-parameterized Matrix and Image Recovery
Lijun Ding
Zhen Qin
Liwei Jiang
Jinxin Zhou
Zhihui Zhu
40
13
0
21 Sep 2022
Implicit Bias of Gradient Descent on Reparametrized Models: On
  Equivalence to Mirror Descent
Implicit Bias of Gradient Descent on Reparametrized Models: On Equivalence to Mirror Descent
Zhiyuan Li
Tianhao Wang
Jason D. Lee
Sanjeev Arora
32
27
0
08 Jul 2022
Improved Global Guarantees for the Nonconvex Burer--Monteiro
  Factorization via Rank Overparameterization
Improved Global Guarantees for the Nonconvex Burer--Monteiro Factorization via Rank Overparameterization
Richard Y. Zhang
30
24
0
05 Jul 2022
Neural Networks can Learn Representations with Gradient Descent
Neural Networks can Learn Representations with Gradient Descent
Alexandru Damian
Jason D. Lee
Mahdi Soltanolkotabi
SSL
MLT
14
112
0
30 Jun 2022
Tensor-on-Tensor Regression: Riemannian Optimization,
  Over-parameterization, Statistical-computational Gap, and Their Interplay
Tensor-on-Tensor Regression: Riemannian Optimization, Over-parameterization, Statistical-computational Gap, and Their Interplay
Yuetian Luo
Anru R. Zhang
21
19
0
17 Jun 2022
Understanding the Generalization Benefit of Normalization Layers:
  Sharpness Reduction
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction
Kaifeng Lyu
Zhiyuan Li
Sanjeev Arora
FAtt
35
69
0
14 Jun 2022
Randomly Initialized Alternating Least Squares: Fast Convergence for
  Matrix Sensing
Randomly Initialized Alternating Least Squares: Fast Convergence for Matrix Sensing
Kiryung Lee
Dominik Stöger
23
11
0
25 Apr 2022
Algorithmic Regularization in Model-free Overparametrized Asymmetric
  Matrix Factorization
Algorithmic Regularization in Model-free Overparametrized Asymmetric Matrix Factorization
Liwei Jiang
Yudong Chen
Lijun Ding
31
26
0
06 Mar 2022
Robust Training under Label Noise by Over-parameterization
Robust Training under Label Noise by Over-parameterization
Sheng Liu
Zhihui Zhu
Qing Qu
Chong You
NoLa
OOD
19
105
0
28 Feb 2022
Geometric Regularization from Overparameterization
Geometric Regularization from Overparameterization
Nicholas J. Teague
17
1
0
18 Feb 2022
More is Less: Inducing Sparsity via Overparameterization
More is Less: Inducing Sparsity via Overparameterization
H. Chou
J. Maly
Holger Rauhut
30
25
0
21 Dec 2021
12
Next