Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic
Objectives with Skewed Hessian Spectrums

Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums

27 October 2021

Tong Zhang

Papers citing "Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums"

8 / 8 papers shown

Title
The Optimality of (Accelerated) SGD for High-Dimensional Quadratic Optimization Haihan Zhang Yuanshi Liu Qianwen Chen Cong Fang 38 0 0 15 Sep 2024
Large Batch Analysis for Adagrad Under Anisotropic Smoothness Yuxing Liu Rui Pan Tong Zhang 26 5 0 21 Jun 2024
Directional Smoothness and Gradient Methods: Convergence and Adaptivity Aaron Mishkin Ahmed Khaled Yuanhao Wang Aaron Defazio Robert Mansel Gower 44 6 0 06 Mar 2024
Accelerated Convergence of Stochastic Heavy Ball Method under Anisotropic Gradient Noise Rui Pan Yuxing Liu Xiaoyu Wang Tong Zhang 23 5 0 22 Dec 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 30 41 0 07 Apr 2023
ExtremeBERT: A Toolkit for Accelerating Pretraining of Customized BERT Rui Pan Shizhe Diao Jianlin Chen Tong Zhang VLM 12 7 0 30 Nov 2022
Last Iterate Risk Bounds of SGD with Decaying Stepsize for Overparameterized Linear Regression Jingfeng Wu Difan Zou Vladimir Braverman Quanquan Gu Sham Kakade 104 20 0 12 Oct 2021
Stochastic Gradient Descent for Non-smooth Optimization: Convergence Results and Optimal Averaging Schemes Ohad Shamir Tong Zhang 101 570 0 08 Dec 2012