ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2002.05273
  4. Cited By
A Second look at Exponential and Cosine Step Sizes: Simplicity,
  Adaptivity, and Performance
v1v2v3v4 (latest)

A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance

International Conference on Machine Learning (ICML), 2020
12 February 2020
Xiaoyun Li
Zhenxun Zhuang
Francesco Orabona
ArXiv (abs)PDFHTML

Papers citing "A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance"

16 / 16 papers shown
Gradient Methods with Online Scaling Part I. Theoretical Foundations
Gradient Methods with Online Scaling Part I. Theoretical Foundations
Wenzhi Gao
Ya-Chi Chu
Yinyu Ye
Madeleine Udell
356
4
0
29 May 2025
Sharpness-Aware Minimization with Adaptive Regularization for Training
  Deep Neural Networks
Sharpness-Aware Minimization with Adaptive Regularization for Training Deep Neural NetworksIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2024
Jinping Zou
Xiaoge Deng
Tao Sun
371
1
0
22 Dec 2024
A Generalized Version of Chung's Lemma and its Applications
A Generalized Version of Chung's Lemma and its Applications
Li Jiang
Xiao Li
Andre Milzarek
Junwen Qiu
263
2
0
09 Jun 2024
New logarithmic step size for stochastic gradient descent
New logarithmic step size for stochastic gradient descent
M. S. Shamaee
S. F. Hafshejani
Z. Saeidian
270
3
0
01 Apr 2024
Shuffling Momentum Gradient Algorithm for Convex Optimization
Shuffling Momentum Gradient Algorithm for Convex Optimization
Trang H. Tran
Quoc Tran-Dinh
Lam M. Nguyen
288
2
0
05 Mar 2024
(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
Anh Dang
Reza Babanezhad
Sharan Vaswani
338
0
0
12 Jan 2024
Based on What We Can Control Artificial Neural Networks
Based on What We Can Control Artificial Neural Networks
Cheng Kang
Xujing Yao
217
0
0
09 Oct 2023
Modified Step Size for Enhanced Stochastic Gradient Descent: Convergence
  and Experiments
Modified Step Size for Enhanced Stochastic Gradient Descent: Convergence and Experiments
M. S. Shamaee
S. F. Hafshejani
270
1
0
03 Sep 2023
Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex ConversionInternational Conference on Machine Learning (ICML), 2023
Ashok Cutkosky
Harsh Mehta
Francesco Orabona
520
51
0
07 Feb 2023
Target-based Surrogates for Stochastic Optimization
Target-based Surrogates for Stochastic OptimizationInternational Conference on Machine Learning (ICML), 2023
J. Lavington
Sharan Vaswani
Reza Babanezhad
Mark Schmidt
Nicolas Le Roux
335
6
0
06 Feb 2023
Understanding AdamW through Proximal Methods and Scale-Freeness
Understanding AdamW through Proximal Methods and Scale-Freeness
Zhenxun Zhuang
Mingrui Liu
Ashok Cutkosky
Francesco Orabona
335
107
0
31 Jan 2022
On Uniform Boundedness Properties of SGD and its Momentum Variants
On Uniform Boundedness Properties of SGD and its Momentum Variants
Xiaoyu Wang
M. Johansson
234
3
0
25 Jan 2022
An Optimization Framework for Federated Edge Learning
An Optimization Framework for Federated Edge LearningIEEE Transactions on Wireless Communications (IEEE TWC), 2021
Yangchen Li
Ying Cui
Vincent K. N. Lau
FedML
230
9
0
26 Nov 2021
Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic
  Objectives with Skewed Hessian Spectrums
Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums
Boyao Wang
Haishan Ye
Tong Zhang
466
19
0
27 Oct 2021
Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization
Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization
Xiaoyu Wang
M. Johansson
285
2
0
05 Jun 2021
On the Convergence of Step Decay Step-Size for Stochastic Optimization
On the Convergence of Step Decay Step-Size for Stochastic OptimizationNeural Information Processing Systems (NeurIPS), 2021
Xiaoyu Wang
Sindri Magnússon
M. Johansson
271
31
0
18 Feb 2021
1
Page 1 of 1