ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2002.05273
  4. Cited By
A Second look at Exponential and Cosine Step Sizes: Simplicity,
  Adaptivity, and Performance
v1v2v3v4 (latest)

A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance

International Conference on Machine Learning (ICML), 2020
12 February 2020
Xiaoyun Li
Zhenxun Zhuang
Francesco Orabona
ArXiv (abs)PDFHTML

Papers citing "A Second look at Exponential and Cosine Step Sizes: Simplicity, Adaptivity, and Performance"

16 / 16 papers shown
Gradient Methods with Online Scaling Part I. Theoretical Foundations
Gradient Methods with Online Scaling Part I. Theoretical Foundations
Wenzhi Gao
Ya-Chi Chu
Yinyu Ye
Madeleine Udell
358
4
0
29 May 2025
Sharpness-Aware Minimization with Adaptive Regularization for Training
  Deep Neural Networks
Sharpness-Aware Minimization with Adaptive Regularization for Training Deep Neural NetworksIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2024
Jinping Zou
Xiaoge Deng
Tao Sun
377
1
0
22 Dec 2024
A Generalized Version of Chung's Lemma and its Applications
A Generalized Version of Chung's Lemma and its Applications
Li Jiang
Xiao Li
Andre Milzarek
Junwen Qiu
267
2
0
09 Jun 2024
New logarithmic step size for stochastic gradient descent
New logarithmic step size for stochastic gradient descent
M. S. Shamaee
S. F. Hafshejani
Z. Saeidian
272
3
0
01 Apr 2024
Shuffling Momentum Gradient Algorithm for Convex Optimization
Shuffling Momentum Gradient Algorithm for Convex Optimization
Trang H. Tran
Quoc Tran-Dinh
Lam M. Nguyen
298
2
0
05 Mar 2024
(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
(Accelerated) Noise-adaptive Stochastic Heavy-Ball Momentum
Anh Dang
Reza Babanezhad
Sharan Vaswani
351
0
0
12 Jan 2024
Based on What We Can Control Artificial Neural Networks
Based on What We Can Control Artificial Neural Networks
Cheng Kang
Xujing Yao
223
0
0
09 Oct 2023
Modified Step Size for Enhanced Stochastic Gradient Descent: Convergence
  and Experiments
Modified Step Size for Enhanced Stochastic Gradient Descent: Convergence and Experiments
M. S. Shamaee
S. F. Hafshejani
287
1
0
03 Sep 2023
Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion
Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex ConversionInternational Conference on Machine Learning (ICML), 2023
Ashok Cutkosky
Harsh Mehta
Francesco Orabona
529
51
0
07 Feb 2023
Target-based Surrogates for Stochastic Optimization
Target-based Surrogates for Stochastic OptimizationInternational Conference on Machine Learning (ICML), 2023
J. Lavington
Sharan Vaswani
Reza Babanezhad
Mark Schmidt
Nicolas Le Roux
365
6
0
06 Feb 2023
Understanding AdamW through Proximal Methods and Scale-Freeness
Understanding AdamW through Proximal Methods and Scale-Freeness
Zhenxun Zhuang
Mingrui Liu
Ashok Cutkosky
Francesco Orabona
336
110
0
31 Jan 2022
On Uniform Boundedness Properties of SGD and its Momentum Variants
On Uniform Boundedness Properties of SGD and its Momentum Variants
Xiaoyu Wang
M. Johansson
236
3
0
25 Jan 2022
An Optimization Framework for Federated Edge Learning
An Optimization Framework for Federated Edge LearningIEEE Transactions on Wireless Communications (IEEE TWC), 2021
Yangchen Li
Ying Cui
Vincent K. N. Lau
FedML
233
9
0
26 Nov 2021
Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic
  Objectives with Skewed Hessian Spectrums
Eigencurve: Optimal Learning Rate Schedule for SGD on Quadratic Objectives with Skewed Hessian Spectrums
Boyao Wang
Haishan Ye
Tong Zhang
477
19
0
27 Oct 2021
Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization
Bandwidth-based Step-Sizes for Non-Convex Stochastic Optimization
Xiaoyu Wang
M. Johansson
293
2
0
05 Jun 2021
On the Convergence of Step Decay Step-Size for Stochastic Optimization
On the Convergence of Step Decay Step-Size for Stochastic OptimizationNeural Information Processing Systems (NeurIPS), 2021
Xiaoyu Wang
Sindri Magnússon
M. Johansson
278
31
0
18 Feb 2021
1
Page 1 of 1