Tuning Learning Rates with the Cumulative-Learning Constant

This paper introduces a novel method for optimizing learning rates in machine learning. A previously unrecognized proportionality between learning rates and dataset sizes is discovered, providing valuable insights into how dataset scale influences training dynamics. Additionally, a cumulative learning constant is identified, offering a framework for designing and optimizing advanced learning rate schedules. These findings have the potential to enhance training efficiency and performance across a wide range of machine learning applications.

@article{faraj2025_2505.13457,
  title={ Tuning Learning Rates with the Cumulative-Learning Constant },
  author={ Nathan Faraj },
  journal={arXiv preprint arXiv:2505.13457},
  year={ 2025 }
}