Accelerated Training via Incrementally Growing Neural Networks using
Variance Transfer and Learning Rate Adaptation

Accelerated Training via Incrementally Growing Neural Networks using Variance Transfer and Learning Rate Adaptation

Neural Information Processing Systems (NeurIPS), 2023

22 June 2023

Pedro H. P. Savarese

ArXiv (abs)PDF HTML

Papers citing "Accelerated Training via Incrementally Growing Neural Networks using Variance Transfer and Learning Rate Adaptation"

4 / 4 papers shown

Title
Growing Efficient Accurate and Robust Neural Networks on the Edge Vignesh Sundaresha Naresh Shanbhag 174 0 0 10 Oct 2024
Modular Growth of Hierarchical Networks: Efficient, General, and Robust Curriculum Learning Mani Hamidi Sina Khajehabdollahi E. Giannakakis Tim Schäfer Anna Levina Charley M. Wu 153 0 0 10 Jun 2024
Stacking Your Transformers: A Closer Look at Model Growth for Efficient LLM Pre-Training Wenyu Du Tongxu Luo Zihan Qiu Zeyu Huang Songlin Yang Reynold Cheng Wenhan Luo Jie Fu 145 32 0 24 May 2024
An Automatic Learning Rate Schedule Algorithm for Achieving Faster Convergence and Steeper Descent Zhao Song Chiwun Yang 201 10 0 17 Oct 2023