Memory-Efficient LLM Training with Online Subspace Descent

23 August 2024

Bo Liu

Papers citing "Memory-Efficient LLM Training with Online Subspace Descent"

3 / 3 papers shown

Title
Memory-Efficient LLM Training by Various-Grained Low-Rank Projection of Gradients Yezhen Wang Zhouhao Yang Brian K Chen Fanyi Pu Bo-wen Li Tianyu Gao Kenji Kawaguchi 34 0 0 03 May 2025
COSMOS: A Hybrid Adaptive Optimizer for Memory-Efficient Training of LLMs Liming Liu Zhenghao Xu Zixuan Zhang Hao Kang Zichong Li Chen Liang Weizhu Chen T. Zhao 57 1 0 24 Feb 2025
Cautious Optimizers: Improving Training with One Line of Code Kaizhao Liang Lizhang Chen B. Liu Qiang Liu ODL 98 5 0 25 Nov 2024