DataStates-LLM: Lazy Asynchronous Checkpointing for Large Language Models

15 June 2024

Papers citing "DataStates-LLM: Lazy Asynchronous Checkpointing for Large Language Models"

4 / 4 papers shown

Title
Learning in Chaos: Efficient Autoscaling and Self-healing for Distributed Training at the Edge Wenjiao Feng Rongxing Xiao Zonghang Li Hongfang Yu Gang Sun Long Luo Mohsen Guizani Qirong Ho 21 0 0 19 May 2025
I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey Noah Lewis J. L. Bez Suren Byna 62 0 0 16 Apr 2024
DeepSpeed4Science Initiative: Enabling Large-Scale Scientific Discovery through Sophisticated AI System Technologies Shuaiwen Leon Song Bonnie Kruft Minjia Zhang Conglong Li Shiyang Chen ... Arash Vahdat Chaowei Xiao Thomas Gibbs Anima Anandkumar R. Stevens 54 13 0 06 Oct 2023
M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining Junyang Lin An Yang Jinze Bai Chang Zhou Le Jiang ... Jie Zhang Yong Li Wei Lin Jingren Zhou Hongxia Yang MoE 92 43 0 08 Oct 2021