Improving Automatic Parallel Training via Balanced Memory Workload
Optimization

Improving Automatic Parallel Training via Balanced Memory Workload Optimization

5 July 2023

Yujie Wang

Youhe Jiang

Shenhan Zhu

Xiaonan Nie

Papers citing "Improving Automatic Parallel Training via Balanced Memory Workload Optimization"

6 / 6 papers shown

Title
Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving Shihong Gao X. Zhang Yanyan Shen Lei Chen 22 1 0 10 Apr 2025
Malleus: Straggler-Resilient Hybrid Parallel Training of Large-scale Models via Malleable Data and Model Parallelization Haoyang Li Fangcheng Fu Hao Ge Sheng Lin Xuanyu Wang Jiawen Niu Y. Wang Hailin Zhang Xiaonan Nie Bin Cui MoMe 31 2 0 17 Oct 2024
Efficient Training of Large Language Models on Distributed Infrastructures: A Survey Jiangfei Duan Shuo Zhang Zerui Wang Lijuan Jiang Wenwen Qu ... Dahua Lin Yonggang Wen Xin Jin Tianwei Zhang Peng Sun 71 8 0 29 Jul 2024
Angel-PTM: A Scalable and Economical Large-scale Pre-training System in Tencent Xiaonan Nie Yi Liu Fangcheng Fu J. Xue Dian Jiao Xupeng Miao Yangyu Tao Bin Cui MoE 19 16 0 06 Mar 2023
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,764 0 24 Feb 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,453 0 23 Jan 2020