XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training

XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training

24 October 2019

Dongsheng Li

Papers citing "XPipe: Efficient Pipeline Model Parallelism for Multi-GPU DNN Training"

12 / 12 papers shown

Title
Nesterov Method for Asynchronous Pipeline Parallel Optimization Thalaiyasingam Ajanthan Sameera Ramasinghe Yan Zuo Gil Avraham Alexander Long 26 0 0 02 May 2025
PipeOptim: Ensuring Effective 1F1B Schedule with Optimizer-Dependent Weight Prediction Lei Guan Dongsheng Li Jiye Liang Wenjian Wang Wenjian Wang Xicheng Lu 42 1 0 01 Dec 2023
DISCO: Distributed Inference with Sparse Communications Minghai Qin Chaowen Sun Jaco A. Hofmann D. Vučinić FedML 27 1 0 22 Feb 2023
Weight Prediction Boosts the Convergence of AdamW Lei Guan 37 16 0 01 Feb 2023
LOFT: Finding Lottery Tickets through Filter-wise Training Qihan Wang Chen Dun Fangshuo Liao C. Jermaine Anastasios Kyrillidis 30 3 0 28 Oct 2022
PARTIME: Scalable and Parallel Processing Over Time with Deep Neural Networks Enrico Meloni Lapo Faggi Simone Marullo Alessandro Betti Matteo Tiezzi Marco Gori S. Melacci GNN AI4TS 24 1 0 17 Oct 2022
DistrEdge: Speeding up Convolutional Neural Network Inference on Distributed Edge Devices Xueyu Hou Yongjie Guan Tao Han Ning Zhang 24 41 0 03 Feb 2022
FTPipeHD: A Fault-Tolerant Pipeline-Parallel Distributed Training Framework for Heterogeneous Edge Devices Yuhao Chen Qianqian Yang Shibo He Zhiguo Shi Jiming Chen 24 3 0 06 Oct 2021
ResIST: Layer-Wise Decomposition of ResNets for Distributed Training Chen Dun Cameron R. Wolfe C. Jermaine Anastasios Kyrillidis 24 21 0 02 Jul 2021
DAPPLE: A Pipelined Data Parallel Approach for Training Large Models Shiqing Fan Yi Rong Chen Meng Zongyan Cao Siyu Wang ... Jun Yang Lixue Xia Lansong Diao Xiaoyong Liu Wei Lin 21 233 0 02 Jul 2020
Taming Momentum in a Distributed Asynchronous Environment Ido Hakimi Saar Barkai Moshe Gabel Assaf Schuster 19 23 0 26 Jul 2019
Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation Yonghui Wu M. Schuster Zhehuai Chen Quoc V. Le Mohammad Norouzi ... Alex Rudnick Oriol Vinyals G. Corrado Macduff Hughes J. Dean AIMat 718 6,750 0 26 Sep 2016