Re-evaluating the Memory-balanced Pipeline Parallelism: BPipe

4 January 2024

Papers citing "Re-evaluating the Memory-balanced Pipeline Parallelism: BPipe"

2 / 2 papers shown

Title
Model Agnostic Hybrid Sharding For Heterogeneous Distributed Inference Claudio Angione Yue Zhao Harry Yang Ahmad Farhan Fielding Johnston James Buban Patrick Colangelo 38 1 0 29 Jul 2024
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,817 0 17 Sep 2019