Breadth-First Pipeline Parallelism

11 November 2022

Papers citing "Breadth-First Pipeline Parallelism"

3 / 3 papers shown

Title
LoRA-FA: Memory-efficient Low-rank Adaptation for Large Language Models Fine-tuning Longteng Zhang Lin Zhang S. Shi X. Chu Bo-wen Li AI4CE 13 91 0 07 Aug 2023
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 228 4,460 0 23 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,817 0 17 Sep 2019