Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large
Language Model Training

v1v2 (latest)

Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training

5 June 2024

Xu Han

Cheng Yang

Zhiyuan Liu

Chuan Shi

ArXiv (abs)PDF HTML Github (17★)

Papers citing "Seq1F1B: Efficient Sequence-Level Pipeline Parallelism for Large Language Model Training"

9 / 9 papers shown

Title
AdaPtis: Reducing Pipeline Bubbles with Adaptive Pipeline Parallelism on Heterogeneous Models Jihu Guo Tenghui Ma Wei Gao Peng Sun Jiaxing Li Xun Chen Yuyang Jin Dahua Lin 64 0 0 28 Sep 2025
Data-Centric Elastic Pipeline Parallelism for Efficient Long-Context LLM Training Shiju Wang Yujie Wang Ao Sun Fangcheng Fu Z. Zhu Huang Leng Xu Han Kaisheng Ma 116 0 0 25 Sep 2025
TokenSmith: Streamlining Data Editing, Search, and Inspection for Large-Scale Language Model Training and Interpretability Mohammad Aflah Khan Ameya Godbole Johnny Tian-Zheng Wei Ryan Yixiang Wang James Flemings Krishna P. Gummadi Willie Neiswanger Robin Jia SyDa 141 0 0 25 Jul 2025
NoLoCo: No-all-reduce Low Communication Training Method for Large Models Jari Kolehmainen Nikolay Blagoev John Donaghy Oğuzhan Ersoy Christopher Nies 242 0 0 12 Jun 2025
SlimPipe: Memory-Thrifty and Efficient Pipeline Parallelism for Long-Context LLM Training Zheng Li Wenshu Fan Wei Zhang Tailing Yuan Bin Chen Chengru Song Chen Zhang 171 1 0 20 Apr 2025
How Social is It? A Benchmark for LLMs' Capabilities in Multi-user Multi-turn Social Agent Tasks Yusen Wu Junwu Xiong Xiaotie Deng LLMAG 226 1 0 04 Apr 2025
PipeOffload: Improving Scalability of Pipeline Parallelism with Memory Optimization Xinyi Wan Penghui Qi Guangxing Huang Jialin Li Jialin Li 157 3 0 03 Mar 2025
APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUsAnnual Meeting of the Association for Computational Linguistics (ACL), 2025 Yuxiang Huang Mingye Li Xu Han Chaojun Xiao Weilin Zhao Sun Ao Hao Zhou Jie Zhou Zhiyuan Liu Maosong Sun 299 1 0 17 Feb 2025
Efficient Training of Large Language Models on Distributed Infrastructures: A Survey Jiangfei Duan Shuo Zhang Zerui Wang Lijuan Jiang Wenwen Qu ... Dahua Lin Yonggang Wen Xin Jin Tianwei Zhang Yang Liu 299 28 0 29 Jul 2024