Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble
Exploitation

Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation

7 August 2024

Haibin Lin

Papers citing "Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation"

6 / 6 papers shown

Title
PipeWeaver: Addressing Data Dynamicity in Large Multimodal Model Training with Dynamic Interleaved Pipeline Zhenliang Xue Hanpeng Hu Xing Chen Yimin Jiang Yixin Song Zeyu Mi Yibo Zhu Daxin Jiang Yubin Xia Haibo Chen 36 0 0 19 Apr 2025
Orchestrate Multimodal Data with Batch Post-Balancing to Accelerate Multimodal Large Language Model Training Yijie Zheng Bangjun Xiao Lei Shi Xiaoyang Li Faming Wu Tianyu Li Xuefeng Xiao Y. Zhang Y. Wang Shouda Liu MLLM MoE 67 1 0 31 Mar 2025
SimpleFSDP: Simpler Fully Sharded Data Parallel with torch.compile Ruisi Zhang Tianyu Liu Will Feng Andrew Gu Sanket Purandare Wanchao Liang Francisco Massa 24 1 0 01 Nov 2024
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 160 440 0 14 Oct 2023
Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines Shigang Li Torsten Hoefler GNN AI4CE LRM 77 131 0 14 Jul 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,817 0 17 Sep 2019