LoongServe: Efficiently Serving Long-context Large Language Models with
Elastic Sequence Parallelism

LoongServe: Efficiently Serving Long-context Large Language Models with Elastic Sequence Parallelism

15 April 2024

Xin Jin

Papers citing "LoongServe: Efficiently Serving Long-context Large Language Models with Elastic Sequence Parallelism"

11 / 11 papers shown

Title
SpecRouter: Adaptive Routing for Multi-Level Speculative Decoding in Large Language Models Hang Wu Jianian Zhu Y. Li Haojie Wang Biao Hou Jidong Zhai 18 0 0 12 May 2025
RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference Y. Chen J. Zhang Baotong Lu Qianxi Zhang Chengruidong Zhang ... Chen Chen Mingxing Zhang Yuqing Yang Fan Yang Mao Yang 32 0 0 05 May 2025
Ascendra: Dynamic Request Prioritization for Efficient LLM Serving Azam Ikram Xiang Li Sameh Elnikety S. Bagchi 47 0 0 29 Apr 2025
Taming the Titans: A Survey of Efficient LLM Inference Serving Ranran Zhen J. Li Yixin Ji Z. Yang Tong Liu Qingrong Xia Xinyu Duan Z. Wang Baoxing Huai M. Zhang LLMAG 77 0 0 28 Apr 2025
GenTorrent: Scaling Large Language Model Serving with An Overley Network Fei Fang Yifan Hua Shengze Wang Ruilin Zhou Y. Liu Chen Qian X. Zhang 46 0 0 27 Apr 2025
LoL-PIM: Long-Context LLM Decoding with Scalable DRAM-PIM System Hyucksung Kwon Kyungmo Koo Janghyeon Kim W. Lee Minjae Lee ... Yongkee Kwon Ilkon Kim Euicheol Lim John Kim Jungwook Choi 51 4 0 28 Dec 2024
Marconi: Prefix Caching for the Era of Hybrid LLMs Rui Pan Zhuang Wang Zhen Jia Can Karakus Luca Zancato Tri Dao Ravi Netravali Yida Wang 87 4 0 28 Nov 2024
Context Parallelism for Scalable Million-Token Inference Amy Yang Jingyi Yang Aya Ibrahim Xinfeng Xie Bangsheng Tang Grigory Sizov Jeremy Reizenstein Jongsoo Park Jianyu Huang MoE LRM 52 5 0 04 Nov 2024
Teola: Towards End-to-End Optimization of LLM-based Applications Xin Tan Yimin Jiang Yitao Yang Hong-Yu Xu 48 4 0 29 Jun 2024
Adaptive Elastic Training for Sparse Deep Learning on Heterogeneous Multi-GPU Servers Yujing Ma Florin Rusu Kesheng Wu A. Sim 22 3 0 13 Oct 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,791 0 17 Sep 2019