Stateful Large Language Model Serving with Pensieve

Stateful Large Language Model Serving with Pensieve

9 December 2023

Papers citing "Stateful Large Language Model Serving with Pensieve"

8 / 8 papers shown

Title
Alignment Drift in CEFR-prompted LLMs for Interactive Spanish Tutoring Mina Almasi Ross Deans Kristensen-McLachlan 13 0 0 13 May 2025
RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference Y. Chen J. Zhang Baotong Lu Qianxi Zhang Chengruidong Zhang ... Chen Chen Mingxing Zhang Yuqing Yang Fan Yang Mao Yang 32 0 0 05 May 2025
Improving the Serving Performance of Multi-LoRA Large Language Models via Efficient LoRA and KV Cache Management Hang Zhang Jiuchen Shi Yixiao Wang Quan Chen Yizhou Shan Minyi Guo 25 0 0 19 Apr 2025
Marconi: Prefix Caching for the Era of Hybrid LLMs Rui Pan Zhuang Wang Zhen Jia Can Karakus Luca Zancato Tri Dao Ravi Netravali Yida Wang 90 4 0 28 Nov 2024
EPIC: Efficient Position-Independent Context Caching for Serving Large Language Models Junhao Hu Wenrui Huang H. Wang Weidong Wang Tiancheng Hu Qin Zhang Hao Feng Xusheng Chen Yizhou Shan Tao Xie RALM LLMAG 29 2 0 20 Oct 2024
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU Ying Sheng Lianmin Zheng Binhang Yuan Zhuohan Li Max Ryabinin ... Joseph E. Gonzalez Percy Liang Christopher Ré Ion Stoica Ce Zhang 144 365 0 13 Mar 2023
ZeRO-Offload: Democratizing Billion-Scale Model Training Jie Ren Samyam Rajbhandari Reza Yazdani Aminabadi Olatunji Ruwase Shuangyang Yang Minjia Zhang Dong Li Yuxiong He MoE 160 413 0 18 Jan 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,815 0 17 Sep 2019