Fairness in Serving Large Language Models

Fairness in Serving Large Language Models

31 December 2023

Joseph E. Gonzalez

Papers citing "Fairness in Serving Large Language Models"

14 / 14 papers shown

Title
Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving Shan Yu Jiarong Xing Yifan Qiao Mingyuan Ma Y. Li ... Shiyi Cao Ke Bao Ion Stoica Harry Xu Ying Sheng 29 0 0 06 May 2025
RetroInfer: A Vector-Storage Approach for Scalable Long-Context LLM Inference Y. Chen J. Zhang Baotong Lu Qianxi Zhang Chengruidong Zhang ... Chen Chen Mingxing Zhang Yuqing Yang Fan Yang Mao Yang 34 0 0 05 May 2025
Ascendra: Dynamic Request Prioritization for Efficient LLM Serving Azam Ikram Xiang Li Sameh Elnikety S. Bagchi 72 0 0 29 Apr 2025
GenTorrent: Scaling Large Language Model Serving with An Overley Network Fei Fang Yifan Hua Shengze Wang Ruilin Zhou Y. Liu Chen Qian X. Zhang 46 0 0 27 Apr 2025
Tempo: Application-aware LLM Serving with Mixed SLO Requirements Wei Zhang Zhiyu Wu Yi Mu Banruo Liu Myungjin Lee Fan Lai 51 0 0 24 Apr 2025
Circinus: Efficient Query Planner for Compound ML Serving Banruo Liu Wei-Yu Lin Minghao Fang Yihan Jiang Fan Lai LRM 34 0 0 23 Apr 2025
Mitigating KV Cache Competition to Enhance User Experience in LLM Inference Haiying Shen Tanmoy Sen Masahiro Tanaka 121 0 0 17 Mar 2025
HyGen: Efficient LLM Serving via Elastic Online-Offline Request Co-location Ting Sun Penghan Wang Fan Lai 107 1 0 15 Jan 2025
iServe: An Intent-based Serving System for LLMs Dimitrios Liakopoulos Tianrui Hu Prasoon Sinha N. Yadwadkar VLM 116 0 0 08 Jan 2025
Is the GPU Half-Empty or Half-Full? Practical Scheduling Techniques for LLMs Ferdi Kossmann Bruce Fontaine Daya Khudia Michael Cafarella Samuel Madden 86 2 0 23 Oct 2024
Efficient LLM Scheduling by Learning to Rank Yichao Fu Siqi Zhu Runlong Su Aurick Qiao Ion Stoica Hao Zhang 50 19 0 28 Aug 2024
Teola: Towards End-to-End Optimization of LLM-based Applications Xin Tan Yimin Jiang Yitao Yang Hong-Yu Xu 57 5 0 29 Jun 2024
Splitwise: Efficient generative LLM inference using phase splitting Pratyush Patel Esha Choukse Chaojie Zhang Aashaka Shah Íñigo Goiri Saeed Maleki Ricardo Bianchini 38 196 0 30 Nov 2023
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU Ying Sheng Lianmin Zheng Binhang Yuan Zhuohan Li Max Ryabinin ... Joseph E. Gonzalez Percy Liang Christopher Ré Ion Stoica Ce Zhang 144 366 0 13 Mar 2023