ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference

15 March 2024

Papers citing "ExeGPT: Constraint-Aware Resource Scheduling for LLM Inference"

23 / 23 papers shown

Title
Prism: Unleashing GPU Sharing for Cost-Efficient Multi-LLM Serving Shan Yu Jiarong Xing Yifan Qiao Mingyuan Ma Y. Li ... Shiyi Cao Ke Bao Ion Stoica Harry Xu Ying Sheng 24 0 0 06 May 2025
Ascendra: Dynamic Request Prioritization for Efficient LLM Serving Azam Ikram Xiang Li Sameh Elnikety S. Bagchi 53 0 0 29 Apr 2025
Apt-Serve: Adaptive Request Scheduling on Hybrid Cache for Scalable LLM Inference Serving Shihong Gao X. Zhang Yanyan Shen Lei Chen 22 1 0 10 Apr 2025
SLOs-Serve: Optimized Serving of Multi-SLO LLMs Siyuan Chen Zhipeng Jia S. Khan Arvind Krishnamurthy Phillip B. Gibbons 24 1 0 05 Apr 2025
HERA: Hybrid Edge-cloud Resource Allocation for Cost-Efficient AI Agents Shiyi Liu Haiying Shen Shuai Che Mahdi Ghandi Mingqin Li LLMAG 48 0 0 01 Apr 2025
AccelGen: Heterogeneous SLO-Guaranteed High-Throughput LLM Inference Serving for Diverse Applications Haiying Shen Tanmoy Sen 37 0 0 17 Mar 2025
Mitigating KV Cache Competition to Enhance User Experience in LLM Inference Haiying Shen Tanmoy Sen Masahiro Tanaka 74 0 0 17 Mar 2025
Cyber Defense Reinvented: Large Language Models as Threat Intelligence Copilots Xiaoqun Liu Jiacheng Liang Qiben Yan Muchao Ye Jinyuan Jia Zhaohan Xi Jinyuan Jia Zhaohan Xi 56 0 0 28 Feb 2025
PAPI: Exploiting Dynamic Parallelism in Large Language Model Decoding with a Processing-In-Memory-Enabled Computing System Yintao He Haiyu Mao Christina Giannoula Mohammad Sadrosadati Juan Gómez Luna Huawei Li Xiaowei Li Ying Wang O. Mutlu 38 5 0 21 Feb 2025
iServe: An Intent-based Serving System for LLMs Dimitrios Liakopoulos Tianrui Hu Prasoon Sinha N. Yadwadkar VLM 66 0 0 08 Jan 2025
Chameleon: Adaptive Caching and Scheduling for Many-Adapter LLM Inference Environments Nikoleta Iliakopoulou Jovan Stojkovic Chloe Alverti Tianyin Xu Hubertus Franke Josep Torrellas 70 2 0 24 Nov 2024
Software Performance Engineering for Foundation Model-Powered Software (FMware) Haoxiang Zhang Shi Chang Arthur Leung Kishanthan Thangarajah Boyuan Chen Hanan Lutfiyya Ahmed E. Hassan 52 0 0 14 Nov 2024
Interactive Speculative Planning: Enhance Agent Efficiency through Co-design of System and User Interface Wenyue Hua Mengting Wan Shashank Vadrevu Ryan Nadel Yongfeng Zhang Chi Wang LLMAG 24 1 0 30 Sep 2024
LLMServingSim: A HW/SW Co-Simulation Infrastructure for LLM Inference Serving at Scale Jaehong Cho Minsu Kim Hyunmin Choi Guseul Heo Jongse Park 35 8 0 10 Aug 2024
DynamoLLM: Designing LLM Inference Clusters for Performance and Energy Efficiency Jovan Stojkovic Chaojie Zhang Íñigo Goiri Josep Torrellas Esha Choukse 30 29 0 01 Aug 2024
LLM Inference Serving: Survey of Recent Advances and Opportunities Baolin Li Yankai Jiang V. Gadepally Devesh Tiwari 73 15 0 17 Jul 2024
Leveraging Large Language Models for Integrated Satellite-Aerial-Terrestrial Networks: Recent Advances and Future Directions Shumaila Javaid R. A. Khalil Nasir Saeed Bin He Mohamed-Slim Alouini 32 8 0 05 Jul 2024
PerLLM: Personalized Inference Scheduling with Edge-Cloud Collaboration for Diverse LLM Services Zheming Yang Yuanhao Yang Chang Zhao Qi Guo Wenkai He Wen Ji 41 13 0 23 May 2024
Aladdin: Joint Placement and Scaling for SLO-Aware LLM Serving Chengyi Nie Rodrigo Fonseca Zhenhua Liu 24 4 0 11 May 2024
A Survey on Efficient Inference for Large Language Models Zixuan Zhou Xuefei Ning Ke Hong Tianyu Fu Jiaming Xu ... Shengen Yan Guohao Dai Xiao-Ping Zhang Yuhan Dong Yu-Xiang Wang 46 78 0 22 Apr 2024
Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines Shigang Li Torsten Hoefler GNN AI4CE LRM 77 94 0 14 Jul 2021
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,791 0 17 Sep 2019
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 170 3,504 0 10 Jun 2015