HexGen: Generative Inference of Large Language Model over Heterogeneous
Environment

HexGen: Generative Inference of Large Language Model over Heterogeneous Environment

20 November 2023

Youhe Jiang

Ran Yan

Yang Zhou

Beidi Chen

Papers citing "HexGen: Generative Inference of Large Language Model over Heterogeneous Environment"

8 / 8 papers shown

Title
Taming the Titans: A Survey of Efficient LLM Inference Serving Ranran Zhen J. Li Yixin Ji Z. Yang Tong Liu Qingrong Xia Xinyu Duan Z. Wang Baoxing Huai M. Zhang LLMAG 77 0 0 28 Apr 2025
Prompt Inference Attack on Distributed Large Language Model Inference Frameworks Xinjian Luo Ting Yu X. Xiao AAML SILM 83 1 0 12 Mar 2025
Prompt Inversion Attack against Collaborative Inference of Large Language Models Wenjie Qu Yuguang Zhou Yongji Wu Tingsong Xiao Binhang Yuan Y. Li Jiaheng Zhang 68 0 0 12 Mar 2025
Seesaw: High-throughput LLM Inference via Model Re-sharding Qidong Su Wei Zhao X. Li Muralidhar Andoorveedu Chenhao Jiang Zhanda Zhu Kevin Song Christina Giannoula Gennady Pekhimenko LRM 72 0 0 09 Mar 2025
SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration Jintao Zhang Jia wei Pengle Zhang Jun-Jie Zhu Jun Zhu Jianfei Chen VLM MQ 82 18 0 03 Oct 2024
Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices Yuxiang Huang Binhang Yuan Xu Han Chaojun Xiao Zhiyuan Liu RALM 73 1 0 02 Oct 2024
Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems Xupeng Miao Gabriele Oliaro Zhihao Zhang Xinhao Cheng Hongyi Jin Tianqi Chen Zhihao Jia 61 76 0 23 Dec 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 236 2,232 0 22 Mar 2023