The CAP Principle for LLM Serving: A Survey of Long-Context Large
Language Model Serving

The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Serving

18 May 2024

Zhenyu Ning

Jieru Zhao

Mengwei Xu

Papers citing "The CAP Principle for LLM Serving: A Survey of Long-Context Large Language Model Serving"

15 / 15 papers shown

Title
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 31 37 0 09 Jul 2024
Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention Tsendsuren Munkhdalai Manaal Faruqui Siddharth Gopal LRM LLMAG CLL 79 101 0 10 Apr 2024
InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory Chaojun Xiao Pengle Zhang Xu Han Guangxuan Xiao Yankai Lin Zhengyan Zhang Zhiyuan Liu Maosong Sun LLMAG 28 33 0 07 Feb 2024
Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models Xindi Wang Mahsa Salmani Parsa Omidi Xiangyu Ren Mehdi Rezagholizadeh A. Eshaghi LRM 21 35 0 03 Feb 2024
A Survey of Resource-efficient LLM and Multimodal Foundation Models Mengwei Xu Wangsong Yin Dongqi Cai Rongjie Yi Daliang Xu ... Shangguang Wang Yuanchun Li Yunxin Liu Xin Jin Xuanzhe Liu VLM 67 70 0 16 Jan 2024
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression Huiqiang Jiang Qianhui Wu Xufang Luo Dongsheng Li Chin-Yew Lin Yuqing Yang Lili Qiu RALM 96 179 0 10 Oct 2023
Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading Howard Chen Ramakanth Pasunuru Jason Weston Asli Celikyilmaz RALM 68 72 0 08 Oct 2023
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training Dawei Zhu Nan Yang Liang Wang Yifan Song Wenhao Wu Furu Wei Sujian Li 49 77 0 19 Sep 2023
Generative Agents: Interactive Simulacra of Human Behavior J. Park Joseph C. O'Brien Carrie J. Cai Meredith Ringel Morris Percy Liang Michael S. Bernstein LM&Ro AI4CE 206 1,701 0 07 Apr 2023
SALO: An Efficient Spatial Accelerator Enabling Hybrid Sparse Attention Mechanisms for Long Sequences Guan Shen Jieru Zhao Quan Chen Jingwen Leng C. Li Minyi Guo 24 26 0 29 Jun 2022
Energon: Towards Efficient Acceleration of Transformers Using Dynamic Sparse Attention Zhe Zhou Junling Liu Zhenyu Gu Guangyu Sun 56 39 0 18 Oct 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 234 690 0 27 Aug 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 275 3,784 0 18 Apr 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 246 1,982 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 228 502 0 12 Mar 2020