CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion

26 May 2024

Papers citing "CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion"

7 / 7 papers shown

Title
Taming the Titans: A Survey of Efficient LLM Inference Serving Ranran Zhen J. Li Yixin Ji Z. Yang Tong Liu Qingrong Xia Xinyu Duan Z. Wang Baoxing Huai M. Zhang LLMAG 77 0 0 28 Apr 2025
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 53 1 0 03 Apr 2025
TeleRAG: Efficient Retrieval-Augmented Generation Inference with Lookahead Retrieval Chien-Yu Lin Keisuke Kamahori Yiyu Liu Xiaoxiang Shi Madhav Kashyap ... Stephanie Wang Arvind Krishnamurthy Rohan Kadekodi Luis Ceze Baris Kasikci 3DV VLM 50 0 0 28 Feb 2025
DroidSpeak: KV Cache Sharing for Cross-LLM Communication and Multi-LLM Serving Yuhan Liu Esha Choukse Shan Lu Junchen Jiang Madan Musuvathi ... Yihua Cheng Junchen Jiang Shan Lu Madan Musuvathi Esha Choukse 78 2 0 05 Nov 2024
Parrot: Efficient Serving of LLM-based Applications with Semantic Variable Chaofan Lin Zhenhua Han Chengruidong Zhang Yuqing Yang Fan Yang Chen Chen Lili Qiu 59 35 0 30 May 2024
Cost-Efficient Large Language Model Serving for Multi-turn Conversations with CachedAttention Bin Gao Zhuomin He Puru Sharma Qingxuan Kang Djordje Jevdjic Junbo Deng Xingkun Yang Zhou Yu Pengfei Zuo 53 42 0 23 Mar 2024
Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models Soham De Samuel L. Smith Anushan Fernando Aleksandar Botev George-Christian Muraru ... David Budden Yee Whye Teh Razvan Pascanu Nando de Freitas Çağlar Gülçehre Mamba 51 116 0 29 Feb 2024