ResearchTrend.AI
Inference-Friendly Models With MixAttention

Inference-Friendly Models With MixAttention

Shashank Rajput
Ying Sheng
Sean Owen
Vitaliy Chiley
ArXivPDFHTML

Papers citing "Inference-Friendly Models With MixAttention"

1 / 1 papers shown
Title
Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices
Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices
Yuxiang Huang
Binhang Yuan
Xu Han
Chaojun Xiao
Zhiyuan Liu
RALM
67
1
0
02 Oct 2024