Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2308.16137
Cited By
v1
v2
v3
v4
v5
v6
v7 (latest)
LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models
North American Chapter of the Association for Computational Linguistics (NAACL), 2023
30 August 2023
Chi Han
Qifan Wang
Yuan Yao
Wenhan Xiong
Yu Chen
Heng Ji
Sinong Wang
Re-assign community
ArXiv (abs)
PDF
HTML
HuggingFace (40 upvotes)
Github (144★)
Papers citing
"LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models"
50 / 60 papers shown
Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs
N. Bui
Shubham Sharma
Simran Lamba
Saumitra Mishra
Rex Ying
150
4
0
03 Dec 2025
SPA: Achieving Consensus in LLM Alignment via Self-Priority Optimization
Yue Huang
Xiangqi Wang
Xiangliang Zhang
165
1
0
09 Nov 2025
Federated Attention: A Distributed Paradigm for Collaborative LLM Inference over Edge Networks
Xiumei Deng
Zehui Xiong
Binbin Chen
Dong In Kim
Mérouane Debbah
H. Vincent Poor
FedML
212
2
0
04 Nov 2025
FlashEVA: Accelerating LLM inference via Efficient Attention
Juan Gabriel Kostelec
Qinghai Guo
205
0
0
01 Nov 2025
Long-Context Modeling with Dynamic Hierarchical Sparse Attention for On-Device LLMs
Siheng Xiong
Joe Zou
Faramarz Fekri
Yae Jee Cho
126
1
0
28 Oct 2025
Do LLMs Truly Understand When a Precedent Is Overruled?
Li Zhang
Jaromír Šavelka
Kevin D. Ashley
AILaw
ELM
367
4
0
23 Oct 2025
Alleviating Forgetfulness of Linear Attention by Hybrid Sparse Attention and Contextualized Learnable Token Eviction
Mutian He
Philip N. Garner
CLL
301
2
0
23 Oct 2025
All You Need is One: Capsule Prompt Tuning with a Single Vector
Yiyang Liu
James Chenhao Liang
Heng Fan
Wenhao Yang
Yiming Cui
Xiaotian Han
Lifu Huang
Dongfang Liu
Qifan Wang
Cheng Han
VLM
201
5
0
19 Oct 2025
NOSA: Native and Offloadable Sparse Attention
Yuxiang Huang
Chaojun Xiao
Xu Han
Zhiyuan Liu
Zhou Su
...
Hengyu Zhao
Yudong Wang
Chaojun Xiao
Xu Han
Zhiyuan Liu
MQ
224
0
0
15 Oct 2025
DSAS: A Universal Plug-and-Play Framework for Attention Optimization in Multi-Document Question Answering
Jiakai Li
Rongzheng Wang
Yizhuo Ma
Shuang Liang
Guangchun Luo
Ke Qin
175
3
0
14 Oct 2025
Artificial Hippocampus Networks for Efficient Long-Context Modeling
Yunhao Fang
Weihao Yu
Shu Zhong
Qinghao Ye
Xuehan Xiong
Lai Wei
205
5
0
08 Oct 2025
SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs
Dachuan Shi
Abedelkadir Asi
Keying Li
Xiangchi Yuan
Leyan Pan
Wenke Lee
Wen Xiao
LRM
271
5
0
06 Oct 2025
On the Limitations and Capabilities of Position Embeddings for Length Generalization
Yang Chen
Yitao Liang
Zhouchen Lin
145
0
0
05 Oct 2025
InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptation
Weilin Zhao
Z. Zhou
Zhou Su
Chaojun Xiao
Yuxuan Li
...
Ruoyao Xiao
Yuxiang Huang
Ao Sun
Xu Han
Zhiyuan Liu
VLM
215
16
0
29 Sep 2025
Guard Vector: Beyond English LLM Guardrails with Task-Vector Composition and Streaming-Aware Prefix SFT
Wonhyuk Lee
Youngchol Kim
Yunjin Park
Junhyung Moon
Dongyoung Jeong
Wanjin Park
171
0
0
27 Sep 2025
Mamba Modulation: On the Length Generalization of Mamba
Peng Lu
Jerry Huang
Qiuhao Zeng
X. Wang
Boxing Wang
Philippe Langlais
Yufei Cui
Mamba
370
0
0
23 Sep 2025
Positional Encoding via Token-Aware Phase Attention
Wang
Sheng Shen
Rémi Munos
Hongyuan Zhan
Yuandong Tian
250
1
0
16 Sep 2025
Compact Attention: Exploiting Structured Spatio-Temporal Sparsity for Fast Video Generation
Qirui Li
Guangcong Zheng
Qi Zhao
Jie Li
Bin Dong
Jing Lin
Xi Li
VGen
190
4
0
18 Aug 2025
SmallKV: Small Model Assisted Compensation of KV Cache Compression for Efficient LLM Inference
Yi Zhao
Yajuan Peng
Cam-Tu Nguyen
Zuchao Li
Xiaoliang Wang
Hai Zhao
Xiaoming Fu
324
2
0
03 Aug 2025
LaCache: Ladder-Shaped KV Caching for Efficient Long-Context Modeling of Large Language Models
Dachuan Shi
Y. Fu
Xiangchi Yuan
Zhongzhi Yu
Haoran You
Sixu Li
Xin Dong
Jan Kautz
Pavlo Molchanov
Yingyan
RALM
239
6
0
14 Jul 2025
Prioritizing Alignment Paradigms over Task-Specific Model Customization in Time-Series LLMs
Wei Li
Yunyao Cheng
Xinli Hao
Chaohong Ma
Yuxuan Liang
Bin Yang
Christian S.Jensen
Xiaofeng Meng
AI4TS
199
0
0
13 Jun 2025
Native-Resolution Image Synthesis
Zidong Wang
Mengwei He
Xiangyu Yue
Xuming He
Yiyuan Zhang
370
7
0
03 Jun 2025
SALE : Low-bit Estimation for Efficient Sparse Attention in Long-context LLM Prefilling
Xiaodong Ji
Hailin Zhang
Fangcheng Fu
Huang Leng
267
2
0
30 May 2025
Scaling External Knowledge Input Beyond Context Windows of LLMs via Multi-Agent Collaboration
Zijun Liu
Zhennan Wan
Peng Li
Ming Yan
Ji Zhang
Fei Huang
Wenshu Fan
LLMAG
310
1
0
27 May 2025
Longer Context, Deeper Thinking: Uncovering the Role of Long-Context Ability in Reasoning
Wang Yang
Zirui Liu
Hongye Jin
Qingyu Yin
Vipin Chaudhary
Xiaotian Han
ReLM
LRM
376
5
0
22 May 2025
Scale-invariant Attention
Ben Anson
Xi Wang
Laurence Aitchison
LRM
557
2
0
20 May 2025
SpecMemo: Speculative Decoding is in Your Pocket
Selin Yildirim
Deming Chen
224
0
0
16 May 2025
FreqKV: Key-Value Compression in Frequency Domain for Context Window Extension
Jushi Kai
Boyi Zeng
Longji Xu
Haoli Bai
Ziwei He
Bo Jiang
Zhouhan Lin
546
3
0
01 May 2025
LVLM_CSP: Accelerating Large Vision Language Models via Clustering, Scattering, and Pruning for Reasoning Segmentation
Hanning Chen
Yang Ni
Wenjun Huang
Hyunwoo Oh
Yezi Liu
Tamoghno Das
Mohsen Imani
VLM
LRM
303
0
0
15 Apr 2025
KeepKV: Achieving Periodic Lossless KV Cache Compression for Efficient LLM Inference
Yuxuan Tian
Zihan Wang
Yebo Peng
Aomufei Yuan
Zhaoxiang Wang
Bairen Yi
Xin Liu
Yong Cui
Tong Yang
469
0
0
14 Apr 2025
SWAN-GPT: An Efficient and Scalable Approach for Long-Context Language Modeling
Krishna Puvvada
Faisal Ladhak
Santiago Akle Serrano
Cheng-Ping Hsieh
Shantanu Acharya
...
Fei Jia
Samuel Kriman
Simeng Sun
Dima Rekesh
Boris Ginsburg
RALM
369
9
0
11 Apr 2025
Harnessing the Unseen: The Hidden Influence of Intrinsic Knowledge in Long-Context Language Models
Yu Fu
Haz Sameen Shahgir
Hui Liu
Xianfeng Tang
Qi He
Yue Dong
KELM
524
1
0
11 Apr 2025
Adaptive Computation Pruning for the Forgetting Transformer
Zhixuan Lin
J. Obando-Ceron
Xu Owen He
Rameswar Panda
384
3
0
09 Apr 2025
AccLLM: Accelerating Long-Context LLM Inference Via Algorithm-Hardware Co-Design
Yanbiao Liang
Huihong Shi
Haikuo Shao
Zhongfeng Wang
318
6
0
07 Apr 2025
LagKV: Lag-Relative Information of the KV Cache Tells Which Tokens Are Important
Manlai Liang
JiaMing Zhang
Xiong Li
Jinlong Li
MQ
359
4
0
07 Apr 2025
Skip-Vision: Efficient and Scalable Acceleration of Vision-Language Models via Adaptive Token Skipping
Weili Zeng
Ziyuan Huang
Kaixiang Ji
Manwen Liao
VLM
783
6
0
26 Mar 2025
LongDiff: Training-Free Long Video Generation in One Go
Computer Vision and Pattern Recognition (CVPR), 2025
Zhuoling Li
Hossein Rahmani
Qiuhong Ke
Jing Liu
DiffM
VGen
VLM
336
6
0
23 Mar 2025
ZSMerge: Zero-Shot KV Cache Compression for Memory-Efficient Long-Context LLMs
Xin Liu
Xudong Wang
Pei Liu
Guoming Tang
MoMe
331
0
0
13 Mar 2025
Hierarchical Balance Packing: Towards Efficient Supervised Fine-tuning for Long-Context LLM
Yongqiang Yao
Jingru Tan
Kaihuan Liang
Jiahao Hu
Jiahao Hu
Jiahao Hu
Yazhe Niu
Ruihao Gong
Dahua Lin
Ningyi Xu
598
2
0
10 Mar 2025
WeightedKV: Attention Scores Weighted Key-Value Cache Merging for Large Language Models
IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2025
Jian Yuan
Ziwei He
Haoli Bai
Jingwen Leng
Ziwei He
MoMe
270
3
0
03 Mar 2025
FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
International Conference on Learning Representations (ICLR), 2025
Xunhao Lai
Jianqiao Lu
Yao Luo
Yiyuan Ma
Xun Zhou
358
82
0
28 Feb 2025
InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU
Heejun Lee
G. Park
Jaduk Suh
Sung Ju Hwang
423
8
0
13 Feb 2025
RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval
Di Liu
Meng Chen
Baotong Lu
Huiqiang Jiang
Zhenhua Han
...
Jianchao Tan
Chong Chen
Fan Yang
Yue Yang
Lili Qiu
664
99
0
03 Jan 2025
Hymba: A Hybrid-head Architecture for Small Language Models
International Conference on Learning Representations (ICLR), 2024
Xin Dong
Y. Fu
Shizhe Diao
Wonmin Byeon
Zijia Chen
...
Min-Hung Chen
Yoshi Suhara
Y. Lin
Jan Kautz
Pavlo Molchanov
Mamba
388
75
0
20 Nov 2024
TokenSelect: Efficient Long-Context Inference and Length Extrapolation for LLMs via Dynamic Token-Level KV Cache Selection
Wei Wu
Zhuoshi Pan
Chao Wang
L. Chen
Y. Bai
Kun Fu
Xiping Hu
Hui Xiong
Hui Xiong
LLMAG
652
25
0
05 Nov 2024
MaskControl: Spatio-Temporal Control for Masked Motion Synthesis
Ekkasit Pinyoanuntapong
Muhammad Usama Saleem
Korrawe Karunratanakul
Pu Wang
Hongfei Xue
Chong Chen
Chuan Guo
Junli Cao
J. Ren
Sergey Tulyakov
VGen
561
123
0
14 Oct 2024
Rodimus*: Breaking the Accuracy-Efficiency Trade-Off with Efficient Attentions
International Conference on Learning Representations (ICLR), 2024
Zhihao He
Hang Yu
Zi Gong
Shizhan Liu
Jia-Nan Li
Weiyao Lin
VLM
514
5
0
09 Oct 2024
Towards LifeSpan Cognitive Systems
Yu Wang
Chi Han
Tongtong Wu
Xiaoxin He
Wangchunshu Zhou
...
Zexue He
Wei Wang
Gholamreza Haffari
Heng Ji
Julian McAuley
KELM
CLL
1.1K
11
0
20 Sep 2024
E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoning
Zihan Liao
Ning Yang
Hang Yu
Lingxiao Wei
Jianguo Li
Jun Wang
Wei Zhang
350
10
0
10 Sep 2024
LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2024
Zhiyuan Hu
Yuliang Liu
Jinman Zhao
Suyuchen Wang
Yan Wang
...
Qing Gu
Anh Tuan Luu
See-Kiong Ng
Zhiwei Jiang
Bryan Hooi
408
23
0
31 Aug 2024
1
2
Next
Page 1 of 2
