Unlimiformer: Long-Range Transformers with Unlimited Length Input

2 May 2023

Graham Neubig

Papers citing "Unlimiformer: Long-Range Transformers with Unlimited Length Input"

21 / 21 papers shown

Title
TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generation Shintaro Ozaki Kazuki Hayashi Yusuke Sakai Jingun Kwon Hidetaka Kamigaito Katsuhiko Hayashi Manabu Okumura Taro Watanabe VLM 68 0 0 25 Apr 2025
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 53 1 0 03 Apr 2025
LV-XAttn: Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models Tzu-Tao Chang Shivaram Venkataraman VLM 73 0 0 04 Feb 2025
Context-Aware Hierarchical Merging for Long Document Summarization Litu Ou Mirella Lapata MoMe 73 1 0 03 Feb 2025
ZETA: Leveraging Z-order Curves for Efficient Top-k Attention Qiuhao Zeng Jerry Huang Peng Lu Gezheng Xu Boxing Chen Charles X. Ling Boyu Wang 37 1 0 24 Jan 2025
RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval Di Liu Meng Chen Baotong Lu Huiqiang Jiang Zhenhua Han ... K. Zhang C. L. P. Chen Fan Yang Y. Yang Lili Qiu 29 29 0 03 Jan 2025
How to Train Long-Context Language Models (Effectively) Tianyu Gao Alexander Wettig Howard Yen Danqi Chen RALM 42 36 0 03 Oct 2024
HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly Howard Yen Tianyu Gao Minmin Hou Ke Ding Daniel Fleischer Peter Izsak Moshe Wasserblat Danqi Chen ALM ELM 36 24 0 03 Oct 2024
In-Context Learning with Long-Context Models: An In-Depth Exploration Amanda Bertsch Maor Ivgi Uri Alon Jonathan Berant Matthew R. Gormley Matthew R. Gormley Graham Neubig ReLM AIMat 65 64 0 30 Apr 2024
MEMORY-VQ: Compression for Tractable Internet-Scale Memory Yury Zemlyanskiy Michiel de Jong Luke Vilnis Santiago Ontañón William W. Cohen Sumit Sanghai Joshua Ainslie RALM MQ 12 0 0 28 Aug 2023
Text2KGBench: A Benchmark for Ontology-Driven Knowledge Graph Generation from Text Nandana Mihindukulasooriya Sanju Tiwari Carlos F. Enguix K. Lata 9 51 0 04 Aug 2023
Multilingual Code Co-Evolution Using Large Language Models Jiyang Zhang Pengyu Nie Junyi Jessy Li Miloš Gligorić 14 20 0 27 Jul 2023
You can't pick your neighbors, or can you? When and how to rely on retrieval in the $k$ NN-LM Andrew Drozdov Shufan Wang Razieh Rahimi Andrew McCallum Hamed Zamani Mohit Iyyer RALM 96 17 0 28 Oct 2022
An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics Huan Yee Koh Jiaxin Ju Ming Liu Shirui Pan 60 120 0 03 Jul 2022
Training Language Models with Memory Augmentation Zexuan Zhong Tao Lei Danqi Chen RALM 221 126 0 25 May 2022
Neuro-Symbolic Language Modeling with Automaton-augmented Retrieval Uri Alon Frank F. Xu Junxian He Sudipta Sengupta Dan Roth Graham Neubig RALM 67 62 0 28 Jan 2022
PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization Wen Xiao Iz Beltagy Giuseppe Carenini Arman Cohan CVBM 67 113 0 16 Oct 2021
ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts Yuta Koreeda Christopher D. Manning AILaw 78 96 0 05 Oct 2021
Leveraging Pretrained Models for Automatic Summarization of Doctor-Patient Conversations Longxiang Zhang Renato M. P. Negrinho Arindam Ghosh V. Jagannathan H. Hassanzadeh Thomas Schaaf Matthew R. Gormley LM&MA AI4MH 62 65 0 24 Sep 2021
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 246 1,982 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 228 502 0 12 Mar 2020