Chunk, Align, Select: A Simple Long-sequence Processing Method for
Transformers

Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers

25 August 2023

Papers citing "Chunk, Align, Select: A Simple Long-sequence Processing Method for Transformers"

13 / 13 papers shown

Title
Cognitive Memory in Large Language Models Lianlei Shan Shixian Luo Zezhou Zhu Yu Yuan Yong Wu LLMAG KELM 58 1 0 03 Apr 2025
Shifting Long-Context LLMs Research from Input to Output Yuhao Wu Yushi Bai Zhiqing Hu Shangqing Tu Ming Shan Hee Juanzi Li Roy Ka-Wei Lee 57 0 0 06 Mar 2025
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 91 122 0 02 May 2023
Training Language Models with Memory Augmentation Zexuan Zhong Tao Lei Danqi Chen RALM 224 126 0 25 May 2022
Transkimmer: Transformer Learns to Layer-wise Skim Yue Guan Zhengyi Li Jingwen Leng Zhouhan Lin Minyi Guo 52 31 0 15 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization Wen Xiao Iz Beltagy Giuseppe Carenini Arman Cohan CVBM 67 113 0 16 Oct 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 234 690 0 27 Aug 2021
CTRLsum: Towards Generic Controllable Text Summarization Junxian He Wojciech Kry'sciñski Bryan McCann Nazneen Rajani Caiming Xiong 203 124 0 08 Dec 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 246 1,982 0 28 Jul 2020
Exploring Controllable Text Generation Techniques Shrimai Prabhumoye A. Black Ruslan Salakhutdinov AI4CE 148 83 0 04 May 2020
Spying on your neighbors: Fine-grained probing of contextual embeddings for information about surrounding words Josef Klafka Allyson Ettinger 35 37 0 04 May 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 228 502 0 12 Mar 2020