Efficient Attentions for Long Document Summarization

5 April 2021

L. Huang

Shuyang Cao

Nikolaus Nova Parulian

Heng Ji

Lu Wang

ArXiv PDF HTML

Papers citing "Efficient Attentions for Long Document Summarization"

50 / 72 papers shown

Title
A Split-then-Join Approach to Abstractive Summarization for Very Long Documents in a Low Resource Setting Lhuqita Fazry VLM 35 0 0 11 May 2025
Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions Yiming Du Wenyu Huang Danna Zheng Zhaowei Wang Sébastien Montella Mirella Lapata Kam-Fai Wong Jeff Z. Pan KELM MU 83 2 0 01 May 2025
WindowKV: Task-Adaptive Group-Wise KV Cache Window Selection for Efficient LLM Inference Youhui Zuo Sibo Wei C. Zhang Zhuorui Liu Wenpeng Lu Dawei Song VLM 56 0 0 23 Mar 2025
GPU-Accelerated Motion Planning of an Underactuated Forestry Crane in Cluttered Environments M. Vu Gerald Ebmer Alexander Watcher Marc-Philip Ecker Giang Nguyen Tobias Glueck 74 2 0 18 Mar 2025
LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning Zhibin Lan Liqiang Niu Fandong Meng Jie Zhou Jinsong Su VLM 69 2 0 04 Mar 2025
Neural Attention Search Difan Deng Marius Lindauer 93 0 0 21 Feb 2025
Identify Critical KV Cache in LLM Inference from an Output Perturbation Perspective Yuan Feng Junlin Lv Yuhang Cao Xike Xie S.Kevin Zhou 79 2 0 06 Feb 2025
Twilight: Adaptive Attention Sparsity with Hierarchical Top- $p$ Pruning C. Lin Jiaming Tang Shuo Yang Hanshuo Wang Tian Tang Boyu Tian Ion Stoica Enze Xie Mingyu Gao 97 2 0 04 Feb 2025
Context-Aware Hierarchical Merging for Long Document Summarization Litu Ou Mirella Lapata MoMe 209 1 0 03 Feb 2025
Learning to Summarize from LLM-generated Feedback Hwanjun Song Taewon Yun Yuho Lee Jihwan Oh Gihun Lee Jason (Jinglun) Cai Hang Su 73 3 0 28 Jan 2025
Ada-KV: Optimizing KV Cache Eviction by Adaptive Budget Allocation for Efficient LLM Inference Yuan Feng Junlin Lv Yukun Cao Xike Xie S. K. Zhou VLM 61 27 0 28 Jan 2025
Efficient Prompt Compression with Evaluator Heads for Long-Context Transformer Inference Weizhi Fei Xueyan Niu Guoqing Xie Yingqing Liu Bo Bai Wei Han 33 1 0 22 Jan 2025
Revisiting In-Context Learning with Long Context Language Models Jinheon Baek Sun Jae Lee Prakhar Gupta Geunseob Oh Siddharth Dalmia 193 1 0 22 Dec 2024
The State and Fate of Summarization Datasets: A Survey Noam Dahan Gabriel Stanovsky HILM 52 0 0 07 Nov 2024
TODO: Enhancing LLM Alignment with Ternary Preferences Yuxiang Guo Lu Yin Bo Jiang Jiaqi Zhang 38 1 0 02 Nov 2024
What is Wrong with Perplexity for Long-context Language Modeling? Lizhe Fang Yifei Wang Zhaoyang Liu Chenheng Zhang Stefanie Jegelka Jinyang Gao Bolin Ding Yisen Wang 69 6 0 31 Oct 2024
MDCure: A Scalable Pipeline for Multi-Document Instruction-Following Gabrielle Kaili-May Liu Bowen Shi Avi Caciularu Idan Szpektor Arman Cohan 72 4 0 30 Oct 2024
HDT: Hierarchical Document Transformer Haoyu He Markus Flicke Jan Buchmann Iryna Gurevych Andreas Geiger 43 0 0 11 Jul 2024
Applicability of Large Language Models and Generative Models for Legal Case Judgement Summarization Aniket Deroy Kripabandhu Ghosh Saptarshi Ghosh ELM AILaw 50 16 0 06 Jul 2024
Short-Long Convolutions Help Hardware-Efficient Linear Attention to Focus on Long Sequences Zicheng Liu Siyuan Li Li Wang Zedong Wang Yunfan Liu Stan Z. Li 35 7 0 12 Jun 2024
Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling Liliang Ren Yang Liu Yadong Lu Yelong Shen Chen Liang Weizhu Chen Mamba 74 56 0 11 Jun 2024
Chain of Agents: Large Language Models Collaborating on Long-Context Tasks Yusen Zhang Ruoxi Sun Yanfei Chen Tomas Pfister Rui Zhang Sercan Ö. Arik RALM AI4CE LLMAG 54 28 0 04 Jun 2024
PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling Zefan Cai Yichi Zhang Bofei Gao Yuliang Liu Yong Li ... Wayne Xiong Yue Dong Baobao Chang Junjie Hu Wen Xiao 62 84 0 04 Jun 2024
A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods Hanlei Jin Yang Zhang Dan Meng Jun Wang Jinghua Tan 68 80 0 05 Mar 2024
Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Tasks Huajian Zhang Yumo Xu Laura Perez-Beltrachini HILM 32 9 0 27 Feb 2024
SportsMetrics: Blending Text and Numerical Data to Understand Information Fusion in LLMs Yebowen Hu Kaiqiang Song Sangwoo Cho Xiaoyang Wang H. Foroosh Dong Yu Fei Liu 23 8 0 15 Feb 2024
Zebra: Extending Context Window with Layerwise Grouped Local-Global Attention Kaiqiang Song Xiaoyang Wang Sangwoo Cho Xiaoman Pan Dong Yu 34 7 0 14 Dec 2023
SCCA: Shifted Cross Chunk Attention for long contextual semantic expansion Yuxiang Guo 17 0 0 12 Dec 2023
Universal Self-Consistency for Large Language Model Generation Xinyun Chen Renat Aksitov Uri Alon Jie Jessie Ren Kefan Xiao Pengcheng Yin Sushant Prakash Charles Sutton Xuezhi Wang Denny Zhou LRM 26 66 0 29 Nov 2023
Prompt Cache: Modular Attention Reuse for Low-Latency Inference In Gim Guojun Chen Seung-seob Lee Nikhil Sarda Anurag Khandelwal Lin Zhong 42 75 0 07 Nov 2023
Surveying the Landscape of Text Summarization with Deep Learning: A Comprehensive Review Guanghua Wang Weili Wu AI4TS AILaw 38 3 0 13 Oct 2023
InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining Wei Ping Ming-Yu Liu Lawrence C. McAfee Peng Xu Bo Li M. Shoeybi Bryan Catanzaro RALM 16 46 0 11 Oct 2023
BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model Nolan Dey Daria Soboleva Faisal Al-Khateeb Bowen Yang Ribhu Pathria ... Robert Myers Jacob Robert Steeves Natalia Vassilieva Marvin Tom Joel Hestness MoE 24 14 0 20 Sep 2023
Retentive Network: A Successor to Transformer for Large Language Models Yutao Sun Li Dong Shaohan Huang Shuming Ma Yuqing Xia Jilong Xue Jianyong Wang Furu Wei LRM 78 301 0 17 Jul 2023
Summarization from Leaderboards to Practice: Choosing A Representation Backbone and Ensuring Robustness David Demeter Oshin Agarwal Simon Ben Igeri Marko Sterbentz Neil P. Molino John M. Conroy A. Nenkova 21 1 0 18 Jun 2023
ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding Uri Shaham Maor Ivgi Avia Efrat Jonathan Berant Omer Levy VLM 41 126 0 23 May 2023
DLUE: Benchmarking Document Language Understanding Ruoxi Xu Hongyu Lin Xinyan Guan Xianpei Han Yingfei Sun Le Sun ELM 24 0 0 16 May 2023
A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization Chenhui Shen Liying Cheng Xuan-Phi Nguyen Yang You Lidong Bing 25 10 0 15 May 2023
Vcc: Scaling Transformers to 128K Tokens or More by Prioritizing Important Tokens Zhanpeng Zeng Cole Hawkins Min-Fong Hong Aston Zhang Nikolaos Pappas Vikas Singh Shuai Zheng 21 6 0 07 May 2023
CoLT5: Faster Long-Range Transformers with Conditional Computation Joshua Ainslie Tao Lei Michiel de Jong Santiago Ontañón Siddhartha Brahma ... Mandy Guo James Lee-Thorp Yi Tay Yun-hsuan Sung Sumit Sanghai LLMAG 33 63 0 17 Mar 2023
ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models Jonas Belouadi Steffen Eger 54 24 0 20 Dec 2022
RISE: Leveraging Retrieval Techniques for Summarization Evaluation David C. Uthus Jianmo Ni RALM 16 0 0 17 Dec 2022
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 125 36 0 15 Dec 2022
Document-Level Abstractive Summarization Gonçalo Raposo Afonso Raposo Ana Sofia Carmo 24 1 0 06 Dec 2022
Long-Document Cross-Lingual Summarization Shaohui Zheng Zhixu Li Jiaan Wang Jianfeng Qu An Liu Lei Zhao Zhigang Chen RALM 25 9 0 01 Dec 2022
GoSum: Extractive Summarization of Long Documents by Reinforcement Learning and Graph Organized discourse state Junyi Bian Xiaodi Huang Hong Zhou Shanfeng Zhu 30 11 0 18 Nov 2022
Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attention Wenhao Li Xiaoyuan Yi Jinyi Hu Maosong Sun Xing Xie 29 0 0 14 Nov 2022
How Far are We from Robust Long Abstractive Summarization? Huan Yee Koh Jiaxin Ju He Zhang Ming Liu Shirui Pan HILM 25 39 0 30 Oct 2022
Toward Unifying Text Segmentation and Long Document Summarization Sangwoo Cho Kaiqiang Song Xiaoyang Wang Fei Liu Dong Yu 24 33 0 28 Oct 2022
Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost Sungjun Cho Seonwoo Min Jinwoo Kim Moontae Lee Honglak Lee Seunghoon Hong 40 3 0 27 Oct 2022