Poolingformer: Long Document Modeling with Pooling Attention

10 May 2021

Papers citing "Poolingformer: Long Document Modeling with Pooling Attention"

22 / 22 papers shown

Title
Paths-over-Graph: Knowledge Graph Empowered Large Language Model Reasoning Xingyu Tan Xiaoyang Wang Qing Liu Xiwei Xu Xin Yuan Wenjie Zhang LRM 59 4 0 18 Oct 2024
Target conversation extraction: Source separation using turn-taking dynamics Tuochao Chen Qirui Wang Bohan Wu Malek Itani Sefik Emre Eskimez Takuya Yoshioka Shyamnath Gollakota 20 4 0 15 Jul 2024
Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling Mahdi Karami Ali Ghodsi VLM 36 6 0 28 Feb 2024
Dynamic Multi-Scale Context Aggregation for Conversational Aspect-Based Sentiment Quadruple Analysis Yuqing Li Wenyuan Zhang Binbin Li Siyu Jia Zisen Qi Xingbang Tan 32 3 0 27 Sep 2023
SummaryMixing: A Linear-Complexity Alternative to Self-Attention for Speech Recognition and Understanding Titouan Parcollet Rogier van Dalen Shucong Zhang S. Bhattacharya 16 6 0 12 Jul 2023
Plug-and-Play Document Modules for Pre-trained Models Chaojun Xiao Zhengyan Zhang Xu Han Chi-Min Chan Yankai Lin Zhiyuan Liu Xiangyang Li Zhonghua Li Zhao Cao Maosong Sun KELM 22 5 0 28 May 2023
Learning to Compress Prompts with Gist Tokens Jesse Mu Xiang Lisa Li Noah D. Goodman VLM 34 204 0 17 Apr 2023
Modeling Sequential Sentence Relation to Improve Cross-lingual Dense Retrieval Shunyu Zhang Yaobo Liang Ming Gong Daxin Jiang Nan Duan 16 4 0 03 Feb 2023
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 120 36 0 15 Dec 2022
Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation Botao Yu Peiling Lu Rui Wang Wei Hu Xu Tan Wei Ye Shikun Zhang Tao Qin Tie-Yan Liu MGen 25 54 0 19 Oct 2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Lingpeng Kong 3DV 39 9 0 14 Oct 2022
Adapting Pretrained Text-to-Text Models for Long Text Sequences Wenhan Xiong Anchit Gupta Shubham Toshniwal Yashar Mehdad Wen-tau Yih RALM VLM 49 30 0 21 Sep 2022
GAAMA 2.0: An Integrated System that Answers Boolean and Extractive Questions Scott McCarley Mihaela A. Bornea Sara Rosenthal Anthony Ferritto Md Arafat Sultan Avirup Sil Radu Florian 12 1 0 16 Jun 2022
Fastformer: Additive Attention Can Be All You Need Chuhan Wu Fangzhao Wu Tao Qi Yongfeng Huang Xing Xie 33 117 0 20 Aug 2021
A Survey of Transformers Tianyang Lin Yuxin Wang Xiangyang Liu Xipeng Qiu ViT 27 1,084 0 08 Jun 2021
Improving Multilingual Models with Language-Clustered Vocabularies Hyung Won Chung Dan Garrette Kiat Chuan Tan Jason Riesa VLM 58 65 0 24 Oct 2020
Efficient Transformers: A Survey Yi Tay Mostafa Dehghani Dara Bahri Donald Metzler VLM 69 1,098 0 14 Sep 2020
Sparsifying Transformer Models with Trainable Representation Pooling Michal Pietruszka Łukasz Borchmann Lukasz Garncarek 11 10 0 10 Sep 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 251 2,012 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 238 579 0 12 Mar 2020
On Extractive and Abstractive Neural Document Summarization with Transformer Language Models Sandeep Subramanian Raymond Li Jonathan Pilault C. Pal 233 214 0 07 Sep 2019
A Decomposable Attention Model for Natural Language Inference Ankur P. Parikh Oscar Täckström Dipanjan Das Jakob Uszkoreit 196 1,363 0 06 Jun 2016