CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling

CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling

14 October 2022

Papers citing "CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling"

15 / 15 papers shown

Title
A Framework to Enable Algorithmic Design Choice Exploration in DNNs Timothy L. Cronin IV Sanmukh Kuppannagari 27 0 0 10 Oct 2024
Gated Slot Attention for Efficient Linear-Time Sequence Modeling Yu Zhang Songlin Yang Ruijie Zhu Yue Zhang Leyang Cui ... Freda Shi Bailin Wang Wei Bi P. Zhou Guohong Fu 54 11 0 11 Sep 2024
TaylorShift: Shifting the Complexity of Self-Attention from Squared to Linear (and Back) using Taylor-Softmax Tobias Christian Nauen Sebastián M. Palacio Andreas Dengel 38 3 0 05 Mar 2024
Linear Attention via Orthogonal Memory Jun Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Lingpeng Kong 26 3 0 18 Dec 2023
Attentive Multi-Layer Perceptron for Non-autoregressive Generation Shuyang Jiang Jinchao Zhang Jiangtao Feng Lin Zheng Lingpeng Kong 41 0 0 14 Oct 2023
L-Eval: Instituting Standardized Evaluation for Long Context Language Models Chen An Shansan Gong Ming Zhong Xingjian Zhao Mukai Li Jun Zhang Lingpeng Kong Xipeng Qiu ELM ALM 25 132 0 20 Jul 2023
In-Context Learning with Many Demonstration Examples Mukai Li Shansan Gong Jiangtao Feng Yiheng Xu Jinchao Zhang Zhiyong Wu Lingpeng Kong 28 26 0 09 Feb 2023
PARAGEN : A Parallel Generation Toolkit Jiangtao Feng Yi Zhou Jun Zhang Xian Qian Liwei Wu Zhexi Zhang Yanming Liu Mingxuan Wang Lei Li Hao Zhou VLM 22 3 0 07 Oct 2022
Transformer Quality in Linear Time Weizhe Hua Zihang Dai Hanxiao Liu Quoc V. Le 68 220 0 21 Feb 2022
ABC: Attention with Bounded-memory Control Hao Peng Jungo Kasai Nikolaos Pappas Dani Yogatama Zhaofeng Wu Lingpeng Kong Roy Schwartz Noah A. Smith 53 21 0 06 Oct 2021
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting Haoyi Zhou Shanghang Zhang J. Peng Shuai Zhang Jianxin Li Hui Xiong Wan Zhang AI4TS 159 3,799 0 14 Dec 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 246 1,982 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 228 502 0 12 Mar 2020
A Style-Based Generator Architecture for Generative Adversarial Networks Tero Karras S. Laine Timo Aila 262 10,183 0 12 Dec 2018
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 208 7,687 0 17 Aug 2015