Sketching as a Tool for Understanding and Accelerating Self-attention
for Long Sequences

Sketching as a Tool for Understanding and Accelerating Self-attention for Long Sequences

10 December 2021

Dilek Z. Hakkani-Tür

Heng Ji

Papers citing "Sketching as a Tool for Understanding and Accelerating Self-attention for Long Sequences"

4 / 4 papers shown

Title
Towards LifeSpan Cognitive Systems Yu Wang Chi Han Tongtong Wu Xiaoxin He Wangchunshu Zhou ... Zexue He Wei Wang Gholamreza Haffari Heng Ji Julian McAuley KELM CLL 91 1 0 20 Sep 2024
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting Haoyi Zhou Shanghang Zhang J. Peng Shuai Zhang Jianxin Li Hui Xiong Wan Zhang AI4TS 167 3,855 0 14 Dec 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 2,009 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 238 578 0 12 Mar 2020