Predicting Attention Sparsity in Transformers

24 September 2021

André F. T. Martins

Papers citing "Predicting Attention Sparsity in Transformers"

3 / 3 papers shown

Title
Real-Time Video Generation with Pyramid Attention Broadcast Xuanlei Zhao Xiaolong Jin Kai Wang Yang You VGen DiffM 66 31 0 22 Aug 2024
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 1,982 0 28 Jul 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 228 578 0 12 Mar 2020