Pay Attention when Required

v1v2v3 (latest)

Pay Attention when Required

9 September 2020

ArXiv (abs)PDF HTML

Papers citing "Pay Attention when Required"

5 / 5 papers shown

Title
Efficient Language Modeling for Low-Resource Settings with Hybrid RNN-Transformer Architectures Gabriel Lindenmaier Sean Papay Sebastian Padó 143 0 0 02 Feb 2025
Efficient Sparsely Activated Transformers Salar Latifi Saurav Muralidharan M. Garland MoE 136 2 0 31 Aug 2022
Understanding Robustness of Transformers for Image Classification Srinadh Bhojanapalli Ayan Chakrabarti Daniel Glasner Daliang Li Thomas Unterthiner Andreas Veit ViT 116 392 0 26 Mar 2021
Finetuning Pretrained Transformers into RNNs Jungo Kasai Hao Peng Yizhe Zhang Dani Yogatama Gabriel Ilharco Nikolaos Pappas Yi Mao Weizhu Chen Noah A. Smith 112 67 0 24 Mar 2021
Generating Images with Sparse Representations C. Nash Jacob Menick Sander Dieleman Peter W. Battaglia 93 211 0 05 Mar 2021