Transformer on a Diet

14 February 2020

Papers citing "Transformer on a Diet"

6 / 6 papers shown

Title
Activator: GLU Activation Function as the Core Component of a Vision Transformer Abdullah Nazhat Abdullah Tarkan Aydin ViT 75 0 0 24 May 2024
NiNformer: A Network in Network Transformer with Token Mixing Generated Gating Function Abdullah Nazhat Abdullah Tarkan Aydin 79 0 0 04 Mar 2024
Adaptive Multi-Resolution Attention with Linear Complexity Yao Zhang Yunpu Ma T. Seidl Volker Tresp 35 1 0 10 Aug 2021
A Practical Survey on Faster and Lighter Transformers Quentin Fournier G. Caron Daniel Aloise 137 104 0 26 Mar 2021
Data-Efficient Pretraining via Contrastive Self-Supervision Nils Rethmeier Isabelle Augenstein 109 21 0 02 Oct 2020
SqueezeBERT: What can computer vision teach NLP about efficient neural networks? F. Iandola Albert Eaton Shaw Ravi Krishna Kurt Keutzer VLM 90 128 0 19 Jun 2020