DeLighT: Deep and Light-weight Transformer

DeLighT: Deep and Light-weight Transformer

3 August 2020

Marjan Ghazvininejad

Luke Zettlemoyer

Hannaneh Hajishirzi

Papers citing "DeLighT: Deep and Light-weight Transformer"

10 / 10 papers shown

Title
TranSFormer: Slow-Fast Transformer for Machine Translation Bei Li Yi Jing Xu Tan Zhen Xing Tong Xiao Jingbo Zhu 41 7 0 26 May 2023
Towards Accurate Post-Training Quantization for Vision Transformer Yifu Ding Haotong Qin Qing-Yu Yan Z. Chai Junjie Liu Xiaolin K. Wei Xianglong Liu MQ 54 68 0 25 Mar 2023
Dual Decomposition of Convex Optimization Layers for Consistent Attention in Medical Images Tom Ron M. Weiler-Sagie Tamir Hazan FAtt MedIm 21 6 0 06 Jun 2022
ACORT: A Compact Object Relation Transformer for Parameter Efficient Image Captioning J. Tan Y. Tan C. Chan Joon Huang Chuah VLM ViT 24 15 0 11 Feb 2022
A Survey of Transformers Tianyang Lin Yuxin Wang Xiangyang Liu Xipeng Qiu ViT 32 1,087 0 08 Jun 2021
Defending Against Backdoor Attacks in Natural Language Generation Xiaofei Sun Xiaoya Li Yuxian Meng Xiang Ao Fei Wu Jiwei Li Tianwei Zhang AAML SILM 23 47 0 03 Jun 2021
Learning Light-Weight Translation Models from Deep Transformer Bei Li Ziyang Wang Hui Liu Quan Du Tong Xiao Chunliang Zhang Jingbo Zhu VLM 117 40 0 27 Dec 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 245 1,817 0 17 Sep 2019
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives Elena Voita Rico Sennrich Ivan Titov 193 181 0 03 Sep 2019
Improving neural networks by preventing co-adaptation of feature detectors Geoffrey E. Hinton Nitish Srivastava A. Krizhevsky Ilya Sutskever Ruslan Salakhutdinov VLM 266 7,634 0 03 Jul 2012