A Survey of Transformers

8 June 2021

Tianyang Lin

Yuxin Wang

Xiangyang Liu

Xipeng Qiu

ViT

ArXiv PDF HTML

Papers citing "A Survey of Transformers"

30 / 80 papers shown

Title
VIMA: General Robot Manipulation with Multimodal Prompts Yunfan Jiang Agrim Gupta Zichen Zhang Guanzhi Wang Yongqiang Dou Yanjun Chen Li Fei-Fei Anima Anandkumar Yuke Zhu Linxi Fan LM&Ro 15 332 0 06 Oct 2022
Deep Sparse Conformer for Speech Recognition Xianchao Wu 14 2 0 01 Sep 2022
Do Artificial Intelligence Systems Understand? Eduardo C. Garrido-Merchán Carlos Blanco 6 6 0 22 Jul 2022
Formal Algorithms for Transformers Mary Phuong Marcus Hutter 19 68 0 19 Jul 2022
Multimodal Learning with Transformers: A Survey P. Xu Xiatian Zhu David A. Clifton ViT 41 518 0 13 Jun 2022
CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection Zhen Li Bing Xu Conghui Zhu T. Zhao 20 70 0 12 Apr 2022
EmoCaps: Emotion Capsule based Model for Conversational Emotion Recognition Zaijing Li Fengxiao Tang Ming Zhao Yusen Zhu 14 95 0 25 Mar 2022
Under the Hood of Transformer Networks for Trajectory Forecasting Luca Franco Leonardo Placidi Francesco Giuliari Irtiza Hasan Marco Cristani Fabio Galasso 8 19 0 22 Mar 2022
Video Transformers: A Survey Javier Selva A. S. Johansen Sergio Escalera Kamal Nasrollahi T. Moeslund Albert Clapés ViT 20 101 0 16 Jan 2022
An Ensemble of Pre-trained Transformer Models For Imbalanced Multiclass Malware Classification Ferhat Demirkiran Aykut Çayır U. Ünal Hasan Dag 22 42 0 25 Dec 2021
Trading with the Momentum Transformer: An Intelligent and Interpretable Architecture Kieran Wood Sven Giegerich Stephen J. Roberts S. Zohren AI4TS AIFin 11 21 0 16 Dec 2021
PhysFormer: Facial Video-based Physiological Measurement with Temporal Difference Transformer Zitong Yu Yuming Shen Jingang Shi Hengshuang Zhao Philip H. S. Torr Guoying Zhao ViT MedIm 129 162 0 23 Nov 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Zhongchao Shi Jianping Fan Zhiqiang He 3DGS ViT 66 325 0 11 Nov 2021
Towards Efficient NLP: A Standard Evaluation and A Strong Baseline Xiangyang Liu Tianxiang Sun Junliang He Jiawen Wu Lingling Wu Xinyu Zhang Hao Jiang Zhao Cao Xuanjing Huang Xipeng Qiu ELM 16 46 0 13 Oct 2021
Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning Linyang Li Demin Song Xiaonan Li Jiehang Zeng Ruotian Ma Xipeng Qiu 11 133 0 31 Aug 2021
Dual Aspect Self-Attention based on Transformer for Remaining Useful Life Prediction Zhizheng Zhang Wen Song Qiqiang Li AI4TS 28 162 0 30 Jun 2021
Pre-Trained Models: Past, Present and Future Xu Han Zhengyan Zhang Ning Ding Yuxian Gu Xiao Liu ... Jie Tang Ji-Rong Wen Jinhui Yuan Wayne Xin Zhao Jun Zhu AIFin MQ AI4MH 19 807 0 14 Jun 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 282 1,490 0 27 Feb 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 4,735 0 24 Feb 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir F. Khan M. Shah ViT 225 2,404 0 04 Jan 2021
A Survey on Deep Reinforcement Learning for Audio-Based Applications S. Latif Heriberto Cuayáhuitl Farrukh Pervez Fahad Shamshad Hafiz Shehbaz Ali Erik Cambria OffRL 32 73 0 01 Jan 2021
ERNIE-Doc: A Retrospective Long-Document Modeling Transformer Siyu Ding Junyuan Shang Shuohuan Wang Yu Sun Hao Tian Hua-Hong Wu Haifeng Wang 60 52 0 31 Dec 2020
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting Haoyi Zhou Shanghang Zhang J. Peng Shuai Zhang Jianxin Li Hui Xiong Wan Zhang AI4TS 167 3,799 0 14 Dec 2020
Big Bird: Transformers for Longer Sequences Manzil Zaheer Guru Guruganesh Kumar Avinava Dubey Joshua Ainslie Chris Alberti ... Philip Pham Anirudh Ravula Qifan Wang Li Yang Amr Ahmed VLM 249 1,982 0 28 Jul 2020
Pre-trained Models for Natural Language Processing: A Survey Xipeng Qiu Tianxiang Sun Yige Xu Yunfan Shao Ning Dai Xuanjing Huang LM&MA VLM 241 1,444 0 18 Mar 2020
Efficient Content-Based Sparse Attention with Routing Transformers Aurko Roy M. Saffar Ashish Vaswani David Grangier MoE 238 578 0 12 Mar 2020
Talking-Heads Attention Noam M. Shazeer Zhenzhong Lan Youlong Cheng Nan Ding L. Hou 89 79 0 05 Mar 2020
How Much Position Information Do Convolutional Neural Networks Encode? Md. Amirul Islam Sen Jia Neil D. B. Bruce SSL 194 343 0 22 Jan 2020
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism M. Shoeybi M. Patwary Raul Puri P. LeGresley Jared Casper Bryan Catanzaro MoE 243 1,791 0 17 Sep 2019
OpenNMT: Open-Source Toolkit for Neural Machine Translation Guillaume Klein Yoon Kim Yuntian Deng Jean Senellart Alexander M. Rush 254 1,893 0 10 Jan 2017