A Survey of Transformers

8 June 2021

Tianyang Lin

Yuxin Wang

Xiangyang Liu

Xipeng Qiu

ViT

ArXiv PDF HTML

Papers citing "A Survey of Transformers"

50 / 80 papers shown

Title
Retrieval Augmented Generation Evaluation for Health Documents Mario Ceresa Lorenzo Bertolini Valentin Comte Nicholas Spadaro Barbara Raffael ... Sergio Consoli Amalia Muñoz Piñeiro Alex Patak Maddalena Querci Tobias Wiesenthal RALM 3DV 31 0 1 07 May 2025
Natural Language Generation in Healthcare: A Review of Methods and Applications Mengxian Lyu Xiaohan Li Ziyi Chen Jinqian Pan Cheng Peng Sankalp Talankar Yonghui Wu LM&MA 38 0 0 07 May 2025
Engineering Artificial Intelligence: Framework, Challenges, and Future Direction Jay Lee Hanqi Su Dai-Yan Ji Takanobu Minami AI4CE 46 0 0 03 Apr 2025
Attention Condensation via Sparsity Induced Regularized Training Eli Sason Darya Frolova Boris Nazarov Felix Goldberd 95 0 0 03 Mar 2025
Transformer Meets Twicing: Harnessing Unattended Residual Information Laziz U. Abdullaev Tan M. Nguyen 37 2 0 02 Mar 2025
Personalizing Education through an Adaptive LMS with Integrated LLMs Kyle Spriggs Meng Cheng Lau Kalpdrum Passi AI4Ed 50 0 0 24 Jan 2025
Cooperative and Asynchronous Transformer-based Mission Planning for Heterogeneous Teams of Mobile Robots Milad Farjadnasab Shahin Sirouspour 33 0 0 08 Oct 2024
Attention layers provably solve single-location regression P. Marion Raphael Berthier Gérard Biau Claire Boyer 57 2 0 02 Oct 2024
Sparse Low-Ranked Self-Attention Transformer for Remaining Useful Lifetime Prediction of Optical Fiber Amplifiers Dominic Schneider Lutz Rapp 22 0 0 22 Sep 2024
LLM-3D Print: Large Language Models To Monitor and Control 3D Printing Yayati Jadhav P. Pak Amir Barati Farimani AI4CE 81 8 0 26 Aug 2024
OccMamba: Semantic Occupancy Prediction with State Space Models Heng Li Yuenan Hou Xiaohan Xing Xiao Sun Xiao Sun Yanyong Zhang Mamba 48 2 0 19 Aug 2024
radarODE: An ODE-Embedded Deep Learning Model for Contactless ECG Reconstruction from Millimeter-Wave Radar Yizheng Wu Jun Cen Xingyi Li Rui Yang Yutao Yue Guo-Shing Lin 22 3 0 03 Aug 2024
Rethinking Transformer-based Multi-document Summarization: An Empirical Investigation Congbo Ma Wei Emma Zhang Dileepa Pitawela Haojie Zhuang Yanfeng Shu 19 0 0 16 Jul 2024
Using LLMs to label medical papers according to the CIViC evidence model Markus Hisch Xing David Wang 31 0 0 05 Jul 2024
When Search Engine Services meet Large Language Models: Visions and Challenges Haoyi Xiong Jiang Bian Yuchen Li Xuhong Li Mengnan Du Shuaiqiang Wang Dawei Yin Sumi Helal 43 28 0 28 Jun 2024
SoK: Leveraging Transformers for Malware Analysis Pradip Kunwar Kshitiz Aryal Maanak Gupta Mahmoud Abdelsalam Elisa Bertino 82 0 0 27 May 2024
Spiking-PhysFormer: Camera-Based Remote Photoplethysmography with Parallel Spike-driven Transformer Mingxuan Liu Jiankai Tang Haoxiang Li Jiahao Qi Siwei Li Kegang Wang Yuntao wang Hong Chen Yuntao Wang Hong Chen 89 13 0 07 Feb 2024
CascadedGaze: Efficiency in Global Context Extraction for Image Restoration Amirhosein Ghasemabadi Muhammad Kamran Janjua Mohammad Salameh Chunhua Zhou Fengyu Sun Di Niu 22 11 0 26 Jan 2024
Early and Accurate Detection of Tomato Leaf Diseases Using TomFormer Asim Khan Umair Nawaz K. Lochan Lakmal D. Seneviratne Irfan Hussain MedIm 17 4 0 26 Dec 2023
Learning Human Action Recognition Representations Without Real Humans Howard Zhong Samarth Mishra Donghyun Kim SouYoung Jin Rameswar Panda Hildegard Kuehne Leonid Karlinsky Venkatesh Saligrama Aude Oliva Rogerio Feris 24 3 0 10 Nov 2023
OmniVec: Learning robust representations with cross modal sharing Siddharth Srivastava Gaurav Sharma SSL 16 64 0 07 Nov 2023
Audio-AdapterFusion: A Task-ID-free Approach for Efficient and Non-Destructive Multi-task Speech Recognition Hillary Ngai Rohan Agrawal Neeraj Gaur Ronny Huang Parisa Haghani P. M. Mengibar MoMe 16 0 0 17 Oct 2023
Transformer Choice Net: A Transformer Neural Network for Choice Prediction Hanzhao Wang Xiaocheng Li K. Talluri 19 2 0 12 Oct 2023
Multiscale Contextual Learning for Speech Emotion Recognition in Emergency Call Center Conversations Théo Deschamps-Berger L. Lamel Laurence Devillers 19 2 0 28 Aug 2023
Visual Prompt Flexible-Modal Face Anti-Spoofing Zitong Yu Rizhao Cai Yawen Cui Ajian Liu Changsheng Chen 30 6 0 26 Jul 2023
Solving Large-scale Spatial Problems with Convolutional Neural Networks Damian Owerko Charilaos I. Kanatsoulis Alejandro Ribeiro 14 2 0 14 Jun 2023
Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers Sotiris Anagnostidis Dario Pavllo Luca Biggio Lorenzo Noci Aurélien Lucchi Thomas Hofmann 32 51 0 25 May 2023
F-PABEE: Flexible-patience-based Early Exiting for Single-label and Multi-label text Classification Tasks Xiangxiang Gao Wei-wei Zhu Jiasheng Gao Congrui Yin VLM 19 12 0 21 May 2023
The emergence of clusters in self-attention dynamics Borjan Geshkovski Cyril Letrouit Yury Polyanskiy Philippe Rigollet 19 46 0 09 May 2023
NoiseTrans: Point Cloud Denoising with Transformers Guangzhe Hou G. Qin Minghui Sun Yanhua Liang Jie Yan Zhonghan Zhang 3DPC ViT 15 2 0 24 Apr 2023
DropDim: A Regularization Method for Transformer Networks Hao Zhang Dan Qu Kejia Shao Xu Yang 9 12 0 20 Apr 2023
Prak: An automatic phonetic alignment tool for Czech V. Hanzl Adléta Hanzlová 17 0 0 17 Apr 2023
Experts' cognition-driven safe noisy labels learning for precise segmentation of residual tumor in breast cancer Yongquan Yang Jie Chen Yani Wei Mohammad H. Alobaidi Hong Bu NoLa 40 1 0 13 Apr 2023
SELFormer: Molecular Representation Learning via SELFIES Language Models Atakan Yüksel Erva Ulusoy Atabey Ünlü Tunca Dogan 25 54 0 10 Apr 2023
Transformer Utilization in Medical Image Segmentation Networks Saikat Roy Gregor Koehler Michael Baumgartner Constantin Ulrich Jens Petersen Fabian Isensee Klaus Maier-Hein ViT MedIm 11 2 0 09 Apr 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 22 39 0 07 Apr 2023
Improving Transformer Performance for French Clinical Notes Classification Using Mixture of Experts on a Limited Dataset Thanh-Dung Le P. Jouvet R. Noumeir MoE MedIm 57 5 0 22 Mar 2023
Online Transformers with Spiking Neurons for Fast Prosthetic Hand Control Nathan Leroux Jan Finkbeiner Emre Neftci 18 9 0 21 Mar 2023
ChatGPT: Jack of all trades, master of none Jan Kocoñ Igor Cichecki Oliwier Kaszyca Mateusz Kochanek Dominika Szydło ... Maciej Piasecki Lukasz Radliñski Konrad Wojtasik Stanislaw Wo'zniak Przemyslaw Kazienko AI4MH 15 518 0 21 Feb 2023
Mask-guided BERT for Few Shot Text Classification Wenxiong Liao Zheng Liu Haixing Dai Zihao Wu Yiyang Zhang ... Dajiang Zhu Tianming Liu Sheng R. Li Xiang Li Hongmin Cai VLM 36 39 0 21 Feb 2023
From paintbrush to pixel: A review of deep neural networks in AI-generated art Anne-Sofie Maerten Derya Soydaner 25 22 0 14 Feb 2023
Enhancing Face Recognition with Latent Space Data Augmentation and Facial Posture Reconstruction Soroush Hashemifar Abdolreza Marefat Javad Hassannataj Joloudari H. Hassanpour CVBM 15 11 0 27 Jan 2023
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 112 36 0 15 Dec 2022
a survey on GPT-3 M. Zong Bhaskar Krishnamachari 25 34 0 01 Dec 2022
Parameter-Efficient Transformer with Hybrid Axial-Attention for Medical Image Segmentation Yiyue Hu Lei Zhang Nan Mu Leijun Liu ViT MedIm 12 1 0 17 Nov 2022
Revisiting Softmax for Uncertainty Approximation in Text Classification Andreas Nugaard Holm Dustin Wright Isabelle Augenstein BDL UQCV 14 8 0 25 Oct 2022
Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation Botao Yu Peiling Lu Rui Wang Wei Hu Xu Tan Wei Ye Shikun Zhang Tao Qin Tie-Yan Liu MGen 14 54 0 19 Oct 2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Lingpeng Kong 3DV 39 9 0 14 Oct 2022
LARF: Two-level Attention-based Random Forests with a Mixture of Contamination Models A. Konstantinov Lev V. Utkin 22 0 0 11 Oct 2022
A Transformer-based deep neural network model for SSVEP classification Jianbo Chen Yangsong Zhang Yudong Pan Peng Xu Cuntai Guan 15 50 0 09 Oct 2022