Reformer: The Efficient Transformer

13 January 2020

Papers citing "Reformer: The Efficient Transformer"

50 / 391 papers shown

Title
JEMMA: An Extensible Java Dataset for ML4Code Applications Anjan Karmakar Miltiadis Allamanis Romain Robbes VLM 21 3 0 18 Dec 2022
Convolution-enhanced Evolving Attention Networks Yujing Wang Yaming Yang Zhuowan Li Jiangang Bai Mingliang Zhang Xiangtai Li J. Yu Ce Zhang Gao Huang Yu Tong ViT 24 6 0 16 Dec 2022
Efficient Long Sequence Modeling via State Space Augmented Transformer Simiao Zuo Xiaodong Liu Jian Jiao Denis Xavier Charles Eren Manavoglu Tuo Zhao Jianfeng Gao 122 36 0 15 Dec 2022
Rethinking Vision Transformers for MobileNet Size and Speed Yanyu Li Ju Hu Yang Wen Georgios Evangelidis Kamyar Salahi Yanzhi Wang Sergey Tulyakov Jian Ren ViT 30 159 0 15 Dec 2022
Full Contextual Attention for Multi-resolution Transformers in Semantic Segmentation Loic Themyr Clément Rambour Nicolas Thome Toby Collins Alexandre Hostettler ViT 19 10 0 15 Dec 2022
UNETR++: Delving into Efficient and Accurate 3D Medical Image Segmentation Abdelrahman M. Shaker Muhammad Maaz H. Rasheed Salman Khan Ming Yang F. Khan MedIm 40 129 0 08 Dec 2022
Transformers for End-to-End InfoSec Tasks: A Feasibility Study Ethan M. Rudd Mohammad Saidur Rahman Philip Tully 22 5 0 05 Dec 2022
FECAM: Frequency Enhanced Channel Attention Mechanism for Time Series Forecasting Maowei Jiang Pengyu Zeng Kai-Ming Wang Huan Liu Wenbo Chen Haoran Liu AI4TS 29 50 0 02 Dec 2022
Deep neural network techniques for monaural speech enhancement: state of the art analysis P. Ochieng 28 21 0 01 Dec 2022
sEHR-CE: Language modelling of structured EHR data for efficient and generalizable patient cohort expansion Anna Munoz-Farre Harry Rose S. A. Cakiroglu 9 4 0 30 Nov 2022
FsaNet: Frequency Self-attention for Semantic Segmentation Fengyu Zhang Ashkan Panahi Guangjun Gao AI4TS 26 28 0 28 Nov 2022
Dynamic Feature Pruning and Consolidation for Occluded Person Re-Identification Yuteng Ye Hang Zhou Jiale Cai Chenxing Gao Youjia Zhang Junle Wang Qiang Hu Junqing Yu Wei Yang 23 6 0 27 Nov 2022
Bypass Exponential Time Preprocessing: Fast Neural Network Training via Weight-Data Correlation Preprocessing Josh Alman Jiehao Liang Zhao-quan Song Ruizhe Zhang Danyang Zhuo 71 31 0 25 Nov 2022
MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attention Wenyuan Zeng Meng Li Wenjie Xiong Tong Tong Wen-jie Lu Jin Tan Runsheng Wang Ru Huang 22 20 0 25 Nov 2022
RNTrajRec: Road Network Enhanced Trajectory Recovery with Spatial-Temporal Transformer Yuqi Chen Hanyuan Zhang Weiwei Sun B. Zheng 27 38 0 23 Nov 2022
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention Zineng Tang Jaemin Cho Jie Lei Mohit Bansal VLM 24 9 0 21 Nov 2022
SeDR: Segment Representation Learning for Long Documents Dense Retrieval Junying Chen Qingcai Chen Dongfang Li Yutao Huang 20 6 0 20 Nov 2022
DeepParliament: A Legal domain Benchmark & Dataset for Parliament Bills Prediction Ankit Pal AILaw 28 0 0 15 Nov 2022
Evade the Trap of Mediocrity: Promoting Diversity and Novelty in Text Generation via Concentrating Attention Wenhao Li Xiaoyuan Yi Jinyi Hu Maosong Sun Xing Xie 23 0 0 14 Nov 2022
HigeNet: A Highly Efficient Modeling for Long Sequence Time Series Prediction in AIOps Jiajia Li Feng Tan Cheng He Zikai Wang Haitao Song Lingfei Wu Pengwei Hu 15 0 0 13 Nov 2022
Equivariance with Learned Canonicalization Functions Sekouba Kaba Arnab Kumar Mondal Yan Zhang Yoshua Bengio Siamak Ravanbakhsh 38 61 0 11 Nov 2022
ViTALiTy: Unifying Low-rank and Sparse Approximation for Vision Transformer Acceleration with a Linear Taylor Attention Jyotikrishna Dass Shang Wu Huihong Shi Chaojian Li Zhifan Ye Zhongfeng Wang Yingyan Lin 17 49 0 09 Nov 2022
Efficiently Scaling Transformer Inference Reiner Pope Sholto Douglas Aakanksha Chowdhery Jacob Devlin James Bradbury Anselm Levskaya Jonathan Heek Kefan Xiao Shivani Agrawal J. Dean 32 292 0 09 Nov 2022
How Far are We from Robust Long Abstractive Summarization? Huan Yee Koh Jiaxin Ju He Zhang Ming Liu Shirui Pan HILM 23 39 0 30 Oct 2022
Transformers meet Stochastic Block Models: Attention with Data-Adaptive Sparsity and Cost Sungjun Cho Seonwoo Min Jinwoo Kim Moontae Lee Honglak Lee Seunghoon Hong 32 3 0 27 Oct 2022
Clinically-Inspired Multi-Agent Transformers for Disease Trajectory Forecasting from Multimodal Data Huy Hoang Nguyen Matthew B. Blaschko S. Saarakkala A. Tiulpin MedIm AI4CE 48 15 0 25 Oct 2022
Graphically Structured Diffusion Models Christian Weilbach William Harvey Frank D. Wood DiffM 35 7 0 20 Oct 2022
An efficient graph generative model for navigating ultra-large combinatorial synthesis libraries Aryan Pedawi P. Gniewek Chao-Ling Chang Brandon M. Anderson H. V. D. Bedem 22 5 0 19 Oct 2022
Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation Botao Yu Peiling Lu Rui Wang Wei Hu Xu Tan Wei Ye Shikun Zhang Tao Qin Tie-Yan Liu MGen 25 54 0 19 Oct 2022
The Devil in Linear Transformer Zhen Qin Xiaodong Han Weixuan Sun Dongxu Li Lingpeng Kong Nick Barnes Yiran Zhong 34 70 0 19 Oct 2022
ViTCoD: Vision Transformer Acceleration via Dedicated Algorithm and Accelerator Co-Design Haoran You Zhanyi Sun Huihong Shi Zhongzhi Yu Yang Katie Zhao Yongan Zhang Chaojian Li Baopu Li Yingyan Lin ViT 17 76 0 18 Oct 2022
CAB: Comprehensive Attention Benchmarking on Long Sequence Modeling Jinchao Zhang Shuyang Jiang Jiangtao Feng Lin Zheng Lingpeng Kong 3DV 41 9 0 14 Oct 2022
An Exploration of Hierarchical Attention Transformers for Efficient Long Document Classification Ilias Chalkidis Xiang Dai Manos Fergadiotis Prodromos Malakasiotis Desmond Elliott 34 33 0 11 Oct 2022
Hierarchical3D Adapters for Long Video-to-text Summarization Pinelopi Papalampidi Mirella Lapata VGen 27 12 0 10 Oct 2022
Bird-Eye Transformers for Text Generation Models Lei Sha Yuhang Song Yordan Yordanov Tommaso Salvatori Thomas Lukasiewicz 19 0 0 08 Oct 2022
KG-MTT-BERT: Knowledge Graph Enhanced BERT for Multi-Type Medical Text Classification Yong He Cheng Wang Shun Zhang Na Li Zhao Li Zhenyu Zeng AI4MH 39 10 0 08 Oct 2022
TimesNet: Temporal 2D-Variation Modeling for General Time Series Analysis Haixu Wu Teng Hu Yong Liu Hang Zhou Jianmin Wang Mingsheng Long AI4TS AIFin 48 705 0 05 Oct 2022
Movement Analytics: Current Status, Application to Manufacturing, and Future Prospects from an AI Perspective Peter Baumgartner Daniel V. Smith Mashud Rana Reena Kapoor Elena Tartaglia A. Schutt Ashfaqur Rahman John Taylor S. Dunstall 27 4 0 04 Oct 2022
Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning Weicong Liang Yuhui Yuan Henghui Ding Xiao Luo Weihong Lin Ding Jia Zheng-Wei Zhang Chao Zhang Hanhua Hu 25 25 0 03 Oct 2022
Grouped self-attention mechanism for a memory-efficient Transformer Bumjun Jung Yusuke Mukuta Tatsuya Harada AI4TS 12 3 0 02 Oct 2022
Spiking Neural Networks for event-based action recognition: A new task to understand their advantage Alex Vicente-Sola D. L. Manna Paul Kirkland G. D. Caterina Trevor J. Bihl 21 8 0 29 Sep 2022
Searching a High-Performance Feature Extractor for Text Recognition Network Hui Zhang Quanming Yao James T. Kwok X. Bai 28 7 0 27 Sep 2022
Explainable Graph Pyramid Autoformer for Long-Term Traffic Forecasting Weiheng Zhong Tanwi Mallick Hadi Meidani Jane Macfarlane Prasanna Balaprakash AI4TS 21 5 0 27 Sep 2022
Liquid Structural State-Space Models Ramin Hasani Mathias Lechner Tsun-Hsuan Wang Makram Chahine Alexander Amini Daniela Rus AI4TS 101 95 0 26 Sep 2022
Optimizing DNN Compilation for Distributed Training with Joint OP and Tensor Fusion Xiaodong Yi Shiwei Zhang Lansong Diao Chuan Wu Zhen Zheng Shiqing Fan Siyu Wang Jun Yang W. Lin 25 4 0 26 Sep 2022
Adapting Pretrained Text-to-Text Models for Long Text Sequences Wenhan Xiong Anchit Gupta Shubham Toshniwal Yashar Mehdad Wen-tau Yih RALM VLM 52 30 0 21 Sep 2022
An Efficient End-to-End Transformer with Progressive Tri-modal Attention for Multi-modal Emotion Recognition Yang Wu Pai Peng Zhenyu Zhang Yanyan Zhao Bing Qin 19 1 0 20 Sep 2022
Adaptable Butterfly Accelerator for Attention-based NNs via Hardware and Algorithm Co-design Hongxiang Fan Thomas C. P. Chau Stylianos I. Venieris Royson Lee Alexandros Kouris Wayne Luk Nicholas D. Lane Mohamed S. Abdelfattah 34 56 0 20 Sep 2022
Graph Reasoning Transformer for Image Parsing Dong Zhang Jinhui Tang Kwang-Ting Cheng ViT 24 16 0 20 Sep 2022
Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humans John J. Nay ELM AILaw 88 27 0 14 Sep 2022