Reformer: The Efficient Transformer

13 January 2020

Papers citing "Reformer: The Efficient Transformer"

50 / 391 papers shown

Title
ContextSpeech: Expressive and Efficient Text-to-Speech for Paragraph Reading Yujia Xiao Shaofei Zhang Xi Wang Xuejiao Tan Lei He Sheng Zhao Frank Soong Tan Lee 17 5 0 03 Jul 2023
PaReprop: Fast Parallelized Reversible Backpropagation Tyler Lixuan Zhu K. Mangalam 17 1 0 15 Jun 2023
Training-free Diffusion Model Adaptation for Variable-Sized Text-to-Image Synthesis Zhiyu Jin Xuli Shen Bin Li Xiangyang Xue 24 36 0 14 Jun 2023
Cross-LKTCN: Modern Convolution Utilizing Cross-Variable Dependency for Multivariate Time Series Forecasting Dependency for Multivariate Time Series Forecasting Donghao Luo Xue Wang BDL AI4TS 11 2 0 04 Jun 2023
Koopa: Learning Non-stationary Time Series Dynamics with Koopman Predictors Yong Liu Chenyu Li Jianmin Wang Mingsheng Long AI4TS 28 101 0 30 May 2023
Networked Time Series Imputation via Position-aware Graph Enhanced Variational Autoencoders Dingsu Wang Yuchen Yan Ruizhong Qiu Yada Zhu Kaiyu Guan A. Margenot Hanghang Tong AI4TS 35 28 0 29 May 2023
Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time Zichang Liu Aditya Desai Fangshuo Liao Weitao Wang Victor Xie Zhaozhuo Xu Anastasios Kyrillidis Anshumali Shrivastava 28 202 0 26 May 2023
Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers Sotiris Anagnostidis Dario Pavllo Luca Biggio Lorenzo Noci Aurélien Lucchi Thomas Hofmann 34 53 0 25 May 2023
Fourier Transformer: Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator Ziwei He Meng-Da Yang Minwei Feng Jingcheng Yin X. Wang Jingwen Leng Zhouhan Lin ViT 32 11 0 24 May 2023
RWKV: Reinventing RNNs for the Transformer Era Bo Peng Eric Alcaide Quentin G. Anthony Alon Albalak Samuel Arcadinho ... Qihang Zhao P. Zhou Qinghua Zhou Jian Zhu Rui-Jie Zhu 81 557 0 22 May 2023
FIT: Far-reaching Interleaved Transformers Ting-Li Chen Lala Li 21 12 0 22 May 2023
CageViT: Convolutional Activation Guided Efficient Vision Transformer Hao Zheng Jinbao Wang Xiantong Zhen H. Chen Jingkuan Song Feng Zheng ViT 17 0 0 17 May 2023
SoundStorm: Efficient Parallel Audio Generation Zalan Borsos Matthew Sharifi Damien Vincent Eugene Kharitonov Neil Zeghidour Marco Tagliasacchi 23 97 0 16 May 2023
DLUE: Benchmarking Document Language Understanding Ruoxi Xu Hongyu Lin Xinyan Guan Xianpei Han Yingfei Sun Le Sun ELM 21 0 0 16 May 2023
Treasure What You Have: Exploiting Similarity in Deep Neural Networks for Efficient Video Processing Hadjer Benmeziane Halima Bouzidi Hamza Ouarnoughi Ozcan Ozturk Smail Niar 36 0 0 10 May 2023
Vcc: Scaling Transformers to 128K Tokens or More by Prioritizing Important Tokens Zhanpeng Zeng Cole Hawkins Min-Fong Hong Aston Zhang Nikolaos Pappas Vikas Singh Shuai Zheng 19 6 0 07 May 2023
The Role of Global and Local Context in Named Entity Recognition Arthur Amalvy Vincent Labatut Richard Dufour 38 4 0 04 May 2023
Learning to Extrapolate: A Transductive Approach Aviv Netanyahu Abhishek Gupta Max Simchowitz K. Zhang Pulkit Agrawal 46 15 0 27 Apr 2023
Improving Autoregressive NLP Tasks via Modular Linearized Attention Victor Agostinelli Lizhong Chen 22 1 0 17 Apr 2023
Training Large Language Models Efficiently with Sparsity and Dataflow V. Srinivasan Darshan Gandhi Urmish Thakker R. Prabhakar MoE 30 6 0 11 Apr 2023
On Efficient Training of Large-Scale Deep Learning Models: A Literature Review Li Shen Yan Sun Zhiyuan Yu Liang Ding Xinmei Tian Dacheng Tao VLM 30 40 0 07 Apr 2023
Dialogue-Contextualized Re-ranking for Medical History-Taking Jian Zhu Ilya Valmianski Anitha Kannan 19 1 0 04 Apr 2023
SwiftFormer: Efficient Additive Attention for Transformer-based Real-time Mobile Vision Applications Abdelrahman M. Shaker Muhammad Maaz H. Rasheed Salman Khan Ming Yang F. Khan ViT 48 84 0 27 Mar 2023
You Only Segment Once: Towards Real-Time Panoptic Segmentation Jie Hu Linyan Huang Tianhe Ren Shengchuan Zhang Rongrong Ji Liujuan Cao SSeg 46 54 0 26 Mar 2023
An Evaluation of Memory Optimization Methods for Training Neural Networks Xiaoxuan Liu Siddharth Jha Alvin Cheung 21 0 0 26 Mar 2023
Towards Accurate Post-Training Quantization for Vision Transformer Yifu Ding Haotong Qin Qing-Yu Yan Z. Chai Junjie Liu Xiaolin K. Wei Xianglong Liu MQ 54 68 0 25 Mar 2023
FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization Pavan Kumar Anasosalu Vasu J. Gabriel Jeff J. Zhu Oncel Tuzel Anurag Ranjan ViT 37 152 0 24 Mar 2023
Bridging Stereo Geometry and BEV Representation with Reliable Mutual Interaction for Semantic Scene Completion Bohan Li Yasheng Sun Zhujin Liang Dalong Du Zhengbiao Zhu Xiaoefeng Wang Yunpeng Zhang Han Xiao Wenjun Zeng 3DV 33 10 0 24 Mar 2023
XWikiGen: Cross-lingual Summarization for Encyclopedic Text Generation in Low Resource Languages Dhaval Taunk Shivprasad Sagare Anupam Patil Shivansh Subramanian Manish Gupta Vasudeva Varma 17 3 0 22 Mar 2023
Transformers in Speech Processing: A Survey S. Latif Aun Zaidi Heriberto Cuayáhuitl Fahad Shamshad Moazzam Shoukat Junaid Qadir 42 47 0 21 Mar 2023
HDformer: A Higher Dimensional Transformer for Diabetes Detection Utilizing Long Range Vascular Signals Ella Lan MedIm 20 1 0 17 Mar 2023
Align and Attend: Multimodal Summarization with Dual Contrastive Losses Bo He Jun Wang Jielin Qiu Trung Bui Abhinav Shrivastava Zhaowen Wang 22 65 0 13 Mar 2023
A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPT Yihan Cao Siyu Li Yixin Liu Zhiling Yan Yutong Dai Philip S. Yu Lichao Sun 29 506 0 07 Mar 2023
Efficient and Explicit Modelling of Image Hierarchies for Image Restoration Yawei Li Yuchen Fan Xiaoyu Xiang D. Demandolx Rakesh Ranjan Radu Timofte Luc Van Gool 21 173 0 01 Mar 2023
Single-Cell Multimodal Prediction via Transformers Wenzhuo Tang Haifang Wen Renming Liu Jiayuan Ding Wei Jin Yuying Xie Hui Liu Jiliang Tang AI4CE 24 11 0 01 Mar 2023
Weather2K: A Multivariate Spatio-Temporal Benchmark Dataset for Meteorological Forecasting Based on Real-Time Observation Data from Ground Weather Stations Xun Zhu Yutong Xiong Ming Wu Gaozhen Nie Bin Zhang Ziheng Yang AI4TS 23 17 0 21 Feb 2023
Efficiency 360: Efficient Vision Transformers Badri N. Patro Vijay Srinivas Agneeswaran 26 6 0 16 Feb 2023
The Framework Tax: Disparities Between Inference Efficiency in NLP Research and Deployment Jared Fernandez Jacob Kahn Clara Na Yonatan Bisk Emma Strubell FedML 28 10 0 13 Feb 2023
Efficient Attention via Control Variates Lin Zheng Jianbo Yuan Chong-Jun Wang Lingpeng Kong 28 18 0 09 Feb 2023
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers K. Choromanski Shanda Li Valerii Likhosherstov Kumar Avinava Dubey Shengjie Luo Di He Yiming Yang Tamás Sarlós Thomas Weingarten Adrian Weller 31 8 0 03 Feb 2023
Mnemosyne: Learning to Train Transformers with Transformers Deepali Jain K. Choromanski Kumar Avinava Dubey Sumeet Singh Vikas Sindhwani Tingnan Zhang Jie Tan OffRL 33 9 0 02 Feb 2023
FV-MgNet: Fully Connected V-cycle MgNet for Interpretable Time Series Forecasting Jianqing Zhu Juncai He Lian Zhang Jinchao Xu 26 3 0 02 Feb 2023
Exploring Attention Map Reuse for Efficient Transformer Neural Networks Kyuhong Shim Jungwook Choi Wonyong Sung ViT 22 3 0 29 Jan 2023
A Comparative Study of Pretrained Language Models for Long Clinical Text Yikuan Li R. M. Wehbe F. Ahmad Hanyin Wang Yuan Luo LM&MA ELM VLM MedIm 24 79 0 27 Jan 2023
Open Problems in Applied Deep Learning M. Raissi AI4CE 39 2 0 26 Jan 2023
Understanding and Improving Deep Graph Neural Networks: A Probabilistic Graphical Model Perspective Jiayuan Chen Xiang Zhang Yinfei Xu Tianli Zhao Renjie Xie Wei Xu GNN BDL 21 0 0 25 Jan 2023
Effective End-to-End Vision Language Pretraining with Semantic Visual Loss Xiaofeng Yang Fayao Liu Guosheng Lin VLM 19 7 0 18 Jan 2023
Dynamic Grained Encoder for Vision Transformers Lin Song Songyang Zhang Songtao Liu Zeming Li Xuming He Hongbin Sun Jian-jun Sun Nanning Zheng ViT 26 34 0 10 Jan 2023
Automating Nearest Neighbor Search Configuration with Constrained Optimization Philip Sun Ruiqi Guo Surinder Kumar 23 7 0 04 Jan 2023
Hungry Hungry Hippos: Towards Language Modeling with State Space Models Daniel Y. Fu Tri Dao Khaled Kamal Saab A. Thomas Atri Rudra Christopher Ré 67 370 0 28 Dec 2022