Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
1901.10430
Cited By
v1
v2 (latest)
Pay Less Attention with Lightweight and Dynamic Convolutions
International Conference on Learning Representations (ICLR), 2019
29 January 2019
Felix Wu
Angela Fan
Alexei Baevski
Yann N. Dauphin
Michael Auli
Re-assign community
ArXiv (abs)
PDF
HTML
Papers citing
"Pay Less Attention with Lightweight and Dynamic Convolutions"
50 / 337 papers shown
Probabilistic Attention for Interactive Segmentation
Neural Information Processing Systems (NeurIPS), 2021
Prasad Gabbur
Manjot Bilkhu
J. Movellan
183
13
0
23 Jun 2021
LV-BERT: Exploiting Layer Variety for BERT
Findings (Findings), 2021
Weihao Yu
Zihang Jiang
Fei Chen
Qibin Hou
Jiashi Feng
MQ
153
0
0
22 Jun 2021
Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation
Srinadh Bhojanapalli
Ayan Chakrabarti
Himanshu Jain
Sanjiv Kumar
Michal Lukasik
Andreas Veit
110
10
0
16 Jun 2021
HELP: Hardware-Adaptive Efficient Latency Prediction for NAS via Meta-Learning
Hayeon Lee
Sewoong Lee
Song Chong
Sung Ju Hwang
258
31
0
16 Jun 2021
Coreference-Aware Dialogue Summarization
Zhengyuan Liu
Ke Shi
Nancy F. Chen
150
69
0
16 Jun 2021
GroupBERT: Enhanced Transformer Architecture with Efficient Grouped Structures
Ivan Chelombiev
Daniel Justus
Douglas Orr
A. Dietrich
Frithjof Gressmann
A. Koliousis
Carlo Luschi
141
5
0
10 Jun 2021
Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2021
Tyler A. Chang
Yifan Xu
Weijian Xu
Zhuowen Tu
ViT
105
16
0
10 Jun 2021
A Survey of Transformers
AI Open (AO), 2021
Tianyang Lin
Yuxin Wang
Xiangyang Liu
Xipeng Qiu
ViT
441
1,380
0
08 Jun 2021
On the Connection between Local Attention and Dynamic Depth-wise Convolution
International Conference on Learning Representations (ICLR), 2021
Qi Han
Zejia Fan
Jingdong Sun
Lei-huan Sun
Ming-Ming Cheng
Jiaying Liu
Jingdong Wang
ViT
356
132
0
08 Jun 2021
On the Language Coverage Bias for Neural Machine Translation
Findings (Findings), 2021
Shuo Wang
Zhaopeng Tu
Zhixing Tan
Shuming Shi
Maosong Sun
Yang Liu
116
21
0
07 Jun 2021
Neural Implicit 3D Shapes from Single Images with Spatial Patterns
International Conference on Image and Graphics (ICIG), 2021
Yixin Zhuang
Yunzhe Liu
Yujie Wang
Baoquan Chen
3DPC
159
0
0
06 Jun 2021
Attention mechanisms and deep learning for machine vision: A survey of the state of the art
A. M. Hafiz
S. A. Parah
R. A. Bhat
224
56
0
03 Jun 2021
Container: Context Aggregation Network
Neural Information Processing Systems (NeurIPS), 2021
Peng Gao
Jiasen Lu
Jiaming Song
Roozbeh Mottaghi
Aniruddha Kembhavi
ViT
271
81
0
02 Jun 2021
Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translation
Annual Meeting of the Association for Computational Linguistics (ACL), 2021
Wenxiang Jiao
Xing Wang
Zhaopeng Tu
Shuming Shi
Michael R. Lyu
Irwin King
UQLM
147
39
0
02 Jun 2021
DLA-Net: Learning Dual Local Attention Features for Semantic Segmentation of Large-Scale Building Facade Point Clouds
Pattern Recognition (Pattern Recogn.), 2021
Yanfei Su
Weiquan Liu
Zhimin Yuan
Ming Cheng
Zhihong Zhang
Xuelun Shen
Cheng-Yu Wang
3DPC
274
49
0
01 Jun 2021
Memory-Efficient Differentiable Transformer Architecture Search
Findings (Findings), 2021
Yuekai Zhao
Li Dong
Yelong Shen
Zhihua Zhang
Furu Wei
Weizhu Chen
ViT
217
20
0
31 May 2021
Good for Misconceived Reasons: An Empirical Revisiting on the Need for Visual Context in Multimodal Machine Translation
Annual Meeting of the Association for Computational Linguistics (ACL), 2021
Zhiyong Wu
Lingpeng Kong
W. Bi
Xiang Li
B. Kao
LRM
134
97
0
30 May 2021
NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search
Knowledge Discovery and Data Mining (KDD), 2021
Jin Xu
Xu Tan
Renqian Luo
Kaitao Song
Jian Li
Tao Qin
Tie-Yan Liu
MQ
139
89
0
30 May 2021
An Attention Free Transformer
Shuangfei Zhai
Walter A. Talbott
Nitish Srivastava
Chen Huang
Hanlin Goh
Ruixiang Zhang
J. Susskind
ViT
381
162
0
28 May 2021
Controllable Abstractive Dialogue Summarization with Sketch Supervision
Findings (Findings), 2021
Chien-Sheng Wu
Linqing Liu
Wenhao Liu
Pontus Stenetorp
Caiming Xiong
216
57
0
28 May 2021
TranSmart: A Practical Interactive Machine Translation System
Guoping Huang
Lemao Liu
Xing Wang
Longyue Wang
Huayang Li
Zhaopeng Tu
Chengyang Huang
Shuming Shi
170
36
0
27 May 2021
Learning Language Specific Sub-network for Multilingual Machine Translation
Annual Meeting of the Association for Computational Linguistics (ACL), 2021
Zehui Lin
Liwei Wu
Mingxuan Wang
Lei Li
274
87
0
19 May 2021
Pay Attention to MLPs
Neural Information Processing Systems (NeurIPS), 2021
Hanxiao Liu
Zihang Dai
David R. So
Quoc V. Le
AI4CE
567
796
0
17 May 2021
Dynamic Pooling Improves Nanopore Base Calling Accuracy
IEEE/ACM Transactions on Computational Biology & Bioinformatics (TCBB), 2021
V. Boža
Peter Perešíni
Broňa Brejová
T. Vinař
160
4
0
16 May 2021
The Volctrans Neural Speech Translation System for IWSLT 2021
International Workshop on Spoken Language Translation (IWSLT), 2021
Chengqi Zhao
Zhicheng Liu
Jian-Fei Tong
Tao Wang
Mingxuan Wang
Rong Ye
Qianqian Dong
Jun Cao
Lei Li
207
9
0
16 May 2021
Not All Memories are Created Equal: Learning to Forget by Expiring
International Conference on Machine Learning (ICML), 2021
Sainbayar Sukhbaatar
Da Ju
Spencer Poff
Stephen Roller
Arthur Szlam
Jason Weston
Angela Fan
CLL
239
36
0
13 May 2021
Poolingformer: Long Document Modeling with Pooling Attention
International Conference on Machine Learning (ICML), 2021
Hang Zhang
Yeyun Gong
Yelong Shen
Weisheng Li
Jiancheng Lv
Nan Duan
Weizhu Chen
183
115
0
10 May 2021
Are Pre-trained Convolutions Better than Pre-trained Transformers?
Annual Meeting of the Association for Computational Linguistics (ACL), 2021
Yi Tay
Mostafa Dehghani
J. Gupta
Dara Bahri
V. Aribandi
Zhen Qin
Donald Metzler
AI4CE
177
51
0
07 May 2021
MLP-Mixer: An all-MLP Architecture for Vision
Neural Information Processing Systems (NeurIPS), 2021
Ilya O. Tolstikhin
N. Houlsby
Alexander Kolesnikov
Lucas Beyer
Xiaohua Zhai
...
Andreas Steiner
Daniel Keysers
Jakob Uszkoreit
Mario Lucic
Alexey Dosovitskiy
1.2K
3,284
0
04 May 2021
Review of end-to-end speech synthesis technology based on deep learning
Zhaoxi Mu
Xinyu Yang
Yizhuo Dong
AuLLM
ALM
202
31
0
20 Apr 2021
Knowledge Neurons in Pretrained Transformers
Annual Meeting of the Association for Computational Linguistics (ACL), 2021
Damai Dai
Li Dong
Y. Hao
Zhifang Sui
Baobao Chang
Furu Wei
KELM
MU
538
574
0
18 Apr 2021
How to Train BERT with an Academic Budget
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
Peter Izsak
Moshe Berchansky
Omer Levy
338
128
0
15 Apr 2021
UniDrop: A Simple yet Effective Technique to Improve Transformer without Extra Cost
North American Chapter of the Association for Computational Linguistics (NAACL), 2021
Zhen Wu
Lijun Wu
Qi Meng
Ziheng Lu
Shufang Xie
Tao Qin
Xinyu Dai
Tie-Yan Liu
201
25
0
11 Apr 2021
Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog
North American Chapter of the Association for Computational Linguistics (NAACL), 2021
Arun Babu
Akshat Shrivastava
Armen Aghajanyan
Ahmed Aly
Angela Fan
Marjan Ghazvininejad
134
21
0
11 Apr 2021
Learning Graph Structures with Transformer for Multivariate Time Series Anomaly Detection in IoT
IEEE Internet of Things Journal (IEEE IoT Journal), 2021
Zekai Chen
Dingshuo Chen
Xiao Zhang
Zixuan Yuan
Xiuzhen Cheng
AI4TS
319
456
0
08 Apr 2021
Do We Need Anisotropic Graph Neural Networks?
International Conference on Learning Representations (ICLR), 2021
Shyam A. Tailor
Felix L. Opolka
Pietro Lio
Nicholas D. Lane
348
42
0
03 Apr 2021
Dual Contrastive Loss and Attention for GANs
IEEE International Conference on Computer Vision (ICCV), 2021
Ning Yu
Guilin Liu
Aysegül Dündar
Andrew Tao
Bryan Catanzaro
Larry S. Davis
Mario Fritz
GAN
325
68
0
31 Mar 2021
CvT: Introducing Convolutions to Vision Transformers
IEEE International Conference on Computer Vision (ICCV), 2021
Haiping Wu
Bin Xiao
Noel Codella
Xiyang Dai
Xiyang Dai
Lu Yuan
Lei Zhang
ViT
495
2,273
0
29 Mar 2021
Unified Graph Structured Models for Video Understanding
IEEE International Conference on Computer Vision (ICCV), 2021
Anurag Arnab
Chen Sun
Cordelia Schmid
226
52
0
29 Mar 2021
Mask Attention Networks: Rethinking and Strengthen Transformer
North American Chapter of the Association for Computational Linguistics (NAACL), 2021
Zhihao Fan
Yeyun Gong
Dayiheng Liu
Zhongyu Wei
Siyuan Wang
Jian Jiao
Nan Duan
Ruofei Zhang
Xuanjing Huang
150
78
0
25 Mar 2021
Finetuning Pretrained Transformers into RNNs
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
Jungo Kasai
Hao Peng
Yizhe Zhang
Dani Yogatama
Gabriel Ilharco
Nikolaos Pappas
Yi Mao
Weizhu Chen
Noah A. Smith
317
81
0
24 Mar 2021
IOT: Instance-wise Layer Reordering for Transformer Structures
International Conference on Learning Representations (ICLR), 2021
Jinhua Zhu
Lijun Wu
Ziheng Lu
Shufang Xie
Tao Qin
Wen-gang Zhou
Houqiang Li
Tie-Yan Liu
189
8
0
05 Mar 2021
Random Feature Attention
International Conference on Learning Representations (ICLR), 2021
Hao Peng
Nikolaos Pappas
Dani Yogatama
Roy Schwartz
Noah A. Smith
Lingpeng Kong
329
401
0
03 Mar 2021
Do Transformer Modifications Transfer Across Implementations and Applications?
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
Sharan Narang
Hyung Won Chung
Yi Tay
W. Fedus
Thibault Févry
...
Wei Li
Nan Ding
Jake Marcus
Adam Roberts
Colin Raffel
215
134
0
23 Feb 2021
Axial Residual Networks for CycleGAN-based Voice Conversion
J. You
Gyuhyeon Nam
Dalhyun Kim
Gyeongsu Chae
170
3
0
16 Feb 2021
MUFASA: Multimodal Fusion Architecture Search for Electronic Health Records
AAAI Conference on Artificial Intelligence (AAAI), 2021
Zhen Xu
David R. So
Andrew M. Dai
Mamba
338
65
0
03 Feb 2021
The heads hypothesis: A unifying statistical approach towards understanding multi-headed attention in BERT
AAAI Conference on Artificial Intelligence (AAAI), 2021
Madhura Pande
Aakriti Budhraja
Preksha Nema
Pratyush Kumar
Mitesh M. Khapra
177
20
0
22 Jan 2021
Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformers
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
Machel Reid
Edison Marrese-Taylor
Y. Matsuo
MoE
340
57
0
01 Jan 2021
Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers
Computer Vision and Pattern Recognition (CVPR), 2020
Sixiao Zheng
Jiachen Lu
Hengshuang Zhao
Xiatian Zhu
Zekun Luo
...
Yanwei Fu
Jianfeng Feng
Tao Xiang
Juil Sock
Li Zhang
ViT
534
3,400
0
31 Dec 2020
Neural Machine Translation: A Review of Methods, Resources, and Tools
AI Open (AO), 2020
Zhixing Tan
Shuo Wang
Zonghan Yang
Gang Chen
Xuancheng Huang
Maosong Sun
Yang Liu
3DV
AI4TS
256
123
0
31 Dec 2020
Previous
1
2
3
4
5
6
7
Next
