CoAtNet: Marrying Convolution and Attention for All Data Sizes

9 June 2021

Mingxing Tan

Papers citing "CoAtNet: Marrying Convolution and Attention for All Data Sizes"

50 / 482 papers shown

Title
Attention Distillation: self-supervised vision transformer students need more guidance Kai Wang Fei Yang Joost van de Weijer ViT 17 16 0 03 Oct 2022
An In-depth Study of Stochastic Backpropagation J. Fang Ming Xu Hao Chen Bing Shuai Z. Tu Joseph Tighe BDL 27 1 0 30 Sep 2022
E-Branchformer: Branchformer with Enhanced merging for speech recognition Kwangyoun Kim Felix Wu Yifan Peng Jing Pan Prashant Sridhar Kyu Jeong Han Shinji Watanabe 50 105 0 30 Sep 2022
MobileViTv3: Mobile-Friendly Vision Transformer with Simple and Effective Fusion of Local, Global and Input Features S. Wadekar Abhishek Chaurasia ViT 98 87 0 30 Sep 2022
Exploring the Relationship between Architecture and Adversarially Robust Generalization Aishan Liu Shiyu Tang Siyuan Liang Ruihao Gong Boxi Wu Xianglong Liu Dacheng Tao AAML 21 18 0 28 Sep 2022
Attention is All They Need: Exploring the Media Archaeology of the Computer Vision Research Paper Sam Goree G. Appleby David J. Crandall Norman Su 27 2 0 22 Sep 2022
Mega: Moving Average Equipped Gated Attention Xuezhe Ma Chunting Zhou Xiang Kong Junxian He Liangke Gui Graham Neubig Jonathan May Luke Zettlemoyer 14 182 0 21 Sep 2022
Axially Expanded Windows for Local-Global Interaction in Vision Transformers Zhemin Zhang Xun Gong ViT 13 1 0 19 Sep 2022
VINet: Visual and Inertial-based Terrain Classification and Adaptive Navigation over Unknown Terrain Tianrui Guan Ruitao Song Zhixian Ye Liangjun Zhang 40 10 0 16 Sep 2022
Neural Networks Reduction via Lumping Dalila Ressi Riccardo Romanello S. Rossi Carla Piazza 22 4 0 15 Sep 2022
Joint Debiased Representation and Image Clustering Learning with Self-Supervision Shun Zheng JaeEun Nam Emilio Dorigatti Bernd Bischl Shekoofeh Azizi Mina Rezaei SSL 8 0 0 14 Sep 2022
Revisiting Neural Scaling Laws in Language and Vision Ibrahim M. Alabdulmohsin Behnam Neyshabur Xiaohua Zhai 151 102 0 13 Sep 2022
Socially Enhanced Situation Awareness from Microblogs using Artificial Intelligence: A Survey Rabindra Lamsal Aaron Harwood M. Read 32 20 0 13 Sep 2022
Communication-Efficient and Privacy-Preserving Feature-based Federated Transfer Learning Feng Wang M. C. Gursoy Senem Velipasalar 8 2 0 12 Sep 2022
Statistical Foundation Behind Machine Learning and Its Impact on Computer Vision Lei Zhang H. Shum VLM SSL 8 2 0 06 Sep 2022
AutoPET Challenge: Combining nn-Unet with Swin UNETR Augmented by Maximum Intensity Projection Classifier Lars Heiliger Zdravko Marinov Max Hasin André Ferreira Jana Fragemann ... D. Kersting Victor Alves Rainer Stiefelhagen Jan Egger Jens Kleesiek 11 9 0 02 Sep 2022
MAFormer: A Transformer Network with Multi-scale Attention Fusion for Visual Recognition Y. Wang H. Sun Xiaodi Wang Bin Zhang Chaonan Li Ying Xin Baochang Zhang Errui Ding Shumin Han ViT 23 9 0 31 Aug 2022
MRL: Learning to Mix with Attention and Convolutions Shlok Mohta Hisahiro Suganuma Yoshiki Tanaka 14 2 0 30 Aug 2022
Overparameterization from Computational Constraints Sanjam Garg S. Jha Saeed Mahloujifar Mohammad Mahmoody Mingyuan Wang 15 1 0 27 Aug 2022
Efficient Attention-free Video Shift Transformers Adrian Bulat Brais Martínez Georgios Tzimiropoulos ViT 27 1 0 23 Aug 2022
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks Wenhui Wang Hangbo Bao Li Dong Johan Bjorck Zhiliang Peng ... Kriti Aggarwal O. Mohammed Saksham Singhal Subhojit Som Furu Wei MLLM VLM ViT 49 628 0 22 Aug 2022
TaCo: Textual Attribute Recognition via Contrastive Learning Chang Nie Yiqing Hu Yanqiu Qu Hao Liu Deqiang Jiang Bo Ren 22 0 0 22 Aug 2022
Conviformers: Convolutionally guided Vision Transformer Mohit Vaishnav Thomas Fel I. F. Rodriguez Thomas Serre ViT 30 1 0 17 Aug 2022
SensorSCAN: Self-Supervised Learning and Deep Clustering for Fault Diagnosis in Chemical Processes Maksim Golyadkin Vitaliy Pozdnyakov L. Zhukov Ilya Makarov 17 8 0 17 Aug 2022
In the Eye of Transformer: Global-Local Correlation for Egocentric Gaze Estimation Bolin Lai Miao Liu Fiona Ryan James M. Rehg ViT 32 32 0 08 Aug 2022
Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model Di Wang Qiming Zhang Yufei Xu Jing Zhang Bo Du Dacheng Tao L. Zhang 23 242 0 08 Aug 2022
Combined CNN Transformer Encoder for Enhanced Fine-grained Human Action Recognition M. C. Leong Haosong Zhang Huibin Tan Liyuan Li J. Lim ViT 26 8 0 03 Aug 2022
A Novel Transformer Network with Shifted Window Cross-Attention for Spatiotemporal Weather Forecasting Alabi Bojesomo Hasan Al-Marzouqi P. Liatsis 11 9 0 02 Aug 2022
HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions Yongming Rao Wenliang Zhao Yansong Tang Jie Zhou Ser-Nam Lim Jiwen Lu ViT 20 252 0 28 Jul 2022
Convolutional Embedding Makes Hierarchical Vision Transformer Stronger Cong Wang Hongmin Xu Xiong Zhang Li Wang Zhitong Zheng Haifeng Liu ViT 12 20 0 27 Jul 2022
TreeSketchNet: From Sketch To 3D Tree Parameters Generation Gilda Manfredi N. Capece U. Erra M. Gruosso 8 11 0 25 Jul 2022
Online Continual Learning with Contrastive Vision Transformer Zhen Wang Liu Liu Yajing Kong Jiaxian Guo Dacheng Tao CLL 21 29 0 24 Jul 2022
HybMT: Hybrid Meta-Predictor based ML Algorithm for Fast Test Vector Generation Shruti Pandey J. Jayadeva S. Sarangi 20 1 0 22 Jul 2022
Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation Sunghwan Hong Seokju Cho Jisu Nam Stephen Lin Seung Wook Kim ViT 19 122 0 22 Jul 2022
Weakly Supervised Object Localization via Transformer with Implicit Spatial Calibration Haotian Bai Ruimao Zhang Jiong Wang Xiang Wan WSOL 37 35 0 21 Jul 2022
SplitMixer: Fat Trimmed From MLP-like Models Ali Borji Sikun Lin 21 3 0 21 Jul 2022
AutoDiCE: Fully Automated Distributed CNN Inference at the Edge Xiaotian Guo A. Pimentel T. Stefanov 11 1 0 20 Jul 2022
Vision Transformers: From Semantic Segmentation to Dense Prediction Li Zhang Jiachen Lu Sixiao Zheng Xinxuan Zhao Xiatian Zhu Yanwei Fu Tao Xiang Jianfeng Feng Philip H. S. Torr ViT 19 7 0 19 Jul 2022
Towards Trustworthy Healthcare AI: Attention-Based Feature Learning for COVID-19 Screening With Chest Radiography Kai Ma Pengcheng Xi K. Habashy Ashkan Ebadi Stéphane Tremblay Alexander Wong ViT MedIm 6 1 0 19 Jul 2022
Parameterization of Cross-Token Relations with Relative Positional Encoding for Vision MLP Zhicai Wang Y. Hao Xingyu Gao Hao Zhang Shuo Wang Tingting Mu Xiangnan He 16 8 0 15 Jul 2022
Next-ViT: Next Generation Vision Transformer for Efficient Deployment in Realistic Industrial Scenarios Jiashi Li Xin Xia W. Li Huixia Li Xing Wang Xuefeng Xiao Rui Wang Min Zheng Xin Pan ViT 17 149 0 12 Jul 2022
Pure Transformers are Powerful Graph Learners Jinwoo Kim Tien Dat Nguyen Seonwoo Min Sungjun Cho Moontae Lee Honglak Lee Seunghoon Hong 32 187 0 06 Jul 2022
Softmax-free Linear Transformers Jiachen Lu Junge Zhang Xiatian Zhu Jianfeng Feng Tao Xiang Li Zhang ViT 11 7 0 05 Jul 2022
FFCNet: Fourier Transform-Based Frequency Learning and Complex Convolutional Network for Colon Disease Classification Kaini Wang Yuting He Shuaishuai Zhuang Juzheng Miao Xiaopu He Ping Zhou Guanyu Yang Guangquan Zhou Shuo Li 17 15 0 04 Jul 2022
Rethinking Query-Key Pairwise Interactions in Vision Transformers Cheng-rong Li Yangxin Liu 31 0 0 01 Jul 2022
Measuring Forgetting of Memorized Training Examples Matthew Jagielski Om Thakkar Florian Tramèr Daphne Ippolito Katherine Lee ... Eric Wallace Shuang Song Abhradeep Thakurta Nicolas Papernot Chiyuan Zhang TDI 40 102 0 30 Jun 2022
Transfer Learning with Deep Tabular Models Roman Levin Valeriia Cherepanova Avi Schwarzschild Arpit Bansal C. B. Bruss Tom Goldstein A. Wilson Micah Goldblum OOD FedML LMTD 73 58 0 30 Jun 2022
ZoDIAC: Zoneout Dropout Injection Attention Calculation Zanyar Zohourianshahzadi Jugal Kalita 26 0 0 28 Jun 2022
RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network Vitaliy Chiley Vithursan Thangarasa Abhay Gupta Anshul Samar Joel Hestness D. DeCoste 29 8 0 28 Jun 2022
Revisiting Architecture-aware Knowledge Distillation: Smaller Models and Faster Search Taehyeon Kim Heesoo Myeong Se-Young Yun 19 2 0 27 Jun 2022