CoAtNet: Marrying Convolution and Attention for All Data Sizes

9 June 2021

Mingxing Tan

Papers citing "CoAtNet: Marrying Convolution and Attention for All Data Sizes"

50 / 482 papers shown

Title
Exploring Vision Transformers as Diffusion Learners He Cao Jianan Wang Tianhe Ren Xianbiao Qi Yihao Chen Yuan Yao L. Zhang 31 10 0 28 Dec 2022
Representation Separation for Semantic Segmentation with Vision Transformers Yuanduo Hong Huihui Pan Weichao Sun Xinghu Yu Huijun Gao ViT 21 5 0 28 Dec 2022
MixupE: Understanding and Improving Mixup from Directional Derivative Perspective Yingtian Zou Vikas Verma Sarthak Mittal Wai Hoh Tang Hieu H. Pham Juho Kannala Yoshua Bengio Arno Solin Kenji Kawaguchi UQCV 43 8 0 27 Dec 2022
A Close Look at Spatial Modeling: From Attention to Convolution Xu Ma Huan Wang Can Qin Kunpeng Li Xing Zhao Jie Fu Yun Fu ViT 3DPC 17 11 0 23 Dec 2022
What Makes for Good Tokenizers in Vision Transformer? Shengju Qian Yi Zhu Wenbo Li Mu Li Jiaya Jia ViT 29 13 0 21 Dec 2022
Universal Object Detection with Large Vision Model Feng-Huei Lin Wenze Hu Yaowei Wang Yonghong Tian Guangming Lu Fanglin Chen Yong-mei Xu Xiaoyu Wang VLM ObjD 32 8 0 19 Dec 2022
Rethinking Vision Transformers for MobileNet Size and Speed Yanyu Li Ju Hu Yang Wen Georgios Evangelidis Kamyar Salahi Yanzhi Wang Sergey Tulyakov Jian Ren ViT 25 159 0 15 Dec 2022
Comparing the Decision-Making Mechanisms by Transformers and CNNs via Explanation Methods Ming-Xiu Jiang Saeed Khorram Li Fuxin FAtt 14 9 0 13 Dec 2022
What do Vision Transformers Learn? A Visual Exploration Amin Ghiasi Hamid Kazemi Eitan Borgnia Steven Reich Manli Shu Micah Goldblum A. Wilson Tom Goldstein ViT 24 60 0 13 Dec 2022
OAMixer: Object-aware Mixing Layer for Vision Transformers H. Kang Sangwoo Mo Jinwoo Shin VLM 34 4 0 13 Dec 2022
Open Vocabulary Semantic Segmentation with Patch Aligned Contrastive Learning Jishnu Mukhoti Tsung-Yu Lin Omid Poursaeed Rui Wang Ashish Shah Philip H. S. Torr Ser-Nam Lim VLM 30 79 0 09 Dec 2022
Deep Incubation: Training Large Models by Divide-and-Conquering Zanlin Ni Yulin Wang Jiangwei Yu Haojun Jiang Yu Cao Gao Huang VLM 18 11 0 08 Dec 2022
MixBoost: Improving the Robustness of Deep Neural Networks by Boosting Data Augmentation Zhendong Liu Wenyu Jiang Min Guo Chongjun Wang AAML 21 1 0 08 Dec 2022
Lightweight Structure-Aware Attention for Visual Understanding Heeseung Kwon F. M. Castro M. Marín-Jiménez N. Guil Alahari Karteek 26 2 0 29 Nov 2022
Minimal Width for Universal Property of Deep RNN Changhoon Song Geonho Hwang Jun ho Lee Myung-joo Kang 13 8 0 25 Nov 2022
Fast-iTPN: Integrally Pre-Trained Transformer Pyramid Network with Token Migration Yunjie Tian Lingxi Xie Jihao Qiu Jianbin Jiao Yaowei Wang Qi Tian Qixiang Ye ViT 29 6 0 23 Nov 2022
Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition Qibin Hou Cheng Lu Mingg-Ming Cheng Jiashi Feng ViT 23 129 0 22 Nov 2022
Vision Transformer with Super Token Sampling Huaibo Huang Xiaoqiang Zhou Jie Cao Ran He T. Tan ViT 18 55 0 21 Nov 2022
You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model Sheng Tang Yaqing Wang Zhenglun Kong Tianchi Zhang Yao Li Caiwen Ding Yanzhi Wang Yi Liang Dongkuan Xu 25 31 0 21 Nov 2022
Vision Transformers in Medical Imaging: A Review Emerald U. Henry Onyeka Emebob C. Omonhinmin ViT MedIm 22 34 0 18 Nov 2022
Towards All-in-one Pre-training via Maximizing Multi-modal Mutual Information Weijie Su Xizhou Zhu Chenxin Tao Lewei Lu Bin Li Gao Huang Yu Qiao Xiaogang Wang Jie Zhou Jifeng Dai 34 41 0 17 Nov 2022
AligNeRF: High-Fidelity Neural Radiance Fields via Alignment-Aware Training Yifan Jiang Peter Hedman B. Mildenhall Dejia Xu Jonathan T. Barron Zhangyang Wang Tianfan Xue AI4CE 23 37 0 17 Nov 2022
EVA: Exploring the Limits of Masked Visual Representation Learning at Scale Yuxin Fang Wen Wang Binhui Xie Quan-Sen Sun Ledell Yu Wu Xinggang Wang Tiejun Huang Xinlong Wang Yue Cao VLM CLIP 56 673 0 14 Nov 2022
ParCNetV2: Oversized Kernel with Enhanced Attention Ruihan Xu Haokui Zhang Wenze Hu Shiliang Zhang Xiaoyu Wang ViT 25 6 0 14 Nov 2022
BiViT: Extremely Compressed Binary Vision Transformer Yefei He Zhenyu Lou Luoming Zhang Jing Liu Weijia Wu Hong Zhou Bohan Zhuang ViT MQ 18 28 0 14 Nov 2022
A Comprehensive Survey of Transformers for Computer Vision Sonain Jamil Md. Jalil Piran Oh-Jin Kwon ViT 30 46 0 11 Nov 2022
Demystify Transformers & Convolutions in Modern Image Deep Networks Jifeng Dai Min Shi Weiyun Wang Sitong Wu Linjie Xing ... Lewei Lu Jie Zhou Xiaogang Wang Yu Qiao Xiao-hua Hu ViT 26 11 0 10 Nov 2022
InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions Wenhai Wang Jifeng Dai Zhe Chen Zhenhang Huang Zhiqi Li ... Tong Lu Lewei Lu Hongsheng Li Xiaogang Wang Yu Qiao VLM 36 656 0 10 Nov 2022
MogaNet: Multi-order Gated Aggregation Network Siyuan Li Zedong Wang Zicheng Liu Cheng Tan Haitao Lin Di Wu Zhiyuan Chen Jiangbin Zheng Stan Z. Li 26 56 0 07 Nov 2022
SPEAKER VGG CCT: Cross-corpus Speech Emotion Recognition with Speaker Embedding and Vision Transformers Alessandro Arezzo Stefano Berretti ViT 16 15 0 04 Nov 2022
Boosting Binary Neural Networks via Dynamic Thresholds Learning Jiehua Zhang Xueyang Zhang Z. Su Zitong Yu Yanghe Feng Xin Lu M. Pietikäinen Li Liu MQ 25 0 0 04 Nov 2022
Exploring Effects of Computational Parameter Changes to Image Recognition Systems Nikolaos Louloudakis Perry Gibson José Cano A. Rajan 11 6 0 01 Nov 2022
Accelerating Certified Robustness Training via Knowledge Transfer Pratik Vaishnavi Kevin Eykholt Amir Rahmati 16 7 0 25 Oct 2022
The Curious Case of Benign Memorization Sotiris Anagnostidis Gregor Bachmann Lorenzo Noci Thomas Hofmann AAML 39 8 0 25 Oct 2022
DialogConv: A Lightweight Fully Convolutional Network for Multi-view Response Selection Yongkang Liu Shi Feng Wei Gao Daling Wang Yifei Zhang 11 3 0 25 Oct 2022
Synthetic Data Supervised Salient Object Detection Zhenyu Wu Lin Wang W. Wang Tengfei Shi Chenglizhao Chen Aimin Hao Shuo Li 21 22 0 25 Oct 2022
MetaFormer Baselines for Vision Weihao Yu Chenyang Si Pan Zhou Mi Luo Yichen Zhou Jiashi Feng Shuicheng Yan Xinchao Wang MoE 23 156 0 24 Oct 2022
Drastically Reducing the Number of Trainable Parameters in Deep CNNs by Inter-layer Kernel-sharing Alireza Azadbakht Saeed Reza Kheradpisheh Ismail Khalfaoui-Hassani T. Masquelier 23 1 0 23 Oct 2022
Similarity of Neural Architectures using Adversarial Attack Transferability Jaehui Hwang Dongyoon Han Byeongho Heo Song Park Sanghyuk Chun Jong-Seok Lee AAML 24 1 0 20 Oct 2022
A Survey of Computer Vision Technologies In Urban and Controlled-environment Agriculture Jiayun Luo Boyang Albert Li Cyril Leung 46 11 0 20 Oct 2022
Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning Dongze Lian Daquan Zhou Jiashi Feng Xinchao Wang 34 247 0 17 Oct 2022
SWFormer: Sparse Window Transformer for 3D Object Detection in Point Clouds Pei Sun Mingxing Tan Weiyue Wang Chenxi Liu Fei Xia Zhaoqi Leng Drago Anguelov ViT 21 114 0 13 Oct 2022
Vision Transformers provably learn spatial structure Samy Jelassi Michael E. Sander Yuan-Fang Li ViT MLT 32 73 0 13 Oct 2022
Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities Brian Bartoldson B. Kailkhura Davis W. Blalock 29 47 0 13 Oct 2022
Fast-ParC: Capturing Position Aware Global Feature for ConvNets and ViTs Taojiannan Yang Haokui Zhang Wenze Hu C. L. P. Chen Xiaoyu Wang ViT 11 0 0 08 Oct 2022
Visualize Before You Write: Imagination-Guided Open-Ended Text Generation Wanrong Zhu An Yan Yujie Lu Wenda Xu X. Wang Miguel P. Eckstein William Yang Wang 74 37 0 07 Oct 2022
The Lie Derivative for Measuring Learned Equivariance Nate Gruver Marc Finzi Micah Goldblum A. Wilson 16 34 0 06 Oct 2022
MOAT: Alternating Mobile Convolution and Attention Brings Strong Vision Models Chenglin Yang Siyuan Qiao Qihang Yu Xiaoding Yuan Yukun Zhu Alan Yuille Hartwig Adam Liang-Chieh Chen ViT MoE 28 58 0 04 Oct 2022
Towards Flexible Inductive Bias via Progressive Reparameterization Scheduling Yunsung Lee Gyuseong Lee Kwang-seok Ryoo Hyojun Go Jihye Park Seung Wook Kim 24 5 0 04 Oct 2022
Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuning Weicong Liang Yuhui Yuan Henghui Ding Xiao Luo Weihong Lin Ding Jia Zheng-Wei Zhang Chao Zhang Hanhua Hu 22 25 0 03 Oct 2022