Do Vision Transformers See Like Convolutional Neural Networks?

19 August 2021

Alexey Dosovitskiy

Papers citing "Do Vision Transformers See Like Convolutional Neural Networks?"

40 / 440 papers shown

Title
SPViT: Enabling Faster Vision Transformers via Soft Token Pruning Zhenglun Kong Peiyan Dong Xiaolong Ma Xin Meng Mengshu Sun ... Geng Yuan Bin Ren Minghai Qin H. Tang Yanzhi Wang ViT 26 141 0 27 Dec 2021
ELSA: Enhanced Local Self-Attention for Vision Transformer Jingkai Zhou Pichao Wang Fan Wang Qiong Liu Hao Li Rong Jin ViT 29 37 0 23 Dec 2021
SeMask: Semantically Masked Transformers for Semantic Segmentation Jitesh Jain Anukriti Singh Nikita Orlov Zilong Huang Jiachen Li Steven Walton Humphrey Shi ViT 27 92 0 23 Dec 2021
Cost Aggregation Is All You Need for Few-Shot Segmentation Sunghwan Hong Seokju Cho Jisu Nam Seungryong Kim ViT 22 23 0 22 Dec 2021
Learned Queries for Efficient Local Attention Moab Arar Ariel Shamir Amit H. Bermano ViT 36 29 0 21 Dec 2021
On Efficient Transformer-Based Image Pre-training for Low-Level Vision Wenbo Li Xin Lu Shengju Qian Jiangbo Lu X. Zhang Jiaya Jia ViT 32 83 0 19 Dec 2021
A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation Wuyang Chen Xianzhi Du Fan Yang Lucas Beyer Xiaohua Zhai ... Huizhong Chen Jing Li Xiaodan Song Zhangyang Wang Denny Zhou ViT 21 20 0 17 Dec 2021
Towards General and Efficient Active Learning Yichen Xie M. Tomizuka Wei Zhan VLM 30 10 0 15 Dec 2021
Deep ViT Features as Dense Visual Descriptors Shirzad Amir Yossi Gandelsman Shai Bagon Tali Dekel MDE ViT 36 271 0 10 Dec 2021
Joint Global and Local Hierarchical Priors for Learned Image Compression Jun-Hyuk Kim Byeongho Heo Jong-Seok Lee 23 82 0 08 Dec 2021
Attentive Prototypes for Source-free Unsupervised Domain Adaptive 3D Object Detection Deepti Hegde Vishal M. Patel 3DPC 29 14 0 30 Nov 2021
Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis Yucheng Tang Dong Yang Wenqi Li H. Roth Bennett Landman Daguang Xu V. Nath Ali Hatamizadeh ViT MedIm 22 517 0 29 Nov 2021
Self-Distilled Self-Supervised Representation Learning J. Jang Seonhoon Kim Kiyoon Yoo Chaerin Kong Jang-Hyun Kim Nojun Kwak SSL 12 14 0 25 Nov 2021
Exploiting Both Domain-specific and Invariant Knowledge via a Win-win Transformer for Unsupervised Domain Adaptation Wen-hui Ma Jinming Zhang Shuang Li Chi Harold Liu Yulin Wang Wei Li ViT 16 11 0 25 Nov 2021
Pruning Self-attentions into Convolutional Layers in Single Path Haoyu He Jianfei Cai Jing Liu Zizheng Pan Jing Zhang Dacheng Tao Bohan Zhuang ViT 29 40 0 23 Nov 2021
Intriguing Findings of Frequency Selection for Image Deblurring Xintian Mao Yiming Liu Fengze Liu Qingli Li Wei Shen Yan Wang 30 145 0 23 Nov 2021
MetaFormer Is Actually What You Need for Vision Weihao Yu Mi Luo Pan Zhou Chenyang Si Yichen Zhou Xinchao Wang Jiashi Feng Shuicheng Yan 26 872 0 22 Nov 2021
TransMorph: Transformer for unsupervised medical image registration Junyu Chen Eric C. Frey Yufan He W. Paul Segars Ye Li Yong Du ViT MedIm 29 186 0 19 Nov 2021
TransMix: Attend to Mix for Vision Transformers Jieneng Chen Shuyang Sun Ju He Philip H. S. Torr Alan Yuille S. Bai ViT 20 103 0 18 Nov 2021
Hybrid BYOL-ViT: Efficient approach to deal with small datasets Safwen Naimi Rien van Leeuwen W. Souidène S. B. Saoud SSL ViT 25 2 0 08 Nov 2021
Multi-Scale High-Resolution Vision Transformer for Semantic Segmentation Jiaqi Gu Hyoukjun Kwon Dilin Wang Wei Ye Meng Li Yu-Hsin Chen Liangzhen Lai Vikas Chandra D. Pan ViT 19 182 0 01 Nov 2021
The Nuts and Bolts of Adopting Transformer in GANs Rui Xu Xiangyu Xu Kai-xiang Chen Bolei Zhou Chen Change Loy ViT 12 4 0 25 Oct 2021
Adaptive Multi-view and Temporal Fusing Transformer for 3D Human Pose Estimation Hui Shuai Lele Wu Qingshan Liu ViT 15 44 0 11 Oct 2021
Universal Paralinguistic Speech Representations Using Self-Supervised Conformers Joel Shor A. Jansen Wei Han Daniel S. Park Yu Zhang SSL AI4TS 33 54 0 09 Oct 2021
Do Self-Supervised and Supervised Methods Learn Similar Visual Representations? Tom George Grigg Dan Busbridge Jason Ramapuram Russ Webb SSL DRL 19 27 0 01 Oct 2021
Physics-Augmented Learning: A New Paradigm Beyond Physics-Informed Learning Ziming Liu Yunyue Chen Yuanqi Du Max Tegmark PINN AI4CE 40 22 0 28 Sep 2021
TVT: Transferable Vision Transformer for Unsupervised Domain Adaptation Jinyu Yang Jingjing Liu N. Xu Junzhou Huang 20 125 0 12 Aug 2021
Efficient Training of Visual Transformers with Small Datasets Yahui Liu E. Sangineto Wei Bi N. Sebe Bruno Lepri Marco De Nadai ViT 28 164 0 07 Jun 2021
StyTr $^2$ : Image Style Transfer with Transformers Yingying Deng Fan Tang Weiming Dong Chongyang Ma Xingjia Pan Lei Wang Changsheng Xu ViT 25 259 0 30 May 2021
An overview of deep learning techniques for epileptic seizures detection and prediction based on neuroimaging modalities: Methods, challenges, and future works A. Shoeibi Parisa Moridian Marjane Khodatars Navid Ghassemi M. Jafari ... Juan M Gorriz Javier Ramírez Abbas Khosravi S. Nahavandi U. Acharya 24 49 0 29 May 2021
Intriguing Properties of Vision Transformers Muzammal Naseer Kanchana Ranasinghe Salman Khan Munawar Hayat F. Khan Ming-Hsuan Yang ViT 251 620 0 21 May 2021
MLP-Mixer: An all-MLP Architecture for Vision Ilya O. Tolstikhin N. Houlsby Alexander Kolesnikov Lucas Beyer Xiaohua Zhai ... Andreas Steiner Daniel Keysers Jakob Uszkoreit Mario Lucic Alexey Dosovitskiy 250 2,603 0 04 May 2021
Emerging Properties in Self-Supervised Vision Transformers Mathilde Caron Hugo Touvron Ishan Misra Hervé Jégou Julien Mairal Piotr Bojanowski Armand Joulin 303 5,773 0 29 Apr 2021
DA-DETR: Domain Adaptive Detection Transformer with Information Fusion Jingyi Zhang Jiaxing Huang Zhipeng Luo Gongjie Zhang Xiaoqin Zhang Shijian Lu ViT 4 35 0 31 Mar 2021
IMAGO: A family photo album dataset for a socio-historical analysis of the twentieth century L. Stacchio Alessia Angeli G. Lisanti Daniela Calanca Gustavo Marfia 23 2 0 03 Dec 2020
Similarity Analysis of Contextual Word Representation Models John M. Wu Yonatan Belinkov Hassan Sajjad Nadir Durrani Fahim Dalvi James R. Glass 46 73 0 03 May 2020
Rapid Learning or Feature Reuse? Towards Understanding the Effectiveness of MAML Aniruddh Raghu M. Raghu Samy Bengio Oriol Vinyals 172 639 0 19 Sep 2019
Investigating Multilingual NMT Representations at Scale Sneha Kudugunta Ankur Bapna Isaac Caswell N. Arivazhagan Orhan Firat LRM 136 120 0 05 Sep 2019
The Bottom-up Evolution of Representations in the Transformer: A Study with Machine Translation and Language Modeling Objectives Elena Voita Rico Sennrich Ivan Titov 190 181 0 03 Sep 2019
What you can cram into a single vector: Probing sentence embeddings for linguistic properties Alexis Conneau Germán Kruszewski Guillaume Lample Loïc Barrault Marco Baroni 199 882 0 03 May 2018