Conditional Positional Encodings for Vision Transformers

22 February 2021

Chunhua Shen

Papers citing "Conditional Positional Encodings for Vision Transformers"

49 / 99 papers shown

Title
Flow-Guided Transformer for Video Inpainting Kaiwen Zhang Jingjing Fu Dong Liu ViT 21 68 0 14 Aug 2022
PointConvFormer: Revenge of the Point-based Convolution Wenxuan Wu Li Fuxin Qi Shan 3DPC 23 30 0 04 Aug 2022
MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depth Chenjie Cao Xinlin Ren Yanwei Fu 22 44 0 04 Aug 2022
Multi-branch Cascaded Swin Transformers with Attention to k-space Sampling Pattern for Accelerated MRI Reconstruction Mevan Ekanayake Kamlesh Pawar Mehrtash Harandi Gary Egan Zhaolin Chen ViT MedIm 19 7 0 18 Jul 2022
Defect Transformer: An Efficient Hybrid Transformer Architecture for Surface Defect Detection Junpu Wang Guili Xu Fuju Yan Jinjin Wang Zhengsheng Wang ViT MedIm 21 65 0 17 Jul 2022
EATFormer: Improving Vision Transformer Inspired by Evolutionary Algorithm Jiangning Zhang Xiangtai Li Yabiao Wang Chengjie Wang Yibo Yang Yong Liu Dacheng Tao ViT 30 32 0 19 Jun 2022
SP-ViT: Learning 2D Spatial Priors for Vision Transformers Yuxuan Zhou Wangmeng Xiang C. Li Biao Wang Xihan Wei Lei Zhang M. Keuper Xia Hua ViT 29 15 0 15 Jun 2022
HiViT: Hierarchical Vision Transformer Meets Masked Image Modeling Xiaosong Zhang Yunjie Tian Wei Huang QiXiang Ye Qi Dai Lingxi Xie Qi Tian 52 26 0 30 May 2022
EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers Junting Pan Adrian Bulat Fuwen Tan Xiatian Zhu L. Dudziak Hongsheng Li Georgios Tzimiropoulos Brais Martínez ViT 23 180 0 06 May 2022
Deeper Insights into the Robustness of ViTs towards Common Corruptions Rui Tian Zuxuan Wu Qi Dai Han Hu Yu-Gang Jiang ViT AAML 16 4 0 26 Apr 2022
VSA: Learning Varied-Size Window Attention in Vision Transformers Qiming Zhang Yufei Xu Jing Zhang Dacheng Tao 22 53 0 18 Apr 2022
DaViT: Dual Attention Vision Transformers Mingyu Ding Bin Xiao Noel Codella Ping Luo Jingdong Wang Lu Yuan ViT 30 240 0 07 Apr 2022
Multi-scale Context-aware Network with Transformer for Gait Recognition Duo-Lin Zhu Xiaohui Huang Xinggang Wang Bo Yang Botao He Wenyu Liu Bin Feng ViT CVBM 14 15 0 07 Apr 2022
Dynamic Focus-aware Positional Queries for Semantic Segmentation Haoyu He Jianfei Cai Zizheng Pan Jing Liu Jing Zhang Dacheng Tao Bohan Zhuang 34 16 0 04 Apr 2022
SepViT: Separable Vision Transformer Wei Li Xing Wang Xin Xia Jie Wu Jiashi Li Xuefeng Xiao Min Zheng Shiping Wen ViT 26 39 0 29 Mar 2022
ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformer Rui Yang Hailong Ma Jie Wu Yansong Tang Xuefeng Xiao Min Zheng Xiu Li ViT 19 53 0 21 Mar 2022
HIPA: Hierarchical Patch Transformer for Single Image Super Resolution Qing Cai Yiming Qian Jinxing Li Junjie Lv Yee-Hong Yang Feng Wu Dafan Zhang 17 28 0 19 Mar 2022
Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs Xiaohan Ding X. Zhang Yi Zhou Jungong Han Guiguang Ding Jian-jun Sun VLM 47 525 0 13 Mar 2022
Multi-class Token Transformer for Weakly Supervised Semantic Segmentation Lian Xu Wanli Ouyang Bennamoun F. Boussaïd Dan Xu ViT 23 209 0 06 Mar 2022
Correlation-Aware Deep Tracking Fei Xie Chunyu Wang Guangting Wang Yue Cao Wankou Yang Wenjun Zeng VOT 22 118 0 03 Mar 2022
Classification of Long Sequential Data using Circular Dilated Convolutional Neural Networks Lei Cheng Ruslan Khalitov Tong Yu Zhirong Yang 22 32 0 06 Jan 2022
Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attention Sitong Wu Tianyi Wu Hao Hao Tan G. Guo ViT 23 70 0 28 Dec 2021
SeMask: Semantically Masked Transformers for Semantic Segmentation Jitesh Jain Anukriti Singh Nikita Orlov Zilong Huang Jiachen Li Steven Walton Humphrey Shi ViT 27 92 0 23 Dec 2021
FaceFormer: Speech-Driven 3D Facial Animation with Transformers Yingruo Fan Zhaojiang Lin Jun Saito Wenping Wang Taku Komura CVBM 38 194 0 10 Dec 2021
DualFormer: Local-Global Stratified Transformer for Efficient Video Recognition Yuxuan Liang Pan Zhou Roger Zimmermann Shuicheng Yan ViT 21 21 0 09 Dec 2021
A Survey of Visual Transformers Yang Liu Yao Zhang Yixin Wang Feng Hou Jin Yuan Jiang Tian Yang Zhang Zhongchao Shi Jianping Fan Zhiqiang He 3DGS ViT 69 330 0 11 Nov 2021
Blending Anti-Aliasing into Vision Transformer Shengju Qian Hao Shao Yi Zhu Mu Li Jiaya Jia 19 20 0 28 Oct 2021
3D Object Tracking with Transformer Yubo Cui Zheng Fang Jiayao Shan Zuoxu Gu Sifan Zhou ViT 3DPC 13 59 0 28 Oct 2021
SOFT: Softmax-free Transformer with Linear Complexity Jiachen Lu Jinghan Yao Junge Zhang Martin Danelljan Hang Xu Weiguo Gao Chunjing Xu Thomas B. Schon Li Zhang 16 161 0 22 Oct 2021
Study of positional encoding approaches for Audio Spectrogram Transformers L. Pepino Pablo Riera Luciana Ferrer ViT 12 6 0 13 Oct 2021
Global Vision Transformer Pruning with Hessian-Aware Saliency Huanrui Yang Hongxu Yin Maying Shen Pavlo Molchanov Hai Helen Li Jan Kautz ViT 28 38 0 10 Oct 2021
UniNet: Unified Architecture Search with Convolution, Transformer, and MLP Jihao Liu Hongsheng Li Guanglu Song Xin Huang Yu Liu ViT 29 35 0 08 Oct 2021
Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs Philipp Benz Soomin Ham Chaoning Zhang Adil Karjauv In So Kweon AAML ViT 29 78 0 06 Oct 2021
CCTrans: Simplifying and Improving Crowd Counting with Transformer Ye Tian Xiangxiang Chu Hongpeng Wang ViT 13 75 0 29 Sep 2021
Hire-MLP: Vision MLP via Hierarchical Rearrangement Jianyuan Guo Yehui Tang Kai Han Xinghao Chen Han Wu Chao Xu Chang Xu Yunhe Wang 38 105 0 30 Aug 2021
Focal Self-attention for Local-Global Interactions in Vision Transformers Jianwei Yang Chunyuan Li Pengchuan Zhang Xiyang Dai Bin Xiao Lu Yuan Jianfeng Gao ViT 39 428 0 01 Jul 2021
Rethinking Token-Mixing MLP for MLP-based Vision Backbone Tan Yu Xu Li Yunfeng Cai Mingming Sun Ping Li 40 26 0 28 Jun 2021
PVT v2: Improved Baselines with Pyramid Vision Transformer Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT AI4TS 13 1,606 0 25 Jun 2021
P2T: Pyramid Pooling Transformer for Scene Understanding Yu-Huan Wu Yun-Hai Liu Xin Zhan Mingg-Ming Cheng ViT 24 218 0 22 Jun 2021
A Survey of Transformers Tianyang Lin Yuxin Wang Xiangyang Liu Xipeng Qiu ViT 27 1,084 0 08 Jun 2021
ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias Yufei Xu Qiming Zhang Jing Zhang Dacheng Tao ViT 48 329 0 07 Jun 2021
SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers Enze Xie Wenhai Wang Zhiding Yu Anima Anandkumar J. Álvarez Ping Luo ViT 12 4,819 0 31 May 2021
Analogous to Evolutionary Algorithm: Designing a Unified Sequence Model Jiangning Zhang Chao Xu Jian Li Wenzhou Chen Yabiao Wang Ying Tai Shuo Chen Chengjie Wang Feiyue Huang Yong Liu 27 22 0 31 May 2021
Dual-stream Network for Visual Recognition Mingyuan Mao Renrui Zhang Honghui Zheng Peng Gao Teli Ma Yan Peng Errui Ding Baochang Zhang Shumin Han ViT 18 63 0 31 May 2021
BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search Changlin Li Tao Tang Guangrun Wang Jiefeng Peng Bing Wang Xiaodan Liang Xiaojun Chang ViT 35 105 0 23 Mar 2021
Scalable Vision Transformers with Hierarchical Pooling Zizheng Pan Bohan Zhuang Jing Liu Haoyu He Jianfei Cai ViT 25 126 0 19 Mar 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,622 0 24 Feb 2021
LambdaNetworks: Modeling Long-Range Interactions Without Attention Irwan Bello 260 179 0 17 Feb 2021
How Much Position Information Do Convolutional Neural Networks Encode? Md. Amirul Islam Sen Jia Neil D. B. Bruce SSL 199 344 0 22 Jan 2020