VSA: Learning Varied-Size Window Attention in Vision Transformers

18 April 2022

Qiming Zhang

Yufei Xu

Jing Zhang

Dacheng Tao

ArXiv PDF HTML

Papers citing "VSA: Learning Varied-Size Window Attention in Vision Transformers"

42 / 42 papers shown

Title
A Genealogy of Multi-Sensor Foundation Models in Remote Sensing Kevin Lane Morteza Karimzadeh 31 0 0 24 Apr 2025
Locality-aware Cross-modal Correspondence Learning for Dense Audio-Visual Events Localization Ling Xing Hongyu Qu Rui Yan Xiangbo Shu Jinhui Tang 45 0 0 12 Sep 2024
HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model Di Wang Meiqi Hu Yao Jin Yuchun Miao Jiaqi Yang ... Lefei Zhang Chen Wu Bo Du Dacheng Tao Liangpei Zhang 59 19 0 17 Jun 2024
LeMeViT: Efficient Vision Transformer with Learnable Meta Tokens for Remote Sensing Image Interpretation Wentao Jiang Jing Zhang Di Wang Qiming Zhang Zengmao Wang Bo Du 24 5 0 16 May 2024
Enhancing Efficiency in Vision Transformer Networks: Design Techniques and Insights Moein Heidari Reza Azad Sina Ghorbani Kolahi René Arimond Leon Niggemeier ... Afshin Bozorgpour Ehsan Khodapanah Aghdam A. Kazerouni I. Hacihaliloglu Dorit Merhof 36 7 0 28 Mar 2024
MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretraining Di Wang Jing Zhang Minqiang Xu Lin Liu Dongsheng Wang ... Chengxi Han Haonan Guo Bo Du Dacheng Tao L. Zhang 24 42 0 20 Mar 2024
HIRI-ViT: Scaling Vision Transformer with High Resolution Inputs Ting Yao Yehao Li Yingwei Pan Tao Mei ViT 18 14 0 18 Mar 2024
Factorization Vision Transformer: Modeling Long Range Dependency with Local Window Cost Haolin Qin Daquan Zhou Tingfa Xu Ziyang Bian Jianan Li 19 9 0 14 Dec 2023
Cross-level Attention with Overlapped Windows for Camouflaged Object Detection Jiepan Li Fangxiao Lu Nan Xue Zhuo Li Hongyan Zhang Wei He 25 2 0 28 Nov 2023
RMT: Retentive Networks Meet Vision Transformers Qihang Fan Huaibo Huang Mingrui Chen Hongmin Liu Ran He ViT 30 65 0 20 Sep 2023
DAT++: Spatially Dynamic Vision Transformer with Deformable Attention Zhuofan Xia Xuran Pan Shiji Song Li Erran Li Gao Huang ViT 19 22 0 04 Sep 2023
Spatial Transform Decoupling for Oriented Object Detection Hongtian Yu Yunjie Tian QiXiang Ye Yunfan Liu 25 26 0 21 Aug 2023
OAFuser: Towards Omni-Aperture Fusion for Light Field Semantic Segmentation Fei Teng Jiaming Zhang Kunyu Peng Yaonan Wang Rainer Stiefelhagen Kailun Yang 22 4 0 28 Jul 2023
ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution Mingjin Zhang Chi Zhang Qiming Zhang Jie-Ru Guo Xinbo Gao Jing Zhang 11 28 0 26 Jul 2023
Minimalist and High-Quality Panoramic Imaging with PSF-aware Transformers Qi Jiang Shaohua Gao Yao Gao Kailun Yang Zhonghua Yi Haowen Shi Lei Sun Kaiwei Wang 11 10 0 22 Jun 2023
Reciprocal Attention Mixing Transformer for Lightweight Image Restoration Haram Choi Cheolwoong Na Jihyeon Oh Seungjae Lee Jinseop S. Kim Subeen Choe Jeongmin Lee Taehoon Kim Jihoon Yang 44 5 0 19 May 2023
RFR-WWANet: Weighted Window Attention-Based Recovery Feature Resolution Network for Unsupervised Image Registration Mingrui Ma Tao Wang Lei Song Weijie Wang Gui-Xian Liu ViT MedIm 11 1 0 07 May 2023
OSP2B: One-Stage Point-to-Box Network for 3D Siamese Tracking Jiahao Nie Zhiwei He Yuxiang Yang Zhengyi Bao Mingchen Gao Jing Zhang 3DPC 25 24 0 23 Apr 2023
DCN-T: Dual Context Network with Transformer for Hyperspectral Image Classification Di Wang Jing Zhang Bo Du L. Zhang Dacheng Tao 6 49 0 19 Apr 2023
Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understanding Yu-Qi Yang Yu-Xiao Guo Jiangfeng Xiong Yang Liu Hao Pan Peng-Shuai Wang Xin Tong B. Guo ViT 25 75 0 14 Apr 2023
Vision Transformer with Quadrangle Attention Qiming Zhang Jing Zhang Yufei Xu Dacheng Tao ViT 19 38 0 27 Mar 2023
DQnet: Cross-Model Detail Querying for Camouflaged Object Detection Wei Sun Chengao Liu Linyan Zhang Yu Li Pengxu Wei Chang-rui Liu J. Zou Jianbin Jiao QiXiang Ye 32 6 0 16 Dec 2022
ViTPose++: Vision Transformer for Generic Body Pose Estimation Yufei Xu Jing Zhang Qiming Zhang Dacheng Tao ViT 32 40 0 07 Dec 2022
1st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Results Benjamin Kiefer Matej Kristan Janez Pervs Lojze vZust Fabio Poiesi ... Chih-Chung Hsu X. Hou Yu-An Jhang Simon X. Yang Mau-Tsuen Yang 20 21 0 24 Nov 2022
N-Gram in Swin Transformers for Efficient Lightweight Image Super-Resolution Haram Choi Jeong-Sik Lee Jihoon Yang ViT 19 74 0 21 Nov 2022
GLT-T: Global-Local Transformer Voting for 3D Single Object Tracking in Point Clouds Jiahao Nie Zhiwei He Yuxiang Yang Mingyu Gao Jing Zhang 3DPC 11 39 0 20 Nov 2022
DeepSolo: Let Transformer Decoder with Explicit Points Solo for Text Spotting Maoyuan Ye Jing Zhang Shanshan Zhao Juhua Liu Tongliang Liu Bo Du Dacheng Tao 23 70 0 19 Nov 2022
Rethinking Hierarchies in Pre-trained Plain Vision Transformer Yufei Xu Jing Zhang Qiming Zhang Dacheng Tao 11 1 0 03 Nov 2022
Advancing Plain Vision Transformer Towards Remote Sensing Foundation Model Di Wang Qiming Zhang Yufei Xu Jing Zhang Bo Du Dacheng Tao L. Zhang 17 234 0 08 Aug 2022
Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection Gaoang Wang Yibing Zhan Xinchao Wang Min-Gyoo Song K. Nahrstedt 6 11 0 24 Jul 2022
Learning Graph Neural Networks for Image Style Transfer Yongcheng Jing Yining Mao Yiding Yang Yibing Zhan Mingli Song Xinchao Wang Dacheng Tao 26 53 0 24 Jul 2022
DPText-DETR: Towards Better Scene Text Detection with Dynamic Points in Transformer Maoyuan Ye Jing Zhang Shanshan Zhao Juhua Liu Bo Du Dacheng Tao ViT 30 72 0 10 Jul 2022
Multi-Task Learning with Multi-Query Transformer for Dense Prediction Yangyang Xu Xiangtai Li Haobo Yuan Yibo Yang Lefei Zhang ViT 13 45 0 28 May 2022
Fast Vision Transformers with HiLo Attention Zizheng Pan Jianfei Cai Bohan Zhuang 20 148 0 26 May 2022
ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation Yufei Xu Jing Zhang Qiming Zhang Dacheng Tao ViT 8 505 0 26 Apr 2022
Dynamic Focus-aware Positional Queries for Semantic Segmentation Haoyu He Jianfei Cai Zizheng Pan Jing Liu Jing Zhang Dacheng Tao Bohan Zhuang 26 16 0 04 Apr 2022
ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond Qiming Zhang Yufei Xu Jing Zhang Dacheng Tao ViT 6 225 0 21 Feb 2022
Dynamic Token Normalization Improves Vision Transformers Wenqi Shao Yixiao Ge Zhaoyang Zhang Xuyuan Xu Xiaogang Wang Ying Shan Ping Luo ViT 113 11 0 05 Dec 2021
Pruning Self-attentions into Convolutional Layers in Single Path Haoyu He Jianfei Cai Jing Liu Zizheng Pan Jing Zhang Dacheng Tao Bohan Zhuang ViT 23 40 0 23 Nov 2021
CMT: Convolutional Neural Networks Meet Vision Transformers Jianyuan Guo Kai Han Han Wu Yehui Tang Chunjing Xu Yunhe Wang Chang Xu ViT 328 500 0 13 Jul 2021
Transformer in Transformer Kai Han An Xiao Enhua Wu Jianyuan Guo Chunjing Xu Yunhe Wang ViT 282 1,490 0 27 Feb 2021
Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions Wenhai Wang Enze Xie Xiang Li Deng-Ping Fan Kaitao Song Ding Liang Tong Lu Ping Luo Ling Shao ViT 263 3,538 0 24 Feb 2021