ResearchTrend.AI
  • Papers
  • Communities
  • Events
  • Blog
  • Pricing
Papers
Communities
Social Events
Terms and Conditions
Pricing
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2025 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2107.14222
  4. Cited By
Rethinking and Improving Relative Position Encoding for Vision
  Transformer

Rethinking and Improving Relative Position Encoding for Vision Transformer

29 July 2021
Kan Wu
Houwen Peng
Minghao Chen
Jianlong Fu
Hongyang Chao
    ViT
ArXivPDFHTML

Papers citing "Rethinking and Improving Relative Position Encoding for Vision Transformer"

50 / 163 papers shown
Title
Window Attention is Bugged: How not to Interpolate Position Embeddings
Window Attention is Bugged: How not to Interpolate Position Embeddings
Daniel Bolya
Chaitanya K. Ryali
Judy Hoffman
Christoph Feichtenhofer
32
10
0
09 Nov 2023
H-NeXt: The next step towards roto-translation invariant networks
H-NeXt: The next step towards roto-translation invariant networks
Tomáš Karella
F. Šroubek
J. Flusser
Jan Blazek
Vasek Kosik
16
1
0
02 Nov 2023
CROMA: Remote Sensing Representations with Contrastive Radar-Optical
  Masked Autoencoders
CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders
A. Fuller
K. Millard
James R. Green
15
60
0
01 Nov 2023
Real-Time Motion Prediction via Heterogeneous Polyline Transformer with
  Relative Pose Encoding
Real-Time Motion Prediction via Heterogeneous Polyline Transformer with Relative Pose Encoding
Zhejun Zhang
Alexander Liniger
Christos Sakaridis
Fisher Yu
Luc Van Gool
48
30
0
19 Oct 2023
GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformers
GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformers
Takeru Miyato
Bernhard Jaeger
Max Welling
Andreas Geiger
ViT
30
14
0
16 Oct 2023
RoFormer for Position Aware Multiple Instance Learning in Whole Slide
  Image Classification
RoFormer for Position Aware Multiple Instance Learning in Whole Slide Image Classification
Etienne Pochet
Rami Maroun
Roger Trullo
MedIm
23
2
0
03 Oct 2023
Speech Audio Synthesis from Tagged MRI and Non-Negative Matrix
  Factorization via Plastic Transformer
Speech Audio Synthesis from Tagged MRI and Non-Negative Matrix Factorization via Plastic Transformer
Xiaofeng Liu
Fangxu Xing
Maureen Stone
Jiachen Zhuo
S. Fels
Jerry L. Prince
G. El Fakhri
Jonghye Woo
MedIm
8
3
0
26 Sep 2023
Keep It SimPool: Who Said Supervised Transformers Suffer from Attention
  Deficit?
Keep It SimPool: Who Said Supervised Transformers Suffer from Attention Deficit?
Bill Psomas
Ioannis Kakogeorgiou
Konstantinos Karantzalos
Yannis Avrithis
ViT
15
8
0
13 Sep 2023
SPANet: Frequency-balancing Token Mixer using Spectral Pooling
  Aggregation Modulation
SPANet: Frequency-balancing Token Mixer using Spectral Pooling Aggregation Modulation
Guhnoo Yun
J. Yoo
Kijung Kim
Jeongho Lee
Dong Hwan Kim
MoE
11
8
0
22 Aug 2023
Occ$^2$Net: Robust Image Matching Based on 3D Occupancy Estimation for
  Occluded Regions
Occ2^22Net: Robust Image Matching Based on 3D Occupancy Estimation for Occluded Regions
Miao Fan
Ming-lei Chen
Chen Hu
Shuchang Zhou
3DPC
3DV
34
4
0
14 Aug 2023
Look at the Neighbor: Distortion-aware Unsupervised Domain Adaptation
  for Panoramic Semantic Segmentation
Look at the Neighbor: Distortion-aware Unsupervised Domain Adaptation for Panoramic Semantic Segmentation
Xueye Zheng
Tianbo Pan
Yuan Luo
Lin Wang
24
24
0
10 Aug 2023
DETR Doesn't Need Multi-Scale or Locality Design
DETR Doesn't Need Multi-Scale or Locality Design
Yutong Lin
Yuhui Yuan
Zheng-Wei Zhang
Chen Li
Nanning Zheng
Han Hu
25
5
0
03 Aug 2023
Don't be so negative! Score-based Generative Modeling with
  Oracle-assisted Guidance
Don't be so negative! Score-based Generative Modeling with Oracle-assisted Guidance
Saeid Naderiparizi
Xiaoxuan Liang
Berend Zwartsenberg
Frank D. Wood
DiffM
13
4
0
31 Jul 2023
3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding
3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding
Zehan Wang
Haifeng Huang
Yang Zhao
Lin Li
Xize Cheng
Yichen Zhu
Aoxiong Yin
Zhou Zhao
3DPC
25
20
0
25 Jul 2023
Unsupervised Deep Graph Matching Based on Cycle Consistency
Unsupervised Deep Graph Matching Based on Cycle Consistency
Siddharth Tourani
Carsten Rother
M. H. Khan
Bogdan Savchynskkyy
24
3
0
18 Jul 2023
HVTSurv: Hierarchical Vision Transformer for Patient-Level Survival
  Prediction from Whole Slide Image
HVTSurv: Hierarchical Vision Transformer for Patient-Level Survival Prediction from Whole Slide Image
Zhucheng Shao
Yang Chen
Hao Bian
Jian Andrew Zhang
Guojun Liu
Yongbing Zhang
12
18
0
30 Jun 2023
Cross Architecture Distillation for Face Recognition
Cross Architecture Distillation for Face Recognition
Weisong Zhao
Xiangyu Zhu
Zhixiang He
Xiaoyu Zhang
Zhen Lei
CVBM
11
6
0
26 Jun 2023
Learning Probabilistic Coordinate Fields for Robust Correspondences
Learning Probabilistic Coordinate Fields for Robust Correspondences
Weiyue Zhao
Hao Lu
Xinyi Ye
Zhiguo Cao
Xin Li
25
6
0
07 Jun 2023
A2B: Anchor to Barycentric Coordinate for Robust Correspondence
A2B: Anchor to Barycentric Coordinate for Robust Correspondence
Weiyue Zhao
Hao Lu
Zhiguo Cao
Xin Li
24
4
0
05 Jun 2023
The Information Pathways Hypothesis: Transformers are Dynamic
  Self-Ensembles
The Information Pathways Hypothesis: Transformers are Dynamic Self-Ensembles
Md Shamim Hussain
Mohammed J. Zaki
D. Subramanian
29
2
0
02 Jun 2023
Collect-and-Distribute Transformer for 3D Point Cloud Analysis
Collect-and-Distribute Transformer for 3D Point Cloud Analysis
Haibo Qiu
Baosheng Yu
Dacheng Tao
3DPC
ViT
16
5
0
02 Jun 2023
Improving Position Encoding of Transformers for Multivariate Time Series
  Classification
Improving Position Encoding of Transformers for Multivariate Time Series Classification
Navid Mohammadi Foumani
Chang Wei Tan
Geoffrey I. Webb
Mahsa Salehi
AI4TS
22
70
0
26 May 2023
Overcoming Topology Agnosticism: Enhancing Skeleton-Based Action
  Recognition through Redefined Skeletal Topology Awareness
Overcoming Topology Agnosticism: Enhancing Skeleton-Based Action Recognition through Redefined Skeletal Topology Awareness
Yuxuan Zhou
Zhi-Qi Cheng
Ju He
Bin Luo
Yifeng Geng
Xuansong Xie
29
11
0
19 May 2023
Deep Multiple Instance Learning with Distance-Aware Self-Attention
Deep Multiple Instance Learning with Distance-Aware Self-Attention
Georg Wolflein
Lucie Charlotte Magister
Pietro Lio'
David J. Harrison
Ognjen Arandjelovic
17
2
0
17 May 2023
PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer
PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer
Honghui Yang
Wenxiao Wang
Minghao Chen
Binbin Lin
Tong He
Huaguan Chen
Xiaofei He
Wanli Ouyang
3DPC
ViT
27
32
0
11 May 2023
Understanding Gaussian Attention Bias of Vision Transformers Using
  Effective Receptive Fields
Understanding Gaussian Attention Bias of Vision Transformers Using Effective Receptive Fields
Bum Jun Kim
Hyeyeon Choi
Hyeonah Jang
Sang Woo Kim
ViT
10
3
0
08 May 2023
Early Detection of Alzheimer's Disease using Bottleneck Transformers
Early Detection of Alzheimer's Disease using Bottleneck Transformers
Arunima Jaiswal
Ananya Sadana
MedIm
11
2
0
01 May 2023
MH-DETR: Video Moment and Highlight Detection with Cross-modal
  Transformer
MH-DETR: Video Moment and Highlight Detection with Cross-modal Transformer
Yifang Xu
Yunzhuo Sun
Yang Li
Yilei Shi
Xiaoxia Zhu
S. Du
ViT
35
33
0
29 Apr 2023
An Introduction to Transformers
An Introduction to Transformers
Richard E. Turner
ViT
23
0
0
20 Apr 2023
Region-Enhanced Feature Learning for Scene Semantic Segmentation
Region-Enhanced Feature Learning for Scene Semantic Segmentation
Xin Kang
Chaoqun Wang
Xuejin Chen
16
3
0
15 Apr 2023
Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene
  Understanding
Swin3D: A Pretrained Transformer Backbone for 3D Indoor Scene Understanding
Yu-Qi Yang
Yu-Xiao Guo
Jiangfeng Xiong
Yang Liu
Hao Pan
Peng-Shuai Wang
Xin Tong
B. Guo
ViT
28
76
0
14 Apr 2023
Slide-Transformer: Hierarchical Vision Transformer with Local
  Self-Attention
Slide-Transformer: Hierarchical Vision Transformer with Local Self-Attention
Xuran Pan
Tianzhu Ye
Zhuofan Xia
S. Song
Gao Huang
ViT
22
53
0
09 Apr 2023
Visual Dependency Transformers: Dependency Tree Emerges from Reversed
  Attention
Visual Dependency Transformers: Dependency Tree Emerges from Reversed Attention
Mingyu Ding
Yikang Shen
Lijie Fan
Zhenfang Chen
Z. Chen
Ping Luo
J. Tenenbaum
Chuang Gan
ViT
67
14
0
06 Apr 2023
Inductive biases in deep learning models for weather prediction
Inductive biases in deep learning models for weather prediction
Jannik Thümmel
Matthias Karlbauer
S. Otte
C. Zarfl
Georg Martius
...
Thomas Scholten
Ulrich Friedrich
V. Wulfmeyer
B. Goswami
Martin Volker Butz
AI4CE
31
4
0
06 Apr 2023
ViTO: Vision Transformer-Operator
ViTO: Vision Transformer-Operator
O. Ovadia
Adar Kahana
P. Stinis
Eli Turkel
George Karniadakis
10
20
0
15 Mar 2023
Learning Accurate Template Matching with Differentiable Coarse-to-Fine
  Correspondence Refinement
Learning Accurate Template Matching with Differentiable Coarse-to-Fine Correspondence Refinement
Zhirui Gao
Renjiao Yi
Zheng Qin
Yunfan Ye
Chenyang Zhu
Kaiping Xu
16
7
0
15 Mar 2023
PR-MCS: Perturbation Robust Metric for MultiLingual Image Captioning
PR-MCS: Perturbation Robust Metric for MultiLingual Image Captioning
Yongil Kim
Yerin Hwang
Hyeongu Yun
Seunghyun Yoon
Trung Bui
Kyomin Jung
12
6
0
15 Mar 2023
AdPE: Adversarial Positional Embeddings for Pretraining Vision
  Transformers via MAE+
AdPE: Adversarial Positional Embeddings for Pretraining Vision Transformers via MAE+
Xiao Wang
Ying Wang
Ziwei Xuan
Guo-Jun Qi
ViT
23
3
0
14 Mar 2023
Trajectory-Aware Body Interaction Transformer for Multi-Person Pose
  Forecasting
Trajectory-Aware Body Interaction Transformer for Multi-Person Pose Forecasting
Xiaogang Peng
Siyuan Mao
Zizhao Wu
16
18
0
09 Mar 2023
Applying Plain Transformers to Real-World Point Clouds
Applying Plain Transformers to Real-World Point Clouds
Lanxiao Li
M. Heizmann
3DPC
ViT
16
3
0
28 Feb 2023
A Unified View of Long-Sequence Models towards Modeling Million-Scale
  Dependencies
A Unified View of Long-Sequence Models towards Modeling Million-Scale Dependencies
Hongyu Hè
Marko Kabić
25
2
0
13 Feb 2023
PhysFormer++: Facial Video-based Physiological Measurement with SlowFast
  Temporal Difference Transformer
PhysFormer++: Facial Video-based Physiological Measurement with SlowFast Temporal Difference Transformer
Zitong Yu
Yuming Shen
Jingang Shi
Hengshuang Zhao
Yawen Cui
Jiehua Zhang
Philip H. S. Torr
Guoying Zhao
ViT
MedIm
29
80
0
07 Feb 2023
Single Cells Are Spatial Tokens: Transformers for Spatial Transcriptomic
  Data Imputation
Single Cells Are Spatial Tokens: Transformers for Spatial Transcriptomic Data Imputation
Haifang Wen
Wenzhuo Tang
Wei Jin
Jiayuan Ding
Renming Liu
Xinnan Dai
Feng Shi
Lulu Shang
Jiliang Tang
Yuying Xie
12
8
0
06 Feb 2023
Learning a Fourier Transform for Linear Relative Positional Encodings in
  Transformers
Learning a Fourier Transform for Linear Relative Positional Encodings in Transformers
K. Choromanski
Shanda Li
Valerii Likhosherstov
Kumar Avinava Dubey
Shengjie Luo
Di He
Yiming Yang
Tamás Sarlós
Thomas Weingarten
Adrian Weller
17
8
0
03 Feb 2023
DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition
DilateFormer: Multi-Scale Dilated Transformer for Visual Recognition
Jiayu Jiao
Yuyao Tang
Kun-Li Channing Lin
Yipeng Gao
Jinhua Ma
Yaowei Wang
Wei-Shi Zheng
MedIm
ViT
17
136
0
03 Feb 2023
Variation-Aware Semantic Image Synthesis
Variation-Aware Semantic Image Synthesis
Mingle Xu
Jaehwan Lee
Sook Yoon
Hyongsuk Kim
D. Park
24
3
0
25 Jan 2023
Text to Point Cloud Localization with Relation-Enhanced Transformer
Text to Point Cloud Localization with Relation-Enhanced Transformer
Guangzhi Wang
Hehe Fan
Mohan S. Kankanhalli
3DPC
20
13
0
13 Jan 2023
Head-Free Lightweight Semantic Segmentation with Linear Transformer
Head-Free Lightweight Semantic Segmentation with Linear Transformer
B. Dong
Pichao Wang
Fan Wang
ViT
16
64
0
11 Jan 2023
Learning to Exploit Temporal Structure for Biomedical Vision-Language
  Processing
Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing
Shruthi Bannur
Stephanie L. Hyland
Qianchu Liu
Fernando Pérez-García
Maximilian Ilse
...
Maria T. A. Wetscherek
M. Lungren
A. Nori
Javier Alvarez-Valle
Ozan Oktay
27
109
0
11 Jan 2023
Position Embedding Needs an Independent Layer Normalization
Position Embedding Needs an Independent Layer Normalization
Runyi Yu
Zhennan Wang
Yinhuai Wang
Kehan Li
Yian Zhao
Jian Andrew Zhang
Guoli Song
Jie Chen
13
1
0
10 Dec 2022
Previous
1234
Next