Translational Equivariance in Kernelizable Attention

15 February 2021

Max Horn

Kumar Shridhar

Elrich Groenewald

Philipp F. M. Baumann

ArXiv (abs)PDF HTML

Papers citing "Translational Equivariance in Kernelizable Attention"

5 / 5 papers shown

ComRoPE: Scalable and Robust Rotary Position Embedding Parameterized by Trainable Commuting Angle MatricesComputer Vision and Pattern Recognition (CVPR), 2025

313

04 Jun 2025

Linearized Relative Positional Encoding

Zhen Qin

Yuchao Dai

Lingpeng Kong

Yiran Zhong

227

18 Jul 2023

Learning a Fourier Transform for Linear Relative Positional Encodings in TransformersInternational Conference on Artificial Intelligence and Statistics (AISTATS), 2023

K. Choromanski

Shanda Li

Valerii Likhosherstov

371

03 Feb 2023

Simplex Random FeaturesInternational Conference on Machine Learning (ICML), 2023

Isaac Reid

K. Choromanski

Valerii Likhosherstov

Adrian Weller

296

31 Jan 2023

RoFormer: Enhanced Transformer with Rotary Position Embedding

1.2K

4,768

20 Apr 2021