ResearchTrend.AI
  • Communities
  • Connect sessions
  • AI calendar
  • Organizations
  • Join Slack
  • Contact Sales
Papers
Communities
Social Events
Terms and Conditions
Pricing
Contact Sales
Parameter LabParameter LabTwitterGitHubLinkedInBlueskyYoutube

© 2026 ResearchTrend.AI, All rights reserved.

  1. Home
  2. Papers
  3. 2104.08698
  4. Cited By
A Simple and Effective Positional Encoding for Transformers
v1v2 (latest)

A Simple and Effective Positional Encoding for Transformers

Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021
18 April 2021
Pu-Chin Chen
Henry Tsai
Srinadh Bhojanapalli
Hyung Won Chung
Yin-Wen Chang
Chun-Sung Ferng
ArXiv (abs)PDFHTML

Papers citing "A Simple and Effective Positional Encoding for Transformers"

32 / 32 papers shown
GeoPE:A Unified Geometric Positional Embedding for Structured Tensors
GeoPE:A Unified Geometric Positional Embedding for Structured Tensors
Yupu Yao
Bowen Yang
MDE
351
0
0
04 Dec 2025
What is the Best Sequence Length for BABYLM?
What is the Best Sequence Length for BABYLM?
Suchir Salhan
Richard Diehl Martinez
Zébulon Goriely
P. Buttery
148
3
0
22 Oct 2025
NDLPNet: A Location-Aware Nighttime Deraining Network and a Real-World Benchmark Dataset
NDLPNet: A Location-Aware Nighttime Deraining Network and a Real-World Benchmark Dataset
Huichun Liu
Xiaosong Li
Yang Liu
Xiaoqi Cheng
Haishu Tan
123
0
0
17 Sep 2025
FusionMAE: large-scale pretrained model to optimize and simplify diagnostic and control of fusion plasma
FusionMAE: large-scale pretrained model to optimize and simplify diagnostic and control of fusion plasma
Zongyu Yang
Zhenghao Yang
Wenjing Tian
Jiyuan Li
Xiang Sun
...
Zhe Gao
Wei Chen
Xiaoquan Ji
Min Xu
Wulyu Zhong
AI4CE
247
0
0
16 Sep 2025
An Empirical Study on Prompt Compression for Large Language Models
An Empirical Study on Prompt Compression for Large Language Models
Zhenru Zhang
Jinyi Li
Yihuai Lan
Xinze Wang
Hao Wang
MQ
305
5
0
24 Apr 2025
Of All StrIPEs: Investigating Structure-informed Positional Encoding for Efficient Music Generation
Of All StrIPEs: Investigating Structure-informed Positional Encoding for Efficient Music Generation
Manvi Agarwal
Changhong Wang
Gaël Richard
214
0
0
07 Apr 2025
Context-aware Biases for Length Extrapolation
Context-aware Biases for Length Extrapolation
Ali Veisi
Hamidreza Amirzadeh
Amir Mansourian
644
2
0
11 Mar 2025
LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMs
LCIRC: A Recurrent Compression Approach for Efficient Long-form Context and Query Dependent Modeling in LLMsNorth American Chapter of the Association for Computational Linguistics (NAACL), 2025
Sumin An
Junyoung Sung
Wonpyo Park
Chanjun Park
Paul Hongsuck Seo
695
0
0
10 Feb 2025
Learning the RoPEs: Better 2D and 3D Position Encodings with STRING
Learning the RoPEs: Better 2D and 3D Position Encodings with STRING
Connor Schenck
Isaac Reid
M. Jacob
Alex Bewley
Joshua Ainslie
...
Matthias Minderer
Dmitry Kalashnikov
Jonathan Tompson
Vikas Sindhwani
Krzysztof Choromanski
381
13
0
04 Feb 2025
Transforming NLU with Babylon: A Case Study in Development of Real-time,
  Edge-Efficient, Multi-Intent Translation System for Automated Drive-Thru
  Ordering
Transforming NLU with Babylon: A Case Study in Development of Real-time, Edge-Efficient, Multi-Intent Translation System for Automated Drive-Thru Ordering
Mostafa Varzaneh
Pooja Voladoddi
Tanmay Bakshi
Uma Gunturi
251
1
0
22 Nov 2024
DipMe: Haptic Recognition of Granular Media for Tangible Interactive
  Applications
DipMe: Haptic Recognition of Granular Media for Tangible Interactive Applications
Xinkai Wang
Shanghang Zhang
Ziyi Zhao
Lifeng Zhu
Aiguo Song
136
1
0
13 Nov 2024
Enhancing High-order Interaction Awareness in LLM-based Recommender
  Model
Enhancing High-order Interaction Awareness in LLM-based Recommender ModelConference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Xinfeng Wang
Jin Cui
Fumiyo Fukumoto
Yoshimi Suzuki
323
13
0
30 Sep 2024
TeXBLEU: Automatic Metric for Evaluate LaTeX Format
TeXBLEU: Automatic Metric for Evaluate LaTeX FormatIEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2024
Kyudan Jung
N. Kim
Hyongon Ryu
Sieun Hyeon
Seung-jun Lee
Hyeok-jae Lee
398
5
0
10 Sep 2024
A Review of Transformer-Based Models for Computer Vision Tasks:
  Capturing Global Context and Spatial Relationships
A Review of Transformer-Based Models for Computer Vision Tasks: Capturing Global Context and Spatial Relationships
Gracile Astlin Pereira
Muhammad Hussain
ViT
320
38
0
27 Aug 2024
Graph Transformers: A Survey
Graph Transformers: A Survey
Ahsan Shehzad
Xiwei Xu
Shagufta Abid
Ciyuan Peng
Shuo Yu
Dongyu Zhang
Karin Verspoor
AI4CE
451
53
0
13 Jul 2024
Positional encoding is not the same as context: A study on positional encoding for sequential recommendation
Positional encoding is not the same as context: A study on positional encoding for sequential recommendation
Alejo López-Ávila
Jinhua Du
Abbas Shimary
Ze Li
443
6
0
16 May 2024
PriorCLIP: Visual Prior Guided Vision-Language Model for Remote Sensing Image-Text Retrieval
PriorCLIP: Visual Prior Guided Vision-Language Model for Remote Sensing Image-Text Retrieval
Jiancheng Pan
Muyuan Ma
Qing Ma
Cong Bai
Shengyong Chen
325
12
0
16 May 2024
PoPE: Legendre Orthogonal Polynomials Based Position Encoding for Large
  Language Models
PoPE: Legendre Orthogonal Polynomials Based Position Encoding for Large Language Models
Arpit Aggarwal
144
0
0
29 Apr 2024
EulerFormer: Sequential User Behavior Modeling with Complex Vector
  Attention
EulerFormer: Sequential User Behavior Modeling with Complex Vector Attention
Zhen Tian
Wayne Xin Zhao
Changwang Zhang
Xin Zhao
Zhongrui Ma
Ji-Rong Wen
352
8
0
26 Mar 2024
MEP: Multiple Kernel Learning Enhancing Relative Positional Encoding
  Length Extrapolation
MEP: Multiple Kernel Learning Enhancing Relative Positional Encoding Length Extrapolation
Weiguo Gao
248
1
0
26 Mar 2024
PCToolkit: A Unified Plug-and-Play Prompt Compression Toolkit of Large
  Language Models
PCToolkit: A Unified Plug-and-Play Prompt Compression Toolkit of Large Language Models
Jinyi Li
Yihuai Lan
Lei Wang
Hao Wang
257
3
0
26 Mar 2024
Beyond the Limits: A Survey of Techniques to Extend the Context Length
  in Large Language Models
Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models
Xindi Wang
Mahsa Salmani
Parsa Omidi
Xiangyu Ren
Mehdi Rezagholizadeh
A. Eshaghi
LRM
381
97
0
03 Feb 2024
Advancing Transformer Architecture in Long-Context Large Language
  Models: A Comprehensive Survey
Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey
Yunpeng Huang
Jingwei Xu
Junyu Lai
Zixu Jiang
Taolue Chen
...
Xiaoxing Ma
Lijuan Yang
Zhou Xin
Shupeng Li
Penghao Zhao
LLMAGKELM
509
114
0
21 Nov 2023
Long-MIL: Scaling Long Contextual Multiple Instance Learning for
  Histopathology Whole Slide Image Analysis
Long-MIL: Scaling Long Contextual Multiple Instance Learning for Histopathology Whole Slide Image Analysis
Honglin Li
Yunlong Zhang
Chenglu Zhu
Jiatong Cai
Sunyi Zheng
Lin Yang
VLM
382
6
0
21 Nov 2023
Enhancing Pre-Trained Language Models with Sentence Position Embeddings
  for Rhetorical Roles Recognition in Legal Opinions
Enhancing Pre-Trained Language Models with Sentence Position Embeddings for Rhetorical Roles Recognition in Legal Opinions
Anas Belfathi
Nicolas Hernandez
Laura Monceaux
AILaw
276
5
0
08 Oct 2023
Generalized Power Attacks against Crypto Hardware using Long-Range Deep
  Learning
Generalized Power Attacks against Crypto Hardware using Long-Range Deep Learning
Elie Bursztein
Luca Invernizzi
Karel Král
D. Moghimi
J. Picod
Marina Zhang
AAML
241
10
0
12 Jun 2023
On the Design Fundamentals of Diffusion Models: A Survey
On the Design Fundamentals of Diffusion Models: A SurveyPattern Recognition (Pattern Recogn.), 2023
Ziyi Chang
George Alex Koulieris
Hyung Jin Chang
Hubert P. H. Shum
DiffM
805
91
0
07 Jun 2023
Knowledge Distillation in Vision Transformers: A Critical Review
Knowledge Distillation in Vision Transformers: A Critical Review
Gousia Habib
Tausifa Jan Saleem
Brejesh Lall
396
25
0
04 Feb 2023
P-Transformer: Towards Better Document-to-Document Neural Machine
  Translation
P-Transformer: Towards Better Document-to-Document Neural Machine TranslationIEEE/ACM Transactions on Audio Speech and Language Processing (TASLP), 2022
Yachao Li
Junhui Li
Jing Jiang
Shimin Tao
Hao Yang
Hao Fei
ViT
202
17
0
12 Dec 2022
Word Order Matters when you Increase Masking
Word Order Matters when you Increase MaskingConference on Empirical Methods in Natural Language Processing (EMNLP), 2022
Karim Lasri
Alessandro Lenci
Thierry Poibeau
311
8
0
08 Nov 2022
KERPLE: Kernelized Relative Positional Embedding for Length
  Extrapolation
KERPLE: Kernelized Relative Positional Embedding for Length ExtrapolationNeural Information Processing Systems (NeurIPS), 2022
Ta-Chung Chi
Ting-Han Fan
Peter J. Ramadge
Alexander I. Rudnicky
433
95
0
20 May 2022
Decoupled Side Information Fusion for Sequential Recommendation
Decoupled Side Information Fusion for Sequential RecommendationAnnual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2022
Yueqi Xie
Peilin Zhou
Sunghun Kim
438
155
0
23 Apr 2022
1
Page 1 of 1