ResearchTrend.AI
On the token distance modeling ability of higher RoPE attention dimension

On the token distance modeling ability of higher RoPE attention dimension

Xiangyu Hong
Che Jiang
Biqing Qi
Fandong Meng
Mo Yu
Bowen Zhou
Jie Zhou
ArXivPDFHTML

Papers citing "On the token distance modeling ability of higher RoPE attention dimension"

1 / 1 papers shown
Title
Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation
Effective Length Extrapolation via Dimension-Wise Positional Embeddings Manipulation
Yi Lu
Wanxu Zhao
Xin Zhou
Chenxin An
C. Wang
...
Jun Zhao
Tao Ji
Tao Gui
Qi Zhang
Xuanjing Huang
39
0
0
26 Apr 2025