3D-RPE: Enhancing Long-Context Modeling Through 3D Rotary Position Encoding

14 June 2024

Peng Zhang

Papers citing "3D-RPE: Enhancing Long-Context Modeling Through 3D Rotary Position Encoding"

2 / 2 papers shown

Title
Eliminating Position Bias of Language Models: A Mechanistic Approach Ziqi Wang Hanlin Zhang Xiner Li Kuan-Hao Huang Chi Han Shuiwang Ji Sham Kakade Hao Peng Heng Ji 49 12 0 01 Jul 2024
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 234 690 0 27 Aug 2021