A Frustratingly Easy Improvement for Position Embeddings via Random
Padding

A Frustratingly Easy Improvement for Position Embeddings via Random Padding

8 May 2023

Dongyan Zhao

Papers citing "A Frustratingly Easy Improvement for Position Embeddings via Random Padding"

6 / 6 papers shown

Title
The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Survey Saurav Pawar S.M. Towhidul Islam Tonmoy S. M. M. Zaman Vinija Jain Aman Chadha Amitava Das 24 24 0 15 Jan 2024
Advancing Transformer Architecture in Long-Context Large Language Models: A Comprehensive Survey Yunpeng Huang Jingwei Xu Junyu Lai Zixu Jiang Taolue Chen ... Xiaoxing Ma Lijuan Yang Zhou Xin Shupeng Li Penghao Zhao LLMAG KELM 28 53 0 21 Nov 2023
Extending Input Contexts of Language Models through Training on Segmented Sequences Petros Karypis Julian McAuley George Karypis 19 0 0 23 Oct 2023
LM-Infinite: Zero-Shot Extreme Length Generalization for Large Language Models Chi Han Qifan Wang Hao Peng Wenhan Xiong Yu Chen Heng Ji Sinong Wang 37 47 0 30 Aug 2023
Revisiting DocRED -- Addressing the False Negative Problem in Relation Extraction Qingyu Tan Lu Xu Lidong Bing Hwee Tou Ng Sharifah Mahani Aljunied 38 63 0 25 May 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 237 690 0 27 Aug 2021