What Kinds of Tokens Benefit from Distant Text? An Analysis on Long Context Language Modeling

17 June 2024

Papers citing "What Kinds of Tokens Benefit from Distant Text? An Analysis on Long Context Language Modeling"

3 / 3 papers shown

Title
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression Huiqiang Jiang Qianhui Wu Xufang Luo Dongsheng Li Chin-Yew Lin Yuqing Yang Lili Qiu RALM 96 179 0 10 Oct 2023
PoSE: Efficient Context Window Extension of LLMs via Positional Skip-wise Training Dawei Zhu Nan Yang Liang Wang Yifan Song Wenhao Wu Furu Wei Sujian Li 49 77 0 19 Sep 2023
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 234 690 0 27 Aug 2021