Long Context is Not Long at All: A Prospector of Long-Dependency Data
for Large Language Models

Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models

28 May 2024

Yunshui Li

Run Luo

Min Yang

Papers citing "Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models"

10 / 10 papers shown

Title
Evaluating the Goal-Directedness of Large Language Models Tom Everitt Cristina Garbacea Alexis Bellot Jonathan G. Richens Henry Papadatos Simeon Campos Rohin Shah ELM LM&MA LM&Ro LRM 68 0 0 16 Apr 2025
Shifting Long-Context LLMs Research from Input to Output Yuhao Wu Yushi Bai Zhiqing Hu Shangqing Tu Ming Shan Hee Juanzi Li Roy Ka-Wei Lee 57 0 0 06 Mar 2025
LongAttn: Selecting Long-context Training Data via Token-level Attention Longyun Wu Dawei Zhu Guangxiang Zhao Zhuocheng Yu Junfeng Ran Xiangyu Wong Lin Sun Sujian Li 39 0 0 24 Feb 2025
The Energy Loss Phenomenon in RLHF: A New Perspective on Mitigating Reward Hacking Yuchun Miao Sen Zhang Liang Ding Yuqi Zhang L. Zhang Dacheng Tao 81 3 0 31 Jan 2025
LongRecipe: Recipe for Efficient Long Context Generalization in Large Language Models Zhiyuan Hu Yuliang Liu Jinman Zhao Suyuchen Wang Yan Wang ... Qing Gu Anh Tuan Luu See-Kiong Ng Zhiwei Jiang Bryan Hooi 55 11 0 31 Aug 2024
LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs Yushi Bai Jiajie Zhang Xin Lv Linzhi Zheng Siqi Zhu Lei Hou Yuxiao Dong Jie Tang Juanzi Li VGen LLMAG ALM 34 37 0 13 Aug 2024
Benchmarking General-Purpose In-Context Learning Fan Wang Chuan Lin Yang Cao Yu Kang 30 1 0 27 May 2024
With Greater Text Comes Greater Necessity: Inference-Time Training Helps Long Text Generation Y. Wang D. Ma D. Cai RALM 36 17 0 21 Jan 2024
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 245 695 0 27 Aug 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 237 590 0 14 Jul 2021