L-Eval: Instituting Standardized Evaluation for Long Context Language Models

20 July 2023

Lingpeng Kong

Xipeng Qiu

ELM

ALM

ArXiv PDF HTML

Papers citing "L-Eval: Instituting Standardized Evaluation for Long Context Language Models"

29 / 29 papers shown

Title
Rethinking Memory in AI: Taxonomy, Operations, Topics, and Future Directions Yiming Du Wenyu Huang Danna Zheng Zhaowei Wang Sébastien Montella Mirella Lapata Kam-Fai Wong Jeff Z. Pan KELM MU 65 1 0 01 May 2025
Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks Yixin Cao Shibo Hong X. Li Jiahao Ying Yubo Ma ... Juanzi Li Aixin Sun Xuanjing Huang Tat-Seng Chua Yu Jiang ALM ELM 84 0 0 26 Apr 2025
CURIE: Evaluating LLMs On Multitask Scientific Long Context Understanding and Reasoning Hao Cui Zahra Shamsi Gowoon Cheon Xuejian Ma Shutong Li ... Eun-Ah Kim Michael P Brenner Viren Jain Sameera Ponda Subhashini Venugopalan ELM LRM 46 0 0 14 Mar 2025
HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding Shehreen Azad Vibhav Vineet Y. S. Rawat VLM 47 1 0 11 Mar 2025
Shifting Long-Context LLMs Research from Input to Output Yuhao Wu Yushi Bai Zhiqing Hu Shangqing Tu Ming Shan Hee Juanzi Li Roy Ka-Wei Lee 57 0 0 06 Mar 2025
ELITR-Bench: A Meeting Assistant Benchmark for Long-Context Language Models Thibaut Thonet Jos Rozen Laurent Besacier RALM 124 2 0 20 Jan 2025
VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation Manan Suri Puneet Mathur Franck Dernoncourt Kanika Goswami Ryan Rossi Dinesh Manocha 93 3 0 14 Dec 2024
Needle Threading: Can LLMs Follow Threads through Near-Million-Scale Haystacks? Jonathan Roberts Kai Han Samuel Albanie LLMAG 65 0 0 07 Nov 2024
Guide-LLM: An Embodied LLM Agent and Text-Based Topological Map for Robotic Guidance of People with Visual Impairments Sangmim Song S. Kodagoda A. Gunatilake Marc G. Carmichael Karthick Thiyagarajan Jodi Martin LM&Ro 21 1 0 28 Oct 2024
ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coverage Taewhoo Lee Chanwoong Yoon Kyochul Jang Donghyeon Lee Minju Song Hyunjae Kim Jaewoo Kang ELM 19 1 0 22 Oct 2024
LongGenBench: Long-context Generation Benchmark Xiang Liu Peijie Dong Xuming Hu Xiaowen Chu RALM 25 8 0 05 Oct 2024
HELMET: How to Evaluate Long-Context Language Models Effectively and Thoroughly Howard Yen Tianyu Gao Minmin Hou Ke Ding Daniel Fleischer Peter Izsak Moshe Wasserblat Danqi Chen ALM ELM 43 24 0 03 Oct 2024
How to Train Long-Context Language Models (Effectively) Tianyu Gao Alexander Wettig Howard Yen Danqi Chen RALM 52 36 0 03 Oct 2024
Locret: Enhancing Eviction in Long-Context LLM Inference with Trained Retaining Heads on Consumer-Grade Devices Yuxiang Huang Binhang Yuan Xu Han Chaojun Xiao Zhiyuan Liu RALM 67 1 0 02 Oct 2024
Bridging Context Gaps: Leveraging Coreference Resolution for Long Contextual Understanding Yanming Liu Xinyue Peng Jiannan Cao Shi Bo Yanxin Shen Tianyu Du Sheng Cheng Xun Wang Jianwei Yin Xuhong Zhang 45 9 0 02 Oct 2024
Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems Philippe Laban Alexander R. Fabbri Caiming Xiong Chien-Sheng Wu RALM 33 41 0 01 Jul 2024
Leave No Document Behind: Benchmarking Long-Context LLMs with Extended Multi-Doc QA Minzheng Wang Longze Chen Cheng Fu Shengyi Liao Xinghua Zhang ... Run Luo Yunshui Li Min Yang Fei Huang Yongbin Li RALM 18 41 0 25 Jun 2024
BABILong: Testing the Limits of LLMs with Long Context Reasoning-in-a-Haystack Yuri Kuratov Aydar Bulatov Petr Anokhin Ivan Rodkin Dmitry Sorokin Artyom Sorokin Mikhail Burtsev RALM ALM LRM ReLM ELM 29 57 0 14 Jun 2024
CS-Bench: A Comprehensive Benchmark for Large Language Models towards Computer Science Mastery Xiaoshuai Song Muxi Diao Guanting Dong Zhengyang Wang Yujia Fu ... Yejie Wang Zhuoma Gongque Jianing Yu Qiuna Tan Weiran Xu ELM 35 10 0 12 Jun 2024
Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Context Understanding Zhihan Zhang Yixin Cao Chenchen Ye Yunshan Ma Lizi Liao Tat-Seng Chua 16 9 0 04 Jun 2024
PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference Dongjie Yang Xiaodong Han Yan Gao Yao Hu Shilin Zhang Hai Zhao 14 49 0 21 May 2024
On the Efficacy of Eviction Policy for Key-Value Constrained Generative Language Model Inference Siyu Ren Kenny Q. Zhu 10 27 0 09 Feb 2024
PROXYQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models Haochen Tan Zhijiang Guo Zhan Shi Lu Xu Zhili Liu ... Xiaoguang Li Yasheng Wang Lifeng Shang Qun Liu Linqi Song 12 12 0 26 Jan 2024
Recursively Summarizing Enables Long-Term Dialogue Memory in Large Language Models Qingyue Wang Y. Fu Yanan Cao Zhiliang Tian Shi Wang Dacheng Tao LLMAG KELM RALM 34 22 0 29 Aug 2023
Unlimiformer: Long-Range Transformers with Unlimited Length Input Amanda Bertsch Uri Alon Graham Neubig Matthew R. Gormley RALM 91 122 0 02 May 2023
SQuALITY: Building a Long-Document Summarization Dataset the Hard Way Alex Jinpeng Wang Richard Yuanzhe Pang Angelica Chen Jason Phang Samuel R. Bowman 69 44 0 23 May 2022
MultiDoc2Dial: Modeling Dialogues Grounded in Multiple Documents Song Feng S. Patel H. Wan Sachindra Joshi 43 66 0 26 Sep 2021
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 234 690 0 27 Aug 2021
Can We Automate Scientific Reviewing? Weizhe Yuan Pengfei Liu Graham Neubig 73 82 0 30 Jan 2021