Adding Error Bars to Evals: A Statistical Approach to Language Model
Evaluations

Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations

1 November 2024

Evan Miller

Papers citing "Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations"

12 / 12 papers shown

Title
LLMs Outperform Experts on Challenging Biology Benchmarks Lennart Justen ELM 30 0 0 09 May 2025
HiBayES: A Hierarchical Bayesian Modeling Framework for AI Evaluation Statistics Lennart Luettgau Harry Coppock Magda Dubois Christopher Summerfield Cozmin Ududec 31 0 0 08 May 2025
Cer-Eval: Certifiable and Cost-Efficient Evaluation Framework for LLMs G. Wang Z. Chen Bo Li Haifeng Xu 126 0 0 02 May 2025
Confidence in Large Language Model Evaluation: A Bayesian Approach to Limited-Sample Challenges Xiao Xiao Yu Su Sijing Zhang Zhang Chen Yadong Chen Tian Liu 39 0 0 30 Apr 2025
ChatBench: From Static Benchmarks to Human-AI Evaluation Serina Chang Ashton Anderson Jake M. Hofman ELM AI4MH 57 2 0 22 Mar 2025
Faithfulness of LLM Self-Explanations for Commonsense Tasks: Larger Is Better, and Instruction-Tuning Allows Trade-Offs but Not Pareto Dominance Noah Y. Siegel N. Heess Maria Perez-Ortiz Oana-Maria Camburu LRM 49 0 0 17 Mar 2025
LAG-MMLU: Benchmarking Frontier LLM Understanding in Latvian and Giriama Naome A. Etori Kevin Lu Randu Karisa Arturs Kanepajs LRM ELM 160 0 0 14 Mar 2025
LLM-Safety Evaluations Lack Robustness Tim Beyer Sophie Xhonneux Simon Geisler Gauthier Gidel Leo Schwinn Stephan Günnemann ALM ELM 171 0 0 04 Mar 2025
What do Large Language Models Say About Animals? Investigating Risks of Animal Harm in Generated Text Arturs Kanepajs Aditi Basu Sankalpa Ghose Constance Li Akshat Mehta Ronak Mehta Samuel David Tucker-Davis Eric Zhou Bob Fischer ALM ELM 43 0 0 03 Mar 2025
Position: Ensuring mutual privacy is necessary for effective external evaluation of proprietary AI systems Ben Bucknall Robert F. Trager Michael A. Osborne 80 0 0 03 Mar 2025
Beyond the Singular: The Essential Role of Multiple Generations in Effective Benchmark Evaluation and Analysis Wenbo Zhang Hengrui Cai Wenyu Chen 82 0 0 17 Feb 2025
Evaluation of Large Language Models via Coupled Token Generation N. C. Benz Stratis Tsirtsis Eleni Straitouri Ivi Chatzi Ander Artola Velasco Suhas Thejaswi Manuel Gomez Rodriguez 51 0 0 03 Feb 2025