Prediction-Powered Ranking of Large Language Models

v1v2 (latest)

Prediction-Powered Ranking of Large Language Models

27 February 2024

Eleni Straitouri

Manuel Gomez Rodriguez

ArXiv (abs)PDF HTML

Papers citing "Prediction-Powered Ranking of Large Language Models"

7 / 7 papers shown

Title
Cost-Optimal Active AI Model Evaluation Anastasios Nikolas Angelopoulos Jacob Eisenstein Jonathan Berant Alekh Agarwal Adam Fisch 77 1 0 09 Jun 2025
Evaluation of Large Language Models via Coupled Token Generation N. C. Benz Stratis Tsirtsis Eleni Straitouri Ivi Chatzi Ander Artola Velasco Suhas Thejaswi Manuel Gomez Rodriguez 146 1 0 03 Feb 2025
Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators Yinhong Liu Han Zhou Zhijiang Guo Ehsan Shareghi Ivan Vulić Anna Korhonen Nigel Collier ALM 363 92 0 20 Jan 2025
PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations Ruosen Li Teerth Patel Xinya Du LLMAG ALM 287 115 0 03 Jan 2025
Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the data Florian E. Dorner Vivian Y. Nastl Moritz Hardt ELM ALM 157 13 0 17 Oct 2024
Can Unconfident LLM Annotations Be Used for Confident Conclusions? Kristina Gligorić Tijana Zrnic Cinoo Lee Emmanuel J. Candès Dan Jurafsky 240 18 0 27 Aug 2024
AutoEval Done Right: Using Synthetic Data for Model Evaluation Pierre Boyeau Anastasios Nikolas Angelopoulos N. Yosef Jitendra Malik Michael I. Jordan SyDa 144 25 0 09 Mar 2024