Large Language Models are Inconsistent and Biased Evaluators

Large Language Models are Inconsistent and Biased Evaluators

2 May 2024

Rickard Stureborg

Dimitris Alikaniotis

Papers citing "Large Language Models are Inconsistent and Biased Evaluators"

11 / 11 papers shown

Title
To Judge or not to Judge: Using LLM Judgements for Advertiser Keyphrase Relevance at eBay Soumik Dey Hansi Wu Binbin Li 36 0 0 07 May 2025
LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning Joy Lim Jia Yin Daniel Zhang-Li Jifan Yu H. Li Shangqing Tu ... Zhiyuan Liu Huiqin Liu Lei Hou Juanzi Li Bin Xu 22 0 0 04 May 2025
Towards Automated Scoping of AI for Social Good Projects Jacob Emmerson Rayid Ghani Zheyuan Ryan Shi 70 0 0 28 Apr 2025
Persona-judge: Personalized Alignment of Large Language Models via Token-level Self-judgment Xiaotian Zhang Ruizhe Chen Yang Feng Zuozhu Liu 40 0 0 17 Apr 2025
SPHERE: An Evaluation Card for Human-AI Systems Qianou Ma Dora Zhao Xinran Zhao Chenglei Si Chenyang Yang Ryan Louie Ehud Reiter Diyi Yang Tongshuang Wu ALM 50 0 0 24 Mar 2025
Self-Generated Critiques Boost Reward Modeling for Language Models Yue Yu Zhengxing Chen Aston Zhang L Tan Chenguang Zhu ... Suchin Gururangan Chao-Yue Zhang Melanie Kambadur Dhruv Mahajan Rui Hou LRM ALM 87 14 0 25 Nov 2024
FactLens: Benchmarking Fine-Grained Fact Verification Kushan Mitra Dan Zhang Sajjadur Rahman Estevam R. Hruschka HILM 32 1 0 08 Nov 2024
From Single to Multi: How LLMs Hallucinate in Multi-Document Summarization Catarina G. Belem Pouya Pezeskhpour Hayate Iso Seiji Maekawa Nikita Bhutani Estevam R. Hruschka HILM 65 1 0 17 Oct 2024
Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templates Hui Wei Shenghua He Tian Xia Andy H. Wong Jingyang Lin Mei Han Mei Han ALM ELM 59 23 0 23 Aug 2024
Inverse Constitutional AI: Compressing Preferences into Principles Arduin Findeis Timo Kaufmann Eyke Hüllermeier Samuel Albanie Robert Mullins SyDa 41 9 0 02 Jun 2024
On the Limitations of Reference-Free Evaluations of Generated Text Daniel Deutsch Rotem Dror Dan Roth 27 44 0 22 Oct 2022