PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations

3 January 2025

Papers citing "PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations"

23 / 73 papers shown

Title
GenRES: Rethinking Evaluation for Generative Relation Extraction in the Era of Large Language Models Pengcheng Jiang Jiacheng Lin Zifeng Wang Jimeng Sun Jiawei Han 23 3 0 16 Feb 2024
LLM-based NLG Evaluation: Current Status and Challenges Mingqi Gao Xinyu Hu Jie Ruan Xiao Pu Xiaojun Wan ELM LM&MA 53 28 0 02 Feb 2024
What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection Shangbin Feng Herun Wan Ningnan Wang Zhaoxuan Tan Minnan Luo Yulia Tsvetkov AAML DeLMO 14 14 0 01 Feb 2024
Can Large Language Models be Trusted for Evaluation? Scalable Meta-Evaluation of LLMs as Evaluators via Agent Debate Steffi Chern Ethan Chern Graham Neubig Pengfei Liu LLMAG ALM ELM 11 23 0 30 Jan 2024
PRE: A Peer Review Based Large Language Model Evaluator Zhumin Chu Qingyao Ai Yiteng Tu Haitao Li Yiqun Liu LRM ALM 22 21 0 28 Jan 2024
Self-Rewarding Language Models Weizhe Yuan Richard Yuanzhe Pang Kyunghyun Cho Xian Li Sainbayar Sukhbaatar Jing Xu Jason Weston ReLM SyDa ALM LRM 215 291 0 18 Jan 2024
Leveraging Large Language Models for NLG Evaluation: Advances and Challenges Zhen Li Xiaohan Xu Tao Shen Can Xu Jia-Chen Gu Yuxuan Lai Chongyang Tao Shuai Ma LM&MA ELM 26 9 0 13 Jan 2024
Large Language Models for Social Networks: Applications, Challenges, and Solutions Jingying Zeng Richard Huang Waleed Malik Langxuan Yin Bojan Babic Danny Shacham Xiao Yan Jaewon Yang Qi He 11 3 0 04 Jan 2024
Beyond Isolation: Multi-Agent Synergy for Improving Knowledge Graph Construction Hongbin Ye Honghao Gui Aijia Zhang Tong Liu Wei Hua Weiqiang Jia LLMAG 15 5 0 05 Dec 2023
Leveraging Large Language Models for Collective Decision-Making Marios Papachristou Longqi Yang Chin-Chia Hsu LLMAG 29 2 0 03 Nov 2023
Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models Raphael Tang Xinyu Crystina Zhang Xueguang Ma Jimmy Lin Ferhan Ture LRM 29 15 0 11 Oct 2023
Evaluating Large Language Models at Evaluating Instruction Following Zhiyuan Zeng Jiatong Yu Tianyu Gao Yu Meng Tanya Goyal Danqi Chen ELM ALM 13 159 0 11 Oct 2023
Benchmarking Cognitive Biases in Large Language Models as Evaluators Ryan Koo Minhwa Lee Vipul Raheja Jong Inn Park Zae Myung Kim Dongyeop Kang ALM 30 71 0 29 Sep 2023
Large Language Model Alignment: A Survey Tianhao Shen Renren Jin Yufei Huang Chuang Liu Weilong Dong Zishan Guo Xinwei Wu Yan Liu Deyi Xiong LM&MA 14 169 0 26 Sep 2023
ALLURE: Auditing and Improving LLM-based Evaluation of Text using Iterative In-Context-Learning Hosein Hasanbeig Hiteshi Sharma Leo Betthauser Felipe Vieira Frujeri Ida Momennejad 24 13 0 24 Sep 2023
EvalLM: Interactive Evaluation of Large Language Model Prompts on User-Defined Criteria Tae Soo Kim Yoonjoo Lee Jamin Shin Young-Ho Kim Juho Kim 19 40 0 24 Sep 2023
Cognitive Mirage: A Review of Hallucinations in Large Language Models Hongbin Ye Tong Liu Aijia Zhang Wei Hua Weiqiang Jia HILM 31 76 0 13 Sep 2023
ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate Chi-Min Chan Weize Chen Yusheng Su Jianxuan Yu Wei Xue Shan Zhang Jie Fu Zhiyuan Liu ELM LLMAG ALM 14 434 0 14 Aug 2023
Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies Liangming Pan Michael Stephen Saxon Wenda Xu Deepak Nathani Xinyi Wang William Yang Wang KELM LRM 28 200 0 06 Aug 2023
LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day Chunyuan Li Cliff Wong Sheng Zhang Naoto Usuyama Haotian Liu Jianwei Yang Tristan Naumann Hoifung Poon Jianfeng Gao LM&MA MedIm 30 674 0 01 Jun 2023
Generative Agents: Interactive Simulacra of Human Behavior J. Park Joseph C. O'Brien Carrie J. Cai Meredith Ringel Morris Percy Liang Michael S. Bernstein LM&Ro AI4CE 209 1,701 0 07 Apr 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 170 3,504 0 10 Jun 2015