Towards a Unified Multi-Dimensional Evaluator for Text Generation

13 October 2022

Yang Liu

Heng Ji

Papers citing "Towards a Unified Multi-Dimensional Evaluator for Text Generation"

44 / 44 papers shown

Title
SEval-Ex: A Statement-Level Framework for Explainable Summarization Evaluation Tanguy Herserant Vincent Guigue ELM 35 0 0 04 May 2025
JaccDiv: A Metric and Benchmark for Quantifying Diversity of Generated Marketing Text in the Music Industry Anum Afzal Alexandre Mercier Florian Matthes 55 0 0 29 Apr 2025
Can LLMs Be Trusted for Evaluating RAG Systems? A Survey of Methods and Datasets Lorenz Brehme Thomas Ströhle Ruth Breu 59 0 0 28 Apr 2025
VIST-GPT: Ushering in the Era of Visual Storytelling with LLMs? Mohamed Gado Towhid Taliee Muhammad Memon D. Ignatov Radu Timofte 63 0 0 27 Apr 2025
Evaluating Evaluation Metrics -- The Mirage of Hallucination Detection Atharva Kulkarni Yuan-kang Zhang Joel Ruben Antony Moniz Xiou Ge Bo-Hsiang Tseng Dhivya Piraviperumal S. Hong-ye Yu HILM 76 0 0 25 Apr 2025
A Scalable Framework for Evaluating Health Language Models Neil Mallinar A. Heydari Xin Liu Anthony Z. Faranesh Brent Winslow ... Mark Malhotra Shwetak N. Patel Javier L. Prieto Daniel J. McDuff Ahmed A. Metwally LM&MA 56 2 0 30 Mar 2025
Think Together and Work Better: Combining Humans' and LLMs' Think-Aloud Outcomes for Effective Text Evaluation SeongYeub Chu JongWoo Kim MunYong Yi 55 1 0 21 Feb 2025
MIH-TCCT: Mitigating Inconsistent Hallucinations in LLMs via Event-Driven Text-Code Cyclic Training Xinxin You Xien Liu Qixin Sun Huan Zhang Kaiyin Zhou Shaohui Liu Guoping Hu Shijin Wang Si Liu Ji Wu 83 0 0 13 Feb 2025
Learning to Summarize from LLM-generated Feedback Hwanjun Song Taewon Yun Yuho Lee Jihwan Oh Gihun Lee Jason (Jinglun) Cai Hang Su 73 2 0 28 Jan 2025
Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators Yinhong Liu Han Zhou Zhijiang Guo Ehsan Shareghi Ivan Vulić Anna Korhonen Nigel Collier ALM 128 67 0 20 Jan 2025
Foundation Models at Work: Fine-Tuning for Fairness in Algorithmic Hiring Buse Sibel Korkmaz Rahul Nair Elizabeth M. Daly Evangelos Anagnostopoulos Christos Varytimidis Antonio del Rio Chanona 40 0 0 13 Jan 2025
PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations Ruosen Li Teerth Patel Xinya Du LLMAG ALM 52 96 0 03 Jan 2025
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models Yulei Qin Yuncheng Yang Pengcheng Guo Gang Li Hang Shao Yuchen Shi Zihan Xu Yun Gu Ke Li Xing Sun ALM 88 11 0 31 Dec 2024
ChipAlign: Instruction Alignment in Large Language Models for Chip Design via Geodesic Interpolation Chenhui Deng Yunsheng Bai Haoxing Ren 26 1 0 31 Dec 2024
Bridging the Gap between Expert and Language Models: Concept-guided Chess Commentary Generation and Evaluation Jaechang Kim Jinmin Goh Inseok Hwang Jaewoong Cho Jungseul Ok ELM 28 1 0 28 Oct 2024
4-LEGS: 4D Language Embedded Gaussian Splatting Gal Fiebelman Tamir Cohen Ayellet Morgenstern Peter Hedman Hadar Averbuch-Elor 3DGS 41 1 0 14 Oct 2024
How to Make the Most of LLMs' Grammatical Knowledge for Acceptability Judgments Yusuke Ide Yuto Nishida Miyu Oba Miyu Oba Justin Vasselli Hidetaka Kamigaito Taro Watanabe 36 2 0 19 Aug 2024
Leveraging Entailment Judgements in Cross-Lingual Summarisation Huajian Zhang Laura Perez-Beltrachini HILM 34 0 0 01 Aug 2024
ECoh: Turn-level Coherence Evaluation for Multilingual Dialogues John Mendonça Isabel Trancoso A. Lavie 29 3 0 16 Jul 2024
FineSurE: Fine-grained Summarization Evaluation using LLMs Hwanjun Song Hang Su Igor Shalyminov Jason (Jinglun) Cai Saab Mansour HILM 26 29 0 01 Jul 2024
Factual Dialogue Summarization via Learning from Large Language Models Rongxin Zhu Jey Han Lau Jianzhong Qi HILM 46 1 0 20 Jun 2024
Holistic Evaluation for Interleaved Text-and-Image Generation Minqian Liu Zhiyang Xu Zihao Lin Trevor Ashby Joy Rimchala Jiaxin Zhang Lifu Huang EGVM 36 7 0 20 Jun 2024
Take the essence and discard the dross: A Rethinking on Data Selection for Fine-Tuning Large Language Models Ziche Liu Rui Ke Feng Jiang Feng Jiang Haizhou Li 61 1 0 20 Jun 2024
Benchmark Data Contamination of Large Language Models: A Survey Cheng Xu Shuhao Guan Derek Greene Mohand-Tahar Kechadi ELM ALM 36 38 0 06 Jun 2024
SLIDE: A Framework Integrating Small and Large Language Models for Open-Domain Dialogues Evaluation Kun Zhao Bohao Yang Chen Tang Chenghua Lin Liang Zhan 41 5 0 24 May 2024
From Matching to Generation: A Survey on Generative Information Retrieval Xiaoxi Li Jiajie Jin Yujia Zhou Yuyao Zhang Peitian Zhang Yutao Zhu Zhicheng Dou 3DV 67 45 0 23 Apr 2024
Semi-Supervised Dialogue Abstractive Summarization via High-Quality Pseudolabel Selection Jianfeng He Hang Su Jason (Jinglun) Cai Igor Shalyminov Hwanjun Song Saab Mansour 24 4 0 06 Mar 2024
How Much Annotation is Needed to Compare Summarization Models? Chantal Shaib Joe Barrow Alexa F. Siu Byron C. Wallace A. Nenkova 36 2 0 28 Feb 2024
To Burst or Not to Burst: Generating and Quantifying Improbable Text Kuleen Sasse Samuel Barham Efsun Sarioglu Kayi Edward W. Staley DeLMO 16 1 0 27 Jan 2024
Investigating Data Contamination for Pre-training Language Models Minhao Jiang Ken Ziyu Liu Ming Zhong Rylan Schaeffer Siru Ouyang Jiawei Han Sanmi Koyejo 23 62 0 11 Jan 2024
X-Eval: Generalizable Multi-aspect Text Evaluation via Augmented Instruction Tuning with Auxiliary Evaluation Aspects Minqian Liu Ying Shen Zhiyang Xu Yixin Cao Eunah Cho Vaibhav Kumar Reza Ghanadan Lifu Huang ELM LM&MA ALM 41 25 0 15 Nov 2023
MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records Scott L. Fleming Alejandro Lozano W. Haberkorn Jenelle A. Jindal E. Reis ... Jonathan H. Chen Keith Morse Emma Brunskill Jason Alan Fries N. Shah LM&MA 28 53 0 27 Aug 2023
DecompEval: Evaluating Generated Texts as Unsupervised Decomposed Question Answering Pei Ke Fei Huang Fei Mi Yasheng Wang Qun Liu Xiaoyan Zhu Minlie Huang ReLM ELM 34 10 0 13 Jul 2023
Information Association for Language Model Updating by Mitigating LM-Logical Discrepancy Pengfei Yu Heng Ji KELM 31 9 0 29 May 2023
A Critical Evaluation of Evaluations for Long-form Question Answering Fangyuan Xu Yixiao Song Mohit Iyyer Eunsol Choi ELM 35 94 0 29 May 2023
G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment Yang Liu Dan Iter Yichong Xu Shuohang Wang Ruochen Xu Chenguang Zhu ELM ALM LM&MA 28 1,068 0 29 Mar 2023
Towards Interpretable and Efficient Automatic Reference-Based Summarization Evaluation Yixin Liu Alexander R. Fabbri Yilun Zhao Pengfei Liu Shafiq R. Joty Chien-Sheng Wu Caiming Xiong Dragomir R. Radev 13 27 0 07 Mar 2023
Towards Fine-Grained Information: Identifying the Type and Location of Translation Errors Keqin Bao Yu Wan Dayiheng Liu Baosong Yang Wenqiang Lei Xiangnan He Derek F.Wong Jun Xie 22 4 0 17 Feb 2023
Toward Human-Like Evaluation for Natural Language Generation with Error Analysis Qingyu Lu Liang Ding Liping Xie Kanjian Zhang Derek F. Wong Dacheng Tao ELM ALM 32 14 0 20 Dec 2022
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation Tianxing He Jingyu Zhang Tianle Wang Sachin Kumar Kyunghyun Cho James R. Glass Yulia Tsvetkov 25 44 0 20 Dec 2022
Bidimensional Leaderboards: Generate and Evaluate Language Hand in Hand Jungo Kasai Keisuke Sakaguchi Ronan Le Bras Lavinia Dunagan Jacob Morrison Alexander R. Fabbri Yejin Choi Noah A. Smith 49 39 0 08 Dec 2021
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies Mor Geva Daniel Khashabi Elad Segal Tushar Khot Dan Roth Jonathan Berant RALM 245 671 0 06 Jan 2021
Factual Error Correction for Abstractive Summarization Models Mengyao Cao Yue Dong Jiapeng Wu Jackie C.K. Cheung HILM KELM 167 159 0 17 Oct 2020
Teaching Machines to Read and Comprehend Karl Moritz Hermann Tomás Kociský Edward Grefenstette L. Espeholt W. Kay Mustafa Suleyman Phil Blunsom 170 3,508 0 10 Jun 2015