A Survey of Evaluation Metrics Used for NLG Systems

27 August 2020

Ananya B. Sai

Akash Kumar Mohankumar

Mitesh M. Khapra

ELM

ArXiv PDF HTML

Papers citing "A Survey of Evaluation Metrics Used for NLG Systems"

23 / 23 papers shown

Title
Enhancing the Learning Experience: Using Vision-Language Models to Generate Questions for Educational Videos Markos Stamatakis Joshua Berger Christian Wartena Ralph Ewerth Anett Hoppe AI4Ed 37 0 0 03 May 2025
Consistency of Responses and Continuations Generated by Large Language Models on Social Media Wenlu Fan Y. X. Zhu Chenyang Wang Bin Wang Wentao Xu 57 1 0 14 Jan 2025
From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge Dawei Li Bohan Jiang Liangjie Huang Alimohammad Beigi Chengshuai Zhao ... Canyu Chen Tianhao Wu Kai Shu Lu Cheng Huan Liu ELM AILaw 108 61 0 25 Nov 2024
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages Ashmal Vayani Dinura Dissanayake Hasindri Watawana Noor Ahsan Nevasini Sasikumar ... Monojit Choudhury Ivan Laptev Mubarak Shah Salman Khan Fahad A Khan 124 8 0 25 Nov 2024
Evaluation of Retrieval-Augmented Generation: A Survey Hao Yu Aoran Gan Kai Zhang Shiwei Tong Qi Liu Zhaofeng Liu 3DV 57 78 0 13 May 2024
Navigating the Path of Writing: Outline-guided Text Generation with Large Language Models Yukyung Lee Soonwon Ka Bokyung Son Pilsung Kang Jaewook Kang LLMAG 42 6 0 22 Apr 2024
The Critique of Critique Shichao Sun Junlong Li Weizhe Yuan Ruifeng Yuan Wenjie Li Pengfei Liu ELM 27 0 0 09 Jan 2024
Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AI Mahyar Abbasian Elahe Khatibi Iman Azimi David Oniani Zahra Shakeri Hossein Abad ... Bryant Lin Olivier Gevaert Li-Jia Li Ramesh C. Jain Amir M. Rahmani LM&MA ELM AI4MH 23 63 0 21 Sep 2023
Rethinking Model Evaluation as Narrowing the Socio-Technical Gap Q. V. Liao Ziang Xiao ALM ELM 43 28 0 01 Jun 2023
APPLS: Evaluating Evaluation Metrics for Plain Language Summarization Yue Guo Tal August Gondy Leroy T. Cohen Lucy Lu Wang 55 8 0 23 May 2023
MAUVE Scores for Generative Models: Theory and Practice Krishna Pillutla Lang Liu John Thickstun Sean Welleck Swabha Swayamdipta Rowan Zellers Sewoong Oh Yejin Choi Zaïd Harchaoui EGVM 23 21 0 30 Dec 2022
Measuring the Measuring Tools: An Automatic Evaluation of Semantic Metrics for Text Corpora George Kour Samuel Ackerman Orna Raz E. Farchi Boaz Carmeli Ateret Anaby-Tavor 18 10 0 29 Nov 2022
Universal Evasion Attacks on Summarization Scoring Wenchuan Mu Kwan Hui Lim AAML 17 1 0 25 Oct 2022
Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implications Kaitlyn Zhou Su Lin Blodgett Adam Trischler Hal Daumé Kaheer Suleman Alexandra Olteanu ELM 94 25 0 13 May 2022
CounterGeDi: A controllable approach to generate polite, detoxified and emotional counterspeech Punyajoy Saha Kanishk Singh Adarsh Kumar Binny Mathew Animesh Mukherjee 6 35 0 09 May 2022
Generating Scientific Claims for Zero-Shot Scientific Fact Checking Dustin Wright David Wadden Kyle Lo Bailey Kuehl Arman Cohan Isabelle Augenstein Lucy Lu Wang MedIm 27 54 0 24 Mar 2022
Towards Explainable Evaluation Metrics for Natural Language Generation Christoph Leiter Piyawat Lertvittayakumjorn M. Fomicheva Wei-Ye Zhao Yang Gao Steffen Eger AAML ELM 16 20 0 21 Mar 2022
Explainable AI (XAI): A Systematic Meta-Survey of Current Challenges and Future Opportunities Waddah Saeed C. Omlin XAI 34 409 0 11 Nov 2021
BEAMetrics: A Benchmark for Language Generation Evaluation Evaluation Thomas Scialom Felix Hill 12 7 0 18 Oct 2021
Perturbation CheckLists for Evaluating NLG Evaluation Metrics Ananya B. Sai Tanay Dixit D. Y. Sheth S. Mohan Mitesh M. Khapra AAML 97 55 0 13 Sep 2021
Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text Yao Dou Maxwell Forbes Rik Koncel-Kedziorski Noah A. Smith Yejin Choi DeLMO 6 125 0 02 Jul 2021
MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers Krishna Pillutla Swabha Swayamdipta Rowan Zellers John Thickstun Sean Welleck Yejin Choi Zaïd Harchaoui 15 340 0 02 Feb 2021
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding Akira Fukui Dong Huk Park Daylen Yang Anna Rohrbach Trevor Darrell Marcus Rohrbach 144 1,458 0 06 Jun 2016