Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale

International Conference on Computational Linguistics (COLING), 2025

26 October 2020

Pranava Madhyastha

Papers citing "Curious Case of Language Generation Evaluation Metrics: A Cautionary Tale"

24 / 24 papers shown

Title
References Matter: Investigating the Impact of Reference Set Variation on Summarization Evaluation Silvia Casola Teli Ma Siyao Peng Oliver Kraus Albert Gatt Barbara Plank 124 0 0 17 Jun 2025
Analyzing The Language of Visual Tokens David M. Chan Rodolfo Corona J. S. Park Cheol Jun Cho Yutong Bai Trevor Darrell 61 8 0 07 Nov 2024
Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models Eddie L. Ungless Nikolas Vitsakis Zeerak Talat James Garforth Bjorn Ross Arno Onken Atoosa Kasirzadeh Alexandra Birch 117 3 0 17 Oct 2024
CaT-BENCH: Benchmarking Language Model Understanding of Causal and Temporal Dependencies in Plans Yash Kumar Lal Vanya Cohen Nathanael Chambers Niranjan Balasubramanian Raymond Mooney ELM LRM ReLM 138 10 0 22 Jun 2024
Creating Suspenseful Stories: Iterative Planning with Large Language Models Kaige Xie Mark Riedl LRM 123 15 0 27 Feb 2024
LUNA: A Framework for Language Understanding and Naturalness Assessment Marat Saidov A. Bakalova Ekaterina Taktasheva Vladislav Mikhailov Ekaterina Artemova ELM 107 3 0 09 Jan 2024
Tailoring with Targeted Precision: Edit-Based Agents for Open-Domain Procedure Customization Yash Kumar Lal Li Zhang Faeze Brahman Bodhisattwa Prasad Majumder Peter Clark Niket Tandon KELM 142 7 0 16 Nov 2023
Evaluating Evaluation Metrics: A Framework for Analyzing NLG Evaluation Metrics using Measurement Theory Ziang Xiao Susu Zhang Vivian Lai Q. V. Liao ELM 165 36 0 24 May 2023
NLG Evaluation Metrics Beyond Correlation Analysis: An Empirical Metric Preference Checklist Iftitahu Ni'mah Meng Fang Vlado Menkovski Mykola Pechenizkiy 114 16 0 15 May 2023
SEM-POS: Grammatically and Semantically Correct Video Captioning Asmar Nadeem A. Hilton R. Dawes Graham A. Thomas A. Mustafa 117 9 0 26 Mar 2023
IC3: Image Captioning by Committee Consensus David M. Chan Austin Myers Sudheendra Vijayanarasimhan David A. Ross John F. Canny 158 20 0 02 Feb 2023
Towards a Unified Model for Generating Answers and Explanations in Visual Question Answering Chenxi Whitehouse Tillman Weyde Pranava Madhyastha LRM 127 4 0 25 Jan 2023
On the Blind Spots of Model-Based Evaluation Metrics for Text Generation Tianxing He Jingyu Zhang Tianle Wang Sachin Kumar Kyunghyun Cho James R. Glass Yulia Tsvetkov 179 54 0 20 Dec 2022
Distribution Aware Metrics for Conditional Natural Language Generation David M. Chan Yiming Ni David A. Ross Sudheendra Vijayanarasimhan Austin Myers John F. Canny 172 4 0 15 Sep 2022
Spurious Correlations in Reference-Free Evaluation of Text Generation Esin Durmus Faisal Ladhak Tatsunori Hashimoto 114 35 0 21 Apr 2022
Uniform Complexity for Text Generation Joseph Marvin Imperial Harish Tayyar Madabushi 106 4 0 11 Apr 2022
Supervised Visual Attention for Simultaneous Multimodal Machine Translation Veneta Haralampieva Ozan Caglayan Lucia Specia LRM 137 4 0 23 Jan 2022
Guiding Visual Question Generation Nihir Vedd Zixu Wang Marek Rei Yishu Miao Lucia Specia 188 24 0 15 Oct 2021
Does Summary Evaluation Survive Translation to Other Languages? Neslihan Iskender Oleg V. Vasilyev Neslihan Iskender John Bohannon 106 4 0 16 Sep 2021
Investigating Text Simplification Evaluation Laura Vásquez-Rodríguez Matthew Shardlow Piotr Przybyła Sophia Ananiadou 68 10 0 28 Jul 2021
From Show to Tell: A Survey on Deep Learning-based Image Captioning Matteo Stefanini Marcella Cornia Lorenzo Baraldi S. Cascianelli G. Fiameni Rita Cucchiara 3DV VLM MLLM 235 290 0 14 Jul 2021
What Context Features Can Transformer Language Models Use? J. O'Connor Jacob Andreas KELM 96 79 0 15 Jun 2021
TellMeWhy: A Dataset for Answering Why-Questions in Narratives Yash Kumar Lal Nathanael Chambers Raymond J. Mooney Niranjan Balasubramanian 116 50 0 11 Jun 2021
Inferring the Reader: Guiding Automated Story Generation with Commonsense Reasoning Xiangyu Peng Siyan Li Sarah Wiegreffe Mark O. Riedl LRM 181 40 0 04 May 2021