Evaluating the Factual Consistency of Abstractive Text Summarization

Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019

28 October 2019

Papers citing "Evaluating the Factual Consistency of Abstractive Text Summarization"

50 / 491 papers shown

Title
Efficient LLM Comparative Assessment: a Product of Experts Framework for Pairwise ComparisonsConference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Adian Liusie Vatsal Raina Yassir Fathullah Mark Gales 155 15 0 09 May 2024
QANA: LLM-based Question Generation and Network Analysis for Zero-shot Key Point Analysis and Beyond Tomoki Fukuma Koki Noda Toshihide Ubukata Kousuke Hoso Yoshiharu Ichikawa Kyosuke Kambe Yu Masubuchi F. Toriumi 155 2 0 29 Apr 2024
Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratings Olivia Wiles Chuhan Zhang Isabela Albuquerque Ivana Kajić Su Wang ... Jordi Pont-Tuset Aida Nematzadeh Anant Nawalgaria Jordi Pont-Tuset Aida Nematzadeh EGVM 560 30 0 25 Apr 2024
A Survey of Automatic Hallucination Evaluation on Natural Language Generation Siya Qi Petr Slovak Yulan He Zheng Yuan LRM HILM 195 5 0 18 Apr 2024
FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document Joonho Yang Seunghyun Yoon Byeongjeong Kim Hwanhee Lee HILM 201 11 0 17 Apr 2024
MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents Liyan Tang Philippe Laban Greg Durrett HILM SyDa 163 145 0 16 Apr 2024
Benchmarking Llama2, Mistral, Gemma and GPT for Factuality, Toxicity, Bias and Propensity for Hallucinations David Nadeau Mike Kroutikov Karen McNeil Simon Baribeau HILM 112 8 0 15 Apr 2024
Mitigating Hallucination in Abstractive Summarization with Domain-Conditional Mutual Information Kyubyung Chae Jaepill Choi Yohan Jo Taesup Kim HILM 138 2 0 15 Apr 2024
WikiSplit++: Easy Data Refinement for Split and Rephrase Hayato Tsukagoshi Tsutomu Hirao Makoto Morishita Katsuki Chousa Ryohei Sasano Koichi Takeda 149 1 0 13 Apr 2024
The Hallucinations Leaderboard -- An Open Effort to Measure Hallucinations in Large Language Models Giwon Hong Aryo Pradipta Gema Rohit Saxena Xiaotang Du Ping Nie ... Laura Perez-Beltrachini Max Ryabinin Xuanli He Clémentine Fourrier Pasquale Minervini LRM HILM 197 18 0 08 Apr 2024
Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators Yann Dubois Balázs Galambosi Abigail Z. Jacobs Tatsunori Hashimoto ALM 286 548 0 06 Apr 2024
Schroedinger's Threshold: When the AUC doesn't predict AccuracyInternational Conference on Language Resources and Evaluation (LREC), 2024 Juri Opitz UQCV 183 1 0 04 Apr 2024
Evaluating Document Simplification: On the Importance of Separately Assessing Simplicity and Meaning Preservation Liam Cripwell Joël Legrand Claire Gardent 120 5 0 04 Apr 2024
ALOHa: A New Measure for Hallucination in Captioning ModelsNorth American Chapter of the Association for Computational Linguistics (NAACL), 2024 Suzanne Petryk David M. Chan Anish Kachinthaya Haodi Zou John F. Canny Joseph E. Gonzalez Trevor Darrell HILM 152 26 0 03 Apr 2024
Multi-Review Fusion-in-Context Aviv Slobodkin Ori Shapira Ran Levy Ido Dagan 595 1 0 22 Mar 2024
SIFiD: Reassess Summary Factual Inconsistency Detection with LLM Jiuding Yang Hui Liu Weidong Guo Zhuwei Rao Yu-Syuan Xu Di Niu HILM 121 1 0 12 Mar 2024
ROUGE-K: Do Your Summaries Have Keywords? Sotaro Takeshita Simone Paolo Ponzetto Kai Eckert 115 2 0 08 Mar 2024
Semi-Supervised Dialogue Abstractive Summarization via High-Quality Pseudolabel Selection Jianfeng He Hang Su Jason (Jinglun) Cai Igor Shalyminov Hwanjun Song Saab Mansour 122 5 0 06 Mar 2024
A Modular Approach for Multimodal Summarization of TV Shows Louis Mahon Mirella Lapata 264 10 0 06 Mar 2024
In Search of Truth: An Interrogation Approach to Hallucination Detection Yakir Yehuda Itzik Malkiel Oren Barkan Jonathan Weill Royi Ronen Noam Koenigstein HILM 93 17 0 05 Mar 2024
A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods Hanlei Jin Yang Zhang Dan Meng Jun Wang Jinghua Tan 360 137 0 05 Mar 2024
FENICE: Factuality Evaluation of summarization based on Natural language Inference and Claim Extraction Alessandro Sciré Karim Ghonim Roberto Navigli HILM 178 17 0 04 Mar 2024
Self-Consistent Decoding for More Factual Open Responses Christopher Malon Xiaodan Zhu HILM 182 3 0 01 Mar 2024
Reducing Hallucinations in Entity Abstract Summarization with Facts-Template Decomposition Fangwei Zhu Peiyi Wang Zhifang Sui HILM 88 3 0 29 Feb 2024
Fine-Grained Natural Language Inference Based Faithfulness Evaluation for Diverse Summarisation Tasks Huajian Zhang Yumo Xu Laura Perez-Beltrachini HILM 129 22 0 27 Feb 2024
Successfully Guiding Humans with Imperfect Instructions by Highlighting Potential Errors and Suggesting Corrections Lingjun Zhao Khanh Nguyen Hal Daumé 159 4 0 26 Feb 2024
Entity-level Factual Adaptiveness of Fine-tuning based Abstractive Summarization Models Jongyoon Song Nohil Park Bongkyu Hwang Jaewoong Yun Seongho Joe Youngjune Gwon Sungroh Yoon KELM HILM 120 2 0 23 Feb 2024
Rethinking Scientific Summarization Evaluation: Grounding Explainable Metrics on Facet-aware Benchmark Preslav Nakov Tairan Wang Qingqing Zhu Taicheng Guo Shen Gao Zhiyong Lu Xin Gao Xiangliang Zhang 281 3 0 22 Feb 2024
SYNFAC-EDIT: Synthetic Imitation Edit Feedback for Factual Alignment in Clinical Summarization Prakamya Mishra Zonghai Yao Parth Vashisht Feiyun Ouyang Beining Wang Vidhi Mody Hong-ye Yu SyDa MedIm 136 9 0 21 Feb 2024
Factual consistency evaluation of summarization in the Era of large language models Zheheng Luo Qianqian Xie Sophia Ananiadou HILM 110 5 0 21 Feb 2024
TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization Liyan Tang Igor Shalyminov Amy Wing-mei Wong Jon Burnsky Jake W. Vincent ... Hang Su Lijia Sun Yi Zhang Saab Mansour Kathleen McKeown HILM 126 68 0 20 Feb 2024
Identifying Factual Inconsistencies in Summaries: Grounding Model Inference via Task Taxonomy Liyan Xu Zhenlin Su Mo Yu Jin Xu Jinho D. Choi Jie Zhou Fei Liu HILM 192 4 0 20 Feb 2024
GenAudit: Fixing Factual Errors in Language Model Outputs with Evidence Kundan Krishna S. Ramprasad Prakhar Gupta Byron C. Wallace Zachary Chase Lipton Jeffrey P. Bigham HILM KELM SyDa 244 14 0 19 Feb 2024
Are LLM-based Evaluators Confusing NLG Quality Criteria? Xinyu Hu Mingqi Gao Sen Hu Yang Zhang Yicheng Chen Teng Xu Xiaojun Wan AAML ELM 208 33 0 19 Feb 2024
Small Models, Big Insights: Leveraging Slim Proxy Models To Decide When and What to Retrieve for LLMs Jiejun Tan Zhicheng Dou Yutao Zhu Peidong Guo Kun Fang Ji-Rong Wen 206 48 0 19 Feb 2024
DELL: Generating Reactions and Explanations for LLM-Based Misinformation Detection Herun Wan Shangbin Feng Zhaoxuan Tan Heng Wang Yulia Tsvetkov Minnan Luo 175 56 0 16 Feb 2024
InstructGraph: Boosting Large Language Models via Graph-centric Instruction Tuning and Preference Alignment Jianing Wang Junda Wu Yupeng Hou Yao Liu Ming Gao Julian McAuley 136 50 0 13 Feb 2024
Plausible Extractive Rationalization through Semi-Supervised Entailment Signal Yeo Wei Jie Frank Xing Xiaoshi Zhong 163 7 0 13 Feb 2024
Merging Facts, Crafting Fallacies: Evaluating the Contradictory Nature of Aggregated Factual Claims in Long-Form Generations Cheng-Han Chiang Hung-yi Lee HILM 227 11 0 08 Feb 2024
Source Identification in Abstractive Summarization Yoshi Suhara Dimitris Alikaniotis 85 2 0 07 Feb 2024
Evaluating the Factuality of Zero-shot Summarizers Across Varied DomainsConference of the European Chapter of the Association for Computational Linguistics (EACL), 2024 S. Ramprasad Kundan Krishna Zachary Chase Lipton Byron C. Wallace HILM 120 6 0 05 Feb 2024
Unified Hallucination Detection for Multimodal Large Language ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 Xiang Chen Chenxi Wang Yida Xue Ningyu Zhang Xiaoyan Yang Qian Li Yue Shen Lei Liang Jinjie Gu Huajun Chen HILM 249 61 0 05 Feb 2024
PRE: A Peer Review Based Large Language Model Evaluator Zhumin Chu Jiaxin Mao Yiteng Tu Haitao Li Yiqun Liu LRM ALM 170 25 0 28 Jan 2024
Are self-explanations from Large Language Models faithful?Annual Meeting of the Association for Computational Linguistics (ACL), 2024 Andreas Madsen Sarath Chandar Siva Reddy LRM 233 54 0 15 Jan 2024
Inroads to a Structured Data Natural Language Bijection and the role of LLM annotation Blake Vente 92 0 0 14 Jan 2024
The Critique of CritiqueAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 Shichao Sun Junlong Li Weizhe Yuan Ruifeng Yuan Wenjie Li Pengfei Liu ELM 142 0 0 09 Jan 2024
Do Androids Know They're Only Dreaming of Electric Sheep?Annual Meeting of the Association for Computational Linguistics (ACL), 2023 Sky CH-Wang Benjamin Van Durme Jason Eisner Chris Kedzie HILM 178 51 0 28 Dec 2023
DelucionQA: Detecting Hallucinations in Domain-specific Question AnsweringConference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Mobashir Sadat Zhengyu Zhou Lukas Lange Jun Araki Arsalan Gundroo Bingqing Wang Rakesh R Menon Md. Rizwan Parvez Zhe Feng HILM 135 50 0 08 Dec 2023
CODEX: A Cluster-Based Method for Explainable Reinforcement Learning Timothy K. Mathes Jessica Inman Andrés Colón Simon Khan OffRL 116 1 0 07 Dec 2023
Building Trustworthy NeuroSymbolic AI Systems: Consistency, Reliability, Explainability, and SafetyThe AI Magazine (AI Mag.), 2023 Manas Gaur Amit P. Sheth 92 22 0 05 Dec 2023