Evaluating the Factual Consistency of Abstractive Text Summarization

Conference on Empirical Methods in Natural Language Processing (EMNLP), 2019

28 October 2019

Papers citing "Evaluating the Factual Consistency of Abstractive Text Summarization"

50 / 491 papers shown

Title
Hallucination Detection in Large Language Models with Metamorphic Relations Borui Yang Md Afif Al Mamun Jie M. Zhang Gias Uddin HILM 272 14 0 20 Feb 2025
SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text GenerationInternational Conference on Learning Representations (ICLR), 2025 Song Duong Florian Le Bronnec Alexandre Allauzen Vincent Guigue Alberto Lumbreras Laure Soulier Patrick Gallinari HILM 193 2 0 20 Feb 2025
Self-Rationalization in the Wild: A Large Scale Out-of-Distribution Evaluation on NLI-related tasksTransactions of the Association for Computational Linguistics (TACL), 2024 Jing Yang Max Glockner Anderson de Rezende Rocha Iryna Gurevych LRM 241 1 0 07 Feb 2025
FactCG: Enhancing Fact Checkers with Graph-Based Multi-Hop DataNorth American Chapter of the Association for Computational Linguistics (NAACL), 2025 Deren Lei Yaxi Li Siyao Li Mengya Hu Rui Xu Ken Archer Mingyu Wang Emily Ching Alex Deng SyDa HILM LRM 186 3 0 28 Jan 2025
Fact-Preserved Personalized News Headline GenerationIndustrial Conference on Data Mining (IDM), 2023 Zhao Yang Junhong Lian Xiang Ao 691 3 0 21 Jan 2025
RLPF: Reinforcement Learning from Prediction Feedback for User Summarization with LLMsAAAI Conference on Artificial Intelligence (AAAI), 2024 Jiaxing Wu Lin Ning Luyang Liu Harrison Lee Neo Wu Chao Wang Sushant Prakash S. O’Banion Bradley Green Jun Xie 259 2 0 20 Jan 2025
PRD: Peer Rank and Discussion Improve Large Language Model based Evaluations Ruosen Li Teerth Patel Xinya Du LLMAG ALM 351 122 0 03 Jan 2025
A review of faithfulness metrics for hallucination assessment in Large Language ModelsIEEE Journal on Selected Topics in Signal Processing (JSTSP), 2024 Ben Malin Tatiana Kalganova Nikoloas Boulgouris HILM 259 5 0 03 Jan 2025
Fine-grained and Explainable Factuality Evaluation for Multimodal Summarization Liqiang Jing Jingxuan Zuo Yue Zhang 158 13 0 31 Dec 2024
A Survey of Calibration Process for Black-Box LLMs Liangru Xie Hui Liu Jingying Zeng Xianfeng Tang Yan Han Chen Luo Jing Huang Zhen Li Suhang Wang Qi He 210 8 0 17 Dec 2024
Learning to Verify Summary Facts with Fine-Grained LLM FeedbackInternational Conference on Computational Linguistics (COLING), 2024 Jihwan Oh J. Choi Nicole Hee-Yeon Kim Taewon Yun Hwanjun Song SyDa ALM HILM 198 1 0 14 Dec 2024
Do Automatic Factuality Metrics Measure Factuality? A Critical Evaluation S. Ramprasad Byron C. Wallace LLMAG HILM 331 6 0 25 Nov 2024
Domain-specific Guided Summarization for Mental Health PostsPacific Asia Conference on Language, Information and Computation (PACLIC), 2024 Lu Qian Yuqi Wang Xiping Hu H. Zhang Wei Wang Ting Yu Anh Nguyen AI4MH 195 3 0 03 Nov 2024
Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance Omer Nahum Nitay Calderon Orgad Keller Idan Szpektor Roi Reichart 135 8 0 24 Oct 2024
VERITAS-NLI : Validation and Extraction of Reliable Information Through Automated Scraping and Natural Language Inference Arjun Shah Hetansh Shah Vedica Bafna Charmi Khandor Sindhu Nair 72 1 0 12 Oct 2024
Measuring the Groundedness of Legal Question-Answering Systems Dietrich Trautmann Natalia Ostapuk Quentin Grail Adrian Alan Pol Guglielmo Bonifazi Shang Gao Martin Gajek HILM AILaw ELM 66 0 0 11 Oct 2024
LLMs Know More Than They Show: On the Intrinsic Representation of LLM HallucinationsInternational Conference on Learning Representations (ICLR), 2024 Hadas Orgad Michael Toker Zorik Gekhman Roi Reichart Idan Szpektor Hadas Kotek Yonatan Belinkov HILM AIFin 390 101 0 03 Oct 2024
A Critical Look at Meta-evaluating Summarisation Evaluation MetricsConference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Xiang Dai Sarvnaz Karimi Biaoyan Fang 133 0 0 29 Sep 2024
Model-based Preference Optimization in Abstractive Summarization without Human FeedbackConference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Jaepill Choi Kyubyung Chae Jiwoo Song Yohan Jo Taesup Kim 224 3 0 27 Sep 2024
Leveraging Long-Context Large Language Models for Multi-Document Understanding and Summarization in Enterprise Applications Aditi Godbole Jabin Geevarghese George Smita Shandilya 155 10 0 27 Sep 2024
Using Similarity to Evaluate Factual Consistency in Summaries Yuxuan Ye Edwin Simpson Raul Santos Rodriguez HILM 104 3 0 23 Sep 2024
Can pre-trained language models generate titles for research papers?International Conference on Asian Digital Libraries (ICADL), 2024 Tohida Rehman Debarshi Kumar Sanyal S. Chattopadhyay 169 3 0 22 Sep 2024
A Dataset for Evaluating LLM-based Evaluation Functions for Research Question Extraction Task Yuya Fujisaki Shiro Takagi Hideki Asoh Wataru Kumagai 111 0 0 10 Sep 2024
Hallucination Detection in LLMs: Fast and Memory-Efficient Finetuned Models Gabriel Y. Arteaga Thomas B. Schon Nicolas Pielawski 190 20 0 04 Sep 2024
Broadening Access to Simulations for End-Users via Large Language Models: Challenges and OpportunitiesOnline World Conference on Soft Computing in Industrial Applications (WSCIA), 2024 Philippe J. Giabbanelli Jose J. Padilla Ameeta Agrawal 141 2 0 03 Sep 2024
Measuring text summarization factuality using atomic facts entailment metrics in the context of retrieval augmented generation N. E. Kriman HILM 145 2 0 27 Aug 2024
SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detection Mengya Hu Rui Xu Deren Lei Yaxi Li Mingyu Wang Emily Ching Eslam Kamal Alex Deng 126 6 0 22 Aug 2024
A Comparative Analysis of Faithfulness Metrics and Humans in Citation Evaluation Weijia Zhang Mohammad Aliannejadi Jiahuan Pei Yifei Yuan Jia-Hong Huang Evangelos Kanoulas HILM 120 5 0 22 Aug 2024
Effective Demonstration Annotation for In-Context Learning via Language Model-Based Determinantal Point ProcessConference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Peng Wang Xiaobin Wang Chao Lou Shengyu Mao Pengjun Xie Yong Jiang 139 7 0 04 Aug 2024
Lynx: An Open Source Hallucination Evaluation Model Selvan Sunitha Ravi B. Mielczarek Anand Kannappan Douwe Kiela Rebecca Qian VLM RALM HILM 199 37 0 11 Jul 2024
STORYSUMM: Evaluating Faithfulness in Story Summarization Melanie Subbiah Faisal Ladhak Akankshya Mishra Griffin Adams Lydia B. Chilton Kathleen McKeown 282 7 0 09 Jul 2024
Towards Enhancing Coherence in Extractive Summarization: Dataset and Experiments with LLMs Mihir Parmar Hanieh Deilamsalehy Franck Dernoncourt Seunghyun Yoon Ryan Rossi Trung Bui 148 5 0 05 Jul 2024
Face4RAG: Factual Consistency Evaluation for Retrieval Augmented Generation in Chinese Yunqi Xu Tianchi Cai Jiyan Jiang Xierui Song 171 8 0 01 Jul 2024
FineSurE: Fine-grained Summarization Evaluation using LLMs Hwanjun Song Hang Su Igor Shalyminov Jason (Jinglun) Cai Saab Mansour HILM 220 65 0 01 Jul 2024
Detection and Measurement of Syntactic Templates in Generated Text Chantal Shaib Yanai Elazar Junyi Jessy Li Byron C. Wallace 184 33 0 28 Jun 2024
Towards Fine-Grained Citation Evaluation in Generated Text: A Comparative Analysis of Faithfulness Metrics Weijia Zhang Mohammad Aliannejadi Yifei Yuan Jiahuan Pei Jia-Hong Huang Evangelos Kanoulas HILM 192 16 0 21 Jun 2024
Factual Dialogue Summarization via Learning from Large Language Models Rongxin Zhu Jey Han Lau Jianzhong Qi HILM 167 5 0 20 Jun 2024
FoRAG: Factuality-optimized Retrieval Augmented Generation for Web-enhanced Long-form Question Answering Tianchi Cai Zhiwen Tan Xierui Song Tao Sun Jiyan Jiang Yunqi Xu Yinger Zhang Jinjie Gu 145 13 0 19 Jun 2024
Learning to Generate Answers with Citations via Factual Consistency ModelsAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 Rami Aly Zhiqiang Tang Samson Tan George Karypis HILM 146 8 0 19 Jun 2024
Detecting Errors through Ensembling Prompts (DEEP): An End-to-End LLM Framework for Detecting Factual ErrorsConference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Alex Chandler Devesh Surve Hui Su HILM UQCV 84 2 0 18 Jun 2024
EMO-KNOW: A Large Scale Dataset on Emotion and Emotion-cause M. Nguyen Yasith Samaradivakara P. Sasikumar Chitralekha Gupta Suranga Nanayakkara 125 1 0 18 Jun 2024
A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models Haopeng Zhang Philip S. Yu Jiawei Zhang 208 72 0 17 Jun 2024
Mitigating Large Language Model Hallucination with Faithful Finetuning Minda Hu Bowei He Yufei Wang Liangyou Li Chen Ma Irwin King HILM 162 19 0 17 Jun 2024
GLIMPSE: Pragmatically Informative Multi-Document Summarization for Scholarly Reviews Maxime Darrin Ines Arous Pablo Piantanida Jackie CK Cheung 111 4 0 11 Jun 2024
Key-Element-Informed sLLM Tuning for Document SummarizationInterspeech (Interspeech), 2024 Sangwon Ryu Heejin Do Yunsu Kim G. G. Lee Jungseul Ok 168 9 0 07 Jun 2024
PatentEval: Understanding Errors in Patent Generation You Zuo Kim Gerdes Eric Villemonte de la Clergerie Benoît Sagot 150 2 0 05 Jun 2024
Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework Xiaoxi Sun Jinpeng Li Yan Zhong Dongyan Zhao Rui Yan LLMAG HILM 152 17 0 05 Jun 2024
TruthEval: A Dataset to Evaluate LLM Truthfulness and Reliability Aisha Khatun Daniel G. Brown HILM 95 6 0 04 Jun 2024
Facilitating Human-LLM Collaboration through Factuality Scores and Source Attributions Hyo Jin Do Rachel Ostrand Justin D. Weisz Casey Dugan P. Sattigeri Dennis L. Wei K. Murugesan Werner Geyer HILM 134 15 0 30 May 2024
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Zorik Gekhman G. Yona Roee Aharoni Matan Eyal Amir Feder Roi Reichart Jonathan Herzig 282 204 0 09 May 2024