Measuring Reliability of Large Language Models through Semantic Consistency

10 November 2022

Papers citing "Measuring Reliability of Large Language Models through Semantic Consistency"

24 / 24 papers shown

Title
Consistency in Language Models: Current Landscape, Challenges, and Future Directions Jekaterina Novikova Carol Anderson Borhane Blili-Hamelin Subhabrata Majumdar HILM 69 0 0 01 May 2025
On the Robustness of Agentic Function Calling Ella Rabinovich Ateret Anaby-Tavor LLMAG 50 0 0 01 Apr 2025
Self-Reported Confidence of Large Language Models in Gastroenterology: Analysis of Commercial, Open-Source, and Quantized Models Nariman Naderi Seyed Amir Ahmad Safavi-Naini Thomas Savage Zahra Atf Peter Lewis Girish Nadkarni Ali Soroush ELM 94 1 0 24 Mar 2025
Improving Consistency in Large Language Models through Chain of Guidance Harsh Raj Vipul Gupta Domenic Rosati Subhabrata Majumdar LLMAG LRM 54 3 0 21 Feb 2025
Enhancing Semantic Consistency of Large Language Models through Model Editing: An Interpretability-Oriented Approach J. Yang Dapeng Chen Yajing Sun Rongjun Li Zhiyong Feng Wei Peng 46 5 0 19 Jan 2025
Evaluating Consistencies in LLM responses through a Semantic Clustering of Question Answering Yanggyu Lee Jihie Kim 23 1 0 20 Oct 2024
One Language, Many Gaps: Evaluating Dialect Fairness and Robustness of Large Language Models in Reasoning Tasks Fangru Lin Shaoguang Mao Emanuele La Malfa Valentin Hofmann Adrian de Wynter Jing Yao Si-Qing Chen Michael Wooldridge Furu Wei Furu Wei 46 2 0 14 Oct 2024
A Novel Metric for Measuring the Robustness of Large Language Models in Non-adversarial Scenarios Samuel Ackerman Ella Rabinovich E. Farchi Ateret Anaby-Tavor 23 1 0 04 Aug 2024
SimCT: A Simple Consistency Test Protocol in LLMs Development Lifecycle Fufangchen Zhao Guoqiang Jin Rui Zhao Jiangheng Huang Fei Tan 29 1 0 24 Jul 2024
Knowledge Mechanisms in Large Language Models: A Survey and Perspective Meng Wang Yunzhi Yao Ziwen Xu Shuofei Qiao Shumin Deng ... Yong-jia Jiang Pengjun Xie Fei Huang Huajun Chen Ningyu Zhang 47 27 0 22 Jul 2024
Knowledge Conflicts for LLMs: A Survey Rongwu Xu Zehan Qi Zhijiang Guo Cunxiang Wang Hongru Wang Yue Zhang Wei Xu 198 91 0 13 Mar 2024
SaGE: Evaluating Moral Consistency in Large Language Models Vamshi Bonagiri Sreeram Vennam Priyanshul Govil Ponnurangam Kumaraguru Manas Gaur ELM 41 0 0 21 Feb 2024
Predicting Question-Answering Performance of Large Language Models through Semantic Consistency Ella Rabinovich Samuel Ackerman Orna Raz E. Farchi Ateret Anaby-Tavor 211 17 0 02 Nov 2023
LUNA: A Model-Based Universal Analysis Framework for Large Language Models Da Song Xuan Xie Jiayang Song Derui Zhu Yuheng Huang Felix Juefei Xu Lei Ma ALM 22 3 0 22 Oct 2023
Self-Consistency of Large Language Models under Ambiguity Henning Bartsch Ole Jorgensen Domenic Rosati Jason Hoelscher-Obermaier Jacob Pfau HILM 22 4 0 20 Oct 2023
Semantic Consistency for Assuring Reliability of Large Language Models Harsh Raj Vipul Gupta Domenic Rosati S. Majumdar HILM 102 14 0 17 Aug 2023
Two Failures of Self-Consistency in the Multi-Step Reasoning of LLMs Angelica Chen Jason Phang Alicia Parrish Vishakh Padmakumar Chen Zhao Sam Bowman Kyunghyun Cho ReLM LRM 17 28 0 23 May 2023
Statistical Knowledge Assessment for Large Language Models Qingxiu Dong Jingjing Xu Lingpeng Kong Zhifang Sui Lei Li HILM 33 6 0 17 May 2023
Language Model Behavior: A Comprehensive Survey Tyler A. Chang Benjamin Bergen VLM LRM LM&MA 27 102 0 20 Mar 2023
Consistency Analysis of ChatGPT Myeongjun Jang Thomas Lukasiewicz 14 53 0 11 Mar 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
Novelty Controlled Paraphrase Generation with Retrieval Augmented Conditional Prompt Tuning Jishnu Ray Chowdhury Yong Zhuang Shuyi Wang 129 39 0 01 Feb 2022
Measuring and Improving Consistency in Pretrained Language Models Yanai Elazar Nora Kassner Shauli Ravfogel Abhilasha Ravichander Eduard H. Hovy Hinrich Schütze Yoav Goldberg HILM 258 343 0 01 Feb 2021
Language Models as Knowledge Bases? Fabio Petroni Tim Rocktaschel Patrick Lewis A. Bakhtin Yuxiang Wu Alexander H. Miller Sebastian Riedel KELM AI4MH 404 2,576 0 03 Sep 2019