Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models

30 May 2023

Jimeng Sun

Papers citing "Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models"

30 / 30 papers shown

Title
Calibrating Uncertainty Quantification of Multi-Modal LLMs using Grounding Trilok Padhi R. Kaur Adam D. Cobb Manoj Acharya Anirban Roy Colin Samplawski Brian Matejek Alexander M. Berenbeim Nathaniel D. Bastian Susmit Jha 20 0 0 30 Apr 2025
Uncertainty Quantification for Language Models: A Suite of Black-Box, White-Box, LLM Judge, and Ensemble Scorers Dylan Bouchard Mohit Singh Chauhan HILM 70 0 0 27 Apr 2025
Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Review Toghrul Abbasli Kentaroh Toyoda Yuan Wang Leon Witt Muhammad Asif Ali Yukai Miao Dan Li Qingsong Wei UQCV 82 0 0 25 Apr 2025
Data-Driven Calibration of Prediction Sets in Large Vision-Language Models Based on Inductive Conformal Prediction Yuanchang Ye Weiyan Wen VLM 56 0 0 24 Apr 2025
TruthPrInt: Mitigating LVLM Object Hallucination Via Latent Truthful-Guided Pre-Intervention Jinhao Duan Fei Kong Hao-Ran Cheng James Diffenderfer B. Kailkhura Lichao Sun Xiaofeng Zhu Xiaoshuang Shi Kaidi Xu 65 0 0 13 Mar 2025
Semantic Volume: Quantifying and Detecting both External and Internal Uncertainty in LLMs Xiaomin Li Zhou Yu Ziji Zhang Yingying Zhuang S. Narayanan Sadagopan Anurag Beniwal HILM 58 0 0 28 Feb 2025
Monte Carlo Temperature: a robust sampling strategy for LLM's uncertainty quantification methods Nicola Cecere Andrea Bacciu Ignacio Fernández Tobías Amin Mantrach 61 1 0 25 Feb 2025
Large Language Model Confidence Estimation via Black-Box Access Tejaswini Pedapati Amit Dhurandhar Soumya Ghosh Soham Dan P. Sattigeri 84 3 0 21 Feb 2025
Cost-Saving LLM Cascades with Early Abstention Michael J. Zellinger Rex Liu Matt Thomson 98 0 0 13 Feb 2025
Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluation Yiming Wang Pei Zhang Baosong Yang Derek F. Wong Rui-cang Wang LRM 40 4 0 17 Oct 2024
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs Ruijia Niu D. Wu Rose Yu Yi-An Ma 18 1 0 09 Oct 2024
Conformal Generative Modeling with Improved Sample Efficiency through Sequential Greedy Filtering K. K. Bernhard Schölkopf Michael Muehlebach 24 0 0 02 Oct 2024
TrustNavGPT: Modeling Uncertainty to Improve Trustworthiness of Audio-Guided LLM-Based Robot Navigation Xingpeng Sun Yiran Zhang Xindi Tang Amrit Singh Bedi Aniket Bera 40 4 0 03 Aug 2024
Cost-Effective Hallucination Detection for LLMs Simon Valentin Jinmiao Fu Gianluca Detommaso Shaoyuan Xu Giovanni Zappella Bryan Wang HILM 33 4 0 31 Jul 2024
Benchmarking Uncertainty Quantification Methods for Large Language Models with LM-Polygraph Roman Vashurin Ekaterina Fadeeva Artem Vazhentsev Akim Tsvigun Daniil Vasilev ... Timothy Baldwin Timothy Baldwin Maxim Panov Artem Shelmanov Artem Shelmanov HILM 61 8 0 21 Jun 2024
CSS: Contrastive Semantic Similarity for Uncertainty Quantification of LLMs Shuang Ao Stefan Rueger Advaith Siddharthan 23 1 0 05 Jun 2024
BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models Yu Feng Ben Zhou Weidong Lin Dan Roth 56 4 0 18 Apr 2024
Confidence Calibration and Rationalization for LLMs via Multi-Agent Deliberation Ruixin Yang Dheeraj Rajagopal S. Hayati Bin Hu Dongyeop Kang LLMAG 30 3 0 14 Apr 2024
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward Xuan Xie Jiayang Song Zhehua Zhou Yuheng Huang Da Song Lei Ma OffRL 35 6 0 12 Apr 2024
Multicalibration for Confidence Scoring in LLMs Gianluca Detommaso Martín Bertrán Riccardo Fogliato Aaron Roth 24 12 0 06 Apr 2024
Methods to Estimate Large Language Model Confidence Maia Kotelanski Robert Gallo Ashwin Nayak Thomas Savage LM&MA 16 6 0 28 Nov 2023
Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QA Neeraj Varshney Chitta Baral 31 13 0 02 May 2023
Out-of-Distribution Detection and Selective Generation for Conditional Language Models Jie Jessie Ren Jiaming Luo Yao-Min Zhao Kundan Krishna Mohammad Saleh Balaji Lakshminarayanan Peter J. Liu OODD 64 92 0 30 Sep 2022
Re-Examining Calibration: The Case of Question Answering Chenglei Si Chen Zhao Sewon Min Jordan L. Boyd-Graber 51 30 0 25 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
DEUP: Direct Epistemic Uncertainty Prediction Salem Lahlou Moksh Jain Hadi Nekoei V. Butoi Paul Bertin Jarrid Rector-Brooks Maksym Korablyov Yoshua Bengio PER UQLM UQCV UD 195 80 0 16 Feb 2021
Reducing conversational agents' overconfidence through linguistic calibration Sabrina J. Mielke Arthur Szlam Emily Dinan Y-Lan Boureau 200 152 0 30 Dec 2020
Calibration of Pre-trained Transformers Shrey Desai Greg Durrett UQLM 234 288 0 17 Mar 2020
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles Balaji Lakshminarayanan Alexander Pritzel Charles Blundell UQCV BDL 268 5,635 0 05 Dec 2016
Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning Y. Gal Zoubin Ghahramani UQCV BDL 247 9,042 0 06 Jun 2015