Evaluating model calibration in classification

19 February 2019

Fredrik Lindsten

Papers citing "Evaluating model calibration in classification"

50 / 127 papers shown

Title
Aligning Evaluation with Clinical Priorities: Calibration, Label Shift, and Error Costs Gerardo Flores Alyssa H. Smith Julia A Fukuyama Ashia C. Wilson 24 0 0 17 Jun 2025
Beyond Segmentation: Confidence-Aware and Debiased Estimation of Ratio-based Biomarkers Jiameng Li Teodora Popordanoska Sebastian G. Gruber F. Maes Matthew B. Blaschko 43 0 0 26 May 2025
Performance Estimation in Binary Classification Using Calibrated Confidence Juhani Kivimäki Jakub Białek W. Kuberski J. Nurminen 97 0 0 08 May 2025
Beyond One-Hot Labels: Semantic Mixing for Model Calibration Haoyang Luo Linwei Tao Minjing Dong Chang Xu 141 0 0 18 Apr 2025
Similarity-Distance-Magnitude Universal Verification Allen Schmaltz UQCV AAML 552 0 0 27 Feb 2025
Optimizing Estimators of Squared Calibration Errors in Classification Sebastian G. Gruber Francis Bach 235 2 0 24 Feb 2025
A calibration test for evaluating set-based epistemic uncertainty representations Mira Jürgens Thomas Mortier Eyke Hüllermeier Viktor Bengs Willem Waegeman 95 1 0 22 Feb 2025
Early Stopping in Contextual Bandits and Inferences Zihan Cui 81 0 0 05 Feb 2025
Rethinking Early Stopping: Refine, Then Calibrate Eugene Berta David Holzmüller Michael I. Jordan Francis Bach 141 1 0 31 Jan 2025
The Curious Case of Arbitrariness in Machine Learning Prakhar Ganesh Afaf Taik G. Farnadi 105 3 0 28 Jan 2025
On Calibration in Multi-Distribution Learning Rajeev Verma Volker Fischer Eric Nalisnick 79 1 0 18 Dec 2024
Labels in Extremes: How Well Calibrated are Extreme Multi-label Classifiers? Nasib Ullah Erik Schultheis Jinbin Zhang Rohit Babbar 45 0 0 06 Nov 2024
Confidence Calibration of Classifiers with Many Classes Adrien LeCoz Stéphane Herbin Faouzi Adjed UQCV 82 1 0 05 Nov 2024
Calibrating Expressions of Certainty Peiqi Wang Barbara D. Lam Yingcheng Liu Ameneh Asgari-Targhi Yikang Shen W. Wells Tina Kapur Polina Golland 118 2 0 06 Oct 2024
ReliOcc: Towards Reliable Semantic Occupancy Prediction via Uncertainty Learning Song Wang Zhongdao Wang Jiawei Yu Wentong Li Bailan Feng Junbo Chen Jianke Zhu UQCV 84 3 0 26 Sep 2024
Mind the Uncertainty in Human Disagreement: Evaluating Discrepancies between Model Predictions and Human Responses in VQA Jian Lan Diego Frassinelli Barbara Plank 47 1 0 17 Sep 2024
Crowd-Calibrator: Can Annotator Disagreement Inform Calibration in Subjective Tasks? Urja Khurana Eric T. Nalisnick Antske Fokkens Swabha Swayamdipta 107 4 0 26 Aug 2024
Evaluating Posterior Probabilities: Decision Theory, Proper Scoring Rules, and Calibration Luciana Ferrer Daniel Ramos UQCV 61 4 0 05 Aug 2024
Superior Scoring Rules for Probabilistic Evaluation of Single-Label Multi-Class Classification Tasks Rouhollah Ahmadian Mehdi Ghatee Johan Wahlström 82 2 0 25 Jul 2024
Robust quantum dots charge autotuning using neural networks uncertainty Victor Yon Bastien Galaup Claude Rohrbacher J. Rivard Clément Godfrin ... Louis Gaudreau Eva Dupont-Ferrier Y. Beilliard R. Melko Dominique Drouin 107 1 0 07 Jun 2024
Reassessing How to Compare and Improve the Calibration of Machine Learning Models M. Chidambaram Rong Ge 133 2 0 06 Jun 2024
Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models Yuchen Hu Chen Chen Chao-Han Huck Yang Chengwei Qin Pin-Yu Chen Chng Eng Siong Chao Zhang VLM 71 4 0 23 May 2024
Calibration of Continual Learning Models Lanpei Li Elia Piccoli Andrea Cossu Davide Bacciu Vincenzo Lomonaco CLL 115 2 0 11 Apr 2024
A Short Survey on Importance Weighting for Machine Learning Masanari Kimura H. Hino 90 8 0 15 Mar 2024
Predict the Next Word: Humans exhibit uncertainty in this task and language models _____ Evgenia Ilia Wilker Aziz 60 2 0 27 Feb 2024
Interpreting Predictive Probabilities: Model Confidence or Human Label Variation? Joris Baan Raquel Fernández Barbara Plank Wilker Aziz 110 11 0 25 Feb 2024
How Flawed Is ECE? An Analysis via Logit Smoothing Muthu Chidambaram Holden Lee Colin McSwiggen Semon Rezchikov UQCV 62 3 0 15 Feb 2024
AI, Meet Human: Learning Paradigms for Hybrid Decision Making Systems Clara Punzi Roberto Pellungrini Mattia Setzu F. Giannotti D. Pedreschi 67 6 0 09 Feb 2024
Automated Clinical Coding for Outpatient Departments Viktor Schlegel Abhinav Ramesh Kashyap Thanh-Tung Nguyen Tsung-Han Yang Vijay Prakash Dwivedi Wei-Hsian Yin Jeng Wei Stefan Winkle 81 1 0 21 Dec 2023
Consistent and Asymptotically Unbiased Estimation of Proper Calibration Errors Teodora Popordanoska Sebastian G. Gruber A. Tiulpin Florian Buettner Matthew B. Blaschko 155 5 0 14 Dec 2023
Estimating calibration error under label shift without labels Teodora Popordanoska Gorjan Radevski Tinne Tuytelaars Matthew B. Blaschko 34 1 0 14 Dec 2023
Beyond Classification: Definition and Density-based Estimation of Calibration in Object Detection Teodora Popordanoska A. Tiulpin Matthew B. Blaschko 113 8 0 11 Dec 2023
Classifier Calibration with ROC-Regularized Isotonic Regression Eugene Berta Francis Bach Michael I. Jordan 35 9 0 21 Nov 2023
Calibration by Distribution Matching: Trainable Kernel Calibration Metrics Charles Marx Sofian Zalouk Stefano Ermon 91 10 0 31 Oct 2023
A Primer on Bayesian Neural Networks: Review and Debates Federico Danieli Konstantinos Pitas M. Vladimirova Vincent Fortuin BDL AAML 103 20 0 28 Sep 2023
Understanding Calibration of Deep Neural Networks for Medical Image Classification A. Sambyal Usma Niyaz N. C. Krishnan Deepti R. Bathula 42 7 0 22 Sep 2023
Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing Jarosław Błasiok Preetum Nakkiran UQCV 110 26 0 21 Sep 2023
A Benchmark Study on Calibration Linwei Tao Younan Zhu Haolan Guo Minjing Dong Chang Xu 93 9 0 23 Aug 2023
Through the Lens of Core Competency: Survey on Evaluation of Large Language Models Ziyu Zhuang Qiguang Chen Longxuan Ma Mingda Li Yi Han Yushan Qian Haopeng Bai Zixian Feng Weinan Zhang Ting Liu ELM 75 13 0 15 Aug 2023
Calibration in Deep Learning: A Survey of the State-of-the-Art Cheng Wang UQCV 118 43 0 02 Aug 2023
Towards Reliable Rare Category Analysis on Graphs via Individual Calibration Longfeng Wu Bowen Lei Dongkuan Xu Dawei Zhou UQCV CML 77 9 0 19 Jul 2023
Set Learning for Accurate and Calibrated Models Lukas Muttenthaler Robert A. Vandermeulen Qiuyi Zhang Thomas Unterthiner Klaus-Robert Muller 67 2 0 05 Jul 2023
TCE: A Test-Based Approach to Measuring Calibration Error Takuo Matsubara Niek Tax Richard Mudd Ido Guy 97 4 0 25 Jun 2023
Beyond Probability Partitions: Calibrating Neural Networks with Semantic Aware Grouping Jia-Qi Yang De-Chuan Zhan Le Gan UQCV 56 5 0 08 Jun 2023
Minimum-Risk Recalibration of Classifiers Zeyu Sun Dogyoon Song Alfred Hero 90 7 0 18 May 2023
Document Understanding Dataset and Evaluation (DUDE) Jordy Van Landeghem Rubèn Pérez Tito Łukasz Borchmann Michal Pietruszka Pawel Józiak ... Bertrand Ackaert Ernest Valveny Matthew Blaschko Sien Moens Tomasz Stanislawek VGen 93 66 0 15 May 2023
Measuring Classification Decision Certainty and Doubt Alexander M. Berenbeim Iain J. Cruickshank Susmit Jha Robert H. Thomson Nathaniel D. Bastian UQCV 23 4 0 25 Mar 2023
Uncertainty Calibration for Counterfactual Propensity Estimation in Recommendation Wenbo Hu Xin Sun Qiang liu Wenbo Hu Shu Wu 126 1 0 23 Mar 2023
Uncertainty quantification in neural network classifiers -- a local linear approach Magnus Malmström Isaac Skog Daniel Axehill Fredrik K. Gustafsson UQCV 60 1 0 10 Mar 2023
Rethinking Confidence Calibration for Failure Prediction Fei Zhu Zhen Cheng Xu-Yao Zhang Cheng-Lin Liu UQCV 91 41 0 06 Mar 2023