Evaluating model calibration in classification

19 February 2019

Fredrik Lindsten

Papers citing "Evaluating model calibration in classification"

50 / 127 papers shown

Title
What Can We Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifiers Ido Galil Mohammed Dabbah Ran El-Yaniv UQCV 78 31 0 23 Feb 2023
On the Richness of Calibration Benedikt Höltgen Robert C. Williamson 60 10 0 08 Feb 2023
Understanding metric-related pitfalls in image analysis validation Annika Reinke M. Tizabi Michael Baumgartner Matthias Eisenmann Doreen Heckmann-Notzel ... Gaël Varoquaux Manuel Wiesenfarth Ziv R. Yaniv Paul F. Jäger Lena Maier-Hein 96 58 0 03 Feb 2023
Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applications Chengyu Dong OOD UQCV BDL AI4CE 121 0 0 02 Feb 2023
Fast Online Value-Maximizing Prediction Sets with Conformal Cost Control Zhen Lin Shubhendu Trivedi Cao Xiao Jimeng Sung 58 2 0 02 Feb 2023
From Classification Accuracy to Proper Scoring Rules: Elicitability of Probabilistic Top List Predictions Johannes Resin 15 3 0 27 Jan 2023
Calibrating AI Models for Wireless Communications via Conformal Prediction K. Cohen Sangwoo Park Osvaldo Simeone S. Shamai 109 6 0 15 Dec 2022
AdaFocal: Calibration-aware Adaptive Focal Loss Arindam Ghosh Thomas Schaaf Matthew R. Gormley FedML UQCV 97 30 0 21 Nov 2022
A Close Look into the Calibration of Pre-trained Language Models Yangyi Chen Lifan Yuan Ganqu Cui Zhiyuan Liu Heng Ji 153 52 0 31 Oct 2022
Learning to Defer to Multiple Experts: Consistent Surrogate Losses, Confidence Calibration, and Conformal Ensembles Rajeev Verma Daniel Barrejón Eric Nalisnick UQCV 62 35 0 30 Oct 2022
Stop Measuring Calibration When Humans Disagree Joris Baan Wilker Aziz Barbara Plank Raquel Fernández 90 56 0 28 Oct 2022
Beyond calibration: estimating the grouping loss of modern neural networks Alexandre Perez-Lebel Marine Le Morvan Gaël Varoquaux UQCV 72 27 0 28 Oct 2022
Calibration tests beyond classification David Widmann Fredrik Lindsten Dave Zachariah 85 18 0 21 Oct 2022
A Consistent and Differentiable Lp Canonical Calibration Error Estimator Teodora Popordanoska Raphael Sayer Matthew B. Blaschko UQCV 96 35 0 13 Oct 2022
Class-wise and reduced calibration methods Michael Panchenko Anes Benmerzoug Miguel de Benito Delgado 133 1 0 07 Oct 2022
Variable-Based Calibration for Machine Learning Classifiers Mark Kelly Padhraic Smyth 55 4 0 30 Sep 2022
GRASP: A Goodness-of-Fit Test for Classification Learning Adel Javanmard M. Mehrabi 69 0 0 05 Sep 2022
Calibrated Selective Classification Adam Fisch Tommi Jaakkola Regina Barzilay 85 17 0 25 Aug 2022
Adaptive Temperature Scaling for Robust Calibration of Deep Neural Networks Sérgio A. Balanya Juan Maroñas Daniel Ramos OOD 83 15 0 31 Jul 2022
Calibrate: Interactive Analysis of Probabilistic Model Output Peter Xenopoulos Joao Rulff L. G. Nonato Brian Barr Claudio Silva 53 13 0 27 Jul 2022
Which models are innately best at uncertainty estimation? Ido Galil Mohammed Dabbah Ran El-Yaniv UQCV 79 5 0 05 Jun 2022
What is Your Metric Telling You? Evaluating Classifier Calibration under Context-Specific Definitions of Reliability John Kirchenbauer Jacob Oaks Eric Heim UQCV 106 4 0 23 May 2022
On the Calibration of Probabilistic Classifier Sets Thomas Mortier Viktor Bengs Eyke Hüllermeier Stijn Luca Willem Waegeman UQCV 78 7 0 20 May 2022
Metrics of calibration for probabilistic predictions Imanol Arrieta-Ibarra Paman Gujral Jonathan Tannen M. Tygert Cherie Xu 100 22 0 19 May 2022
Training Uncertainty-Aware Classifiers with Conformalized Deep Learning Bat-Sheva Einbinder Yaniv Romano Matteo Sesia Yanfei Zhou UQCV 146 54 0 12 May 2022
On the Usefulness of the Fit-on-the-Test View on Evaluating Calibration of Classifiers Markus Kängsepp Kaspar Valk Meelis Kull 108 3 0 16 Mar 2022
Better Uncertainty Calibration via Proper Scores for Classification and Beyond Sebastian G. Gruber Florian Buettner UQCV 83 50 0 15 Mar 2022
T-Cal: An optimal test for the calibration of predictive models Donghwan Lee Xinmeng Huang Hamed Hassani Yan Sun 128 22 0 03 Mar 2022
Taking a Step Back with KCal: Multi-Class Kernel-Based Calibration for Deep Neural Networks Zhen Lin Shubhendu Trivedi Jimeng Sun 60 5 0 15 Feb 2022
Conformal Prediction Sets with Limited False Positives Adam Fisch Tal Schuster Tommi Jaakkola Regina Barzilay 85 23 0 15 Feb 2022
Calibrated Learning to Defer with One-vs-All Classifiers Rajeev Verma Eric Nalisnick 73 45 0 08 Feb 2022
Hidden Heterogeneity: When to Choose Similarity-Based Calibration K. Wagstaff Thomas G. Dietterich 69 1 0 03 Feb 2022
Classifier Calibration: A survey on how to assess and improve predicted class probabilities Telmo de Menezes e Silva Filho Hao Song Miquel Perelló Nieto Raúl Santos-Rodríguez Meelis Kull Peter A. Flach 179 84 0 20 Dec 2021
Understanding Square Loss in Training Overparametrized Neural Network Classifiers Tianyang Hu Jun Wang Wei Cao Zhenguo Li UQCV AAML 86 19 0 07 Dec 2021
The Box Size Confidence Bias Harms Your Object Detector Johannes Gilg Torben Teepe Fabian Herzog Gerhard Rigoll ObjD 63 4 0 03 Dec 2021
Uncertainty Calibration for Ensemble-Based Debiasing Methods Ruibin Xiong Yimeng Chen Liang Pang Xueqi Chen Yanyan Lan 42 21 0 07 Nov 2021
Robustness and Reliability When Training With Noisy Labels Amanda Olmin Fredrik Lindsten OOD NoLa 80 14 0 07 Oct 2021
Can We Leverage Predictive Uncertainty to Detect Dataset Shift and Adversarial Examples in Android Malware Detection? Deqiang Li Tian Qiu Shuo Chen Qianmu Li Shouhuai Xu AAML 119 13 0 20 Sep 2021
INVIGORATE: Interactive Visual Grounding and Grasping in Clutter Hanbo Zhang Yunfan Lu Cunjun Yu David Hsu Xuguang Lan Nanning Zheng LM&Ro 98 66 0 25 Aug 2021
Top-label calibration and multiclass-to-binary reductions Chirag Gupta Aaditya Ramdas 105 37 0 18 Jul 2021
A Survey of Uncertainty in Deep Neural Networks J. Gawlikowski Cedrique Rovile Njieutcheu Tassi Mohsin Ali Jongseo Lee Matthias Humt ... R. Roscher Muhammad Shahzad Wen Yang R. Bamler Xiaoxiang Zhu BDL UQCV OOD 242 1,174 0 07 Jul 2021
Assessing Generalization of SGD via Disagreement Yiding Jiang Vaishnavh Nagarajan Christina Baek J. Zico Kolter 105 115 0 25 Jun 2021
Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Error Ondrej Bohdal Yongxin Yang Timothy M. Hospedales UQCV OOD 108 23 0 17 Jun 2021
Revisiting the Calibration of Modern Neural Networks Matthias Minderer Josip Djolonga Rob Romijnders F. Hubis Xiaohua Zhai N. Houlsby Dustin Tran Mario Lucic UQCV 110 368 0 15 Jun 2021
Calibrating sufficiently Dirk Tasche UQCV 32 11 0 15 May 2021
Danish Fungi 2020 -- Not Just Another Image Recognition Dataset Lukás Picek Milan Šulc Jirí Matas J. Heilmann‐Clausen T. Jeppesen T. Læssøe T. Frøslev 99 56 0 18 Mar 2021
Distribution-free uncertainty quantification for classification under label shift Aleksandr Podkopaev Aaditya Ramdas UQCV 70 88 0 04 Mar 2021
Prostate Tissue Grading with Deep Quantum Measurement Ordinal Regression Santiago Toledo-Cortés Diego H. Useche Fabio A. González 64 0 0 04 Mar 2021
Parameterized Temperature Scaling for Boosting the Expressive Power in Post-Hoc Uncertainty Calibration Christian Tomani Daniel Cremers Florian Buettner UQCV 54 36 0 24 Feb 2021
On Calibration and Out-of-domain Generalization Yoav Wald Amir Feder D. Greenfeld Uri Shalit OODD 125 158 0 20 Feb 2021