v1v2 (latest)

Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift

Neural Information Processing Systems (NeurIPS), 2022

27 June 2022

Papers citing "Agreement-on-the-Line: Predicting the Performance of Neural Networks under Distribution Shift"

43 / 43 papers shown

Title
ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction Han Yu Kehan Li Dongbai Li Yue He Xingxuan Zhang Peng Cui OODD 298 0 0 31 Oct 2025
Confidence and Dispersity as Signals: Unsupervised Model Evaluation and Ranking Weijian Deng Weijie Tu Ibrahim Radwan Mohammad Abu Alsheikh Stephen Gould Liang Zheng 104 0 0 03 Oct 2025
ALSA: Anchors in Logit Space for Out-of-Distribution Accuracy Estimation Chenzhi Liu Mahsa Baktashmotlagh Yanran Tang Zi Huang Ruihong Qiu 80 0 0 27 Aug 2025
The Ramon Llull's Thinking Machine for Automated Ideation Xinran Zhao Boyuan Zheng Chenglei Si Haofei Yu Ken Liu ... Ruochen Li Tong Chen Xiang Li Yiming Zhang Tongshuang Wu LLMAG LRM 192 0 0 26 Aug 2025
Automated Model Evaluation for Object Detection via Prediction Consistency and Reliability Seungju Yoo Hyuk Kwon Joong-Won Hwang Kibok Lee 148 0 0 16 Aug 2025
Label-free estimation of clinically relevant performance metrics under distribution shifts Tim Flühmann Alceu Bissoto Trung-Dung Hoang Lisa M. Koch OOD 69 0 0 30 Jul 2025
Monitoring Risks in Test-Time Adaptation Mona Schirmer Metod Jazbec C. A. Naesseth Eric T. Nalisnick TTA 436 2 0 11 Jul 2025
On Continuous Monitoring of Risk Violations under Unknown ShiftConference on Uncertainty in Artificial Intelligence (UAI), 2025 Alexander Timans Rajeev Verma Eric T. Nalisnick C. A. Naesseth 135 2 0 19 Jun 2025
ODD: Overlap-aware Estimation of Model Performance under Distribution ShiftConference on Uncertainty in Artificial Intelligence (UAI), 2025 Aayush Mishra Anqi Liu 132 1 0 17 Jun 2025
Suitability Filter: A Statistical Framework for Classifier Evaluation in Real-World Deployment Settings Angéline Pouget Mohammad Yaghini Stephan Rabanser Nicolas Papernot 154 0 0 28 May 2025
Performance Estimation in Binary Classification Using Calibrated Confidence Juhani Kivimäki Jakub Białek W. Kuberski J. Nurminen 233 4 0 08 May 2025
UNEM: UNrolled Generalized EM for Transductive Few-Shot LearningComputer Vision and Pattern Recognition (CVPR), 2024 Long Zhou Fereshteh Shakeri Aymen Sadraoui Mounir Kaaniche J. Pesquet Ismail Ben Ayed VLM 528 1 0 21 Dec 2024
Toward a Holistic Evaluation of Robustness in CLIP ModelsIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2024 Weijie Tu Weijian Deng Tom Gedeon VLM 281 7 0 02 Oct 2024
Poor-Supervised Evaluation for SuperLLM via Mutual ConsistencyAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 Peiwen Yuan Shaoxiong Feng Yiwei Li Xinglin Wang Boyuan Pan Heda Wang Yao Hu Kan Li 193 1 0 25 Aug 2024
Sharpness-diversity tradeoff: improving flat ensembles with SharpBalance Haiquan Lu Xiaotian Liu Yefan Zhou Qunli Li Kurt Keutzer Michael W. Mahoney Yujun Yan Huanrui Yang Yaoqing Yang 135 2 0 17 Jul 2024
Mission Critical -- Satellite Data is a Distinct Modality in Machine Learning Esther Rolf Konstantin Klemmer Caleb Robinson Hannah Kerner 176 67 0 02 Feb 2024
AutoFT: Learning an Objective for Robust Fine-Tuning Caroline Choi Yoonho Lee Annie S. Chen Allan Zhou Aditi Raghunathan Chelsea Finn OOD 269 1 0 18 Jan 2024
Estimating Model Performance Under Covariate Shift Without Labels Jakub Bialek W. Kuberski W. Kuberski Nikolaos Perrakis 244 5 0 16 Jan 2024
Beyond Top-Class Agreement: Using Divergences to Forecast Performance under Distribution Shift Mona Schirmer Dan Zhang Eric T. Nalisnick 113 0 0 13 Dec 2023
On the Out of Distribution Robustness of Foundation Models in Medical Image Segmentation D. M. Nguyen Tan Ngoc Pham Nghiem Tuong Diep Nghi Quoc Phan Quang Pham ... Ngan Hoang Le Nhat Ho Pengtao Xie Daniel Sonntag Mathias Niepert VLM UQCV OOD 151 8 0 18 Nov 2023
It's an Alignment, Not a Trade-off: Revisiting Bias and Variance in Deep Models Lin Chen Michal Lukasik Wittawat Jitkrittum Chong You Sanjiv Kumar 278 1 0 13 Oct 2023
Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz RegularizationNeural Information Processing Systems (NeurIPS), 2023 Mahyar Fazlyab Taha Entesari Aniket Roy Ramalingam Chellappa AAML 548 19 0 29 Sep 2023
Towards Last-layer Retraining for Group Robustness with Fewer AnnotationsNeural Information Processing Systems (NeurIPS), 2023 Tyler LaBonte Vidya Muthukumar Abhishek Kumar 337 54 0 15 Sep 2023
Learning Diverse Features in Vision Transformers for Improved Generalization A. Nicolicioiu Andrei Liviu Nicolicioiu B. Alexe Damien Teney 250 4 0 30 Aug 2023
Distance Matters For Improving Performance Estimation Under Covariate Shift Mélanie Roschewitz Ben Glocker 166 1 0 14 Aug 2023
Does Progress On Object Recognition Benchmarks Improve Real-World Generalization? Megan Richards Polina Kirichenko Diane Bouchacourt Mark Ibrahim VLM 218 14 0 24 Jul 2023
Confidence-Based Model Selection: When to Take Shortcuts for Subpopulation Shifts Annie S. Chen Yoonho Lee Amrith Rajagopal Setlur Sergey Levine Chelsea Finn OOD 211 5 0 19 Jun 2023
(Almost) Provable Error Bounds Under Distribution Shift via Disagreement DiscrepancyNeural Information Processing Systems (NeurIPS), 2023 Elan Rosenfeld Saurabh Garg UQCV 142 12 0 01 Jun 2023
Characterizing Out-of-Distribution Error via Optimal TransportNeural Information Processing Systems (NeurIPS), 2023 Yuzhe Lu Yilong Qin Runtian Zhai Andrew Shen Ketong Chen Zhenlin Wang Soheil Kolouri Simon Stepputtis Joseph Campbell Katia Sycara OODD 177 20 0 25 May 2023
DC4L: Distribution Shift Recovery via Data-Driven Control for Deep Learning ModelsConference on Learning for Dynamics & Control (L4DC), 2023 Vivian Lin Kuk Jin Jang Souradeep Dutta Michele Caprio O. Sokolsky Insup Lee OOD 264 8 0 20 Feb 2023
Predicting Out-of-Distribution Error with Confidence Optimal Transport Yuzhe Lu Zhenlin Wang Runtian Zhai Soheil Kolouri Joseph Campbell Katia Sycara OODD 138 14 0 10 Feb 2023
RLSbench: Domain Adaptation Under Relaxed Label ShiftInternational Conference on Machine Learning (ICML), 2023 Saurabh Garg Nick Erickson James Sharpnack Alexander J. Smola Sivaraman Balakrishnan Zachary Chase Lipton VLM 314 44 0 06 Feb 2023
Effective Robustness against Natural Distribution Shifts for Models with Different Training DataNeural Information Processing Systems (NeurIPS), 2023 Zhouxing Shi Nicholas Carlini Ananth Balashankar Ludwig Schmidt Cho-Jui Hsieh Alex Beutel Yao Qin OOD 199 12 0 02 Feb 2023
Confidence and Dispersity Speak: Characterising Prediction Matrix for Unsupervised Accuracy EstimationInternational Conference on Machine Learning (ICML), 2023 Weijian Deng Yumin Suh Stephen Gould Liang Zheng UQCV 281 20 0 02 Feb 2023
Model Monitoring and Robustness of In-Use Machine Learning Models: Quantifying Data Distribution Shifts Using Population Stability Index A. Khademi M. Hopka Devesh Upadhyay OOD 183 4 0 01 Feb 2023
Demystifying Disagreement-on-the-Line in High DimensionsInternational Conference on Machine Learning (ICML), 2023 Dong-Hwan Lee Behrad Moniri Xinmeng Huang Guang Cheng Hamed Hassani 219 12 0 31 Jan 2023
Empirical Study on Optimizer Selection for Out-of-Distribution Generalization Hiroki Naganuma Kartik Ahuja S. Takagi Tetsuya Motokawa Rio Yokota Kohta Ishikawa I. Sato Ioannis Mitliagkas OOD 327 8 0 15 Nov 2022
Beyond Bayes-optimality: meta-learning what you know you don't know Jordi Grau-Moya Grégoire Delétang M. Kunesch Tim Genewein Elliot Catt ... Jane X. Wang Marcus Hutter Christopher Summerfield Shane Legg Pedro A. Ortega 162 2 0 30 Sep 2022
Assaying Out-Of-Distribution Generalization in Transfer LearningNeural Information Processing Systems (NeurIPS), 2022 F. Wenzel Andrea Dittadi Peter V. Gehler Carl-Johann Simon-Gabriel Max Horn ... Chris Russell Thomas Brox Bernt Schiele Bernhard Schölkopf Francesco Locatello OOD OODD AAML 369 83 0 19 Jul 2022
On the Strong Correlation Between Model Invariance and GeneralizationNeural Information Processing Systems (NeurIPS), 2022 Weijian Deng Stephen Gould Liang Zheng OOD 201 24 0 14 Jul 2022
Predicting Out-of-Domain Generalization with Neighborhood Invariance Nathan Ng Neha Hulkund Dong Wang Marzyeh Ghassemi OOD 277 6 0 05 Jul 2022
A Note on "Assessing Generalization of SGD via Disagreement" Andreas Kirsch Y. Gal FedML UQCV 156 18 0 03 Feb 2022
Xception: Deep Learning with Depthwise Separable ConvolutionsComputer Vision and Pattern Recognition (CVPR), 2016 François Chollet MDE BDL PINN 2.5K 16,459 0 07 Oct 2016