Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings

1 March 2022

Papers citing "Investigating Selective Prediction Approaches Across Several Tasks in IID, OOD, and Adversarial Settings"

50 / 53 papers shown

Title
I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token Roi Cohen Konstantin Dobler Eden Biran Gerard de Melo 93 3 0 09 Dec 2024
Joint Training for Selective Prediction Zhaohui Li Rebecca J. Passonneau 19 0 0 31 Oct 2024
Efficiently Deploying LLMs with Controlled Risk Michael J. Zellinger Matt Thomson 41 1 0 03 Oct 2024
The Craft of Selective Prediction: Towards Reliable Case Outcome Classification -- An Empirical Study on European Court of Human Rights Cases T. Y. S. S. Santosh Irtiza Chowdhury Shanshan Xu Matthias Grabmair AILaw 30 0 0 27 Sep 2024
The Art of Saying No: Contextual Noncompliance in Language Models Faeze Brahman Sachin Kumar Vidhisha Balachandran Pradeep Dasigi Valentina Pyatkin ... Jack Hessel Yulia Tsvetkov Noah A. Smith Yejin Choi Hannaneh Hajishirzi 75 20 0 02 Jul 2024
Certainly Uncertain: A Benchmark and Metric for Multimodal Epistemic and Aleatoric Awareness Khyathi Raghavi Chandu Linjie Li Anas Awadalla Ximing Lu Jae Sung Park Jack Hessel Lijuan Wang Yejin Choi 47 2 0 02 Jul 2024
MFC-Bench: Benchmarking Multimodal Fact-Checking with Large Vision-Language Models Shengkang Wang Hongzhan Lin Ziyang Luo Zhen Ye Guang Chen Jing Ma 68 3 0 17 Jun 2024
Contextualized Sequence Likelihood: Enhanced Confidence Scores for Natural Language Generation Zhen Lin Shubhendu Trivedi Jimeng Sun 16 1 0 03 Jun 2024
Detecting Multimodal Situations with Insufficient Context and Abstaining from Baseless Predictions Junzhang Liu Zhecan Wang Hammad A. Ayyubi Haoxuan You Chris Thomas Rui Sun Shih-Fu Chang Kai-Wei Chang 45 0 0 18 May 2024
Conformal Alignment: Knowing When to Trust Foundation Models with Guarantees Yu Gui Ying Jin Zhimei Ren MedIm 38 18 0 16 May 2024
Out-of-Distribution Data: An Acquaintance of Adversarial Examples -- A Survey Naveen Karunanayake Ravin Gunawardena Suranga Seneviratne Sanjay Chawla OOD 51 5 0 08 Apr 2024
Selective "Selective Prediction": Reducing Unnecessary Abstention in Vision-Language Reasoning Tejas Srinivasan Jack Hessel Tanmay Gupta Bill Yuchen Lin Yejin Choi Jesse Thomason Khyathi Raghavi Chandu 24 7 0 23 Feb 2024
Distinguishing the Knowable from the Unknowable with Language Models Gustaf Ahdritz Tian Qin Nikhil Vyas Boaz Barak Benjamin L. Edelman 26 18 0 05 Feb 2024
The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness Neeraj Varshney Pavel Dolin Agastya Seth Chitta Baral AAML ELM 20 47 0 30 Dec 2023
Adaptation with Self-Evaluation to Improve Selective Prediction in LLMs Jiefeng Chen Jinsung Yoon Sayna Ebrahimi Sercan Ö. Arik Tomas Pfister Somesh Jha 25 31 0 18 Oct 2023
Do Large Language Models Know about Facts? Xuming Hu Junzhe Chen Xiaochuan Li Yingxin Lai Lijie Wen Philip S. Yu Zhijiang Guo HILM KELM 31 49 0 08 Oct 2023
Outlier Robust Adversarial Training Shu Hu Zhenhuan Yang X. Wang Yiming Ying Siwei Lyu AAML 31 9 0 10 Sep 2023
Can NLP Models Ídentify', 'Distinguish', and 'Justify' Questions that Don't have a Definitive Answer? Ayushi Agarwal Nisarg Patel Neeraj Varshney Mihir Parmar Pavan Mallina Aryan Bhavin Shah Srihari Sangaraju Tirth Patel Nihar Thakkar Chitta Baral ELM 10 3 0 08 Sep 2023
Making Pre-trained Language Models both Task-solvers and Self-calibrators Yangyi Chen Xingyao Wang Heng Ji 18 0 0 21 Jul 2023
A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation Neeraj Varshney Wenlin Yao Hongming Zhang Jianshu Chen Dong Yu HILM 42 155 0 08 Jul 2023
Improving Selective Visual Question Answering by Learning from Your Peers Corentin Dancette Spencer Whitehead Rishabh Maheshwary Ramakrishna Vedantam Stefan Scherer Xinlei Chen Matthieu Cord Marcus Rohrbach AAML OOD 38 16 0 14 Jun 2023
Measuring and Modifying Factual Knowledge in Large Language Models Pouya Pezeshkpour KELM 14 17 0 09 Jun 2023
Uncertainty in Natural Language Processing: Sources, Quantification, and Applications Mengting Hu Zhen Zhang Shiwan Zhao Minlie Huang Bingzhe Wu BDL 31 34 0 05 Jun 2023
Generating with Confidence: Uncertainty Quantification for Black-box Large Language Models Zhen Lin Shubhendu Trivedi Jimeng Sun HILM 21 129 0 30 May 2023
Selectively Answering Ambiguous Questions Jeremy R. Cole Michael J.Q. Zhang D. Gillick Julian Martin Eisenschlos Bhuwan Dhingra Jacob Eisenstein UQLM 26 26 0 24 May 2023
LM vs LM: Detecting Factual Errors via Cross Examination Roi Cohen May Hamri Mor Geva Amir Globerson HILM 32 120 0 22 May 2023
Zero-Shot End-to-End Spoken Language Understanding via Cross-Modal Selective Self-Training Jianfeng He Julian Salazar Kaisheng Yao Haoqi Li Jason (Jinglun) Cai VLM 13 7 0 22 May 2023
Learning to Generalize for Cross-domain QA Yingjie Niu Linyi Yang Ruihai Dong Yue Zhang 21 6 0 14 May 2023
Simple Token-Level Confidence Improves Caption Correctness Suzanne Petryk Spencer Whitehead Joseph E. Gonzalez Trevor Darrell Anna Rohrbach Marcus Rohrbach 31 7 0 11 May 2023
A Unified Evaluation Framework for Novelty Detection and Accommodation in NLP with an Instantiation in Authorship Attribution Neeraj Varshney Himanshu Gupta Eric Robertson Bin Liu Chitta Baral 24 1 0 08 May 2023
A Survey on Out-of-Distribution Detection in NLP Hao Lang Yinhe Zheng Yixuan Li Jian Sun Feiling Huang Yongbin Li 29 20 0 05 May 2023
Post-Abstention: Towards Reliably Re-Attempting the Abstained Instances in QA Neeraj Varshney Chitta Baral 39 13 0 02 May 2023
Did You Mean...? Confidence-based Trade-offs in Semantic Parsing Elias Stengel-Eskin Benjamin Van Durme 21 5 0 29 Mar 2023
Finding Competence Regions in Domain Generalization Jens Müller Stefan T. Radev R. Schmier Felix Dräxler Carsten Rother Ullrich Kothe 19 4 0 17 Mar 2023
Crawling the Internal Knowledge-Base of Language Models Roi Cohen Mor Geva Jonathan Berant Amir Globerson 186 77 0 30 Jan 2023
Contrastive Novelty-Augmented Learning: Anticipating Outliers with Large Language Models Albert Xu Xiang Ren Robin Jia OODD 35 2 0 28 Nov 2022
Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans? Neeraj Varshney Man Luo Chitta Baral RALM 21 11 0 23 Nov 2022
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-distribution Generalization Perspective Linyi Yang Shuibai Zhang Libo Qin Yafu Li Yidong Wang Hanmeng Liu Jindong Wang Xingxu Xie Yue Zhang ELM 44 79 0 15 Nov 2022
Calibrated Interpretation: Confidence Estimation in Semantic Parsing Elias Stengel-Eskin Benjamin Van Durme UQLM 41 24 0 14 Nov 2022
DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering Ella Neeman Roee Aharoni Or Honovich Leshem Choshen Idan Szpektor Omri Abend KELM CML 20 77 0 10 Nov 2022
Discover, Explanation, Improvement: An Automatic Slice Detection Framework for Natural Language Processing Wenyue Hua Lifeng Jin Linfeng Song Haitao Mi Yongfeng Zhang Dong Yu 32 1 0 08 Nov 2022
A Close Look into the Calibration of Pre-trained Language Models Yangyi Chen Lifan Yuan Ganqu Cui Zhiyuan Liu Heng Ji 30 43 0 31 Oct 2022
Hardness of Samples Need to be Quantified for a Reliable Evaluation System: Exploring Potential Opportunities with a New Task Swaroop Mishra Anjana Arunkumar Chris Bryan Chitta Baral 22 1 0 14 Oct 2022
Model Cascading: Towards Jointly Improving Efficiency and Accuracy of NLP Systems Neeraj Varshney Chitta Baral 17 27 0 11 Oct 2022
Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis Yuxin Xiao Paul Pu Liang Umang Bhatt W. Neiswanger Ruslan Salakhutdinov Louis-Philippe Morency 175 86 0 10 Oct 2022
Investigating the Failure Modes of the AUC metric and Exploring Alternatives for Evaluating Systems in Safety Critical Applications Swaroop Mishra Anjana Arunkumar Chitta Baral 17 0 0 10 Oct 2022
Language Models (Mostly) Know What They Know Saurav Kadavath Tom Conerly Amanda Askell T. Henighan Dawn Drain ... Nicholas Joseph Benjamin Mann Sam McCandlish C. Olah Jared Kaplan ELM 47 712 0 11 Jul 2022
Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectly Spencer Whitehead Suzanne Petryk Vedaad Shakib Joseph E. Gonzalez Trevor Darrell Anna Rohrbach Marcus Rohrbach 34 54 0 28 Apr 2022
Reasoning over Public and Private Data in Retrieval-Based Systems Simran Arora Patrick Lewis Angela Fan Jacob Kahn Christopher Ré 23 23 0 14 Mar 2022
Will this Question be Answered? Question Filtering via Answer Model Distillation for Efficient Question Answering Siddhant Garg Alessandro Moschitti 29 26 0 14 Sep 2021