The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation

4 November 2022

Papers citing "The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation"

9 / 59 papers shown

Title
LaMP: When Large Language Models Meet Personalization Alireza Salemi Sheshera Mysore Michael Bendersky Hamed Zamani RALM 11 202 0 22 Apr 2023
Uncertainty-Aware Natural Language Inference with Stochastic Weight Averaging Aarne Talman H. Çelikkanat Sami Virpioja Markus Heinonen Jörg Tiedemann BDL UQCV 19 7 0 10 Apr 2023
Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement Gavin Abercrombie Verena Rieser Dirk Hovy 39 15 0 25 Jan 2023
Stop Measuring Calibration When Humans Disagree Joris Baan Wilker Aziz Barbara Plank Raquel Fernández 16 53 0 28 Oct 2022
Cascading Biases: Investigating the Effect of Heuristic Annotation Strategies on Data and Models Chaitanya Malaviya Sudeep Bhatia Mark Yatskar 11 4 0 24 Oct 2022
Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreement Elisa Leonardelli Stefano Menini Alessio Palmero Aprosio Marco Guerini Sara Tonelli 42 97 0 28 Sep 2021
AmbiFC: Fact-Checking Ambiguous Claims with Evidence Max Glockner Ieva Staliunaite James Thorne Gisela Vallejo Andreas Vlachos Iryna Gurevych 16 20 0 01 Apr 2021
Calibration of Pre-trained Transformers Shrey Desai Greg Durrett UQLM 237 288 0 17 Mar 2020
NarrativeTime: Dense Temporal Annotation on a Timeline Anna Rogers Marzena Karpinska Ankita Gupta Vladislav Lialin Gregory Smelkov Anna Rumshisky 11 3 0 29 Aug 2019