Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement

25 January 2023

Papers citing "Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement"

11 / 11 papers shown

Title
Voices in a Crowd: Searching for Clusters of Unique Perspectives Nikolas Vitsakis Amit Parekh Ioannis Konstas 26 0 0 19 Jul 2024
Linguistic Fingerprint in Transformer Models: How Language Variation Influences Parameter Selection in Irony Detection Michele Mastromattei Fabio Massimo Zanzotto 26 0 0 04 Jun 2024
The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human Labels Eve Fleisig Su Lin Blodgett Dan Klein Zeerak Talat 19 13 0 09 May 2024
Malaysian English News Decoded: A Linguistic Resource for Named Entity and Relation Extraction Mohan Raj Chanthran Lay-Ki Soon H. Ong Bhawani Selvaretnam 17 0 0 22 Feb 2024
Conformalized Credal Set Predictors Alireza Javanmardi David Stutz Eyke Hüllermeier 19 10 0 16 Feb 2024
A Taxonomy of Rater Disagreements: Surveying Challenges & Opportunities from the Perspective of Annotating Online Toxicity Wenbo Zhang Hangzhi Guo Ian D Kivlichan Vinodkumar Prabhakaran Davis Yadav Amulya Yadav 17 2 0 07 Nov 2023
Architectural Sweet Spots for Modeling Human Label Variation by the Example of Argument Quality: It's Best to Relate Perspectives! Philipp Heinisch Matthias Orlikowski Julia Romberg Philipp Cimiano 8 2 0 06 Nov 2023
Evaluating AI systems under uncertain ground truth: a case study in dermatology David Stutz A. Cemgil Abhijit Guha Roy Tatiana Matejovicova Melih Barsbey ... Yossi Matias Pushmeet Kohli Yun-hui Liu Arnaud Doucet Alan Karthikesalingam 20 4 0 05 Jul 2023
What about em? How Commercial Machine Translation Fails to Handle (Neo-)Pronouns Anne Lauscher Debora Nozza Archie Crowley E. Miltersen Dirk Hovy 24 21 0 25 May 2023
Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreement Elisa Leonardelli Stefano Menini Alessio Palmero Aprosio Marco Guerini Sara Tonelli 36 97 0 28 Sep 2021
Beyond Fair Pay: Ethical Implications of NLP Crowdsourcing Boaz Shmueli Jan Fell Soumya Ray Lun-Wei Ku 100 71 0 20 Apr 2021