SemEval-2023 Task 11: Learning With Disagreements (LeWiDi)

International Workshop on Semantic Evaluation (SemEval), 2023

28 April 2023

Papers citing "SemEval-2023 Task 11: Learning With Disagreements (LeWiDi)"

34 / 34 papers shown

Title
When Annotators Disagree, Topology Explains: Mapper, a Topological Tool for Exploring Text Embedding Geometry and Ambiguity Nisrine Rair Alban Goupil Valeriu Vrabie Emmanuel Chochoy 77 0 0 20 Oct 2025
Hypernetworks for Perspectivist Adaptation Daniil Ignatev Denis Paperno Massimo Poesio 88 0 0 15 Oct 2025
On the Interplay between Human Label Variation and Model Fairness Kemal Kurniawan Meladel Mistica Timothy Baldwin Jey Han Lau VLM 71 0 0 14 Oct 2025
BoN Appetit Team at LeWiDi-2025: Best-of-N Test-time Scaling Can Not Stomach Annotation Disagreements (Yet) Tomas Ruiz Siyao Peng Barbara Plank Carsten Schwemmer 60 1 0 14 Oct 2025
LeWiDi-2025 at NLPerspectives: The Third Edition of the Learning with Disagreements Shared Task Elisa Leonardelli Silvia Casola Siyao Peng Giulia Rizzi Valerio Basile ... Diego Frassinelli Hyewon Jang Maja Pavlovic Barbara Plank Massimo Poesio 98 5 0 09 Oct 2025
Balancing Quality and Variation: Spam Filtering Distorts Data Label Distributions Eve Fleisig Matthias Orlikowski Philipp Cimiano Dan Klein 76 0 0 10 Sep 2025
The Impact of Annotator Personas on LLM Behavior Across the Perspectivism Spectrum O. O. Sarumi Charles F Welch Daniel Braun Jorg Schlotterer 80 0 0 23 Aug 2025
Can Reasoning Help Large Language Models Capture Human Annotator Disagreement? Jingwei Ni Yu Fan Vilém Zouhar Donya Rooein Alexander Miserlis Hoyle Mrinmaya Sachan Markus Leippold Dirk Hovy Elliott Ash 172 1 0 24 Jun 2025
Threading the Needle: Reweaving Chain-of-Thought Reasoning to Explain Human Label Variation Beiduo Chen Wenshu Fan Anna Korhonen Barbara Plank LRM 481 2 0 29 May 2025
Embracing Diversity: A Multi-Perspective Approach with Soft Labels Benedetta Muscato Praveen Bushipaka Gizem Gezici Lucia Passaro F. Giannotti Tommaso Cucinotta 283 4 0 01 Mar 2025
Towards a Perspectivist Turn in Argument Quality AssessmentNorth American Chapter of the Association for Computational Linguistics (NAACL), 2025 Julia Romberg Maximilian Maurer Henning Wachsmuth Gabriella Lapesa 176 1 0 21 Feb 2025
Is LLM an Overconfident Judge? Unveiling the Capabilities of LLMs in Detecting Offensive Language with Annotation DisagreementAnnual Meeting of the Association for Computational Linguistics (ACL), 2025 Junyu Lu Kai Ma Kaichun Wang Kelaiti Xiao Roy Ka-Wei Lee Bo Xu Liang Yang Hongfei Lin 257 0 0 10 Feb 2025
Training and Evaluating with Human Label Variation: An Empirical Study Kemal Kurniawan Meladel Mistica Timothy Baldwin Jey Han Lau 549 3 0 03 Feb 2025
Aligning NLP Models with Target Population Perspectives using PAIR: Population-Aligned Instance Replication Stephanie Eckman Bolei Ma Christoph Kern Rob Chew Yun Xue Frauke Kreuter 241 0 0 12 Jan 2025
Understanding The Effect Of Temperature On Alignment With Human Opinions Maja Pavlovic Massimo Poesio 241 2 0 15 Nov 2024
Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models Eddie L. Ungless Nikolas Vitsakis Zeerak Talat James Garforth Bjorn Ross Arno Onken Atoosa Kasirzadeh Alexandra Birch 238 3 0 17 Oct 2024
Re-examining Sexism and Misogyny Classification with Annotator AttitudesConference on Empirical Methods in Natural Language Processing (EMNLP), 2024 Aiqi Jiang Nikolas Vitsakis Tanvi Dinkar Gavin Abercrombie Ioannis Konstas 306 11 0 04 Oct 2024
To Aggregate or Not to Aggregate. That is the Question: A Case Study on Annotation Subjectivity in Span PredictionWorkshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA), 2024 Kemal Kurniawan Meladel Mistica Timothy Baldwin Jey Han Lau 200 1 0 05 Aug 2024
Voices in a Crowd: Searching for Clusters of Unique Perspectives Nikolas Vitsakis Amit Parekh Ioannis Konstas 240 6 0 19 Jul 2024
"Seeing the Big through the Small": Can LLMs Approximate Human Judgment Distributions on NLI from a Few Explanations? Beiduo Chen Xinpeng Wang Siyao Peng Robert Litschko Anna Korhonen Barbara Plank 220 16 0 25 Jun 2024
The Effectiveness of LLMs as Annotators: A Comparative Overview and Empirical Analysis of Direct Representation Maja Pavlovic Massimo Poesio 334 30 0 02 May 2024
Improving Health Question Answering with Reliable and Time-Aware Evidence Retrieval Juraj Vladika Florian Matthes RALM 199 13 0 12 Apr 2024
Corpus Considerations for Annotator Modeling and ScalingNorth American Chapter of the Association for Computational Linguistics (NAACL), 2024 O. O. Sarumi Béla Neuendorf Joan Plepi Lucie Flek Jorg Schlotterer Charles F Welch 356 4 0 02 Apr 2024
Text Detoxification as Style Transfer in English and Hindi Sourabrata Mukherjee Akanksha Bansal Atul Kr. Ojha John P. Mccrae Ondrej Dusek 275 15 0 12 Feb 2024
Comparing Knowledge Sources for Open-Domain Scientific Claim Verification Juraj Vladika Florian Matthes 199 7 0 05 Feb 2024
Different Tastes of Entities: Investigating Human Label Variation in Named Entity Annotations Siyao Peng Zihang Sun Sebastian Loftus Barbara Plank 245 4 0 02 Feb 2024
Toxic language detection: a systematic review of Arabic datasets Imene Bensalem Paolo Rosso Hanane Zitouni 328 7 0 12 Dec 2023
Architectural Sweet Spots for Modeling Human Label Variation by the Example of Argument Quality: It's Best to Relate Perspectives!Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023 Philipp Heinisch Matthias Orlikowski Julia Romberg Philipp Cimiano 189 10 0 06 Nov 2023
Evaluating AI systems under uncertain ground truth: a case study in dermatology David Stutz A. Cemgil Abhijit Guha Roy Tatiana Matejovicova Melih Barsbey ... Yossi Matias Pushmeet Kohli Yao Xiao Arnaud Doucet Alan Karthikesalingam 237 3 0 05 Jul 2023
Annotation Imputation to Individualize Predictions: Initial Studies on Distribution Dynamics and Model Predictions London Lowmanstone Ruyuan Wan Risako Owan Jaehyung Kim Luan Tuyen Chau 218 1 0 24 May 2023
iLab at SemEval-2023 Task 11 Le-Wi-Di: Modelling Disagreement or Modelling Perspectives?International Workshop on Semantic Evaluation (SemEval), 2023 Nikolas Vitsakis Amit Parekh Tanvi Dinkar Gavin Abercrombie Ioannis Konstas Verena Rieser 204 12 0 10 May 2023
SafeWebUH at SemEval-2023 Task 11: Learning Annotator Disagreement in Derogatory Text: Comparison of Direct Training vs AggregationInternational Workshop on Semantic Evaluation (SemEval), 2023 Sadat Shahriar Thamar Solorio 153 7 0 01 May 2023
Lon-ea at SemEval-2023 Task 11: A Comparison of Activation Functions for Soft and Hard Label PredictionInternational Workshop on Semantic Evaluation (SemEval), 2023 Peyman Hosseini Mehran Hosseini Sana Al-Azzawi Marcus Liwicki Ignacio Castro Matthew Purver 226 1 0 04 Mar 2023
AmbiFC: Fact-Checking Ambiguous Claims with EvidenceTransactions of the Association for Computational Linguistics (TACL), 2021 Max Glockner Ieva Staliunaite James Thorne Gisela Vallejo Andreas Vlachos Iryna Gurevych 285 39 0 01 Apr 2021