Don't Blame the Annotator: Bias Already Starts in the Annotation
Instructions

Don't Blame the Annotator: Bias Already Starts in the Annotation Instructions

1 May 2022

Papers citing "Don't Blame the Annotator: Bias Already Starts in the Annotation Instructions"

12 / 12 papers shown

Title
Human and LLM Biases in Hate Speech Annotations: A Socio-Demographic Analysis of Annotators and Targets Tommaso Giorgi Lorenzo Cima T. Fagni M. Avvenuti S. Cresci 40 9 0 10 Oct 2024
Context Does Matter: Implications for Crowdsourced Evaluation Labels in Task-Oriented Dialogue Systems Clemencia Siro Mohammad Aliannejadi Maarten de Rijke 27 3 0 15 Apr 2024
Evaluating Webcam-based Gaze Data as an Alternative for Human Rationale Annotations Stephanie Brandl Oliver Eberle Tiago F. R. Ribeiro Anders Søgaard Nora Hollenstein 30 1 0 29 Feb 2024
What's the Meaning of Superhuman Performance in Today's NLU? Simone Tedeschi Johan Bos T. Declerck Jan Hajic Daniel Hershcovich ... Simon Krek Steven Schockaert Rico Sennrich Ekaterina Shutova Roberto Navigli ELM LM&MA VLM ReLM LRM 24 26 0 15 May 2023
LINGO : Visually Debiasing Natural Language Instructions to Support Task Diversity Anjana Arunkumar Shubham Sharma Rakhi Agrawal Sriramakrishnan Chandrasekaran Chris Bryan 18 0 0 12 Apr 2023
Evaluating Human-Language Model Interaction Mina Lee Megha Srivastava Amelia Hardy John Thickstun Esin Durmus ... Hancheng Cao Tony Lee Rishi Bommasani Michael S. Bernstein Percy Liang LM&MA ALM 46 99 0 19 Dec 2022
BioTABQA: Instruction Learning for Biomedical Table Question Answering Man Luo S. Saxena Swaroop Mishra Mihir Parmar Chitta Baral LMTD 149 15 0 06 Jul 2022
Less is More: Summary of Long Instructions is Better for Program Synthesis Kirby Kuznia Swaroop Mishra Mihir Parmar Chitta Baral AIMat 14 22 0 16 Mar 2022
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts Stephen H. Bach Victor Sanh Zheng-Xin Yong Albert Webson Colin Raffel ... Khalid Almubarak Xiangru Tang Dragomir R. Radev Mike Tian-Jian Jiang Alexander M. Rush VLM 225 338 0 02 Feb 2022
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies Mor Geva Daniel Khashabi Elad Segal Tushar Khot Dan Roth Jonathan Berant RALM 245 671 0 06 Jan 2021
Are We Modeling the Task or the Annotator? An Investigation of Annotator Bias in Natural Language Understanding Datasets Mor Geva Yoav Goldberg Jonathan Berant 235 319 0 21 Aug 2019
Hypothesis Only Baselines in Natural Language Inference Adam Poliak Jason Naradowsky Aparajita Haldar Rachel Rudinger Benjamin Van Durme 187 576 0 02 May 2018