When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset

12 June 2023

Papers citing "When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset"

26 / 26 papers shown

Title
Conflicts in Texts: Data, Implications and Challenges Siyi Liu Dan Roth 81 0 0 28 Apr 2025
HALO: Human-Aligned End-to-end Image Retargeting with Layered Transformations Yiran Xu Siqi Xie Zhuofang Li Harris Shadmany Yinxiao Li ... Jesse Berent Ming Yang Irfan Essa Jia-Bin Huang Feng Yang VOS 51 0 0 03 Apr 2025
Evaluating how LLM annotations represent diverse views on contentious topics Megan A. Brown Shubham Atreja Libby Hemphill Patrick Y. Wu 76 0 0 29 Mar 2025
Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptions Matthias Orlikowski Jiaxin Pei Paul Röttger Philipp Cimiano David Jurgens Dirk Hovy 54 1 0 28 Feb 2025
Hope vs. Hate: Understanding User Interactions with LGBTQ+ News Content in Mainstream US News Media through the Lens of Hope Speech Jonathan Pofcher Christopher Homan Randall Sell Ashiqur R. KhudaBukhsh 91 0 0 13 Feb 2025
Personas with Attitudes: Controlling LLMs for Diverse Data Annotation Leon Fröhling Gianluca Demartini Dennis Assenmacher 24 2 0 15 Oct 2024
Which Demographics do LLMs Default to During Annotation? Johannes Schäfer Aidan Combs Christopher Bagdon Jiahui Li Nadine Probol ... Yarik Menchaca Resendiz Aswathy Velutharambath Amelie Wuhrl Sabine Weber Roman Klinger 30 2 0 11 Oct 2024
Re-examining Sexism and Misogyny Classification with Annotator Attitudes Aiqi Jiang Nikolas Vitsakis Tanvi Dinkar Gavin Abercrombie Ioannis Konstas 37 1 0 04 Oct 2024
Prompts Are Programs Too! Understanding How Developers Build Software Containing Prompts Jenny T Liang Melissa Lin Nikitha Rao Brad A. Myers 75 5 0 19 Sep 2024
Rater Cohesion and Quality from a Vicarious Perspective Deepak Pandita Tharindu Cyril Weerasooriya Sujan Dutta Sarah K. K. Luger Tharindu Ranasinghe Ashiqur R. KhudaBukhsh Marcos Zampieri Christopher M. Homan 14 1 0 15 Aug 2024
Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art Chen Cecilia Liu Iryna Gurevych Anna Korhonen 33 5 0 06 Jun 2024
The Unseen Targets of Hate -- A Systematic Review of Hateful Communication Datasets Zehui Yu Indira Sen Dennis Assenmacher Mattia Samory Leon Fröhling Christina Dahn Debora Nozza Claudia Wagner 30 5 0 14 May 2024
Beyond Accuracy: Investigating Error Types in GPT-4 Responses to USMLE Questions Soumyadeep Roy A. Khatua Fatemeh Ghoochani Uwe Hadler Wolfgang Nejdl Niloy Ganguly ELM LM&MA 33 8 0 20 Apr 2024
D3CODE: Disentangling Disagreements in Data across Cultures on Offensiveness Detection and Evaluation Aida Mostafazadeh Davani Mark Díaz Dylan K. Baker Vinodkumar Prabhakaran 19 8 0 16 Apr 2024
Towards Measuring and Modeling "Culture" in LLMs: A Survey Muhammad Farid Adilazuarda Sagnik Mukherjee Pradhyumna Lavania Siddhant Singh Alham Fikri Aji Jacki OÑeill Ashutosh Modi Monojit Choudhury 50 53 0 05 Mar 2024
Quantifying the Persona Effect in LLM Simulations Tiancheng Hu Nigel Collier 22 49 0 16 Feb 2024
Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions Sachin Kumar Chan Young Park Yulia Tsvetkov VLM 12 2 0 13 Nov 2023
GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectives Vinodkumar Prabhakaran Christopher Homan Lora Aroyo Aida Mostafazadeh Davani Alicia Parrish Alex S. Taylor Mark Díaz Ding Wang Greg Serapio-García 29 9 0 09 Nov 2023
Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting Tilman Beck Hendrik Schuff Anne Lauscher Iryna Gurevych 15 32 0 13 Sep 2023
Exploring Cross-Cultural Differences in English Hate Speech Annotations: From Dataset Construction to Analysis Nayeon Lee Chani Jung Jun-Hee Myung Jiho Jin Jose Camacho-Collados Juho Kim Alice H. Oh 24 13 0 31 Aug 2023
BiasX: "Thinking Slow" in Toxic Content Moderation with Explanations of Implied Social Biases Yiming Zhang Sravani Nanduri Liwei Jiang Tongshuang Wu Maarten Sap 20 7 0 23 May 2023
"HOT" ChatGPT: The promise of ChatGPT in detecting and discriminating hateful, offensive, and toxic comments on social media Lingyao Li Lizhou Fan Shubham Atreja Libby Hemphill AI4MH 34 83 0 20 Apr 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communications Jiaxin Pei David Jurgens 23 28 0 30 Sep 2021
Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators' Disagreement Elisa Leonardelli Stefano Menini Alessio Palmero Aprosio Marco Guerini Sara Tonelli 45 97 0 28 Sep 2021
Towards generalisable hate speech detection: a review on obstacles and solutions Wenjie Yin A. Zubiaga 117 164 0 17 Feb 2021