NLPositionality: Characterizing Design Biases of Datasets and Models

Annual Meeting of the Association for Computational Linguistics (ACL), 2023

2 June 2023

Papers citing "NLPositionality: Characterizing Design Biases of Datasets and Models"

50 / 67 papers shown

Title
Culture Cartography: Mapping the Landscape of Cultural Knowledge Caleb Ziems William B. Held Jane A. Yu Amir Goldberg David Grusky Diyi Yang 108 0 0 31 Oct 2025
Somatic in the East, Psychological in the West?: Investigating Clinically-Grounded Cross-Cultural Depression Symptom Expression in LLMs Shintaro Sakai Jisun An Migyeong Kang Haewoon Kwak 57 0 0 05 Aug 2025
Isolating Culture Neurons in Multilingual Large Language Models Danial Namazifard Lukas Galke 115 1 0 04 Aug 2025
NUTMEG: Separating Signal From Noise in Annotator Disagreement Jonathan Ivey Susan Gauch David Jurgens 107 0 0 25 Jul 2025
Objectifying the Subjective: Cognitive Biases in Topic Interpretations Swapnil Hingmire Ze Shi Li Shiyu Zeng Ahmed Musa Awon Luiz Franciscatto Guerra Neil A. Ernst 83 0 0 25 Jul 2025
"Just a strange pic": Evaluating 'safety' in GenAI Image safety annotation tasks from diverse annotators' perspectives Ding Wang Mark Díaz Charvi Rastogi Aida Mostafazadeh Davani Vinodkumar Prabhakaran ... Zoe C. Ashwood M. Paganini Tian Huey Teh Verena Rieser Lora Aroyo 77 0 0 21 Jul 2025
Can Reasoning Help Large Language Models Capture Human Annotator Disagreement? Jingwei Ni Yu Fan Vilém Zouhar Donya Rooein Alexander Miserlis Hoyle Mrinmaya Sachan Markus Leippold Dirk Hovy Elliott Ash 176 1 0 24 Jun 2025
Low-resource domain adaptation while minimizing energy and hardware resource consumption Hernán Maina Nicolás Wolovick Luciana Benotti 144 0 0 10 Jun 2025
We Need to Measure Data Diversity in NLP -- Better and Broader Dong Nguyen Esther Ploeger 223 1 0 26 May 2025
Will AI Tell Lies to Save Sick Children? Litmus-Testing AI Values Prioritization with AIRiskDilemmas Yu Ying Chiu Zhilin Wang Sharan Maiya Yejin Choi Kyle Fish Sydney Levine Evan Hubinger 220 6 0 20 May 2025
CLASH: Evaluating Language Models on Judging High-Stakes Dilemmas from Multiple Perspectives Ayoung Lee Ryan Sungmo Kwon Peter Railton Lu Wang ELM 409 3 0 15 Apr 2025
AI as a deliberative partner fosters intercultural empathy for Americans but fails for Latin American participants Isabel Villanueva Tara Bobinac Binwei Yao Junjie Hu Kaiping Chen 146 3 0 04 Apr 2025
Evaluating how LLM annotations represent diverse views on contentious topics Megan A. Brown Shubham Atreja Libby Hemphill Patrick Y. Wu 872 3 0 29 Mar 2025
AI Biases as Asymmetries: A Review to Guide PracticeFrontiers in Big Data (FBD), 2025 Gabriella Waters Phillip Honenberger 118 2 0 10 Mar 2025
Leveraging Large Language Models and Topic Modeling for Toxicity ClassificationInternational Conference on Computing, Networking and Communications (ICNC), 2024 Haniyeh Ehsani Oskouie Christina Chance Claire Huang Margaret Capetz Elizabeth Eyeson Majid Sarrafzadeh 177 5 0 26 Nov 2024
What Really is Commonsense Knowledge? Quyet V. Do Junze Li Tung-Duong Vuong Zhaowei Wang Yangqiu Song Xiaojuan Ma 188 2 0 06 Nov 2024
Diverging Preferences: When do Annotators Disagree and do Models Know? Michael J.Q. Zhang Zhilin Wang Jena D. Hwang Yi Dong Olivier Delalleau Yejin Choi Eunsol Choi Xiang Ren Valentina Pyatkin 234 26 0 18 Oct 2024
Ethics Whitepaper: Whitepaper on Ethical Research into Large Language Models Eddie L. Ungless Nikolas Vitsakis Zeerak Talat James Garforth Bjorn Ross Arno Onken Atoosa Kasirzadeh Alexandra Birch 238 3 0 17 Oct 2024
Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying QuestionsInternational Conference on Learning Representations (ICLR), 2024 Michael J.Q. Zhang W. Bradley Knox Eunsol Choi 362 31 0 17 Oct 2024
Building Better: Avoiding Pitfalls in Developing Language Resources when Data is ScarceAnnual Meeting of the Association for Computational Linguistics (ACL), 2024 N. Ousidhoum Meriem Beloucif Saif M. Mohammad 524 1 0 16 Oct 2024
Disentangling Hate Across Target Identities Yiping Jin Leo Wanner Aneesh Moideen Koya 149 0 0 14 Oct 2024
Which Demographics do LLMs Default to During Annotation?Annual Meeting of the Association for Computational Linguistics (ACL), 2024 Johannes Schäfer Aidan Combs Christopher Bagdon Jiahui Li Nadine Probol ... Yarik Menchaca Resendiz Aswathy Velutharambath Amelie Wuhrl Sabine Weber Roman Klinger 305 5 0 11 Oct 2024
Intuitions of Compromise: Utilitarianism vs. Contractualism Jared Moore Yejin Choi Sydney Levine 194 1 0 07 Oct 2024
DailyDilemmas: Revealing Value Preferences of LLMs with Quandaries of Daily LifeInternational Conference on Learning Representations (ICLR), 2024 Yu Ying Chiu Liwei Jiang Yejin Choi 272 24 0 03 Oct 2024
CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeaming Yu Ying Chiu Liwei Jiang Bill Yuchen Lin Chan Young Park Shuyue Stella Li ... Mehar Bhatia Maria Antoniak Yulia Tsvetkov Vered Shwartz Yejin Choi ELM ALM 313 22 0 03 Oct 2024
'Simulacrum of Stories': Examining Large Language Models as Qualitative Research ParticipantsInternational Conference on Human Factors in Computing Systems (CHI), 2024 Shivani Kapania William Agnew Motahhare Eslami Hoda Heidari Sarah E Fox 167 18 0 28 Sep 2024
Prompts Are Programs Too! Understanding How Developers Build Software Containing Prompts Jenny T Liang Melissa Lin Nikitha Rao Brad A. Myers 256 24 0 19 Sep 2024
Can Unconfident LLM Annotations Be Used for Confident Conclusions?North American Chapter of the Association for Computational Linguistics (NAACL), 2024 Kristina Gligorić Tijana Zrnic Cinoo Lee Emmanuel J. Candès Dan Jurafsky 348 25 0 27 Aug 2024
Stark: Social Long-Term Multi-Modal Conversation with Persona Commonsense Knowledge Young-Jun Lee Dokyong Lee Junyoung Youn Kyeongjin Oh ByungSoo Ko Jonghwan Hyeon Ho-Jin Choi 257 7 0 04 Jul 2024
Native Design Bias: Studying the Impact of English Nativeness on Language Model Performance Manon Reusens Philipp Borchert Jochen De Weerdt Bart Baesens 360 4 0 25 Jun 2024
Let Guidelines Guide You: A Prescriptive Guideline-Centered Data Annotation Methodology Federico Ruggeri Eleonora Misino Arianna Muti Katerina Korre Paolo Torroni Alberto Barrón-Cedeño 294 3 0 20 Jun 2024
Cultural Conditioning or Placebo? On the Effectiveness of Socio-Demographic Prompting Sagnik Mukherjee Muhammad Farid Adilazuarda Sunayana Sitaram Kalika Bali Alham Fikri Aji Monojit Choudhury 220 19 0 17 Jun 2024
See It from My Perspective: How Language Affects Cultural Bias in Image Understanding Amith Ananthram Elias Stengel-Eskin Carl Vondrick Joey Tianyi Zhou VLM 320 7 0 17 Jun 2024
A Taxonomy of Challenges to Curating Fair Datasets Dora Zhao M. Scheuerman Pooja Chitre Jerone T. A. Andrews Georgia Panagiotidou Shawn Walker Kathleen H. Pine Alice Xiang 249 3 0 10 Jun 2024
Annotation alignment: Comparing LLM and human annotations of conversational safety Rajiv Movva Pang Wei Koh Emma Pierson ALM 287 20 0 10 Jun 2024
Culturally Aware and Adapted NLP: A Taxonomy and a Survey of the State of the Art Chen Cecilia Liu Iryna Gurevych Anna Korhonen 518 14 0 06 Jun 2024
Multilingual Diversity Improves Vision-Language Representations Thao Nguyen Matthew Wallingford Sebastin Santy Wei-Chiu Ma Sewoong Oh Ludwig Schmidt Pang Wei Koh Ranjay Krishna VLM 283 15 0 27 May 2024
CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Models Giada Pistilli Alina Leidinger Yacine Jernite Atoosa Kasirzadeh A. Luccioni Margaret Mitchell 283 6 0 22 May 2024
Facilitating Opinion Diversity through Hybrid NLP ApproachesNorth American Chapter of the Association for Computational Linguistics (NAACL), 2024 Michiel van der Meer 265 3 0 15 May 2024
BLIP: Facilitating the Exploration of Undesirable Consequences of Digital Technologies Rock Yuren Pang Sebastin Santy René Just Katharina Reinecke 176 19 0 10 May 2024
The Perspectivist Paradigm Shift: Assumptions and Challenges of Capturing Human LabelsNorth American Chapter of the Association for Computational Linguistics (NAACL), 2024 Eve Fleisig Su Lin Blodgett Dan Klein Zeerak Talat 205 34 0 09 May 2024
ChatGPT and Vaccine Hesitancy: A Comparison of English, Spanish, and French Responses Using a Validated Scale Saubhagya Joshi Eunbin Ha Yonaira M Rivera Vivek K Singh LM&MA 45 1 0 06 May 2024
Modeling the Sacred: Considerations when Using Religious Texts in Natural Language Processing Ben Hutchinson 253 0 0 23 Apr 2024
Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions Leena Mathur Paul Pu Liang Louis-Philippe Morency LLMAG 241 21 0 17 Apr 2024
CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledge Yu Ying Chiu Amirhossein Ajalloeian Maria Antoniak Chan Young Park Shuyue Stella Li Mehar Bhatia Sahithya Ravi Yulia Tsvetkov Vered Shwartz Yejin Choi 170 35 0 10 Apr 2024
Do Large Language Models Rank Fairly? An Empirical Study on the Fairness of LLMs as RankersNorth American Chapter of the Association for Computational Linguistics (NAACL), 2024 Yuan Wang Xuyang Wu Hsin-Tai Wu Zhiqiang Tao Yi Fang ALM 226 18 0 04 Apr 2024
NLP Systems That Can't Tell Use from Mention Censor Counterspeech, but Teaching the Distinction HelpsNorth American Chapter of the Association for Computational Linguistics (NAACL), 2024 Kristina Gligorić Myra Cheng Lucia Zheng Esin Durmus Dan Jurafsky 167 14 0 02 Apr 2024
Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation Yixin Wan Arjun Subramonian Anaelia Ovalle Zongyu Lin Ashima Suvarna Christina Chance Hritik Bansal Rebecca Pattichis Kai-Wei Chang EGVM 466 62 0 01 Apr 2024
Particip-AI: A Democratic Surveying Framework for Anticipating Future AI Use Cases, Harms and Benefits Kumail Alhamoud Liwei Jiang Jenny T Liang Inyoung Cheong Nicole DeCario Yejin Choi Tadayoshi Kohno Maarten Sap 167 11 0 21 Mar 2024
Towards Measuring and Modeling "Culture" in LLMs: A Survey Muhammad Farid Adilazuarda Sagnik Mukherjee Pradhyumna Lavania Siddhant Singh Alham Fikri Aji Jacki OÑeill Ashutosh Modi Monojit Choudhury 545 118 0 05 Mar 2024