Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models

8 February 2021

Aleksandar Shtedritski

Yuki M. Asano

ArXiv PDF HTML

Papers citing "Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models"

30 / 30 papers shown

Title
Does Training on Synthetic Data Make Models Less Robust? Lingze Zhang Ellie Pavlick SyDa 84 0 0 11 Feb 2025
Fairness through Difference Awareness: Measuring Desired Group Discrimination in LLMs Angelina Wang Michelle Phan Daniel E. Ho Sanmi Koyejo 49 2 0 04 Feb 2025
Surveying Attitudinal Alignment Between Large Language Models Vs. Humans Towards 17 Sustainable Development Goals Qingyang Wu Ying Xu Tingsong Xiao Yunze Xiao Yitong Li ... Yichi Zhang Shanghai Zhong Yuwei Zhang Wei Lu Yifan Yang 73 1 0 17 Jan 2025
Explicit vs. Implicit: Investigating Social Bias in Large Language Models through Self-Reflection Yachao Zhao Bo Wang Yan Wang 48 2 0 04 Jan 2025
Exposing Assumptions in AI Benchmarks through Cognitive Modelling Jonathan H. Rystrøm Kenneth C. Enevoldsen 32 0 0 25 Sep 2024
AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances Dhruv Agarwal Mor Naaman Aditya Vashistha 34 16 0 17 Sep 2024
Are Generative Language Models Multicultural? A Study on Hausa Culture and Emotions using ChatGPT Ibrahim Said Ahmad Shiran Dudy R. Ramachandranpillai Kenneth Church 31 4 0 27 Jun 2024
Synthetic Data Generation for Intersectional Fairness by Leveraging Hierarchical Group Structure Gaurav Maheshwari A. Bellet Pascal Denis Mikaela Keller 46 1 0 23 May 2024
Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPT Shucheng Zhu Weikang Wang Ying Liu 29 5 0 11 May 2024
Laissez-Faire Harms: Algorithmic Biases in Generative Language Models Evan Shieh Faye-Marie Vassel Cassidy R. Sugimoto T. Monroe-White 32 3 0 11 Apr 2024
SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety Paul Röttger Fabio Pernisi Bertie Vidgen Dirk Hovy ELM KELM 58 30 0 08 Apr 2024
Evaluating and Mitigating Discrimination in Language Model Decisions Alex Tamkin Amanda Askell Liane Lovitt Esin Durmus Nicholas Joseph Shauna Kravec Karina Nguyen Jared Kaplan Deep Ganguli 36 66 0 06 Dec 2023
ViPE: Visualise Pretty-much Everything Hassan Shahmohammadi Adhiraj Ghosh Hendrik P. A. Lensch DiffM 23 1 0 16 Oct 2023
The Unequal Opportunities of Large Language Models: Revealing Demographic Bias through Job Recommendations A. Salinas Parth Vipul Shah Yuzhong Huang Robert McCormack Fred Morstatter 26 32 0 03 Aug 2023
VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution S. Hall F. G. Abrantes Hanwen Zhu Grace A. Sodunke Aleksandar Shtedritski Hannah Rose Kirk CoGe 13 39 0 21 Jun 2023
ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languages Sourojit Ghosh Aylin Caliskan 33 69 0 17 May 2023
Should ChatGPT be Biased? Challenges and Risks of Bias in Large Language Models Emilio Ferrara SILM 19 247 0 07 Apr 2023
Assessing Language Model Deployment with Risk Cards Leon Derczynski Hannah Rose Kirk Vidhisha Balachandran Sachin Kumar Yulia Tsvetkov M. Leiser Saif Mohammad 20 42 0 31 Mar 2023
Koala: An Index for Quantifying Overlaps with Pre-training Corpora Thuy-Trang Vu Xuanli He Gholamreza Haffari Ehsan Shareghi CLL 19 12 0 26 Mar 2023
Auditing large language models: a three-layered approach Jakob Mokander Jonas Schuett Hannah Rose Kirk Luciano Floridi AILaw MLAU 37 194 0 16 Feb 2023
Bringing the State-of-the-Art to Customers: A Neural Agent Assistant Framework for Customer Service Support Stephen Obadinma Faiza Khan Khattak Shi Wang Tania Sidhom Elaine Lau ... Jaswinder Narain D. Pandya Xiao-Dan Zhu Frank Rudzicz Elham Dolatabadi AI4TS 24 3 0 07 Feb 2023
The Infinite Index: Information Retrieval on Generative Text-To-Image Models Niklas Deckers Maik Frobe Johannes Kiesel G. Pandolfo Christopher Schröder Benno Stein Martin Potthast DiffM 34 16 0 14 Dec 2022
Validating Large Language Models with ReLM Michael Kuchnik Virginia Smith George Amvrosiadis 21 27 0 21 Nov 2022
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks Nikil Selvam Sunipa Dev Daniel Khashabi Tushar Khot Kai-Wei Chang ALM 13 25 0 18 Oct 2022
The Lifecycle of "Facts": A Survey of Social Bias in Knowledge Graphs Angelie Kraft Ricardo Usbeck KELM 20 9 0 07 Oct 2022
Individual Fairness Guarantees for Neural Networks Elias Benussi A. Patané Matthew Wicker Luca Laurenti Marta Kwiatkowska University of Oxford 20 21 0 11 May 2022
Handling and Presenting Harmful Text in NLP Research Hannah Rose Kirk Abeba Birhane Bertie Vidgen Leon Derczynski 13 47 0 29 Apr 2022
A Survey on Gender Bias in Natural Language Processing Karolina Stañczak Isabelle Augenstein 28 109 0 28 Dec 2021
The Woman Worked as a Babysitter: On Biases in Language Generation Emily Sheng Kai-Wei Chang Premkumar Natarajan Nanyun Peng 208 616 0 03 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 297 6,950 0 20 Apr 2018