This Prompt is Measuring <MASK>: Evaluating Bias Evaluation in Language
Models

This Prompt is Measuring <MASK>: Evaluating Bias Evaluation in Language Models

22 May 2023

Seraphina Goldfarb-Tarrant

Eddie L. Ungless

Su Lin Blodgett

Papers citing "This Prompt is Measuring <MASK>: Evaluating Bias Evaluation in Language Models"

12 / 12 papers shown

Title
Agree to Disagree? A Meta-Evaluation of LLM Misgendering Arjun Subramonian Vagrant Gautam Preethi Seshadri Dietrich Klakow Kai-Wei Chang Yizhou Sun 22 1 0 23 Apr 2025
Examples as the Prompt: A Scalable Approach for Efficient LLM Adaptation in E-Commerce Jingying Zeng Zhenwei Dai Hui Liu Samarth Varshney Zhiji Liu Chen Luo Zhen Li Qi He X. Tang 38 0 0 14 Mar 2025
Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models Luiza Amador Pozzobon B. Ermiş Patrick Lewis Sara Hooker 15 20 0 11 Oct 2023
Mind vs. Mouth: On Measuring Re-judge Inconsistency of Social Bias in Large Language Models Yachao Zhao Bo Wang Dongming Zhao Kun Huang Yan Wang Ruifang He Yuexian Hou 24 4 0 24 Aug 2023
Evaluating the Social Impact of Generative AI Systems in Systems and Society Irene Solaiman Zeerak Talat William Agnew Lama Ahmad Dylan K. Baker ... Marie-Therese Png Shubham Singh A. Strait Lukas Struppek Arjun Subramonian ELM EGVM 18 101 0 09 Jun 2023
Undesirable Biases in NLP: Addressing Challenges of Measurement Oskar van der Wal Dominik Bachmann Alina Leidinger L. Maanen Willem H. Zuidema K. Schulz 11 6 0 24 Nov 2022
The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarks Nikil Selvam Sunipa Dev Daniel Khashabi Tushar Khot Kai-Wei Chang ALM 9 25 0 18 Oct 2022
Quantifying Social Biases Using Templates is Unreliable P. Seshadri Pouya Pezeshkpour Sameer Singh 42 28 0 09 Oct 2022
Challenges in Measuring Bias via Open-Ended Language Generation Afra Feyza Akyürek Muhammed Yusuf Kocyigit Sejin Paik Derry Wijaya 32 21 0 23 May 2022
Unpacking the Interdependent Systems of Discrimination: Ableist Bias in NLP Systems through an Intersectional Lens Saad Hassan Matt Huenerfauth Cecilia Ovesdotter Alm 36 38 0 01 Oct 2021
Mitigating Language-Dependent Ethnic Bias in BERT Jaimeen Ahn Alice H. Oh 118 90 0 13 Sep 2021
The Woman Worked as a Babysitter: On Biases in Language Generation Emily Sheng Kai-Wei Chang Premkumar Natarajan Nanyun Peng 198 607 0 03 Sep 2019