"What We Can't Measure, We Can't Understand": Challenges to Demographic Data Procurement in the Pursuit of Fairness

30 October 2020

Papers citing ""What We Can't Measure, We Can't Understand": Challenges to Demographic Data Procurement in the Pursuit of Fairness"

50 / 56 papers shown

Title
Talking About the Assumption in the Room Ramaravind Kommiya Mothilal Faisal M. Lalani Syed Ishtiaque Ahmed Shion Guha Sharifa Sultana 56 0 0 20 Feb 2025
Auditing a Dutch Public Sector Risk Profiling Algorithm Using an Unsupervised Bias Detection Tool Floris Holstege Mackenzie Jorgensen Kirtan Padh Jurriaan Parie Joel Persson Krsto Prorokovic Lukas Snoek 102 0 0 03 Feb 2025
Accurate and Data-Efficient Toxicity Prediction when Annotators Disagree Harbani Jaggi Kashyap Murali Eve Fleisig Erdem Bıyık 29 1 0 16 Oct 2024
Minimum Viable Ethics: From Institutionalizing Industry AI Governance to Product Impact Archana Ahlawat Amy Winecoff Jonathan Mayer 51 1 0 11 Sep 2024
Position: Measure Dataset Diversity, Don't Just Claim It Dora Zhao Jerone T. A. Andrews Orestis Papakyriakopoulos Alice Xiang 64 14 0 11 Jul 2024
FairJob: A Real-World Dataset for Fairness in Online Systems Mariia Vladimirova Federico Pavone Eustache Diemert 37 1 0 03 Jul 2024
A Taxonomy of Challenges to Curating Fair Datasets Dora Zhao M. Scheuerman Pooja Chitre Jerone T. A. Andrews Georgia Panagiotidou Shawn Walker Kathleen H. Pine Alice Xiang 39 2 0 10 Jun 2024
Addressing Discretization-Induced Bias in Demographic Prediction Evan Dong Aaron Schein Yixin Wang Nikhil Garg 32 3 0 27 May 2024
Interpretability Needs a New Paradigm Andreas Madsen Himabindu Lakkaraju Siva Reddy Sarath Chandar 39 4 0 08 May 2024
Lazy Data Practices Harm Fairness Research Jan Simson Alessandro Fabris Christoph Kern 23 5 0 26 Apr 2024
Fairness Without Harm: An Influence-Guided Active Sampling Approach Jinlong Pang Jialu Wang Zhaowei Zhu Yuanshun Yao Chen Qian Yang Liu TDI 46 2 0 20 Feb 2024
AboutMe: Using Self-Descriptions in Webpages to Document the Effects of English Pretraining Data Filters L. Lucy Suchin Gururangan Luca Soldaini Emma Strubell David Bamman Lauren Klein Jesse Dodge 26 14 0 12 Jan 2024
Modeling subjectivity (by Mimicking Annotator Annotation) in toxic comment identification across diverse communities Senjuti Dutta Sid Mittal Sherol Chen Deepak Ramachandran Ravi Rajakumar Ian D Kivlichan Sunny Mak Alena Butryna Praveen Paritosh University of Tennessee 37 5 0 01 Nov 2023
"One-Size-Fits-All"? Examining Expectations around What Constitute "Fair" or "Good" NLG System Behaviors Li Lucy Su Lin Blodgett Milad Shokouhi Hanna M. Wallach Alexandra Olteanu 31 7 0 23 Oct 2023
Estimating and Implementing Conventional Fairness Metrics With Probabilistic Protected Features Hadi Elzayn Emily Black Patrick Vossler Nathanael Jo Jacob Goldin Daniel E. Ho 18 2 0 02 Oct 2023
Beyond Skin Tone: A Multidimensional Measure of Apparent Skin Color William Thong Przemyslaw K. Joniak Alice Xiang 30 19 0 10 Sep 2023
TIDE: Textual Identity Detection for Evaluating and Augmenting Classification and Language Models Emmanuel Klu Sameer Sethi 22 0 0 07 Sep 2023
When Fair Classification Meets Noisy Protected Attributes Avijit Ghosh Pablo Kvitca Chris L. Wilson FaML 19 6 0 06 Jul 2023
Evaluating the Social Impact of Generative AI Systems in Systems and Society Irene Solaiman Zeerak Talat William Agnew Lama Ahmad Dylan K. Baker ... Marie-Therese Png Shubham Singh A. Strait Lukas Struppek Arjun Subramonian ELM EGVM 31 104 0 09 Jun 2023
Disentangling and Operationalizing AI Fairness at LinkedIn Joaquin Quiñonero Candela Yuwen Wu Brian Hsu Sakshi Jain Jennifer Ramos Jon Adams R. Hallman Kinjal Basu 21 9 0 30 May 2023
A View From Somewhere: Human-Centric Face Representations Jerone T. A. Andrews Przemyslaw K. Joniak Alice Xiang CVBM 24 9 0 30 Mar 2023
Can Workers Meaningfully Consent to Workplace Wellbeing Technologies? Shreyans Chowdhary Anna Kawakami Mary L. Gray J. Suh Alexandra Olteanu Koustuv Saha 36 30 0 13 Mar 2023
Overwriting Pretrained Bias with Finetuning Data Angelina Wang Olga Russakovsky 21 29 0 10 Mar 2023
Multi-dimensional discrimination in Law and Machine Learning -- A comparative overview Arjun Roy J. Horstmann Eirini Ntoutsi FaML 13 20 0 12 Feb 2023
Ethical Considerations for Responsible Data Curation Jerone T. A. Andrews Dora Zhao William Thong Apostolos Modas Orestis Papakyriakopoulos Alice Xiang 17 19 0 07 Feb 2023
An Operational Perspective to Fairness Interventions: Where and How to Intervene Brian Hsu Xiaotong Chen Ying Han Hongseok Namkoong Kinjal Basu 24 1 0 03 Feb 2023
Simplicity Bias Leads to Amplified Performance Disparities Samuel J. Bell Levent Sagun 19 12 0 13 Dec 2022
Fair Ranking with Noisy Protected Attributes Anay Mehrotra Nisheeth K. Vishnoi 23 16 0 30 Nov 2022
A Keyword Based Approach to Understanding the Overpenalization of Marginalized Groups by English Marginal Abuse Models on Twitter Kyra Yee Alice Schoenauer Sebag Olivia Redfield Emily Sheng Matthias Eck Luca Belli 20 2 0 07 Oct 2022
Weak Proxies are Sufficient and Preferable for Fairness with Missing Sensitive Attributes Zhaowei Zhu Yuanshun Yao Jiankai Sun Hanguang Li Y. Liu 26 21 0 06 Oct 2022
Representing Marginalized Populations: Challenges in Anthropographics Priya Dhawka H. He Wesley Willett 11 2 0 06 Oct 2022
From plane crashes to algorithmic harm: applicability of safety engineering frameworks for responsible ML Shalaleh Rismani Renee Shelby A. Smart Edgar W. Jatho Joshua A. Kroll AJung Moon Negar Rostamzadeh 34 36 0 06 Oct 2022
Metadata Archaeology: Unearthing Data Subsets by Leveraging Training Dynamics Shoaib Ahmed Siddiqui Nitarshan Rajkumar Tegan Maharaj David M. Krueger Sara Hooker 37 27 0 20 Sep 2022
FAT Forensics: A Python Toolbox for Implementing and Deploying Fairness, Accountability and Transparency Algorithms in Predictive Systems Kacper Sokol Alexander Hepburn Rafael Poyiadzi M. Clifford Raúl Santos-Rodríguez Peter A. Flach 27 29 0 08 Sep 2022
What-is and How-to for Fairness in Machine Learning: A Survey, Reflection, and Perspective Zeyu Tang Jiji Zhang Kun Zhang FaML 25 26 0 08 Jun 2022
Regulating Facial Processing Technologies: Tensions Between Legal and Technical Considerations in the Application of Illinois BIPA Rui-Jie Yew Alice X. Xiang 11 12 0 15 May 2022
Exploring How Machine Learning Practitioners (Try To) Use Fairness Toolkits Wesley Hanwen Deng Manish Nagireddy M. S. Lee Jatinder Singh Zhiwei Steven Wu Kenneth Holstein Haiyi Zhu 38 88 0 13 May 2022
Evaluation Gaps in Machine Learning Practice Ben Hutchinson Negar Rostamzadeh Christina Greer Katherine A. Heller Vinodkumar Prabhakaran ELM 28 56 0 11 May 2022
Subverting Fair Image Search with Generative Adversarial Perturbations A. Ghosh Matthew Jagielski Chris L. Wilson 17 7 0 05 May 2022
Demographic-Reliant Algorithmic Fairness: Characterizing the Risks of Demographic Data Collection in the Pursuit of Fairness Mckane Andrus Sarah Villeneuve FaML 21 50 0 18 Apr 2022
Model Positionality and Computational Reflexivity: Promoting Reflexivity in Data Science S. Cambo Darren Gergle 15 34 0 08 Mar 2022
Language technology practitioners as language managers: arbitrating data bias and predictive bias in ASR Nina Markl S. McNulty 22 9 0 25 Feb 2022
Fair ranking: a critical review, challenges, and future directions Gourab K. Patro Lorenzo Porcaro Laura Mitchell Qiuyue Zhang Meike Zehlike Nikhil Garg 18 51 0 29 Jan 2022
Assessing the Fairness of AI Systems: AI Practitioners' Processes, Challenges, and Needs for Support Michael A. Madaio Lisa Egede Hariharan Subramonyam Jennifer Wortman Vaughan Hanna M. Wallach 25 141 0 10 Dec 2021
Causal-BALD: Deep Bayesian Active Learning of Outcomes to Infer Treatment-Effects from Observational Data Andrew Jesson P. Tigas Joost R. van Amersfoort Andreas Kirsch Uri Shalit Y. Gal CML 36 30 0 03 Nov 2021
Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach Alessandro Fabris Andrea Esuli Alejandro Moreo Fabrizio Sebastiani 18 18 0 17 Sep 2021
Challenges in Detoxifying Language Models Johannes Welbl Amelia Glaese J. Uesato Sumanth Dathathri John F. J. Mellor Lisa Anne Hendricks Kirsty Anderson Pushmeet Kohli Ben Coppin Po-Sen Huang LM&MA 242 193 0 15 Sep 2021
Demystifying the Draft EU Artificial Intelligence Act Michael Veale Frederik J. Zuiderveen Borgesius 24 333 0 08 Jul 2021
Targeted Active Learning for Bayesian Decision-Making Louis Filstroff Iiris Sundin P. Mikkola A. Tiulpin Juuso Kylmäoja Samuel Kaski 11 4 0 08 Jun 2021
Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency Kyra Yee U. Tantipongpipat Shubhanshu Mishra 13 46 0 18 May 2021