Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification

11 March 2019

Daniel Borkan

Lucas Dixon

Jeffrey Scott Sorensen

Nithum Thain

Lucy Vasserman

ArXiv PDF HTML

Papers citing "Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification"

25 / 125 papers shown

Title
Mandoline: Model Evaluation under Distribution Shift Mayee F. Chen Karan Goel N. Sohoni Fait Poms Kayvon Fatahalian Christopher Ré 41 69 0 01 Jul 2021
Quantifying Social Biases in NLP: A Generalization and Empirical Comparison of Extrinsic Fairness Metrics Paula Czarnowska Yogarshi Vyas Kashif Shah 21 104 0 28 Jun 2021
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization Yi Tay Vinh Q. Tran Sebastian Ruder Jai Gupta Hyung Won Chung Dara Bahri Zhen Qin Simon Baumgartner Cong Yu Donald Metzler 51 153 0 23 Jun 2021
DORO: Distributional and Outlier Robust Optimization Runtian Zhai Chen Dan J. Zico Kolter Pradeep Ravikumar 33 60 0 11 Jun 2021
Leveraging Sparse Linear Layers for Debuggable Deep Networks Eric Wong Shibani Santurkar A. Madry FAtt 22 88 0 11 May 2021
UTNLP at SemEval-2021 Task 5: A Comparative Analysis of Toxic Span Detection using Attention-based, Named Entity Recognition, and Ensemble Models Alireza Salemi Nazanin Sabri Emad Kebriaei B. Bahrak A. Shakery 14 3 0 10 Apr 2021
IITK@Detox at SemEval-2021 Task 5: Semi-Supervised Learning and Dice Loss for Toxic Spans Detection Archit Bansal Abhay Kaushik Ashutosh Modi 21 3 0 04 Apr 2021
Generalizing to Unseen Domains: A Survey on Domain Generalization Jindong Wang Cuiling Lan Chang-Shu Liu Yidong Ouyang Tao Qin Wang Lu Yiqiang Chen Wenjun Zeng Philip S. Yu OOD 59 1,179 0 02 Mar 2021
ToxCCIn: Toxic Content Classification with Interpretability Tong Xiang Sean MacAvaney Eugene Yang Nazli Goharian 82 15 0 01 Mar 2021
On Calibration and Out-of-domain Generalization Yoav Wald Amir Feder D. Greenfeld Uri Shalit OODD 30 153 0 20 Feb 2021
MUDES: Multilingual Detection of Offensive Spans Tharindu Ranasinghe Marcos Zampieri 27 41 0 18 Feb 2021
HateXplain: A Benchmark Dataset for Explainable Hate Speech Detection Binny Mathew Punyajoy Saha Seid Muhie Yimam Chris Biemann Pawan Goyal Animesh Mukherjee 49 551 0 18 Dec 2020
WILDS: A Benchmark of in-the-Wild Distribution Shifts Pang Wei Koh Shiori Sagawa Henrik Marklund Sang Michael Xie Marvin Zhang ... A. Kundaje Emma Pierson Sergey Levine Chelsea Finn Percy Liang OOD 106 1,386 0 14 Dec 2020
Towards Ethics by Design in Online Abusive Content Detection S. Kiritchenko I. Nejadgholi 29 13 0 28 Oct 2020
Selective Classification Can Magnify Disparities Across Groups Erik Jones Shiori Sagawa Pang Wei Koh Ananya Kumar Percy Liang 39 46 0 27 Oct 2020
GeDi: Generative Discriminator Guided Sequence Generation Ben Krause Akhilesh Deepak Gotmare Bryan McCann N. Keskar Chenyu You R. Socher Nazneen Rajani 56 392 0 14 Sep 2020
Toxicity Detection: Does Context Really Matter? John Pavlopoulos Jeffrey Scott Sorensen Lucas Dixon Nithum Thain Ion Androutsopoulos 18 158 0 01 Jun 2020
Cyberbullying Detection with Fairness Constraints O. Gencoglu 23 48 0 09 May 2020
Social Biases in NLP Models as Barriers for Persons with Disabilities Ben Hutchinson Vinodkumar Prabhakaran Emily L. Denton Kellie Webster Yu Zhong Stephen Denuyl 28 302 0 02 May 2020
Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech Recognition Xiaolei Huang Linzi Xing Franck Dernoncourt Michael J. Paul 16 87 0 24 Feb 2020
Perturbation Sensitivity Analysis to Detect Unintended Model Biases Vinodkumar Prabhakaran Ben Hutchinson Margaret Mitchell 22 117 0 09 Oct 2019
Empirical Analysis of Multi-Task Learning for Reducing Model Bias in Toxic Comment Detection Ameya Vaidya Feng Mai Yue Ning 115 21 0 21 Sep 2019
Pairwise Fairness for Ranking and Regression Harikrishna Narasimhan Andrew Cotter Maya R. Gupta S. Wang 22 112 0 12 Jun 2019
Limitations of Pinned AUC for Measuring Unintended Bias Daniel Borkan Lucas Dixon John Li Jeffrey Scott Sorensen Nithum Thain Lucy Vasserman 25 16 0 05 Mar 2019
Fairness in Recommendation Ranking through Pairwise Comparisons Alex Beutel Jilin Chen Tulsee Doshi Hai Qian Li Wei ... Lukasz Heldt Zhe Zhao Lichan Hong Ed H. Chi Cristos Goodrow FaML 36 373 0 02 Mar 2019