Jury Learning: Integrating Dissenting Voices into Machine Learning Models

7 February 2022

Kayur Patel

Tatsunori Hashimoto

Papers citing "Jury Learning: Integrating Dissenting Voices into Machine Learning Models"

38 / 88 papers shown

Title
Annotation Imputation to Individualize Predictions: Initial Studies on Distribution Dynamics and Model Predictions London Lowmanstone Ruyuan Wan Risako Owan Jaehyung Kim Dongyeop Kang 21 1 0 24 May 2023
You Are What You Annotate: Towards Better Models through Annotator Representations Naihao Deng Xinliang Frederick Zhang Siyang Liu Winston Wu Lu Wang Rada Mihalcea 23 20 0 24 May 2023
What's the Meaning of Superhuman Performance in Today's NLU? Simone Tedeschi Johan Bos T. Declerck Jan Hajic Daniel Hershcovich ... Simon Krek Steven Schockaert Rico Sennrich Ekaterina Shutova Roberto Navigli ELM LM&MA VLM ReLM LRM 34 26 0 15 May 2023
When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks Eve Fleisig Rediet Abebe Dan Klein 28 43 0 11 May 2023
iLab at SemEval-2023 Task 11 Le-Wi-Di: Modelling Disagreement or Modelling Perspectives? Nikolas Vitsakis Amit Parekh Tanvi Dinkar Gavin Abercrombie Ioannis Konstas Verena Rieser 51 10 0 10 May 2023
Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation Patrick Fernandes Aman Madaan Emmy Liu António Farinhas Pedro Henrique Martins ... José G. C. de Souza Shuyan Zhou Tongshuang Wu Graham Neubig André F. T. Martins ALM 117 56 0 01 May 2023
We're Afraid Language Models Aren't Modeling Ambiguity Alisa Liu Zhaofeng Wu Julian Michael Alane Suhr Peter West Alexander Koller Swabha Swayamdipta Noah A. Smith Yejin Choi 63 90 0 27 Apr 2023
LaMP: When Large Language Models Meet Personalization Alireza Salemi Sheshera Mysore Michael Bendersky Hamed Zamani RALM 16 206 0 22 Apr 2023
Learning Personalized Decision Support Policies Umang Bhatt Valerie Chen Katherine M. Collins Parameswaran Kamalaruban Emma Kallina Adrian Weller Ameet Talwalkar OffRL 56 10 0 13 Apr 2023
Whose Opinions Do Language Models Reflect? Shibani Santurkar Esin Durmus Faisal Ladhak Cinoo Lee Percy Liang Tatsunori Hashimoto 21 385 0 30 Mar 2023
Model Sketching: Centering Concepts in Early-Stage Machine Learning Model Design Michelle S. Lam Zixian Ma Anne Li Izequiel Freitas Dakuo Wang James A. Landay Michael S. Bernstein 164 22 0 06 Mar 2023
Rethinking "Risk" in Algorithmic Systems Through A Computational Narrative Analysis of Casenotes in Child-Welfare Devansh Saxena Erin Moon Aryan Chaurasia Yixin Guan Shion Guha 27 26 0 16 Feb 2023
Same Same, But Different: Conditional Multi-Task Learning for Demographic-Specific Toxicity Detection Soumyajit Gupta Sooyong Lee Maria De-Arteaga Matthew Lease 27 13 0 14 Feb 2023
Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making Luke M. Guerdan Amanda Coston Zhiwei Steven Wu Kenneth Holstein CML 27 26 0 13 Feb 2023
Ethical Considerations for Responsible Data Curation Jerone T. A. Andrews Dora Zhao William Thong Apostolos Modas Orestis Papakyriakopoulos Alice Xiang 17 19 0 07 Feb 2023
Lived Experience Matters: Automatic Detection of Stigma on Social Media Toward People Who Use Substances Salvatore Giorgi Douglas Bellew Daniel Roy Sadek Habib G. Sherman Joao Sedoc Chase Smitterberg Amanda Devoto McKenzie Himelein-Wachowiak Brenda L. Curtis 22 3 0 04 Feb 2023
Everyone's Voice Matters: Quantifying Annotation Disagreement Using Demographic Information Ruyuan Wan Jaehyung Kim Dongyeop Kang 9 36 0 12 Jan 2023
Human-Guided Fair Classification for Natural Language Processing Florian E.Dorner Momchil Peychev Nikola Konstantinov Naman Goel Elliott Ash Martin Vechev FaML 19 3 0 20 Dec 2022
SeedBERT: Recovering Annotator Rating Distributions from an Aggregated Label A. Sampath Victoria Lin Louis-Philippe Morency 6 2 0 23 Nov 2022
The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation Barbara Plank 30 97 0 04 Nov 2022
"It's Not Just Hate'': A Multi-Dimensional Perspective on Detecting Harmful Speech Online Federico Bianchi S. A. Hills Patrícia G. C. Rossini Dirk Hovy Rebekah Tromble N. Tintarev 28 14 0 28 Oct 2022
Trustworthy Human Computation: A Survey H. Kashima S. Oyama Hiromi Arai Junichiro Mori 27 0 0 22 Oct 2022
Codes, Patterns and Shapes of Contemporary Online Antisemitism and Conspiracy Narratives -- an Annotation Guide and Labeled German-Language Dataset in the Context of COVID-19 Elisabeth Steffen Helena Mihaljević Milena Pustet Nyco Bischoff María do Mar Castro Varela Yener Bayramoğlu Bahar Oghalai 21 3 0 13 Oct 2022
Understanding Practices, Challenges, and Opportunities for User-Engaged Algorithm Auditing in Industry Practice Wesley Hanwen Deng B. Guo Alicia DeVrio Hong Shen Motahhare Eslami Kenneth Holstein MLAU 19 58 0 07 Oct 2022
Identifying the Context Shift between Test Benchmarks and Production Data Matthew Groh OOD 13 8 0 03 Jul 2022
Eliciting and Learning with Soft Labels from Every Annotator K. M. Collins Umang Bhatt Adrian Weller 19 44 0 02 Jul 2022
ToKen: Task Decomposition and Knowledge Infusion for Few-Shot Hate Speech Detection Badr AlKhamissi Faisal Ladhak Srini Iyer Ves Stoyanov Zornitsa Kozareva Xian Li Pascale Fung Lambert Mathias Asli Celikyilmaz Mona T. Diab 46 18 0 25 May 2022
Perspectives on Incorporating Expert Feedback into Model Updates Valerie Chen Umang Bhatt Hoda Heidari Adrian Weller Ameet Talwalkar 30 11 0 13 May 2022
Sensible AI: Re-imagining Interpretability and Explainability using Sensemaking Theory Harmanpreet Kaur Eytan Adar Eric Gilbert Cliff Lampe 10 59 0 10 May 2022
Challenges and Strategies in Cross-Cultural NLP Daniel Hershcovich Stella Frank Heather Lent Miryam de Lhoneux Mostafa Abdou ... Ruixiang Cui Constanza Fierro Katerina Margatina Phillip Rust Anders Søgaard 43 163 0 18 Mar 2022
Remove, Reduce, Inform: What Actions do People Want Social Media Platforms to Take on Potentially Misleading Content? Shubham Atreja Libby Hemphill Paul Resnick 18 14 0 01 Feb 2022
Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks Paul Röttger Bertie Vidgen Dirk Hovy J. Pierrehumbert 18 11 0 14 Dec 2021
Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection Maarten Sap Swabha Swayamdipta Laura Vianna Xuhui Zhou Yejin Choi Noah A. Smith 43 267 0 15 Nov 2021
Toward a Perspectivist Turn in Ground Truthing for Predictive Computing Valerio Basile F. Cabitza Andrea Campagner Michael Fell 31 151 0 09 Sep 2021
Searching For or Reviewing Evidence Improves Crowdworkers' Misinformation Judgments and Reduces Partisan Bias Paul Resnick Aljoharah Alfayez Jane Im Eric Gilbert 10 9 0 17 Aug 2021
Misinfo Reaction Frames: Reasoning about Readers' Reactions to News Headlines Saadia Gabriel Skyler Hallinan Maarten Sap Pemi Nguyen Franziska Roesner Eunsol Choi Yejin Choi 16 40 0 18 Apr 2021
A Survey on Bias and Fairness in Machine Learning Ninareh Mehrabi Fred Morstatter N. Saxena Kristina Lerman Aram Galstyan SyDa FaML 323 4,212 0 23 Aug 2019
Capturing Ambiguity in Crowdsourcing Frame Disambiguation Anca Dumitrache Lora Aroyo Chris Welty FedML 30 31 0 01 May 2018