Debiasing Vision-Language Models via Biased Prompts

31 January 2023

Ching-Yao Chuang

Varun Jampani

Yuanzhen Li

Antonio Torralba

Stefanie Jegelka

VLM

ArXiv PDF HTML

Papers citing "Debiasing Vision-Language Models via Biased Prompts"

50 / 76 papers shown

Title
Implicit Bias Injection Attacks against Text-to-Image Diffusion Models Huayang Huang Xiangye Jin Jiaxu Miao Yu Wu 29 0 0 02 Apr 2025
Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization Zefeng Zhang Hengzhu Tang Jiawei Sheng Zhenyu Zhang Yiming Ren Zhenyang Li Dawei Yin Duohe Ma Tingwen Liu 45 0 0 23 Mar 2025
Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Models Davide Berasi Matteo Farina Massimiliano Mancini Elisa Ricci Nicola Strisciuglio CoGe 66 0 0 21 Mar 2025
Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness B. Zhu Jiequan Cui H. Zhang Chi Zhang 71 0 0 12 Mar 2025
Debiased Prompt Tuning in Vision-Language Model without Annotations Chaoquan Jiang Yunfan Yang Rui Hu Jitao Sang VLM 57 0 0 11 Mar 2025
ACE: Concept Editing in Diffusion Models without Performance Degradation Ruipeng Wang Junfeng Fang Jiaqi Li Hao Chen Jie Shi K. Wang X. Wang DiffM 51 2 0 11 Mar 2025
Personalize Your LLM: Fake it then Align it Yijing Zhang Dyah Adila Changho Shin Frederic Sala 86 0 0 02 Mar 2025
Is Free Self-Alignment Possible? Dyah Adila Changho Shin Yijing Zhang Frederic Sala MoMe 105 2 0 24 Feb 2025
Black Sheep in the Herd: Playing with Spuriously Correlated Attributes for Vision-Language Recognition Xinyu Tian Shu Zou Zhaoyuan Yang Mengqi He Jing Zhang VLM 43 0 0 19 Feb 2025
Refining Skewed Perceptions in Vision-Language Models through Visual Representations Haocheng Dai Sarang Joshi VLM 55 0 0 03 Jan 2025
Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuning Zhifang Zhang Shuo He Bingquan Shen Lei Feng Lei Feng AAML 38 0 0 29 Dec 2024
Adaptive Concept Bottleneck for Foundation Models Under Distribution Shifts Jihye Choi Jayaram Raghuram Yixuan Li Somesh Jha 95 4 0 18 Dec 2024
DoubleCCA: Improving Foundation Model Group Robustness with Random Sentence Embeddings Hong Liu Yitong Lu 73 0 0 25 Nov 2024
Image Generation Diversity Issues and How to Tame Them Mischa Dombrowski Weitong Zhang Sarah Cechnicka Hadrien Reynaud Bernhard Kainz 67 0 0 25 Nov 2024
Joint Vision-Language Social Bias Removal for CLIP Haoyu Zhang Yangyang Guo Mohan S. Kankanhalli VLM 62 0 0 19 Nov 2024
Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesis Taihang Hu Linxuan Li Joost van de Weijer Hongcheng Gao Fahad Shahbaz Khan Jian Yang Ming-Ming Cheng Kai Wang Yaxing Wang DiffM 43 4 0 11 Nov 2024
FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generation Christopher T. H. Teo Milad Abdollahzadeh Xinda Ma Ngai-man Cheung DiffM 16 1 0 24 Oct 2024
Debiasing Large Vision-Language Models by Ablating Protected Attribute Representations Neale Ratzlaff Matthew Lyle Olson Musashi Hinck Shao-Yen Tseng Vasudev Lal Phillip Howard 27 0 0 17 Oct 2024
Embedding an Ethical Mind: Aligning Text-to-Image Synthesis via Lightweight Value Optimization Xingqi Wang Xiaoyuan Yi Xing Xie Jia Jia 14 1 0 16 Oct 2024
Debiasing Vison-Language Models with Text-Only Training Yunfan Yang Chaoquan Jiang Zhiyu Lin Jinlin Xiao Jiaming Zhang Jitao Sang VLM 23 1 0 12 Oct 2024
A Unified Debiasing Approach for Vision-Language Models across Modalities and Tasks Hoin Jung T. Jang Xiaoqian Wang VLM 21 2 0 10 Oct 2024
Bias Begets Bias: The Impact of Biased Embeddings on Diffusion Models Sahil Kuchlous Marvin Li Jeffrey G. Wang 26 0 0 15 Sep 2024
Evaluating Attribute Comprehension in Large Vision-Language Models Haiwen Zhang Zixi Yang Yuanzhi Liu Xinran Wang Zheqi He Kongming Liang Zhanyu Ma ELM 24 0 0 25 Aug 2024
Prompt and Prejudice Lorenzo Berlincioni Luca Cultrera Federico Becattini Marco Bertini A. Bimbo 33 0 0 07 Aug 2024
Fairness and Bias Mitigation in Computer Vision: A Survey Sepehr Dehdashtian Ruozhen He Yi Li Guha Balakrishnan Nuno Vasconcelos Vicente Ordonez Vishnu Naresh Boddeti 29 4 0 05 Aug 2024
Reproducibility Study of "ITI-GEN: Inclusive Text-to-Image Generation" Daniel Gallo Fernández Ruazvan-Andrei Matisan Alejandro Monroy Muñoz Janusz Partyka 19 0 0 29 Jul 2024
Evaluating Model Bias Requires Characterizing its Mistakes Isabela Albuquerque Jessica Schrouff David Warde-Farley Ali Taylan Cemgil Sven Gowal Olivia Wiles 42 2 0 15 Jul 2024
NODE-Adapter: Neural Ordinary Differential Equations for Better Vision-Language Reasoning Yi Zhang Chun-Wun Cheng Ke Yu Zhihai He Carola-Bibiane Schonlieb Angelica I Aviles-Rivero VLM 39 2 0 11 Jul 2024
Diminishing Stereotype Bias in Image Generation Model using Reinforcemenlent Learning Feedback Xin Chen Virgile Foussereau EGVM 38 0 0 27 Jun 2024
MoESD: Mixture of Experts Stable Diffusion to Mitigate Gender Bias Guorun Wang Lucia Specia DiffM MoE 32 0 0 25 Jun 2024
They're All Doctors: Synthesizing Diverse Counterfactuals to Mitigate Associative Bias Salma Abdel Magid Jui-Hsien Wang Kushal Kafle Hanspeter Pfister 23 1 0 17 Jun 2024
Latent Directions: A Simple Pathway to Bias Mitigation in Generative AI Carolina Lopez Olmos A. Neophytou Sunando Sengupta Dim P. Papadopoulos EGVM 31 2 0 10 Jun 2024
Improving Geo-diversity of Generated Images with Contextualized Vendi Score Guidance Reyhane Askari Hemmat Melissa Hall Alicia Sun Candace Ross M. Drozdzal Adriana Romero Soriano 35 4 0 06 Jun 2024
Embracing Diversity: Interpretable Zero-shot classification beyond one vector per class Mazda Moayeri Michael G. Rabbat Mark Ibrahim Diane Bouchacourt VLM 41 1 0 25 Apr 2024
FairDeDup: Detecting and Mitigating Vision-Language Fairness Disparities in Semantic Dataset Deduplication Eric Slyman Stefan Lee Scott D. Cohen Kushal Kafle VLM 23 5 0 24 Apr 2024
Would Deep Generative Models Amplify Bias in Future Models? Tianwei Chen Yusuke Hirota Mayu Otani Noa Garcia Yuta Nakashima 19 11 0 04 Apr 2024
Relation Rectification in Diffusion Model Yinwei Wu Xingyi Yang Xinchao Wang 28 6 0 29 Mar 2024
MIST: Mitigating Intersectional Bias with Disentangled Cross-Attention Editing in Text-to-Image Diffusion Models Hidir Yesiltepe Kiymet Akdemir Pinar Yanardag 24 3 0 28 Mar 2024
FairerCLIP: Debiasing CLIP's Zero-Shot Predictions using Functions in RKHSs Sepehr Dehdashtian Lan Wang Vishnu Naresh Boddeti VLM 22 11 0 22 Mar 2024
Mitigating the Impact of Attribute Editing on Face Recognition Sudipta Banerjee Sai Pranaswi Mullangi Shruti Wagle Chinmay Hegde Nasir D. Memon CVBM 31 1 0 12 Mar 2024
Debiasing Multimodal Large Language Models Yi-Fan Zhang Weichen Yu Qingsong Wen Xue Wang Zhang Zhang Liang Wang Rong Jin Tien-Ping Tan 28 4 0 08 Mar 2024
Spurious Feature Eraser: Stabilizing Test-Time Adaptation for Vision-Language Foundation Model Huan Ma Yan Zhu Changqing Zhang Peilin Zhao Baoyuan Wu Long-Kai Huang Qinghua Hu Bing Wu VLM 64 1 0 01 Mar 2024
Balancing Act: Distribution-Guided Debiasing in Diffusion Models Rishubh Parihar Abhijnya Bhat Abhipsa Basu Saswat Mallick Jogendra Nath Kundu R. V. Babu 43 18 0 28 Feb 2024
Aligning Modalities in Vision Large Language Models via Preference Fine-tuning Yiyang Zhou Chenhang Cui Rafael Rafailov Chelsea Finn Huaxiu Yao VLM MLLM 27 89 0 18 Feb 2024
Examining Gender and Racial Bias in Large Vision-Language Models Using a Novel Dataset of Parallel Images Kathleen C. Fraser S. Kiritchenko 33 31 0 08 Feb 2024
Cross-modality debiasing: using language to mitigate sub-population shifts in imaging Yijiang Pang Hoang Bao Jiayu Zhou 9 0 0 02 Feb 2024
Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help You Felix Friedrich Katharina Hämmerl P. Schramowski Manuel Brack Jindrich Libovický Kristian Kersting Alexander M. Fraser EGVM 22 10 0 29 Jan 2024
Generalized People Diversity: Learning a Human Perception-Aligned Diversity Representation for People Images Hansa Srinivasan Candice Schumann Aradhana Sinha David Madras Gbolahan O. Olanubi Alex Beutel Susanna Ricco Jilin Chen 18 5 0 25 Jan 2024
The Neglected Tails in Vision-Language Models Shubham Parashar Zhiqiu Lin Tian Liu Xiangjue Dong Yanan Li Deva Ramanan James Caverlee Shu Kong VLM 14 33 0 23 Jan 2024
Large-scale Reinforcement Learning for Diffusion Models Yinan Zhang Eric Tzeng Yilun Du Dmitry Kislyuk VLM 26 29 0 20 Jan 2024