Relative Bias: A Comparative Framework for Quantifying Bias in LLMs

22 May 2025

Papers citing "Relative Bias: A Comparative Framework for Quantifying Bias in LLMs"

46 / 46 papers shown

Title
A Human-AI Comparative Analysis of Prompt Sensitivity in LLM-Based Relevance Judgment Negar Arabzadeh Charles L. A. Clarke 80 3 0 16 Apr 2025
Does Prompt Formatting Have Any Impact on LLM Performance? Jia He Mukund Rungta David Koleczek Arshdeep Sekhon Franklin X Wang Sadid Hasan LLMAG LRM 102 59 0 15 Nov 2024
Order Matters in Hallucination: Reasoning Order as Benchmark and Reflexive Prompting for Large-Language-Models Zikai Xie HILM LRM 146 7 0 09 Aug 2024
Non-Determinism of "Deterministic" LLM Settings Berk Atil Alexa Chittams Liseng Fu Ferhan Ture Lixinyu Xu ... Tomasz Tudrej Ferhan Ture Zhe Wu Lixinyu Xu Breck Baldwin 110 6 0 06 Aug 2024
LiveBench: A Challenging, Contamination-Limited LLM Benchmark Colin White Samuel Dooley Manley Roberts Arka Pal Ben Feuer ... Willie Neiswanger Micah Goldblum Tom Goldstein Willie Neiswanger Micah Goldblum ELM 125 20 0 27 Jun 2024
MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark Yubo Wang Xueguang Ma Ge Zhang Yuansheng Ni Abhranil Chandra ... Kai Wang Alex Zhuang Rongqi Fan Xiang Yue Wenhu Chen LRM ELM 156 465 0 03 Jun 2024
OR-Bench: An Over-Refusal Benchmark for Large Language Models Justin Cui Wei-Lin Chiang Ion Stoica Cho-Jui Hsieh ALM 161 55 0 31 May 2024
MBIAS: Mitigating Bias in Large Language Models While Retaining Context Shaina Raza Ananya Raval Veronica Chatrath 130 10 0 18 May 2024
Disclosure and Mitigation of Gender Bias in LLMs Xiangjue Dong Yibo Wang Philip S. Yu James Caverlee 67 39 0 17 Feb 2024
GPTBIAS: A Comprehensive Framework for Evaluating Bias in Large Language Models Jiaxu Zhao Meng Fang Shirui Pan Wenpeng Yin Mykola Pechenizkiy ELM 76 13 0 11 Dec 2023
ROBBIE: Robust Bias Evaluation of Large Generative Language Models David Esiobu X. Tan Saghar Hosseini Megan Ung Yuchen Zhang Jude Fernandes Jane Dwivedi-Yu Eleonora Presani Adina Williams Eric Michael Smith 65 57 0 29 Nov 2023
The Perils & Promises of Fact-checking with Large Language Models Dorian Quelle Alexandre Bovet 79 26 0 20 Oct 2023
NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails Traian Rebedea R. Dinu Makesh Narsimhan Sreedhar Christopher Parisien Jonathan Cohen KELM 99 152 0 16 Oct 2023
Large Language Model Alignment: A Survey Tianhao Shen Renren Jin Yufei Huang Chuang Liu Weilong Dong Zishan Guo Xinwei Wu Yan Liu Deyi Xiong LM&MA 112 205 0 26 Sep 2023
Explainability for Large Language Models: A Survey Haiyan Zhao Hanjie Chen Fan Yang Ninghao Liu Huiqi Deng Hengyi Cai Shuaiqiang Wang Dawei Yin Jundong Li LRM 106 469 0 02 Sep 2023
Bias and Fairness in Large Language Models: A Survey Isabel O. Gallegos Ryan Rossi Joe Barrow Md Mehrab Tanjim Sungchul Kim Franck Dernoncourt Tong Yu Ruiyi Zhang Nesreen Ahmed AILaw 140 606 0 02 Sep 2023
Wider and Deeper LLM Networks are Fairer LLM Evaluators Xinghua Zhang Yu Bowen Haiyang Yu Yangyu Lv Tingwen Liu Fei Huang Hongbo Xu Yongbin Li ALM 143 90 0 03 Aug 2023
Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias Yue Yu Yuchen Zhuang Jieyu Zhang Yu Meng Alexander Ratner Ranjay Krishna Jiaming Shen Chao Zhang ALM 108 236 0 28 Jun 2023
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena Lianmin Zheng Wei-Lin Chiang Ying Sheng Siyuan Zhuang Zhanghao Wu ... Dacheng Li Eric Xing Haotong Zhang Joseph E. Gonzalez Ion Stoica ALM OSLM ELM 531 4,453 0 09 Jun 2023
Large Language Models are not Fair Evaluators Peiyi Wang Lei Li Liang Chen Zefan Cai Dawei Zhu Binghuai Lin Yunbo Cao Qi Liu Tianyu Liu Zhifang Sui ALM 147 575 0 29 May 2023
AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback Yann Dubois Xuechen Li Rohan Taori Tianyi Zhang Ishaan Gulrajani Jimmy Ba Carlos Guestrin Percy Liang Tatsunori B. Hashimoto ALM 152 608 0 22 May 2023
Pretraining Language Models with Human Preferences Tomasz Korbak Kejian Shi Angelica Chen Rasika Bhalerao C. L. Buckley Jason Phang Sam Bowman Ethan Perez ALM SyDa 99 231 0 16 Feb 2023
Self-Instruct: Aligning Language Models with Self-Generated Instructions Yizhong Wang Yeganeh Kordi Swaroop Mishra Alisa Liu Noah A. Smith Daniel Khashabi Hannaneh Hajishirzi ALM SyDa LRM 198 2,264 0 20 Dec 2022
One Embedder, Any Task: Instruction-Finetuned Text Embeddings Hongjin Su Weijia Shi Jungo Kasai Yizhong Wang Yushi Hu Mari Ostendorf Wen-tau Yih Noah A. Smith Luke Zettlemoyer Tao Yu 115 302 0 19 Dec 2022
Large Language Models Are Human-Level Prompt Engineers Yongchao Zhou Andrei Ioan Muresanu Ziwen Han Keiran Paster Silviu Pitis Harris Chan Jimmy Ba ALM LLMAG 195 904 0 03 Nov 2022
A Robust Bias Mitigation Procedure Based on the Stereotype Content Model Eddie L. Ungless Amy Rafferty Hrichika Nag Bjorn Ross 62 30 0 26 Oct 2022
Prompting GPT-3 To Be Reliable Chenglei Si Zhe Gan Zhengyuan Yang Shuohang Wang Jianfeng Wang Jordan L. Boyd-Graber Lijuan Wang KELM LRM 113 303 0 17 Oct 2022
Toy Models of Superposition Nelson Elhage Tristan Hume Catherine Olsson Nicholas Schiefer T. Henighan ... Sam McCandlish Jared Kaplan Dario Amodei Martin Wattenberg C. Olah AAML MILM 203 380 0 21 Sep 2022
Mitigating Language-Dependent Ethnic Bias in BERT Jaimeen Ahn Alice Oh 229 101 0 13 Sep 2021
Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models Jianmo Ni Gustavo Hernández Ábrego Noah Constant Ji Ma Keith B. Hall Daniel Cer Yinfei Yang 257 568 0 19 Aug 2021
Towards Understanding and Mitigating Social Biases in Language Models Paul Pu Liang Chiyu Wu Louis-Philippe Morency Ruslan Salakhutdinov 102 399 0 24 Jun 2021
FairFil: Contrastive Neural Debiasing Method for Pretrained Text Encoders Pengyu Cheng Weituo Hao Siyang Yuan Shijing Si Lawrence Carin 77 105 0 11 Mar 2021
Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP Timo Schick Sahana Udupa Hinrich Schütze 319 388 0 28 Feb 2021
BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation Jwala Dhamala Tony Sun Varun Kumar Satyapriya Krishna Yada Pruksachatkun Kai-Wei Chang Rahul Gupta 92 402 0 27 Jan 2021
Debiasing Pre-trained Contextualised Embeddings Masahiro Kaneko Danushka Bollegala 269 143 0 23 Jan 2021
Measuring and Reducing Gendered Correlations in Pre-trained Models Kellie Webster Xuezhi Wang Ian Tenney Alex Beutel Emily Pitler Ellie Pavlick Jilin Chen Ed Chi Slav Petrov FaML 93 260 0 12 Oct 2020
CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models Nikita Nangia Clara Vania Rasika Bhalerao Samuel R. Bowman 138 690 0 30 Sep 2020
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models Samuel Gehman Suchin Gururangan Maarten Sap Yejin Choi Noah A. Smith 202 1,222 0 24 Sep 2020
Measuring Massive Multitask Language Understanding Dan Hendrycks Collin Burns Steven Basart Andy Zou Mantas Mazeika Basel Alomair Jacob Steinhardt ELM RALM 204 4,580 0 07 Sep 2020
Towards Debiasing Sentence Representations Paul Pu Liang Irene Li Emily Zheng Y. Lim Ruslan Salakhutdinov Louis-Philippe Morency 103 242 0 16 Jul 2020
Language Models are Few-Shot Learners Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan ... Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever Dario Amodei BDL 1.0K 42,651 0 28 May 2020
Reducing Sentiment Bias in Language Models via Counterfactual Evaluation Po-Sen Huang Huan Zhang Ray Jiang Robert Stanforth Johannes Welbl Jack W. Rae Vishal Maini Dani Yogatama Pushmeet Kohli 104 217 0 08 Nov 2019
Does Gender Matter? Towards Fairness in Dialogue Systems Haochen Liu Jamell Dacon Wenqi Fan Hui Liu Zitao Liu Jiliang Tang 161 144 0 16 Oct 2019
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks Nils Reimers Iryna Gurevych 1.3K 12,348 0 27 Aug 2019
Measuring Bias in Contextualized Word Representations Keita Kurita Nidhi Vyas Ayush Pareek A. Black Yulia Tsvetkov 116 454 0 18 Jun 2019
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova VLM SSL SSeg 1.9K 95,531 0 11 Oct 2018