RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models

24 September 2020

Yejin Choi

Papers citing "RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models"

50 / 201 papers shown

Title
Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing Huanqian Wang Yang Yue Rui Lu Jingxin Shi Andrew Zhao Shenzhi Wang Shiji Song Gao Huang LM&Ro KELM 49 6 0 11 Jul 2024
Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitation Riccardo Cantini Giada Cosenza A. Orsino Domenico Talia AAML 50 5 0 11 Jul 2024
Composable Interventions for Language Models Arinbjorn Kolbeinsson Kyle O'Brien Tianjin Huang Shanghua Gao Shiwei Liu ... Anurag J. Vaidya Faisal Mahmood Marinka Zitnik Tianlong Chen Thomas Hartvigsen KELM MU 84 5 0 09 Jul 2024
Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression Zhichao Xu Ashim Gupta Tao Li Oliver Bentham Vivek Srikumar 40 8 0 06 Jul 2024
CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models Song Wang Peng Wang Tong Zhou Yushun Dong Zhen Tan Jundong Li CoGe 46 6 0 02 Jul 2024
FrenchToxicityPrompts: a Large Benchmark for Evaluating and Mitigating Toxicity in French Texts Caroline Brun Vassilina Nikoulina 36 1 0 25 Jun 2024
ARES: Alternating Reinforcement Learning and Supervised Fine-Tuning for Enhanced Multi-Modal Chain-of-Thought Reasoning Through Diverse AI Feedback Ju-Seung Byun Jiyun Chun Jihyung Kil Andrew Perrault ReLM LRM 39 1 0 25 Jun 2024
Cascade Reward Sampling for Efficient Decoding-Time Alignment Bolian Li Yifan Wang A. Grama Ruqi Zhang Ruqi Zhang AI4TS 47 9 0 24 Jun 2024
ToVo: Toxicity Taxonomy via Voting Tinh Son Luong Thanh-Thien Le Thang Viet Doan L. Van Thien Huu Nguyen Diep Thi-Ngoc Nguyen 31 0 0 21 Jun 2024
SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Tinghao Xie Xiangyu Qi Yi Zeng Yangsibo Huang Udari Madhushani Sehwag ... Bo Li Kai Li Danqi Chen Peter Henderson Prateek Mittal ALM ELM 50 51 0 20 Jun 2024
Split, Unlearn, Merge: Leveraging Data Attributes for More Effective Unlearning in LLMs S. Kadhe Farhan Ahmed Dennis Wei Nathalie Baracaldo Inkit Padhi MoMe MU 28 6 0 17 Jun 2024
CHiSafetyBench: A Chinese Hierarchical Safety Benchmark for Large Language Models Wenjing Zhang Xuejiao Lei Zhaoxiang Liu Meijuan An Bikun Yang Kaikai Zhao Kai Wang Shiguo Lian ELM 34 7 0 14 Jun 2024
Ask LLMs Directly, "What shapes your bias?": Measuring Social Bias in Large Language Models Jisu Shin Hoyun Song Huije Lee Soyeong Jeong Jong C. Park 38 6 0 06 Jun 2024
TimeChara: Evaluating Point-in-Time Character Hallucination of Role-Playing Large Language Models Jaewoo Ahn Taehyun Lee Junyoung Lim Jin-Hwa Kim Sangdoo Yun Hwaran Lee Gunhee Kim LLMAG HILM 35 12 0 28 May 2024
Aligning to Thousands of Preferences via System Message Generalization Seongyun Lee Sue Hyun Park Seungone Kim Minjoon Seo ALM 36 36 0 28 May 2024
MBIAS: Mitigating Bias in Large Language Models While Retaining Context Shaina Raza Ananya Raval Veronica Chatrath 38 6 0 18 May 2024
Exploring Subjectivity for more Human-Centric Assessment of Social Biases in Large Language Models Paula Akemi Aoyagui Sharon Ferguson Anastasia Kuzminykh 46 0 0 17 May 2024
Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPT Shucheng Zhu Weikang Wang Ying Liu 29 5 0 11 May 2024
"They are uncultured": Unveiling Covert Harms and Social Threats in LLM Generated Conversations Preetam Prabhu Srikar Dammu Hayoung Jung Anjali Singh Monojit Choudhury Tanushree Mitra 32 8 0 08 May 2024
SOUL: Unlocking the Power of Second-Order Optimization for LLM Unlearning Jinghan Jia Yihua Zhang Yimeng Zhang Jiancheng Liu Bharat Runwal James Diffenderfer B. Kailkhura Sijia Liu MU 29 32 0 28 Apr 2024
Online Safety Analysis for LLMs: a Benchmark, an Assessment, and a Path Forward Xuan Xie Jiayang Song Zhehua Zhou Yuheng Huang Da Song Lei Ma OffRL 39 6 0 12 Apr 2024
SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety Paul Röttger Fabio Pernisi Bertie Vidgen Dirk Hovy ELM KELM 58 30 0 08 Apr 2024
"I'm categorizing LLM as a productivity tool": Examining ethics of LLM use in HCI research practices Shivani Kapania Ruiyi Wang Toby Jia-Jun Li Tianshi Li Hong Shen 34 7 0 28 Mar 2024
On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models Xinpeng Wang Shitong Duan Xiaoyuan Yi Jing Yao Shanlin Zhou Zhihua Wei Peng Zhang Dongkuan Xu Maosong Sun Xing Xie OffRL 35 16 0 07 Mar 2024
From One to Many: Expanding the Scope of Toxicity Mitigation in Language Models Luiza Amador Pozzobon Patrick Lewis Sara Hooker B. Ermiş 36 7 0 06 Mar 2024
Large Language Models are Vulnerable to Bait-and-Switch Attacks for Generating Harmful Content Federico Bianchi James Y. Zou 32 4 0 21 Feb 2024
ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs Fengqing Jiang Zhangchen Xu Luyao Niu Zhen Xiang Bhaskar Ramasubramanian Bo Li Radha Poovendran 31 86 0 19 Feb 2024
Trustworthy Distributed AI Systems: Robustness, Privacy, and Governance Wenqi Wei Ling Liu 25 16 0 02 Feb 2024
Time is Encoded in the Weights of Finetuned Language Models Kai Nylund Suchin Gururangan Noah A. Smith AI4TS 23 17 0 20 Dec 2023
A Pseudo-Semantic Loss for Autoregressive Models with Logical Constraints Kareem Ahmed Kai-Wei Chang Guy Van den Broeck 26 10 0 06 Dec 2023
Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks Rahul Ramesh Ekdeep Singh Lubana Mikail Khona Robert P. Dick Hidenori Tanaka CoGe 33 6 0 21 Nov 2023
JAB: Joint Adversarial Prompting and Belief Augmentation Ninareh Mehrabi Palash Goyal Anil Ramakrishna Jwala Dhamala Shalini Ghosh Richard Zemel Kai-Wei Chang Aram Galstyan Rahul Gupta AAML 28 7 0 16 Nov 2023
How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities Lingbo Mo Boshi Wang Muhao Chen Huan Sun 29 27 0 15 Nov 2023
Controlled Text Generation for Black-box Language Models via Score-based Progressive Editor Sangwon Yu Changmin Lee Hojin Lee Sungroh Yoon 24 0 0 13 Nov 2023
Leveraging LLMs for Synthesizing Training Data Across Many Languages in Multilingual Dense Retrieval Nandan Thakur Jianmo Ni Gustavo Hernández Ábrego John Wieting Jimmy J. Lin Daniel Matthew Cer RALM 31 12 0 10 Nov 2023
Model Merging by Uncertainty-Based Gradient Matching Nico Daheim Thomas Möllenhoff E. Ponti Iryna Gurevych Mohammad Emtiyaz Khan MoMe FedML 32 43 0 19 Oct 2023
Privacy Preserving Large Language Models: ChatGPT Case Study Based Vision and Framework Imdad Ullah Najm Hassan S. Gill Basem Suleiman T. Ahanger Zawar Shah Junaid Qadir S. Kanhere 35 16 0 19 Oct 2023
Goodtriever: Adaptive Toxicity Mitigation with Retrieval-augmented Models Luiza Amador Pozzobon B. Ermiş Patrick Lewis Sara Hooker 28 20 0 11 Oct 2023
Beyond Reverse KL: Generalizing Direct Preference Optimization with Diverse Divergence Constraints Chaoqi Wang Yibo Jiang Yuguang Yang Han Liu Yuxin Chen 26 81 0 28 Sep 2023
Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding Jiacheng Liu Andrew Cohen Ramakanth Pasunuru Yejin Choi Hannaneh Hajishirzi Asli Celikyilmaz 16 23 0 26 Sep 2023
Foundation Metrics for Evaluating Effectiveness of Healthcare Conversations Powered by Generative AI Mahyar Abbasian Elahe Khatibi Iman Azimi David Oniani Zahra Shakeri Hossein Abad ... Bryant Lin Olivier Gevaert Li-Jia Li Ramesh C. Jain Amir M. Rahmani LM&MA ELM AI4MH 29 66 0 21 Sep 2023
BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model Nolan Dey Daria Soboleva Faisal Al-Khateeb Bowen Yang Ribhu Pathria ... Robert Myers Jacob Robert Steeves Natalia Vassilieva Marvin Tom Joel Hestness MoE 19 14 0 20 Sep 2023
CMD: a framework for Context-aware Model self-Detoxification Zecheng Tang Keyan Zhou Juntao Li Yuyang Ding Pinzheng Wang Bowen Yan Minzhang MU 23 5 0 16 Aug 2023
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher Youliang Yuan Wenxiang Jiao Wenxuan Wang Jen-tse Huang Pinjia He Shuming Shi Zhaopeng Tu SILM 65 231 0 12 Aug 2023
XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Models Paul Röttger Hannah Rose Kirk Bertie Vidgen Giuseppe Attanasio Federico Bianchi Dirk Hovy ALM ELM AILaw 23 122 0 02 Aug 2023
Of Models and Tin Men: A Behavioural Economics Study of Principal-Agent Problems in AI Alignment using Large-Language Models S. Phelps Rebecca E. Ranson LLMAG 34 1 0 20 Jul 2023
Gender-tuning: Empowering Fine-tuning for Debiasing Pre-trained Language Models Somayeh Ghanbarzadeh Yan-ping Huang Hamid Palangi R. C. Moreno Hamed Khanpour 32 12 0 20 Jul 2023
MasterKey: Automated Jailbreak Across Multiple Large Language Model Chatbots Gelei Deng Yi Liu Yuekang Li Kailong Wang Ying Zhang Zefeng Li Haoyu Wang Tianwei Zhang Yang Liu SILM 35 118 0 16 Jul 2023
Jailbroken: How Does LLM Safety Training Fail? Alexander Wei Nika Haghtalab Jacob Steinhardt 75 832 0 05 Jul 2023
Towards Theory-based Moral AI: Moral AI with Aggregating Models Based on Normative Ethical Theory Masashi Takeshita Rafal Rzepka K. Araki 18 8 0 20 Jun 2023