"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models

7 August 2023

Michael Backes

Papers citing ""Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models"

28 / 178 papers shown

Title
BadLlama: cheaply removing safety fine-tuning from Llama 2-Chat 13B Pranav M. Gade Simon Lermen Charlie Rogers-Smith Jeffrey Ladish ALM AI4MH 9 23 0 31 Oct 2023
From Chatbots to PhishBots? -- Preventing Phishing scams created using ChatGPT, Google Bard and Claude S. Roy Poojitha Thota Krishna Vamsi Naragam Shirin Nilizadeh SILM 36 15 0 29 Oct 2023
Privacy in Large Language Models: Attacks, Defenses and Future Directions Haoran Li Yulin Chen Jinglong Luo Yan Kang Xiaojin Zhang Qi Hu Chunkit Chan Yangqiu Song PILM 38 39 0 16 Oct 2023
Prompt Packer: Deceiving LLMs through Compositional Instruction with Hidden Attacks Shuyu Jiang Xingshu Chen Rui Tang 19 22 0 16 Oct 2023
Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation Yangsibo Huang Samyak Gupta Mengzhou Xia Kai Li Danqi Chen AAML 22 264 0 10 Oct 2023
Multilingual Jailbreak Challenges in Large Language Models Yue Deng Wenxuan Zhang Sinno Jialin Pan Lidong Bing AAML 29 112 0 10 Oct 2023
SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks Alexander Robey Eric Wong Hamed Hassani George J. Pappas AAML 38 215 0 05 Oct 2023
Shadow Alignment: The Ease of Subverting Safely-Aligned Language Models Xianjun Yang Xiao Wang Qi Zhang Linda R. Petzold William Yang Wang Xun Zhao Dahua Lin 18 159 0 04 Oct 2023
Low-Resource Languages Jailbreak GPT-4 Zheng-Xin Yong Cristina Menghini Stephen H. Bach SILM 10 169 0 03 Oct 2023
On the Safety of Open-Sourced Large Language Models: Does Alignment Really Prevent Them From Being Misused? Hangfan Zhang Zhimeng Guo Huaisheng Zhu Bochuan Cao Lu Lin Jinyuan Jia Jinghui Chen Di Wu 70 23 0 02 Oct 2023
Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AI Muhammad Aurangzeb Ahmad Ilker Yaramis Taposh Dutta Roy LM&MA HILM 12 33 0 26 Sep 2023
Can LLM-Generated Misinformation Be Detected? Canyu Chen Kai Shu DeLMO 29 157 0 25 Sep 2023
LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Dataset Lianmin Zheng Wei-Lin Chiang Ying Sheng Tianle Li Siyuan Zhuang ... Zi Lin Eric P. Xing Joseph E. Gonzalez Ion Stoica Haotong Zhang 16 170 0 21 Sep 2023
Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM Bochuan Cao Yu Cao Lu Lin Jinghui Chen AAML 23 129 0 18 Sep 2023
FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Models Dongyu Yao Jianshu Zhang Ian G. Harris Marcel Carlsson 16 30 0 11 Sep 2023
Demystifying RCE Vulnerabilities in LLM-Integrated Apps Tong Liu Zizhuang Deng Guozhu Meng Yuekang Li Kai Chen SILM 29 19 0 06 Sep 2023
Baseline Defenses for Adversarial Attacks Against Aligned Language Models Neel Jain Avi Schwarzschild Yuxin Wen Gowthami Somepalli John Kirchenbauer Ping Yeh-Chiang Micah Goldblum Aniruddha Saha Jonas Geiping Tom Goldstein AAML 10 335 0 01 Sep 2023
Use of LLMs for Illicit Purposes: Threats, Prevention Measures, and Vulnerabilities Maximilian Mozes Xuanli He Bennett Kleinberg Lewis D. Griffin 31 75 0 24 Aug 2023
On the Adversarial Robustness of Multi-Modal Foundation Models Christian Schlarmann Matthias Hein AAML 102 84 0 21 Aug 2023
Robustness Over Time: Understanding Adversarial Examples' Effectiveness on Longitudinal Versions of Large Language Models Yugeng Liu Tianshuo Cong Zhengyu Zhao Michael Backes Yun Shen Yang Zhang AAML 25 6 0 15 Aug 2023
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback Stephen Casper Xander Davies Claudia Shi T. Gilbert Jérémy Scheurer ... Erdem Biyik Anca Dragan David M. Krueger Dorsa Sadigh Dylan Hadfield-Menell ALM OffRL 34 468 0 27 Jul 2023
Jailbreak in pieces: Compositional Adversarial Attacks on Multi-Modal Language Models Erfan Shayegani Yue Dong Nael B. Abu-Ghazaleh 20 126 0 26 Jul 2023
FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets Seonghyeon Ye Doyoung Kim Sungdong Kim Hyeonbin Hwang Seungone Kim Yongrae Jo James Thorne Juho Kim Minjoon Seo ALM 30 96 0 20 Jul 2023
MGTBench: Benchmarking Machine-Generated Text Detection Xinlei He Xinyue Shen Z. Chen Michael Backes Yang Zhang DeLMO 51 99 0 26 Mar 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 242 1,070 0 05 Oct 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,730 0 04 Mar 2022
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 267 1,798 0 14 Dec 2020
Adversarial Example Generation with Syntactically Controlled Paraphrase Networks Mohit Iyyer John Wieting Kevin Gimpel Luke Zettlemoyer AAML GAN 181 708 0 17 Apr 2018