Jailbreaking Black Box Large Language Models in Twenty Queries

12 October 2023

George J. Pappas

Papers citing "Jailbreaking Black Box Large Language Models in Twenty Queries"

47 / 97 papers shown

Title
HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Models Seanie Lee Haebin Seong Dong Bok Lee Minki Kang Xiaoyin Chen Dominik Wagner Yoshua Bengio Juho Lee Sung Ju Hwang 58 2 0 02 Oct 2024
Endless Jailbreaks with Bijection Learning Brian R. Y. Huang Maximilian Li Leonard Tang AAML 66 5 0 02 Oct 2024
Robust LLM safeguarding via refusal feature adversarial training L. Yu Virginie Do Karen Hambardzumyan Nicola Cancedda AAML 51 9 0 30 Sep 2024
PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMs Jiahao Yu Yangguang Shao Hanwen Miao Junzheng Shi SILM AAML 60 3 0 23 Sep 2024
Recent Advances in Attack and Defense Approaches of Large Language Models Jing Cui Yishi Xu Zhewei Huang Shuchang Zhou Jianbin Jiao Junge Zhang PILM AAML 47 1 0 05 Sep 2024
Multi-Turn Context Jailbreak Attack on Large Language Models From First Principles Xiongtao Sun Deyue Zhang Dongdong Yang Quanchen Zou Hui Li AAML 19 11 0 08 Aug 2024
Why Are My Prompts Leaked? Unraveling Prompt Extraction Threats in Customized Large Language Models Zi Liang Haibo Hu Qingqing Ye Yaxin Xiao Haoyang Li AAML ELM SILM 40 4 0 05 Aug 2024
Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification Boyang Zhang Yicong Tan Yun Shen Ahmed Salem Michael Backes Savvas Zannettou Yang Zhang LLMAG AAML 38 12 0 30 Jul 2024
Does Refusal Training in LLMs Generalize to the Past Tense? Maksym Andriushchenko Nicolas Flammarion 42 25 0 16 Jul 2024
Are Large Language Models Really Bias-Free? Jailbreak Prompts for Assessing Adversarial Robustness to Bias Elicitation Riccardo Cantini Giada Cosenza A. Orsino Domenico Talia AAML 45 5 0 11 Jul 2024
T2VSafetyBench: Evaluating the Safety of Text-to-Video Generative Models Yibo Miao Yifan Zhu Yinpeng Dong Lijia Yu Jun Zhu Xiao-Shan Gao EGVM 31 12 0 08 Jul 2024
Badllama 3: removing safety finetuning from Llama 3 in minutes Dmitrii Volkov 24 3 0 01 Jul 2024
Large Language Models Are Involuntary Truth-Tellers: Exploiting Fallacy Failure for Jailbreak Attacks Yue Zhou Henry Peng Zou Barbara Maria Di Eugenio Yang Zhang HILM LRM 37 1 0 01 Jul 2024
Few-shot Personalization of LLMs with Mis-aligned Responses Jaehyung Kim Yiming Yang 33 7 0 26 Jun 2024
Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference Anton Xue Avishree Khare Rajeev Alur Surbhi Goel Eric Wong 41 2 0 21 Jun 2024
"Not Aligned" is Not "Malicious": Being Careful about Hallucinations of Large Language Models' Jailbreak Lingrui Mei Shenghua Liu Yiwei Wang Baolong Bi Jiayi Mao Xueqi Cheng AAML 38 9 0 17 Jun 2024
PRISM: A Design Framework for Open-Source Foundation Model Safety Terrence Neumann Bryan Jones 22 1 0 14 Jun 2024
Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents Avital Shafran R. Schuster Vitaly Shmatikov 37 27 0 09 Jun 2024
SelfDefend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner Xunguang Wang Daoyuan Wu Zhenlan Ji Zongjie Li Pingchuan Ma Shuai Wang Yingjiu Li Yang Liu Ning Liu Juergen Rahmel AAML 66 6 0 08 Jun 2024
Ranking Manipulation for Conversational Search Engines Samuel Pfrommer Yatong Bai Tanmay Gautam Somayeh Sojoudi SILM 39 4 0 05 Jun 2024
Defending Large Language Models Against Attacks With Residual Stream Activation Analysis Amelia Kawasaki Andrew Davis Houssam Abbas AAML KELM 19 2 0 05 Jun 2024
Voice Jailbreak Attacks Against GPT-4o Xinyue Shen Yixin Wu Michael Backes Yang Zhang AuLLM 26 9 0 29 May 2024
Learning diverse attacks on large language models for robust red-teaming and safety tuning Seanie Lee Minsu Kim Lynn Cherif David Dobre Juho Lee ... Kenji Kawaguchi Gauthier Gidel Yoshua Bengio Nikolay Malkin Moksh Jain AAML 48 12 0 28 May 2024
Securing the Future of GenAI: Policy and Technology Mihai Christodorescu Craven S. Feizi Neil Zhenqiang Gong Mia Hoffmann ... Jessica Newman Emelia Probasco Yanjun Qi Khawaja Shams Turek SILM 26 3 0 21 May 2024
Talking Nonsense: Probing Large Language Models' Understanding of Adversarial Gibberish Inputs Valeriia Cherepanova James Zou AAML 28 4 0 26 Apr 2024
Advancing the Robustness of Large Language Models through Self-Denoised Smoothing Jiabao Ji Bairu Hou Zhen Zhang Guanhua Zhang Wenqi Fan Qing Li Yang Zhang Gaowen Liu Sijia Liu Shiyu Chang AAML 24 5 0 18 Apr 2024
Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks Maksym Andriushchenko Francesco Croce Nicolas Flammarion AAML 81 155 0 02 Apr 2024
Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack M. Russinovich Ahmed Salem Ronen Eldan 26 75 0 02 Apr 2024
Automated Black-box Prompt Engineering for Personalized Text-to-Image Generation Yutong He Alexander Robey Naoki Murata Yiding Jiang J. Williams George Pappas Hamed Hassani Yuki Mitsufuji Ruslan Salakhutdinov J. Zico Kolter DiffM 91 4 0 28 Mar 2024
Images are Achilles' Heel of Alignment: Exploiting Visual Vulnerabilities for Jailbreaking Multimodal Large Language Models Yifan Li Hangyu Guo Kun Zhou Wayne Xin Zhao Ji-Rong Wen 40 38 0 14 Mar 2024
Can LLMs Separate Instructions From Data? And What Do We Even Mean By That? Egor Zverev Sahar Abdelnabi Soroush Tabesh Mario Fritz Christoph H. Lampert 41 19 0 11 Mar 2024
Here Comes The AI Worm: Unleashing Zero-click Worms that Target GenAI-Powered Applications Stav Cohen Ron Bitton Ben Nassi 17 17 0 05 Mar 2024
ImgTrojan: Jailbreaking Vision-Language Models with ONE Image Xijia Tao Shuai Zhong Lei Li Qi Liu Lingpeng Kong 30 25 0 05 Mar 2024
Round Trip Translation Defence against Large Language Model Jailbreaking Attacks Canaan Yung H. M. Dolatabadi S. Erfani Christopher Leckie AAML 56 5 0 21 Feb 2024
Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space Leo Schwinn David Dobre Sophie Xhonneux Gauthier Gidel Stephan Gunnemann AAML 40 36 0 14 Feb 2024
Comprehensive Assessment of Jailbreak Attacks Against LLMs Junjie Chu Yugeng Liu Ziqing Yang Xinyue Shen Michael Backes Yang Zhang AAML 25 65 0 08 Feb 2024
Black-Box Access is Insufficient for Rigorous AI Audits Stephen Casper Carson Ezell Charlotte Siegmann Noam Kolt Taylor Lynn Curtis ... Michael Gerovitch David Bau Max Tegmark David M. Krueger Dylan Hadfield-Menell AAML 13 75 0 25 Jan 2024
All in How You Ask for It: Simple Black-Box Method for Jailbreak Attacks Kazuhiro Takemoto 32 21 0 18 Jan 2024
AttackEval: How to Evaluate the Effectiveness of Jailbreak Attacking on Large Language Models Dong Shu Mingyu Jin Suiyuan Zhu Beichen Wang Zihao Zhou Chong Zhang Yongfeng Zhang ELM 37 12 0 17 Jan 2024
Bypassing the Safety Training of Open-Source LLMs with Priming Attacks Jason Vega Isha Chaudhary Changming Xu Gagandeep Singh AAML 14 18 0 19 Dec 2023
Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak Yanrui Du Sendong Zhao Ming Ma Yuhan Chen Bing Qin 18 15 0 07 Dec 2023
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 129 116 0 09 Nov 2023
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 225 495 0 28 Sep 2022
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 218 441 0 23 Aug 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,898 0 31 Dec 2020