Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation

10 October 2023

Papers citing "Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation"

45 / 45 papers shown

Title
ACE: A Security Architecture for LLM-Integrated App Systems Evan Li Tushin Mallick Evan Rose William K. Robertson Alina Oprea Cristina Nita-Rotaru 52 0 0 29 Apr 2025
Unified Attacks to Large Language Model Watermarks: Spoofing and Scrubbing in Unauthorized Knowledge Distillation Xin Yi Shunfan Zhengc Linlin Wanga Xiaoling Wang Liang He Liang He AAML 68 0 0 24 Apr 2025
Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Control Hannah Cyberey David E. Evans LLMSV 74 0 0 23 Apr 2025
2D-Curri-DPO: Two-Dimensional Curriculum Learning for Direct Preference Optimization Mengyang Li Zhong Zhang 27 0 0 10 Apr 2025
Foot-In-The-Door: A Multi-turn Jailbreak for LLMs Zixuan Weng Xiaolong Jin Jinyuan Jia X. Zhang AAML 49 0 0 27 Feb 2025
Single-pass Detection of Jailbreaking Input in Large Language Models Leyla Naz Candogan Yongtao Wu Elias Abad Rocamora Grigorios G. Chrysos V. Cevher AAML 45 0 0 24 Feb 2025
A generative approach to LLM harmfulness detection with special red flag tokens Sophie Xhonneux David Dobre Mehrnaz Mohfakhami Leo Schwinn Gauthier Gidel 45 1 0 22 Feb 2025
DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing Yi Wang Fenghua Weng S. Yang Zhan Qin Minlie Huang Wenjie Wang KELM AAML 43 0 0 17 Feb 2025
FLAME: Flexible LLM-Assisted Moderation Engine Ivan Bakulin Ilia Kopanichuk Iaroslav Bespalov Nikita Radchenko V. Shaposhnikov Dmitry V. Dylov Ivan Oseledets 84 0 0 13 Feb 2025
Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks Ang Li Yin Zhou Vethavikashini Chithrra Raghuram Tom Goldstein Micah Goldblum AAML 71 7 0 12 Feb 2025
When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search Xuan Chen Yuzhou Nie Wenbo Guo Xiangyu Zhang 107 9 0 28 Jan 2025
Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment Melissa Kazemi Rad Huy Nghiem Andy Luo Sahil Wadhwa Mohammad Sorower Stephen Rawls AAML 91 2 0 22 Jan 2025
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Kaifeng Lyu Haoyu Zhao Xinran Gu Dingli Yu Anirudh Goyal Sanjeev Arora ALM 75 44 0 20 Jan 2025
Layer-Level Self-Exposure and Patch: Affirmative Token Mitigation for Jailbreak Attack Defense Yang Ouyang Hengrui Gu Shuhang Lin Wenyue Hua Jie Peng B. Kailkhura Tianlong Chen Kaixiong Zhou Kaixiong Zhou AAML 31 1 0 05 Jan 2025
Beyond the Safety Bundle: Auditing the Helpful and Harmless Dataset Khaoula Chehbouni Jonathan Colaço-Carr Yash More Jackie CK Cheung G. Farnadi 73 0 0 12 Nov 2024
Diversity Helps Jailbreak Large Language Models Weiliang Zhao Daniel Ben-Levi Wei Hao Junfeng Yang Chengzhi Mao AAML 67 0 0 06 Nov 2024
Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities Chung-En Sun Xiaodong Liu Weiwei Yang Tsui-Wei Weng Hao Cheng Aidan San Michel Galley Jianfeng Gao 39 2 0 24 Oct 2024
On the Role of Attention Heads in Large Language Model Safety Z. Zhou Haiyang Yu Xinghua Zhang Rongwu Xu Fei Huang Kun Wang Yang Liu Junfeng Fang Yongbin Li 54 5 0 17 Oct 2024
Multi-round jailbreak attack on large language models Yihua Zhou Xiaochuan Shi AAML 16 0 0 15 Oct 2024
Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Jing Jiang Min-Bin Lin 33 8 0 09 Oct 2024
Root Defence Strategies: Ensuring Safety of LLM at the Decoding Level Xinyi Zeng Yuying Shang Yutao Zhu Jingyuan Zhang Yu Tian AAML 54 2 0 09 Oct 2024
Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablation Xinpeng Wang Chengzhi Hu Paul Röttger Barbara Plank 46 6 0 04 Oct 2024
Endless Jailbreaks with Bijection Learning Brian R. Y. Huang Maximilian Li Leonard Tang AAML 70 5 0 02 Oct 2024
An Adversarial Perspective on Machine Unlearning for AI Safety Jakub Łucki Boyi Wei Yangsibo Huang Peter Henderson F. Tramèr Javier Rando MU AAML 71 31 0 26 Sep 2024
Recent Advances in Attack and Defense Approaches of Large Language Models Jing Cui Yishi Xu Zhewei Huang Shuchang Zhou Jianbin Jiao Junge Zhang PILM AAML 47 1 0 05 Sep 2024
Bi-Factorial Preference Optimization: Balancing Safety-Helpfulness in Language Models Wenxuan Zhang Philip H. S. Torr Mohamed Elhoseiny Adel Bibi 48 9 0 27 Aug 2024
Multi-Turn Context Jailbreak Attack on Large Language Models From First Principles Xiongtao Sun Deyue Zhang Dongdong Yang Quanchen Zou Hui Li AAML 19 11 0 08 Aug 2024
Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification Boyang Zhang Yicong Tan Yun Shen Ahmed Salem Michael Backes Savvas Zannettou Yang Zhang LLMAG AAML 40 12 0 30 Jul 2024
Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference Anton Xue Avishree Khare Rajeev Alur Surbhi Goel Eric Wong 43 2 0 21 Jun 2024
SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Tinghao Xie Xiangyu Qi Yi Zeng Yangsibo Huang Udari Madhushani Sehwag ... Bo Li Kai Li Danqi Chen Peter Henderson Prateek Mittal ALM ELM 47 51 0 20 Jun 2024
"Not Aligned" is Not "Malicious": Being Careful about Hallucinations of Large Language Models' Jailbreak Lingrui Mei Shenghua Liu Yiwei Wang Baolong Bi Jiayi Mao Xueqi Cheng AAML 42 9 0 17 Jun 2024
Voice Jailbreak Attacks Against GPT-4o Xinyue Shen Yixin Wu Michael Backes Yang Zhang AuLLM 26 9 0 29 May 2024
Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models Chia-Yi Hsu Yu-Lin Tsai Chih-Hsun Lin Pin-Yu Chen Chia-Mu Yu Chun-ying Huang 42 30 0 27 May 2024
SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety Paul Röttger Fabio Pernisi Bertie Vidgen Dirk Hovy ELM KELM 53 30 0 08 Apr 2024
Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack M. Russinovich Ahmed Salem Ronen Eldan 31 76 0 02 Apr 2024
Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs Aly M. Kassem Omar Mahmoud Niloofar Mireshghallah Hyunwoo J. Kim Yulia Tsvetkov Yejin Choi Sherif Saad Santu Rana 47 18 0 05 Mar 2024
Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space Leo Schwinn David Dobre Sophie Xhonneux Gauthier Gidel Stephan Gunnemann AAML 47 36 0 14 Feb 2024
Comprehensive Assessment of Jailbreak Attacks Against LLMs Junjie Chu Yugeng Liu Ziqing Yang Xinyue Shen Michael Backes Yang Zhang AAML 25 65 0 08 Feb 2024
How Trustworthy are Open-Source LLMs? An Assessment under Malicious Demonstrations Shows their Vulnerabilities Lingbo Mo Boshi Wang Muhao Chen Huan Sun 21 25 0 15 Nov 2023
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 129 117 0 09 Nov 2023
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher Youliang Yuan Wenxiang Jiao Wenxuan Wang Jen-tse Huang Pinjia He Shuming Shi Zhaopeng Tu SILM 53 231 0 12 Aug 2023
Improving alignment of dialogue agents via targeted human judgements Amelia Glaese Nat McAleese Maja Trkebacz John Aslanides Vlad Firoiu ... John F. J. Mellor Demis Hassabis Koray Kavukcuoglu Lisa Anne Hendricks G. Irving ALM AAML 225 495 0 28 Sep 2022
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 218 441 0 23 Aug 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,730 0 04 Mar 2022
Gradient-based Adversarial Attacks against Text Transformers Chuan Guo Alexandre Sablayrolles Hervé Jégou Douwe Kiela SILM 98 225 0 15 Apr 2021