Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks

2 April 2024

Maksym Andriushchenko

Papers citing "Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks"

50 / 124 papers shown

Title
Demystifying optimized prompts in language models Rimon Melamed Lucas H. McCabe H. H. Huang 31 0 0 04 May 2025
Transferable Adversarial Attacks on Black-Box Vision-Language Models Kai Hu Weichen Yu L. Zhang Alexander Robey Andy Zou Chengming Xu Haoqi Hu Matt Fredrikson AAML VLM 47 0 0 02 May 2025
A Cryptographic Perspective on Mitigation vs. Detection in Machine Learning Greg Gluch Shafi Goldwasser AAML 33 0 0 28 Apr 2025
RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models Bang An Shiyue Zhang Mark Dredze 54 0 0 25 Apr 2025
MR. Guard: Multilingual Reasoning Guardrail using Curriculum Learning Yahan Yang Soham Dan Shuo Li Dan Roth Insup Lee LRM 17 0 0 21 Apr 2025
DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification Yu Li Han Jiang Zhihua Wei AAML 24 0 0 18 Apr 2025
The Jailbreak Tax: How Useful are Your Jailbreak Outputs? Kristina Nikolić Luze Sun Jie Zhang F. Tramèr 18 0 0 14 Apr 2025
The Structural Safety Generalization Problem Julius Broomfield Tom Gibbs Ethan Kosak-Hine George Ingebretsen Tia Nasir Jason Zhang Reihaneh Iranmanesh Sara Pieri Reihaneh Rabbany Kellin Pelrine AAML 23 0 0 13 Apr 2025
A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models Carlos Peláez-González Andrés Herrera-Poyatos Cristina Zuheros David Herrera-Poyatos Virilo Tejedor F. Herrera AAML 19 0 0 07 Apr 2025
Emerging Cyber Attack Risks of Medical AI Agents Jianing Qiu Lin Li Jiankai Sun Hao Wei Zhe Xu K. Lam Wu Yuan AAML 21 1 0 02 Apr 2025
LightDefense: A Lightweight Uncertainty-Driven Defense against Jailbreaks via Shifted Token Distribution Zhuoran Yang Jie Peng Zhen Tan Tianlong Chen Yanyong Zhang AAML 39 0 0 02 Apr 2025
Representation Bending for Large Language Model Safety Ashkan Yousefpour Taeheon Kim Ryan S. Kwon Seungbeen Lee Wonje Jeung Seungju Han Alvin Wan Harrison Ngan Youngjae Yu Jonghyun Choi AAML ALM KELM 52 0 0 02 Apr 2025
Exposing the Ghost in the Transformer: Abnormal Detection for Large Language Models via Hidden State Forensics Shide Zhou K. Wang Ling Shi H. Wang 39 0 0 01 Apr 2025
Output Constraints as Attack Surface: Exploiting Structured Generation to Bypass LLM Safety Mechanisms Shuoming Zhang Jiacheng Zhao Ruiyuan Xu Xiaobing Feng Huimin Cui AAML 29 0 0 31 Mar 2025
Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy Joonhyun Jeong Seyun Bae Yeonsung Jung Jaeryong Hwang Eunho Yang AAML 43 0 0 26 Mar 2025
STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Models Xunguang Wang Wenxuan Wang Zhenlan Ji Zongjie Li Pingchuan Ma Daoyuan Wu Shuai Wang 46 0 0 23 Mar 2025
Gender and content bias in Large Language Models: a case study on Google Gemini 2.0 Flash Experimental Roberto Balestri 37 0 0 18 Mar 2025
AI Companies Should Report Pre- and Post-Mitigation Safety Evaluations Dillon Bowen Ann-Kathrin Dombrowski Adam Gleave Chris Cundy ELM 43 0 0 17 Mar 2025
Multi-Agent Systems Execute Arbitrary Malicious Code Harold Triedman Rishi Jha Vitaly Shmatikov LLMAG AAML 83 2 0 15 Mar 2025
Siege: Autonomous Multi-Turn Jailbreaking of Large Language Models with Tree Search Andy Zhou MU 64 0 0 13 Mar 2025
Backtracking for Safety Bilgehan Sel Dingcheng Li Phillip Wallis Vaishakh Keshava Ming Jin Siddhartha Reddy Jonnalagadda KELM 50 0 0 11 Mar 2025
Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulation Wenlong Meng Fan Zhang Wendao Yao Zhenyuan Guo Y. Li Chengkun Wei Wenzhi Chen AAML 31 1 0 11 Mar 2025
Improving LLM Safety Alignment with Dual-Objective Optimization Xuandong Zhao Will Cai Tianneng Shi David Huang Licong Lin Song Mei Dawn Song AAML MU 49 1 0 05 Mar 2025
LLM-Safety Evaluations Lack Robustness Tim Beyer Sophie Xhonneux Simon Geisler Gauthier Gidel Leo Schwinn Stephan Günnemann ALM ELM 85 0 0 04 Mar 2025
À la recherche du sens perdu: your favourite LLM might have more to say than you can understand K. O. T. Erziev 26 0 0 28 Feb 2025
Steering Dialogue Dynamics for Robustness against Multi-turn Jailbreaking Attacks Hanjiang Hu Alexander Robey Changliu Liu AAML LLMSV 36 1 0 28 Feb 2025
UDora: A Unified Red Teaming Framework against LLM Agents by Dynamically Hijacking Their Own Reasoning J. Zhang Shuang Yang B. Li AAML LLMAG 48 0 0 28 Feb 2025
The Role of Sparsity for Length Generalization in Transformers Noah Golowich Samy Jelassi David Brandfonbrener Sham Kakade Eran Malach 32 0 0 24 Feb 2025
Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignment Pedram Zaree Md Abdullah Al Mamun Quazi Mishkatul Alam Yue Dong Ihsen Alouani Nael B. Abu-Ghazaleh AAML 33 0 0 24 Feb 2025
GuidedBench: Equipping Jailbreak Evaluation with Guidelines Ruixuan Huang Xunguang Wang Zongjie Li Daoyuan Wu Shuai Wang ALM ELM 50 0 0 24 Feb 2025
REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective Simon Geisler Tom Wollschlager M. H. I. Abdalla Vincent Cohen-Addad Johannes Gasteiger Stephan Günnemann AAML 70 2 0 24 Feb 2025
AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement Zhexin Zhang Leqi Lei Junxiao Yang Xijie Huang Yida Lu ... Xianqi Lei C. Pan Lei Sha H. Wang Minlie Huang AAML 38 0 0 24 Feb 2025
Single-pass Detection of Jailbreaking Input in Large Language Models Leyla Naz Candogan Yongtao Wu Elias Abad Rocamora Grigorios G. Chrysos V. Cevher AAML 45 0 0 24 Feb 2025
A generative approach to LLM harmfulness detection with special red flag tokens Sophie Xhonneux David Dobre Mehrnaz Mohfakhami Leo Schwinn Gauthier Gidel 43 1 0 22 Feb 2025
TurboFuzzLLM: Turbocharging Mutation-based Fuzzing for Effectively Jailbreaking Large Language Models in Practice Aman Goel Xian Carrie Wu Zhe Wang Dmitriy Bespalov Yanjun Qi 44 0 0 21 Feb 2025
Fast Proxies for LLM Robustness Evaluation Tim Beyer Jan Schuchardt Leo Schwinn Stephan Günnemann AAML 39 0 0 14 Feb 2025
FLAME: Flexible LLM-Assisted Moderation Engine Ivan Bakulin Ilia Kopanichuk Iaroslav Bespalov Nikita Radchenko V. Shaposhnikov Dmitry V. Dylov Ivan Oseledets 78 0 0 13 Feb 2025
Commercial LLM Agents Are Already Vulnerable to Simple Yet Dangerous Attacks Ang Li Yin Zhou Vethavikashini Chithrra Raghuram Tom Goldstein Micah Goldblum AAML 48 7 0 12 Feb 2025
Jailbreaking to Jailbreak Jeremy Kritz Vaughn Robinson Robert Vacareanu Bijan Varjavand Michael Choi Bobby Gogov Scale Red Team Summer Yue Willow Primack Zifan Wang 93 0 0 09 Feb 2025
Adversarial ML Problems Are Getting Harder to Solve and to Evaluate Javier Rando Jie Zhang Nicholas Carlini F. Tramèr AAML ELM 45 3 0 04 Feb 2025
Model Tampering Attacks Enable More Rigorous Evaluations of LLM Capabilities Zora Che Stephen Casper Robert Kirk Anirudh Satheesh Stewart Slocum ... Zikui Cai Bilal Chughtai Y. Gal Furong Huang Dylan Hadfield-Menell MU AAML ELM 66 2 0 03 Feb 2025
Trading Inference-Time Compute for Adversarial Robustness Wojciech Zaremba Evgenia Nitishinskaya Boaz Barak Stephanie Lin Sam Toyer ... Rachel Dias Eric Wallace Kai Y. Xiao Johannes Heidecke Amelia Glaese LRM AAML 83 15 0 31 Jan 2025
Smoothed Embeddings for Robust Language Models Ryo Hase Md. Rafi Ur Rashid Ashley Lewis Jing Liu T. Koike-Akino K. Parsons Y. Wang AAML 39 0 0 27 Jan 2025
HumorReject: Decoupling LLM Safety from Refusal Prefix via A Little Humor Zihui Wu Haichang Gao Jiacheng Luo Zhaoxiang Liu 28 0 0 23 Jan 2025
Diverse and Effective Red Teaming with Auto-generated Rewards and Multi-step Reinforcement Learning Alex Beutel Kai Y. Xiao Johannes Heidecke Lilian Weng AAML 36 3 0 24 Dec 2024
Human-Readable Adversarial Prompts: An Investigation into LLM Vulnerabilities Using Situational Context Nilanjana Das Edward Raff Manas Gaur AAML 101 1 0 20 Dec 2024
Targeting the Core: A Simple and Effective Method to Attack RAG-based Agents via Direct LLM Manipulation Xuying Li Zhuo Li Yuji Kosuga Yasuhiro Yoshida Victor Bian AAML 76 2 0 05 Dec 2024
Jailbreak Defense in a Narrow Domain: Limitations of Existing Methods and a New Transcript-Classifier Approach T. T. Wang John Hughes Henry Sleight Rylan Schaeffer Rajashree Agrawal Fazl Barez Mrinank Sharma Jesse Mu Nir Shavit Ethan Perez AAML 76 4 0 03 Dec 2024
Examining Multimodal Gender and Content Bias in ChatGPT-4o Roberto Balestri 71 2 0 28 Nov 2024
Preventing Jailbreak Prompts as Malicious Tools for Cybercriminals: A Cyber Defense Perspective Jean Marie Tshimula Xavier Ndona D'Jeff K. Nkashama Pierre Martin Tardif F. Kabanza Marc Frappier Shengrui Wang SILM 74 0 0 25 Nov 2024