Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations

10 October 2023

Zeming Wei

Papers citing "Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations"

50 / 189 papers shown

Title
One Trigger Token Is Enough: A Defense Strategy for Balancing Safety and Usability in Large Language Models Haoran Gu Handing Wang Yi Mei Mengjie Zhang Yaochu Jin 11 0 0 12 May 2025
Cannot See the Forest for the Trees: Invoking Heuristics and Biases to Elicit Irrational Choices of LLMs Haoming Yang Ke Ma X. Jia Yingfei Sun Qianqian Xu Q. Huang AAML 45 0 0 03 May 2025
Transferable Adversarial Attacks on Black-Box Vision-Language Models Kai Hu Weichen Yu L. Zhang Alexander Robey Andy Zou Chengming Xu Haoqi Hu Matt Fredrikson AAML VLM 47 0 0 02 May 2025
Steering the CensorShip: Uncovering Representation Vectors for LLM "Thought" Control Hannah Cyberey David E. Evans LLMSV 69 0 0 23 Apr 2025
WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks Ivan Evtimov Arman Zharmagambetov Aaron Grattafiori Chuan Guo Kamalika Chaudhuri AAML 30 0 0 22 Apr 2025
DETAM: Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification Yu Li Han Jiang Zhihua Wei AAML 29 0 0 18 Apr 2025
AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defender Weixiang Zhao Jiahe Guo Yulin Hu Yang Deng An Zhang ... Xinyang Han Yanyan Zhao Bing Qin Tat-Seng Chua Ting Liu AAML LLMSV 37 0 0 13 Apr 2025
X-Guard: Multilingual Guard Agent for Content Moderation Bibek Upadhayay Vahid Behzadan Ph.D 23 1 0 11 Apr 2025
A Domain-Based Taxonomy of Jailbreak Vulnerabilities in Large Language Models Carlos Peláez-González Andrés Herrera-Poyatos Cristina Zuheros David Herrera-Poyatos Virilo Tejedor F. Herrera AAML 19 0 0 07 Apr 2025
Retrieval-Augmented Purifier for Robust LLM-Empowered Recommendation Liangbo Ning Wenqi Fan Qing Li AAML 33 1 0 03 Apr 2025
LightDefense: A Lightweight Uncertainty-Driven Defense against Jailbreaks via Shifted Token Distribution Zhuoran Yang Jie Peng Zhen Tan Tianlong Chen Yanyong Zhang AAML 41 0 0 02 Apr 2025
Evolving Security in LLMs: A Study of Jailbreak Attacks and Defenses Zhengchun Shang Wenlan Wei AAML 38 0 0 02 Apr 2025
Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy Joonhyun Jeong Seyun Bae Yeonsung Jung Jaeryong Hwang Eunho Yang AAML 43 0 0 26 Mar 2025
FLEX: A Benchmark for Evaluating Robustness of Fairness in Large Language Models Dahyun Jung Seungyoon Lee Hyeonseok Moon Chanjun Park Heuiseok Lim AAML ALM ELM 48 0 0 25 Mar 2025
STShield: Single-Token Sentinel for Real-Time Jailbreak Detection in Large Language Models Xunguang Wang Wenxuan Wang Zhenlan Ji Zongjie Li Pingchuan Ma Daoyuan Wu Shuai Wang 48 0 0 23 Mar 2025
AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integration Andy Zhou Kevin E. Wu Francesco Pinto Z. Chen Yi Zeng Yu Yang Shuang Yang Sanmi Koyejo James Zou Bo Li LLMAG AAML 68 0 0 20 Mar 2025
Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents Juhee Kim Woohyuk Choi Byoungyoung Lee LLMAG 74 1 0 17 Mar 2025
Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-tuning Yiwei Chen Yuguang Yao Yihua Zhang Bingquan Shen Gaowen Liu Sijia Liu AAML MU 54 1 0 14 Mar 2025
Dialogue Injection Attack: Jailbreaking LLMs through Context Manipulation Wenlong Meng Fan Zhang Wendao Yao Zhenyuan Guo Y. Li Chengkun Wei Wenzhi Chen AAML 36 1 0 11 Mar 2025
Adversarial Training for Multimodal Large Language Models against Jailbreak Attacks Liming Lu Shuchao Pang Siyuan Liang Haotian Zhu Xiyu Zeng Aishan Liu Yunhuai Liu Yongbin Zhou AAML 49 1 0 05 Mar 2025
Foot-In-The-Door: A Multi-turn Jailbreak for LLMs Zixuan Weng Xiaolong Jin Jinyuan Jia X. Zhang AAML 43 0 0 27 Feb 2025
Beyond Surface-Level Patterns: An Essence-Driven Defense Framework Against Jailbreak Attacks in LLMs Shiyu Xiang Ansen Zhang Yanfei Cao Yang Fan Ronghao Chen AAML 55 0 0 26 Feb 2025
On the Robustness of Transformers against Context Hijacking for Linear Classification Tianle Li Chenyang Zhang Xingwu Chen Yuan Cao Difan Zou 62 0 0 24 Feb 2025
GuidedBench: Equipping Jailbreak Evaluation with Guidelines Ruixuan Huang Xunguang Wang Zongjie Li Daoyuan Wu Shuai Wang ALM ELM 53 0 0 24 Feb 2025
Attention Eclipse: Manipulating Attention to Bypass LLM Safety-Alignment Pedram Zaree Md Abdullah Al Mamun Quazi Mishkatul Alam Yue Dong Ihsen Alouani Nael B. Abu-Ghazaleh AAML 38 0 0 24 Feb 2025
AISafetyLab: A Comprehensive Framework for AI Safety Evaluation and Improvement Zhexin Zhang Leqi Lei Junxiao Yang Xijie Huang Yida Lu ... Xianqi Lei C. Pan Lei Sha H. Wang Minlie Huang AAML 43 0 0 24 Feb 2025
SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention Jiaqi Wu Chen Chen Chunyan Hou Xiaojie Yuan AAML 54 0 0 24 Feb 2025
Making Them a Malicious Database: Exploiting Query Code to Jailbreak Aligned Large Language Models Qingsong Zou Jingyu Xiao Qing Li Zhi Yan Y. Wang Li Xu Wenxuan Wang Kuofeng Gao Ruoyu Li Yong-jia Jiang AAML 79 0 0 21 Feb 2025
A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos Yang Yao Xuan Tong Ruofan Wang Yixu Wang Lujundong Li Liang Liu Yan Teng Y. Wang LRM 38 2 0 19 Feb 2025
SafeEraser: Enhancing Safety in Multimodal Large Language Models through Multimodal Machine Unlearning Junkai Chen Zhijie Deng Kening Zheng Yibo Yan Shuliang Liu PeiJun Wu Peijie Jiang J. Liu Xuming Hu MU 46 3 0 18 Feb 2025
When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search Xuan Chen Yuzhou Nie Wenbo Guo Xiangyu Zhang 101 9 0 28 Jan 2025
Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment Melissa Kazemi Rad Huy Nghiem Andy Luo Sahil Wadhwa Mohammad Sorower Stephen Rawls AAML 83 2 0 22 Jan 2025
You Can't Eat Your Cake and Have It Too: The Performance Degradation of LLMs with Jailbreak Defense Wuyuao Mai Geng Hong Pei Chen Xudong Pan Baojun Liu Y. Zhang Haixin Duan Min Yang AAML 63 1 0 21 Jan 2025
Episodic memory in AI agents poses risks that should be studied and mitigated Chad DeChant 53 1 0 20 Jan 2025
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Kaifeng Lyu Haoyu Zhao Xinran Gu Dingli Yu Anirudh Goyal Sanjeev Arora ALM 73 41 0 20 Jan 2025
LLM-Virus: Evolutionary Jailbreak Attack on Large Language Models Miao Yu Junfeng Fang Yingjie Zhou Xing Fan Kun Wang Shirui Pan Qingsong Wen AAML 56 0 0 03 Jan 2025
No Free Lunch for Defending Against Prefilling Attack by In-Context Learning Zhiyu Xue Guangliang Liu Bocheng Chen K. Johnson Ramtin Pedarsani AAML 68 0 0 13 Dec 2024
FlexLLM: Exploring LLM Customization for Moving Target Defense on Black-Box LLMs Against Jailbreak Attacks Bocheng Chen Hanqing Guo Qiben Yan AAML 63 0 0 10 Dec 2024
DROJ: A Prompt-Driven Attack against Large Language Models Leyang Hu Boran Wang 19 0 0 14 Nov 2024
Exploring Response Uncertainty in MLLMs: An Empirical Evaluation under Misleading Scenarios Yunkai Dang Mengxi Gao Yibo Yan Xin Zou Yanggan Gu Aiwei Liu Xuming Hu 37 4 0 05 Nov 2024
SQL Injection Jailbreak: A Structural Disaster of Large Language Models Jiawei Zhao Kejiang Chen W. Zhang Nenghai Yu AAML 36 0 0 03 Nov 2024
Defense Against Prompt Injection Attack by Leveraging Attack Techniques Yulin Chen Haoran Li Zihao Zheng Y. Song Dekai Wu Bryan Hooi SILM AAML 47 4 0 01 Nov 2024
What is Wrong with Perplexity for Long-context Language Modeling? Lizhe Fang Yifei Wang Zhaoyang Liu Chenheng Zhang Stefanie Jegelka Jinyang Gao Bolin Ding Yisen Wang 47 4 0 31 Oct 2024
AmpleGCG-Plus: A Strong Generative Model of Adversarial Suffixes to Jailbreak LLMs with Higher Success Rates in Fewer Attempts Vishal Kumar Zeyi Liao Jaylen Jones Huan Sun AAML 23 1 0 29 Oct 2024
Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring Honglin Mu Han He Yuxin Zhou Yunlong Feng Yang Xu ... Zeming Liu Xudong Han Qi Shi Qingfu Zhu Wanxiang Che AAML 23 1 0 28 Oct 2024
Dynamic Guided and Domain Applicable Safeguards for Enhanced Security in Large Language Models He Cao Weidi Luo Zijing Liu Yu Wang Bing Feng Yuan Yao Yuan Yao Yu Li AAML 45 2 0 23 Oct 2024
LLMScan: Causal Scan for LLM Misbehavior Detection Mengdi Zhang Kai Kiat Goh Peixin Zhang Jun Sun 13 0 0 22 Oct 2024
Bayesian scaling laws for in-context learning Aryaman Arora Dan Jurafsky Christopher Potts Noah D. Goodman 19 2 0 21 Oct 2024
SMILES-Prompting: A Novel Approach to LLM Jailbreak Attacks in Chemical Synthesis Aidan Wong He Cao Zijing Liu Yu Li 31 2 0 21 Oct 2024
Faster-GCG: Efficient Discrete Optimization Jailbreak Attacks against Aligned Large Language Models Xiao-Li Li Zhuhong Li Qiongxiu Li Bingze Lee Jinghao Cui Xiaolin Hu AAML 17 2 0 20 Oct 2024