Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations

10 October 2023

Zeming Wei

Papers citing "Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations"

39 / 189 papers shown

Title
COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability Xing-ming Guo Fangxu Yu Huan Zhang Lianhui Qin Bin Hu AAML 109 69 0 13 Feb 2024
Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast Xiangming Gu Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Ye Wang Jing Jiang Min-Bin Lin LLMAG LM&Ro 35 47 0 13 Feb 2024
StruQ: Defending Against Prompt Injection with Structured Queries Sizhe Chen Julien Piet Chawin Sitawarin David A. Wagner SILM AAML 17 65 0 09 Feb 2024
Fight Back Against Jailbreaking via Prompt Adversarial Tuning Yichuan Mo Yuji Wang Zeming Wei Yisen Wang AAML SILM 47 11 0 09 Feb 2024
In-Context Learning Can Re-learn Forbidden Tasks Sophie Xhonneux David Dobre Jian Tang Gauthier Gidel Dhanya Sridhar 11 3 0 08 Feb 2024
Prioritizing Safeguarding Over Autonomy: Risks of LLM Agents for Science Xiangru Tang Qiao Jin Kunlun Zhu Tongxin Yuan Yichi Zhang ... Jian Tang Zhuosheng Zhang Arman Cohan Zhiyong Lu Mark B. Gerstein LLMAG ELM 17 39 0 06 Feb 2024
GUARD: Role-playing to Generate Natural-language Jailbreakings to Test Guideline Adherence of Large Language Models Haibo Jin Ruoxi Chen Andy Zhou Yang Zhang Haohan Wang LLMAG 11 19 0 05 Feb 2024
Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks Andy Zhou Bo Li Haohan Wang AAML 22 71 0 30 Jan 2024
Weak-to-Strong Jailbreaking on Large Language Models Xuandong Zhao Xianjun Yang Tianyu Pang Chao Du Lei Li Yu-Xiang Wang William Yang Wang 19 49 0 30 Jan 2024
Red-Teaming for Generative AI: Silver Bullet or Security Theater? Michael Feffer Anusha Sinha Wesley Hanwen Deng Zachary Chase Lipton Hoda Heidari AAML 25 66 0 29 Jan 2024
How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs Yi Zeng Hongpeng Lin Jingwen Zhang Diyi Yang Ruoxi Jia Weiyan Shi 10 179 0 12 Jan 2024
Intention Analysis Makes LLMs A Good Jailbreak Defender Yuqi Zhang Liang Ding Lefei Zhang Dacheng Tao LLMSV 12 15 0 12 Jan 2024
Universal Vulnerabilities in Large Language Models: Backdoor Attacks for In-context Learning Shuai Zhao Meihuizi Jia Anh Tuan Luu Fengjun Pan Jinming Wen AAML 19 34 0 11 Jan 2024
MLLM-Protector: Ensuring MLLM's Safety without Hurting Performance Renjie Pi Tianyang Han Jianshu Zhang Yueqi Xie Rui Pan Qing Lian Hanze Dong Jipeng Zhang Tong Zhang AAML 23 53 0 05 Jan 2024
The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness Neeraj Varshney Pavel Dolin Agastya Seth Chitta Baral AAML ELM 14 46 0 30 Dec 2023
Jatmo: Prompt Injection Defense by Task-Specific Finetuning Julien Piet Maha Alrashed Chawin Sitawarin Sizhe Chen Zeming Wei Elizabeth Sun Basel Alomair David A. Wagner AAML SyDa 73 50 0 29 Dec 2023
Safety Alignment in NLP Tasks: Weakly Aligned Summarization as an In-Context Attack Yu Fu Yufei Li Wen Xiao Cong Liu Yue Dong AAML 21 5 0 12 Dec 2023
A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly Yifan Yao Jinhao Duan Kaidi Xu Yuanfang Cai Eric Sun Yue Zhang PILM ELM 24 463 0 04 Dec 2023
MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Models Xin Liu Yichen Zhu Jindong Gu Yunshi Lan Chao Yang Yu Qiao 19 80 0 29 Nov 2023
How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs Haoqin Tu Chenhang Cui Zijun Wang Yiyang Zhou Bingchen Zhao Junlin Han Wangchunshu Zhou Huaxiu Yao Cihang Xie MLLM 36 70 0 27 Nov 2023
Hijacking Large Language Models via Adversarial In-Context Learning Yao Qiang Xiangyu Zhou Dongxiao Zhu 30 27 0 16 Nov 2023
Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking Nan Xu Fei Wang Ben Zhou Bangzheng Li Chaowei Xiao Muhao Chen 19 55 0 16 Nov 2023
Stealthy and Persistent Unalignment on Large Language Models via Backdoor Injections Yuanpu Cao Bochuan Cao Jinghui Chen 10 23 0 15 Nov 2023
Fake Alignment: Are LLMs Really Aligned Well? Yixu Wang Yan Teng Kexin Huang Chengqi Lyu Songyang Zhang Wenwei Zhang Xingjun Ma Yu-Gang Jiang Yu Qiao Yingchun Wang 14 14 0 10 Nov 2023
DeepInception: Hypnotize Large Language Model to Be Jailbreaker Xuan Li Zhanke Zhou Jianing Zhu Jiangchao Yao Tongliang Liu Bo Han 29 149 0 06 Nov 2023
AutoDAN: Interpretable Gradient-Based Adversarial Attacks on Large Language Models Sicheng Zhu Ruiyi Zhang Bang An Gang Wu Joe Barrow Zichao Wang Furong Huang A. Nenkova Tong Sun SILM AAML 19 40 0 23 Oct 2023
Privacy in Large Language Models: Attacks, Defenses and Future Directions Haoran Li Yulin Chen Jinglong Luo Yan Kang Xiaojin Zhang Qi Hu Chunkit Chan Yangqiu Song PILM 38 39 0 16 Oct 2023
Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM Bochuan Cao Yu Cao Lu Lin Jinghui Chen AAML 15 129 0 18 Sep 2023
Enhancing Adversarial Attacks: The Similar Target Method Shuo Zhang Ziruo Wang Zikai Zhou Huanran Chen AAML 34 1 0 21 Aug 2023
Robust Classification via a Single Diffusion Model Huanran Chen Yinpeng Dong Zhengyi Wang X. Yang Chen-Dong Duan Hang Su Jun Zhu 61 41 0 24 May 2023
$k$ NN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference Benfeng Xu Quan Wang Zhendong Mao Yajuan Lyu Qiaoqiao She Yongdong Zhang 83 52 0 24 Mar 2023
Rethinking Model Ensemble in Transfer-based Adversarial Attacks Huanran Chen Yichi Zhang Yinpeng Dong Xiao Yang Hang Su Junyi Zhu AAML 13 54 0 16 Mar 2023
The Learnability of In-Context Learning Noam Wies Yoav Levine Amnon Shashua 108 89 0 14 Mar 2023
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned Deep Ganguli Liane Lovitt John Kernion Amanda Askell Yuntao Bai ... Nicholas Joseph Sam McCandlish C. Olah Jared Kaplan Jack Clark 216 327 0 23 Aug 2022
Instruction Induction: From Few Examples to Natural Language Task Descriptions Or Honovich Uri Shaham Samuel R. Bowman Omer Levy ELM LRM 107 133 0 22 May 2022
Diffusion Models for Adversarial Purification Weili Nie Brandon Guo Yujia Huang Chaowei Xiao Arash Vahdat Anima Anandkumar WIGM 184 410 0 16 May 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Gradient-based Adversarial Attacks against Text Transformers Chuan Guo Alexandre Sablayrolles Hervé Jégou Douwe Kiela SILM 95 225 0 15 Apr 2021
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018