Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)

20 July 2024

Papers citing "Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)"

39 / 39 papers shown

Title
RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models Bang An Shiyue Zhang Mark Dredze 54 0 0 25 Apr 2025
Steering Without Side Effects: Improving Post-Deployment Control of Language Models Asa Cooper Stickland Alexander Lyzhov Jacob Pfau Salsabila Mahdi Samuel R. Bowman LLMSV AAML 37 17 0 21 Jun 2024
SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Tinghao Xie Xiangyu Qi Yi Zeng Yangsibo Huang Udari Madhushani Sehwag ... Bo Li Kai Li Danqi Chen Peter Henderson Prateek Mittal ALM ELM 39 50 0 20 Jun 2024
The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions Eric Wallace Kai Y. Xiao R. Leike Lilian Weng Johannes Heidecke Alex Beutel SILM 47 113 0 19 Apr 2024
Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices Sara Abdali Richard Anarfi C. Barberan Jia He PILM 43 22 0 19 Mar 2024
What Was Your Prompt? A Remote Keylogging Attack on AI Assistants Roy Weiss Daniel Ayzenshteyn Guy Amit Yisroel Mirsky 37 12 0 14 Mar 2024
Here's a Free Lunch: Sanitizing Backdoored Models with Model Merge Ansh Arora Xuanli He Maximilian Mozes Srinibas Swain Mark Dras Qiongkai Xu SILM MoMe AAML 34 3 0 29 Feb 2024
Rainbow Teaming: Open-Ended Generation of Diverse Adversarial Prompts Mikayel Samvelyan Sharath Chandra Raparthy Andrei Lupu Eric Hambro Aram H. Markosyan ... Minqi Jiang Jack Parker-Holder Jakob Foerster Tim Rocktaschel Roberta Raileanu SyDa 62 61 0 26 Feb 2024
Immunization against harmful fine-tuning attacks Domenic Rosati Jan Wehner Kai Williams Lukasz Bartoszcze Jan Batzner Hassan Sajjad Frank Rudzicz AAML 34 15 0 26 Feb 2024
Semantic Mirror Jailbreak: Genetic Algorithm Based Jailbreak Prompts Against Open-source LLMs Xiaoxia Li Siyuan Liang Jiyi Zhang Hansheng Fang Aishan Liu Ee-Chien Chang 51 23 0 21 Feb 2024
Using Hallucinations to Bypass GPT4's Filter Benjamin Lemkin 25 2 0 16 Feb 2024
When "Competency" in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Complex Ciphers Divij Handa Advait Chirmule Bimal Gajera Chitta Baral Chitta Baral 31 18 0 16 Feb 2024
Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey Zhichen Dong Zhanhui Zhou Chao Yang Jing Shao Yu Qiao ELM 37 55 0 14 Feb 2024
Self-Alignment for Factuality: Mitigating Hallucinations in LLMs via Self-Evaluation Xiaoying Zhang Baolin Peng Ye Tian Jingyan Zhou Lifeng Jin Linfeng Song Haitao Mi Helen Meng HILM 28 42 0 14 Feb 2024
SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding Zhangchen Xu Fengqing Jiang Luyao Niu Jinyuan Jia Bill Yuchen Lin Radha Poovendran AAML 129 82 0 14 Feb 2024
COLD-Attack: Jailbreaking LLMs with Stealthiness and Controllability Xing-ming Guo Fangxu Yu Huan Zhang Lianhui Qin Bin Hu AAML 109 69 0 13 Feb 2024
OS-Copilot: Towards Generalist Computer Agents with Self-Improvement Zhiyong Wu Chengcheng Han Zichen Ding Zhenmin Weng Zhoumianze Liu Shunyu Yao Tao Yu Lingpeng Kong LLMAG LM&Ro 107 83 0 12 Feb 2024
Fight Back Against Jailbreaking via Prompt Adversarial Tuning Yichuan Mo Yuji Wang Zeming Wei Yisen Wang AAML SILM 41 11 0 09 Feb 2024
When Large Language Models Meet Vector Databases: A Survey Zhi Jing Yongye Su Yikun Han Bo Yuan Haiyun Xu Chunjiang Liu Kehai Chen Min Zhang 33 32 0 30 Jan 2024
Pruning for Protection: Increasing Jailbreak Resistance in Aligned LLMs Without Fine-Tuning Adib Hasan Ileana Rugina Alex Wang AAML 44 22 0 19 Jan 2024
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 127 116 0 09 Nov 2023
Robust Safety Classifier for Large Language Models: Adversarial Prompt Shield Jinhwa Kim Ali Derakhshan Ian G. Harris AAML 56 16 0 31 Oct 2023
Language Model Unalignment: Parametric Red-Teaming to Expose Hidden Harms and Biases Rishabh Bhardwaj Soujanya Poria ALM 37 14 0 22 Oct 2023
Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks Erfan Shayegani Md Abdullah Al Mamun Yu Fu Pedram Zaree Yue Dong Nael B. Abu-Ghazaleh AAML 135 139 0 16 Oct 2023
Large Language Models Can Be Good Privacy Protection Learners Yijia Xiao Yiqiao Jin Yushi Bai Yue Wu Xianjun Yang ... Xujiang Zhao Yanchi Liu Haifeng Chen Wei Wang Wei Cheng PILM 95 17 0 03 Oct 2023
GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts Jiahao Yu Xingwei Lin Zheng Yu Xinyu Xing SILM 110 292 0 19 Sep 2023
Large Language Models Understand and Can be Enhanced by Emotional Stimuli Cheng-rong Li Jindong Wang Yixuan Zhang Kaijie Zhu Wenxin Hou Jianxun Lian Fang Luo Qiang Yang Xingxu Xie LRM 53 116 0 14 Jul 2023
Sentence Embedding Leaks More Information than You Expect: Generative Embedding Inversion Attack to Recover the Whole Sentence Haoran Li Mingshi Xu Yangqiu Song 54 26 0 04 May 2023
Poisoning Language Models During Instruction Tuning Alexander Wan Eric Wallace Sheng Shen Dan Klein SILM 90 124 0 01 May 2023
Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond Jingfeng Yang Hongye Jin Ruixiang Tang Xiaotian Han Qizhang Feng Haoming Jiang Bing Yin Xia Hu LM&MA 123 593 0 26 Apr 2023
ReAct: Synergizing Reasoning and Acting in Language Models Shunyu Yao Jeffrey Zhao Dian Yu Nan Du Izhak Shafran Karthik Narasimhan Yuan Cao LLMAG ReLM LRM 208 2,413 0 06 Oct 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer Fanchao Qi Yangyi Chen Xurui Zhang Mukai Li Zhiyuan Liu Maosong Sun AAML SILM 75 171 0 14 Oct 2021
Differentially Private Fine-tuning of Language Models Da Yu Saurabh Naik A. Backurs Sivakanth Gopi Huseyin A. Inan ... Y. Lee Andre Manoel Lukas Wutschitz Sergey Yekhanin Huishuai Zhang 128 258 0 13 Oct 2021
Deduplicating Training Data Makes Language Models Better Katherine Lee Daphne Ippolito A. Nystrom Chiyuan Zhang Douglas Eck Chris Callison-Burch Nicholas Carlini SyDa 234 447 0 14 Jul 2021
Killing One Bird with Two Stones: Model Extraction and Attribute Inference Attacks against BERT-based APIs Chen Chen Xuanli He Lingjuan Lyu Fangzhao Wu SILM MIACV 33 5 0 23 May 2021
Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models Alex Tamkin Miles Brundage Jack Clark Deep Ganguli AILaw ELM 192 248 0 04 Feb 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 261 1,798 0 14 Dec 2020
Categorical Reparameterization with Gumbel-Softmax Eric Jang S. Gu Ben Poole BDL 69 5,262 0 03 Nov 2016