Automatically Auditing Large Language Models via Discrete Optimization

8 March 2023

Papers citing "Automatically Auditing Large Language Models via Discrete Optimization"

36 / 36 papers shown

Title
Adaptive Stress Testing Black-Box LLM Planners Neeloy Chakraborty John Pohovey Melkior Ornik Katherine Driggs-Campbell 26 0 0 08 May 2025
JailbreaksOverTime: Detecting Jailbreak Attacks Under Distribution Shift Julien Piet Xiao Huang Dennis Jacob Annabella Chow Maha Alrashed Geng Zhao Zhanhao Hu Chawin Sitawarin Basel Alomair David A. Wagner AAML 63 0 0 28 Apr 2025
Graph of Attacks: Improved Black-Box and Interpretable Jailbreaks for LLMs Mohammad Akbar-Tajari Mohammad Taher Pilehvar Mohammad Mahmoody AAML 46 0 0 26 Apr 2025
Foot-In-The-Door: A Multi-turn Jailbreak for LLMs Zixuan Weng Xiaolong Jin Jinyuan Jia X. Zhang AAML 99 0 0 27 Feb 2025
Shh, don't say that! Domain Certification in LLMs Cornelius Emde Alasdair Paren Preetham Arvind Maxime Kayser Tom Rainforth Thomas Lukasiewicz Bernard Ghanem Philip H. S. Torr Adel Bibi 45 1 0 26 Feb 2025
Making Them a Malicious Database: Exploiting Query Code to Jailbreak Aligned Large Language Models Qingsong Zou Jingyu Xiao Qing Li Zhi Yan Y. Wang Li Xu Wenxuan Wang Kuofeng Gao Ruoyu Li Yong-jia Jiang AAML 141 0 0 21 Feb 2025
Universal Adversarial Attack on Aligned Multimodal LLMs Temurbek Rahmatullaev Polina Druzhinina Matvey Mikhalchuk Andrey Kuznetsov Anton Razzhigaev AAML 98 0 0 11 Feb 2025
Refining Input Guardrails: Enhancing LLM-as-a-Judge Efficiency Through Chain-of-Thought Fine-Tuning and Alignment Melissa Kazemi Rad Huy Nghiem Andy Luo Sahil Wadhwa Mohammad Sorower Stephen Rawls AAML 91 2 0 22 Jan 2025
Differentiable Adversarial Attacks for Marked Temporal Point Processes Pritish Chakraborty Vinayak Gupta R. Raj Srikanta J. Bedathur A. De AAML 134 0 0 17 Jan 2025
Dynamics of Adversarial Attacks on Large Language Model-Based Search Engines Xiyang Hu AAML 31 1 0 03 Jan 2025
Human-Readable Adversarial Prompts: An Investigation into LLM Vulnerabilities Using Situational Context Nilanjana Das Edward Raff Manas Gaur AAML 101 1 0 20 Dec 2024
Diversity Helps Jailbreak Large Language Models Weiliang Zhao Daniel Ben-Levi Wei Hao Junfeng Yang Chengzhi Mao AAML 105 0 0 06 Nov 2024
Deciphering the Chaos: Enhancing Jailbreak Attacks via Adversarial Prompt Translation Qizhang Li Xiaochen Yang W. Zuo Yiwen Guo AAML 61 0 0 15 Oct 2024
RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Process Peiran Wang Xiaogeng Liu Chaowei Xiao AAML 29 3 0 11 Oct 2024
Cheating Automatic LLM Benchmarks: Null Models Achieve High Win Rates Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Jing Jiang Min-Bin Lin 33 8 0 09 Oct 2024
Non-Halting Queries: Exploiting Fixed Points in LLMs Ghaith Hammouri Kemal Derya B. Sunar 28 0 0 08 Oct 2024
Human-Interpretable Adversarial Prompt Attack on Large Language Models with Situational Context Nilanjana Das Edward Raff Manas Gaur AAML 35 2 0 19 Jul 2024
Logicbreaks: A Framework for Understanding Subversion of Rule-based Inference Anton Xue Avishree Khare Rajeev Alur Surbhi Goel Eric Wong 48 2 0 21 Jun 2024
Merging Improves Self-Critique Against Jailbreak Attacks Victor Gallego AAML MoMe 36 3 0 11 Jun 2024
Ranking Manipulation for Conversational Search Engines Samuel Pfrommer Yatong Bai Tanmay Gautam Somayeh Sojoudi SILM 39 4 0 05 Jun 2024
Navigating LLM Ethics: Advancements, Challenges, and Future Directions Junfeng Jiao S. Afroogh Yiming Xu Connor Phillips AILaw 60 19 0 14 May 2024
Optimization-based Prompt Injection Attack to LLM-as-a-Judge Jiawen Shi Zenghui Yuan Yinuo Liu Yue Huang Pan Zhou Lichao Sun Neil Zhenqiang Gong AAML 43 39 0 26 Mar 2024
Alpaca against Vicuna: Using LLMs to Uncover Memorization of LLMs Aly M. Kassem Omar Mahmoud Niloofar Mireshghallah Hyunwoo J. Kim Yulia Tsvetkov Yejin Choi Sherif Saad Santu Rana 47 18 0 05 Mar 2024
ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs Fengqing Jiang Zhangchen Xu Luyao Niu Zhen Xiang Bhaskar Ramasubramanian Bo Li Radha Poovendran 26 86 0 19 Feb 2024
Gradient-Based Language Model Red Teaming Nevan Wichers Carson E. Denison Ahmad Beirami 8 25 0 30 Jan 2024
Black-Box Access is Insufficient for Rigorous AI Audits Stephen Casper Carson Ezell Charlotte Siegmann Noam Kolt Taylor Lynn Curtis ... Michael Gerovitch David Bau Max Tegmark David M. Krueger Dylan Hadfield-Menell AAML 13 76 0 25 Jan 2024
Stability Analysis of ChatGPT-based Sentiment Analysis in AI Quality Assurance Tinghui Ouyang AprilPyone Maungmaung Koichi Konishi Yoshiki Seo Isao Echizen AI4MH 21 5 0 15 Jan 2024
Analyzing the Inherent Response Tendency of LLMs: Real-World Instructions-Driven Jailbreak Yanrui Du Sendong Zhao Ming Ma Yuhan Chen Bing Qin 26 15 0 07 Dec 2023
Universal Jailbreak Backdoors from Poisoned Human Feedback Javier Rando Florian Tramèr 13 60 0 24 Nov 2023
GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher Youliang Yuan Wenxiang Jiao Wenxuan Wang Jen-tse Huang Pinjia He Shuming Shi Zhaopeng Tu SILM 61 231 0 12 Aug 2023
Jailbroken: How Does LLM Safety Training Fail? Alexander Wei Nika Haghtalab Jacob Steinhardt 50 829 0 05 Jul 2023
Diffusion-LM Improves Controllable Text Generation Xiang Lisa Li John Thickstun Ishaan Gulrajani Percy Liang Tatsunori B. Hashimoto AI4CE 171 772 0 27 May 2022
Gradient-based Adversarial Attacks against Text Transformers Chuan Guo Alexandre Sablayrolles Hervé Jégou Douwe Kiela SILM 98 227 0 15 Apr 2021
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 267 1,808 0 14 Dec 2020
The Woman Worked as a Babysitter: On Biases in Language Generation Emily Sheng Kai-Wei Chang Premkumar Natarajan Nanyun Peng 206 616 0 03 Sep 2019
Generating Natural Language Adversarial Examples M. Alzantot Yash Sharma Ahmed Elgohary Bo-Jhang Ho Mani B. Srivastava Kai-Wei Chang AAML 243 914 0 21 Apr 2018