StruQ: Defending Against Prompt Injection with Structured Queries

9 February 2024

Papers citing "StruQ: Defending Against Prompt Injection with Structured Queries"

50 / 50 papers shown

Title
AgentXploit: End-to-End Redteaming of Black-Box AI Agents Zhun Wang Vincent Siu Zhe Ye Tianneng Shi Yuzhou Nie Xuandong Zhao Chenguang Wang Wenbo Guo Dawn Song LLMAG AAML 27 0 0 09 May 2025
Defending against Indirect Prompt Injection by Instruction Detection Tongyu Wen Chenglong Wang Xiyuan Yang Haoyu Tang Yueqi Xie Lingjuan Lyu Zhicheng Dou Fangzhao Wu AAML 12 0 0 08 May 2025
The Illusion of Role Separation: Hidden Shortcuts in LLM Role Learning (and How to Fix Them) Zihao Wang Yibo Jiang Jiahao Yu Heqing Huang 28 0 0 01 May 2025
Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instruction Y. Chen Haoran Li Yuan Sui Y. Liu Yufei He Y. Song Bryan Hooi AAML SILM 61 0 0 29 Apr 2025
ACE: A Security Architecture for LLM-Integrated App Systems Evan Li Tushin Mallick Evan Rose William K. Robertson Alina Oprea Cristina Nita-Rotaru 47 0 0 29 Apr 2025
Prompt Injection Attack to Tool Selection in LLM Agents Jiawen Shi Zenghui Yuan Guiyao Tie Pan Zhou Neil Zhenqiang Gong Lichao Sun LLMAG 51 0 0 28 Apr 2025
Adversarial Attacks on LLM-as-a-Judge Systems: Insights from Prompt Injections Narek Maloyan Dmitry Namiot SILM AAML ELM 70 0 0 25 Apr 2025
WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks Ivan Evtimov Arman Zharmagambetov Aaron Grattafiori Chuan Guo Kamalika Chaudhuri AAML 30 0 0 22 Apr 2025
Exploring the Role of Large Language Models in Cybersecurity: A Systematic Survey Shuang Tian Tao Zhang J. Liu Jiacheng Wang Xuangou Wu ... Ruichen Zhang W. Zhang Zhenhui Yuan Shiwen Mao Dong In Kim 48 0 0 22 Apr 2025
Progent: Programmable Privilege Control for LLM Agents Tianneng Shi Jingxuan He Zhun Wang Linyu Wu Hongwei Li Wenbo Guo Dawn Song LLMAG 31 0 0 16 Apr 2025
DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks Yupei Liu Yuqi Jia Jinyuan Jia Dawn Song Neil Zhenqiang Gong AAML 31 0 0 15 Apr 2025
Separator Injection Attack: Uncovering Dialogue Biases in Large Language Models Caused by Role Separators Xitao Li H. Wang Jiang Wu Ting Liu AAML 23 0 0 08 Apr 2025
Frontier AI's Impact on the Cybersecurity Landscape Wenbo Guo Yujin Potter Tianneng Shi Zhun Wang Andy Zhang Dawn Song 31 1 0 07 Apr 2025
Exploiting Fine-Grained Skip Behaviors for Micro-Video Recommendation Sanghyuck Lee Sangkeun Park Jaesung Lee 43 0 0 04 Apr 2025
Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents Juhee Kim Woohyuk Choi Byoungyoung Lee LLMAG 74 1 0 17 Mar 2025
Multi-Agent Systems Execute Arbitrary Malicious Code Harold Triedman Rishi Jha Vitaly Shmatikov LLMAG AAML 86 2 0 15 Mar 2025
ASIDE: Architectural Separation of Instructions and Data in Language Models Egor Zverev Evgenii Kortukov Alexander Panfilov Soroush Tabesh Alexandra Volkova Sebastian Lapuschkin Wojciech Samek Christoph H. Lampert AAML 52 1 0 13 Mar 2025
Can Indirect Prompt Injection Attacks Be Detected and Removed? Yulin Chen Haoran Li Yuan Sui Yufei He Yue Liu Y. Song Bryan Hooi AAML 34 3 0 23 Feb 2025
Control Illusion: The Failure of Instruction Hierarchies in Large Language Models Yilin Geng H. Li Honglin Mu Xudong Han Timothy Baldwin Omri Abend Eduard H. Hovy Lea Frermann 31 2 0 21 Feb 2025
UniGuardian: A Unified Defense for Detecting Prompt Injection, Backdoor Attacks and Adversarial Attacks in Large Language Models Huawei Lin Yingjie Lao Tong Geng Tan Yu Weijie Zhao AAML SILM 73 2 0 18 Feb 2025
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates Kaifeng Lyu Haoyu Zhao Xinran Gu Dingli Yu Anirudh Goyal Sanjeev Arora ALM 73 41 0 20 Jan 2025
The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agents Feiran Jia Tong Wu Xin Qin Anna Squicciarini LLMAG AAML 80 4 0 21 Dec 2024
RAG-Thief: Scalable Extraction of Private Data from Retrieval-Augmented Generation Applications with Agent-based Attacks Changyue Jiang Xudong Pan Geng Hong Chenfu Bao Min Yang SILM 69 7 0 21 Nov 2024
Defense Against Prompt Injection Attack by Leveraging Attack Techniques Yulin Chen Haoran Li Zihao Zheng Y. Song Dekai Wu Bryan Hooi SILM AAML 39 4 0 01 Nov 2024
Attention Tracker: Detecting Prompt Injection Attacks in LLMs Kuo-Han Hung Ching-Yun Ko Ambrish Rawat I-Hsin Chung Winston H. Hsu Pin-Yu Chen 41 7 0 01 Nov 2024
InjecGuard: Benchmarking and Mitigating Over-defense in Prompt Injection Guardrail Models H. Li Xiaogeng Liu SILM 24 4 0 30 Oct 2024
FATH: Authentication-based Test-time Defense against Indirect Prompt Injection Attacks Jiongxiao Wang Fangzhou Wu Wendi Li Jinsheng Pan Edward Suh Zhuoqing Mao Muhao Chen Chaowei Xiao AAML 26 6 0 28 Oct 2024
Prompt Infection: LLM-to-LLM Prompt Injection within Multi-Agent Systems Donghyun Lee Mo Tiwari LLMAG 18 8 0 09 Oct 2024
Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy Tong Wu Shujian Zhang Kaiqiang Song Silei Xu Sanqiang Zhao Ravi Agrawal Sathish Indurthi Chong Xiang Prateek Mittal Wenxuan Zhou 16 7 0 09 Oct 2024
VLMGuard: Defending VLMs against Malicious Prompts via Unlabeled Data Xuefeng Du Reshmi Ghosh Robert Sim Ahmed Salem Vitor Carvalho Emily Lawton Yixuan Li Jack W. Stokes VLM AAML 29 5 0 01 Oct 2024
PROMPTFUZZ: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMs Jiahao Yu Yangguang Shao Hanwen Miao Junzheng Shi SILM AAML 53 3 0 23 Sep 2024
Soft Begging: Modular and Efficient Shielding of LLMs against Prompt Injection and Jailbreaking based on Prompt Tuning Simon Ostermann Kevin Baum Christoph Endres Julia Masloh P. Schramowski AAML 28 1 0 03 Jul 2024
Adversarial Search Engine Optimization for Large Language Models Fredrik Nestaas Edoardo Debenedetti Florian Tramèr AAML 18 4 0 26 Jun 2024
AgentDojo: A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents Edoardo Debenedetti Jie Zhang Mislav Balunović Luca Beurer-Kellner Marc Fischer Florian Tramèr LLMAG AAML 37 25 1 19 Jun 2024
Security of AI Agents Yifeng He Ethan Wang Yuyang Rong Zifei Cheng Hao Chen LLMAG 21 7 0 12 Jun 2024
Machine Against the RAG: Jamming Retrieval-Augmented Generation with Blocker Documents Avital Shafran R. Schuster Vitaly Shmatikov 29 27 0 09 Jun 2024
A Survey of Language-Based Communication in Robotics William Hunt Sarvapali D. Ramchurn Mohammad D. Soorati LM&Ro 38 11 0 06 Jun 2024
Ranking Manipulation for Conversational Search Engines Samuel Pfrommer Yatong Bai Tanmay Gautam Somayeh Sojoudi SILM 28 4 0 05 Jun 2024
Voice Jailbreak Attacks Against GPT-4o Xinyue Shen Yixin Wu Michael Backes Yang Zhang AuLLM 18 9 0 29 May 2024
Lockpicking LLMs: A Logit-Based Jailbreak Using Token-level Manipulation Yuxi Li Yi Liu Yuekang Li Ling Shi Gelei Deng Shengquan Chen Kailong Wang 24 12 0 20 May 2024
The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions Eric Wallace Kai Y. Xiao R. Leike Lilian Weng Johannes Heidecke Alex Beutel SILM 47 113 0 19 Apr 2024
GoEX: Perspectives and Designs Towards a Runtime for Autonomous LLM Applications Shishir G. Patil Tianjun Zhang Vivian Fang Noppapon C Roy Huang Uc Berkeley Aaron Hao Martin Casado Joseph E. Gonzalez Raluca Ada Popa Ion Stoica ALM 21 9 0 10 Apr 2024
Optimization-based Prompt Injection Attack to LLM-as-a-Judge Jiawen Shi Zenghui Yuan Yinuo Liu Yue Huang Pan Zhou Lichao Sun Neil Zhenqiang Gong AAML 23 38 0 26 Mar 2024
Can LLMs Separate Instructions From Data? And What Do We Even Mean By That? Egor Zverev Sahar Abdelnabi Soroush Tabesh Mario Fritz Christoph H. Lampert 38 19 0 11 Mar 2024
InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents Qiusi Zhan Zhixiang Liang Zifan Ying Daniel Kang LLMAG 42 72 0 05 Mar 2024
Jatmo: Prompt Injection Defense by Task-Specific Finetuning Julien Piet Maha Alrashed Chawin Sitawarin Sizhe Chen Zeming Wei Elizabeth Sun Basel Alomair David A. Wagner AAML SyDa 70 50 0 29 Dec 2023
Formalizing and Benchmarking Prompt Injection Attacks and Defenses Yupei Liu Yuqi Jia Runpeng Geng Jinyuan Jia Neil Zhenqiang Gong SILM LLMAG 16 57 0 19 Oct 2023
Privacy in Large Language Models: Attacks, Defenses and Future Directions Haoran Li Yulin Chen Jinglong Luo Yan Kang Xiaojin Zhang Qi Hu Chunkit Chan Yangqiu Song PILM 30 39 0 16 Oct 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
Extracting Training Data from Large Language Models Nicholas Carlini Florian Tramèr Eric Wallace Matthew Jagielski Ariel Herbert-Voss ... Tom B. Brown D. Song Ulfar Erlingsson Alina Oprea Colin Raffel MLAU SILM 264 1,798 0 14 Dec 2020