Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection

23 February 2023

Sahar Abdelnabi

Mario Fritz

Papers citing "Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection"

50 / 288 papers shown

Title
LM-Scout: Analyzing the Security of Language Model Integration in Android Apps Muhammad Ibrahim Gűliz Seray Tuncay Z. Berkay Celik Aravind Machiry Antonio Bianchi 26 0 0 13 May 2025
Securing RAG: A Risk Assessment and Mitigation Framework Lukas Ammann Sara Ott Christoph R. Landolt Marco P. Lehmann SILM 14 0 0 13 May 2025
GRADA: Graph-based Reranker against Adversarial Documents Attack Jingjie Zheng Aryo Pradipta Gema Giwon Hong Xuanli He Pasquale Minervini Youcheng Sun Qiongkai Xu 16 0 0 12 May 2025
SecReEvalBench: A Multi-turned Security Resilience Evaluation Benchmark for Large Language Models Huining Cui Wei Liu AAML ELM 23 0 0 12 May 2025
Practical Reasoning Interruption Attacks on Reasoning Large Language Models Yu Cui Cong Zuo SILM AAML LRM 24 0 0 10 May 2025
System Prompt Poisoning: Persistent Attacks on Large Language Models Beyond User Injection Jiawei Guo Haipeng Cai SILM AAML 18 0 0 10 May 2025
Threat Modeling for AI: The Case for an Asset-Centric Approach Jose Sanchez Vicarte Marcin Spoczynski Mostafa Elsaid 14 0 0 08 May 2025
Defending against Indirect Prompt Injection by Instruction Detection Tongyu Wen Chenglong Wang Xiyuan Yang Haoyu Tang Yueqi Xie Lingjuan Lyu Zhicheng Dou Fangzhao Wu AAML 24 0 0 08 May 2025
Safeguard-by-Development: A Privacy-Enhanced Development Paradigm for Multi-Agent Collaboration Systems Jian Cui Zichuan Li Luyi Xing Xiaojing Liao 17 0 0 07 May 2025
Unveiling the Landscape of LLM Deployment in the Wild: An Empirical Study Xinyi Hou Jiahao Han Yanjie Zhao Haoyu Wang 32 0 0 05 May 2025
A Survey on Progress in LLM Alignment from the Perspective of Reward Design Miaomiao Ji Yanqiu Wu Zhibin Wu Shoujin Wang Jian Yang Mark Dras Usman Naseem 37 0 0 05 May 2025
Attack and defense techniques in large language models: A survey and new perspectives Zhiyu Liao Kang Chen Yuanguo Lin Kangkang Li Yunxuan Liu Hefeng Chen Xingwang Huang Yuanhui Yu AAML 54 0 0 02 May 2025
LLM Security: Vulnerabilities, Attacks, Defenses, and Countermeasures Francisco Aguilera-Martínez Fernando Berzal PILM 48 0 0 02 May 2025
Generative AI in Financial Institution: A Global Survey of Opportunities, Threats, and Regulation Bikash Saha Nanda Rani Sandeep K. Shukla 91 0 0 30 Apr 2025
Traceback of Poisoning Attacks to Retrieval-Augmented Generation Baolei Zhang Haoran Xin Minghong Fang Zhuqing Liu Biao Yi Tong Li Zheli Liu SILM AAML 62 0 0 30 Apr 2025
ACE: A Security Architecture for LLM-Integrated App Systems Evan Li Tushin Mallick Evan Rose William K. Robertson Alina Oprea Cristina Nita-Rotaru 52 0 0 29 Apr 2025
Robustness via Referencing: Defending against Prompt Injection Attacks by Referencing the Executed Instruction Y. Chen Haoran Li Yuan Sui Y. Liu Yufei He Y. Song Bryan Hooi AAML SILM 61 0 0 29 Apr 2025
CachePrune: Neural-Based Attribution Defense Against Indirect Prompt Injection Attacks Rui Wang Junda Wu Yu Xia Tong Yu R. Zhang Ryan Rossi Lina Yao Julian McAuley AAML SILM 46 0 0 29 Apr 2025
Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in Large Language Models against Reference Corruption Wenxiao Wang Parsa Hosseini S. Feizi LRM AI4CE 53 0 0 29 Apr 2025
Prompt Injection Attack to Tool Selection in LLM Agents Jiawen Shi Zenghui Yuan Guiyao Tie Pan Zhou Neil Zhenqiang Gong Lichao Sun LLMAG 51 0 0 28 Apr 2025
Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling Ishan Kavathekar Raghav Donakanti Ponnurangam Kumaraguru Karthik Vaidhyanathan 52 0 0 27 Apr 2025
RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models Bang An Shiyue Zhang Mark Dredze 54 0 0 25 Apr 2025
Information Leakage of Sentence Embeddings via Generative Embedding Inversion Attacks Antonios Tragoudaras Theofanis Aslanidis Emmanouil Georgios Lionis Marina Orozco González Panagiotis Eustratiadis MIACV SILM 54 0 0 23 Apr 2025
WASP: Benchmarking Web Agent Security Against Prompt Injection Attacks Ivan Evtimov Arman Zharmagambetov Aaron Grattafiori Chuan Guo Kamalika Chaudhuri AAML 33 0 0 22 Apr 2025
Jailbreak Detection in Clinical Training LLMs Using Feature-Based Predictive Models Tri Nguyen Lohith Srikanth Pentapalli Magnus Sieverding Laurah Turner Seth Overla ... Michael Gharib Matt Kelleher Michael Shukis Cameron Pawlik Kelly Cohen 51 0 0 21 Apr 2025
Manipulating Multimodal Agents via Cross-Modal Prompt Injection Le Wang Zonghao Ying Tianyuan Zhang Siyuan Liang Shengshan Hu Mingchuan Zhang A. Liu Xianglong Liu AAML 31 1 0 19 Apr 2025
Progent: Programmable Privilege Control for LLM Agents Tianneng Shi Jingxuan He Zhun Wang Linyu Wu Hongwei Li Wenbo Guo Dawn Song LLMAG 34 0 0 16 Apr 2025
DataSentinel: A Game-Theoretic Detection of Prompt Injection Attacks Yupei Liu Yuqi Jia Jinyuan Jia Dawn Song Neil Zhenqiang Gong AAML 34 0 0 15 Apr 2025
StruPhantom: Evolutionary Injection Attacks on Black-Box Tabular Agents Powered by Large Language Models Yang Feng Xudong Pan AAML 31 0 0 14 Apr 2025
You've Changed: Detecting Modification of Black-Box Large Language Models Alden Dima James R. Foulds Shimei Pan Philip G. Feldman 30 0 0 14 Apr 2025
AttentionDefense: Leveraging System Prompt Attention for Explainable Defense Against Novel Jailbreaks Charlotte Siska Anush Sankaran AAML 45 0 0 10 Apr 2025
Separator Injection Attack: Uncovering Dialogue Biases in Large Language Models Caused by Role Separators Xitao Li H. Wang Jiang Wu Ting Liu AAML 26 0 0 08 Apr 2025
Frontier AI's Impact on the Cybersecurity Landscape Wenbo Guo Yujin Potter Tianneng Shi Zhun Wang Andy Zhang Dawn Song 52 1 0 07 Apr 2025
Practical Poisoning Attacks against Retrieval-Augmented Generation Baolei Zhang Y. Chen Minghong Fang Zhuqing Liu Lihai Nie Tong Li Zheli Liu SILM AAML 57 0 0 04 Apr 2025
Exploiting Fine-Grained Skip Behaviors for Micro-Video Recommendation Sanghyuck Lee Sangkeun Park Jaesung Lee 48 0 0 04 Apr 2025
The H-Elena Trojan Virus to Infect Model Weights: A Wake-Up Call on the Security Risks of Malicious Fine-Tuning Virilo Tejedor Cristina Zuheros Carlos Peláez-González David Herrera-Poyatos Andrés Herrera-Poyatos F. Herrera 24 0 0 04 Apr 2025
Evolving Security in LLMs: A Study of Jailbreak Attacks and Defenses Zhengchun Shang Wenlan Wei AAML 38 0 0 02 Apr 2025
Large Language Models are Unreliable for Cyber Threat Intelligence Emanuele Mezzi Fabio Massacci Katja Tuma 31 0 0 29 Mar 2025
Encrypted Prompt: Securing LLM Applications Against Unauthorized Actions Shih-Han Chan AAML 46 0 0 29 Mar 2025
Data Poisoning in Deep Learning: A Survey Pinlong Zhao Weiyao Zhu Pengfei Jiao Di Gao Ou Wu AAML 39 0 0 27 Mar 2025
Tricking Retrievers with Influential Tokens: An Efficient Black-Box Corpus Poisoning Attack Cheng Wang Yiwei Wang Yujun Cai Bryan Hooi AAML 49 0 0 27 Mar 2025
FLEX: A Benchmark for Evaluating Robustness of Fairness in Large Language Models Dahyun Jung Seungyoon Lee Hyeonseok Moon Chanjun Park Heuiseok Lim AAML ALM ELM 53 0 0 25 Mar 2025
Efficient but Vulnerable: Benchmarking and Defending LLM Batch Prompting Attack Murong Yue Ziyu Yao SILM AAML 53 0 0 18 Mar 2025
Prompt Flow Integrity to Prevent Privilege Escalation in LLM Agents Juhee Kim Woohyuk Choi Byoungyoung Lee LLMAG 79 1 0 17 Mar 2025
Multi-Agent Systems Execute Arbitrary Malicious Code Harold Triedman Rishi Jha Vitaly Shmatikov LLMAG AAML 89 2 0 15 Mar 2025
ASIDE: Architectural Separation of Instructions and Data in Language Models Egor Zverev Evgenii Kortukov Alexander Panfilov Soroush Tabesh Alexandra Volkova Sebastian Lapuschkin Wojciech Samek Christoph H. Lampert AAML 52 1 0 13 Mar 2025
Cats Confuse Reasoning LLM: Query Agnostic Adversarial Triggers for Reasoning Models Meghana Arakkal Rajeev Rajkumar Ramamurthy Prapti Trivedi Vikas Yadav Oluwanifemi Bamgbose Sathwik Tejaswi Madhusudan James Y. Zou Nazneen Rajani AAML LRM 45 2 0 03 Mar 2025
Zero-Trust Artificial Intelligence Model Security Based on Moving Target Defense and Content Disarm and Reconstruction Daniel Gilkarov Ran Dubin 66 0 0 03 Mar 2025
Building Safe GenAI Applications: An End-to-End Overview of Red Teaming for Large Language Models Alberto Purpura Sahil Wadhwa Jesse Zymet Akshay Gupta Andy Luo Melissa Kazemi Rad Swapnil Shinde Mohammad Sorower AAML 100 0 0 03 Mar 2025
ConvCodeWorld: Benchmarking Conversational Code Generation in Reproducible Feedback Environments Hojae Han Seung-won Hwang Rajhans Samdani Yuxiong He ALM 65 2 0 27 Feb 2025