VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models

21 February 2024

Papers citing "VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models"

38 / 38 papers shown

Title
Natural Reflection Backdoor Attack on Vision Language Model for Autonomous Driving Ming Liu Siyuan Liang Koushik Howlader L. Wang Dacheng Tao Wensheng Zhang AAML 9 0 0 09 May 2025
REVEAL: Multi-turn Evaluation of Image-Input Harms for Vision LLM Madhur Jindal Saurabh Deshpande AAML 40 0 0 07 May 2025
T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Models Siyuan Liang Jiayang Liu Jiecheng Zhai Tianmeng Fang Rongcheng Tu A. Liu Xiaochun Cao Dacheng Tao VGen 49 0 0 22 Apr 2025
Manipulating Multimodal Agents via Cross-Modal Prompt Injection Le Wang Zonghao Ying Tianyuan Zhang Siyuan Liang Shengshan Hu Mingchuan Zhang A. Liu Xianglong Liu AAML 31 1 0 19 Apr 2025
Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots Erfan Shayegani G M Shahariar Sara Abdali Lei Yu Nael B. Abu-Ghazaleh Yue Dong AAML 39 0 0 01 Apr 2025
Lie Detector: Unified Backdoor Detection via Cross-Examination Framework X. U. Wang Siyuan Liang Dongping Liao Han Fang Aishan Liu Xiaochun Cao Yu-liang Lu E. Chang X. Gao AAML 44 1 0 21 Mar 2025
BadToken: Token-level Backdoor Attacks to Multi-modal Large Language Models Zenghui Yuan Jiawen Shi Pan Zhou Neil Zhenqiang Gong Lichao Sun AAML 52 1 0 20 Mar 2025
Survey of Adversarial Robustness in Multimodal Large Language Models Chengze Jiang Zhuangzhuang Wang Minjing Dong Jie Gui AAML 58 0 0 18 Mar 2025
Stealthy Backdoor Attack in Self-Supervised Learning Vision Encoders for Large Vision Language Models Zhaoyi Liu Huan Zhang AAML 68 0 0 25 Feb 2025
ELBA-Bench: An Efficient Learning Backdoor Attacks Benchmark for Large Language Models X. Liu Siyuan Liang M. Han Yong Luo Aishan Liu Xiantao Cai Zheng He Dacheng Tao AAML SILM ELM 29 1 0 22 Feb 2025
Red Pill and Blue Pill: Controllable Website Fingerprinting Defense via Dynamic Backdoor Learning Siyuan Liang Jiajun Gong Tianmeng Fang Aishan Liu Tao Wang X. Liu Xiaochun Cao Dacheng Tao Chang Ee-Chien AAML 75 3 0 16 Dec 2024
CopyrightShield: Spatial Similarity Guided Backdoor Defense against Copyright Infringement in Diffusion Models Zhixiang Guo Siyuan Liang Aishan Liu Dacheng Tao AAML 66 1 0 02 Dec 2024
PSA-VLM: Enhancing Vision-Language Model Safety through Progressive Concept-Bottleneck-Driven Alignment Zhendong Liu Yuanbi Nie Yingshui Tan Xiangyu Yue Qiushi Cui Chongjun Wang Xiaoyong Zhu Bo Zheng Bo Zheng 63 0 0 18 Nov 2024
SoK: Unifying Cybersecurity and Cybersafety of Multimodal Foundation Models with an Information Theory Approach Ruoxi Sun Jiamin Chang Hammond Pearce Chaowei Xiao B. Li Qi Wu Surya Nepal Minhui Xue 30 0 0 17 Nov 2024
Backdooring Vision-Language Models with Out-Of-Distribution Data Weimin Lyu Jiachen Yao Saumya Gupta Lu Pang Tao Sun Lingjie Yi Lijie Hu Haibin Ling Chao Chen VLM AAML 41 2 0 02 Oct 2024
Efficient Backdoor Defense in Multimodal Contrastive Learning: A Token-Level Unlearning Method for Mitigating Threats Kuanrong Liu Siyuan Liang Jiawei Liang Pengwen Dai Xiaochun Cao MU AAML 16 1 0 29 Sep 2024
Towards Robust Object Detection: Identifying and Removing Backdoors via Module Inconsistency Analysis Xianda Zhang Siyuan Liang AAML 16 0 0 24 Sep 2024
Sample-agnostic Adversarial Perturbation for Vision-Language Pre-training Models Haonan Zheng Wen Jiang Xinyang Deng Wenrui Li VLM AAML 13 2 0 06 Aug 2024
Compromising Embodied Agents with Contextual Backdoor Attacks Aishan Liu Yuguang Zhou Xianglong Liu Tianyuan Zhang Siyuan Liang ... Tianlin Li Junqi Zhang Wenbo Zhou Qing-Wu Guo Dacheng Tao LLMAG AAML 21 1 0 06 Aug 2024
Multimodal Unlearnable Examples: Protecting Data against Multimodal Contrastive Learning Xinwei Liu Xiaojun Jia Yuan Xun Siyuan Liang Xiaochun Cao 21 6 0 23 Jul 2024
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective Zhen Qin Daoyuan Chen Wenhao Zhang Liuyi Yao Yilun Huang Bolin Ding Yaliang Li Shuiguang Deng 43 5 0 11 Jul 2024
A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends Daizong Liu Mingyu Yang Xiaoye Qu Pan Zhou Yu Cheng Wei Hu ELM AAML 27 24 0 10 Jul 2024
SPA-VL: A Comprehensive Safety Preference Alignment Dataset for Vision Language Model Yongting Zhang Lu Chen Guodong Zheng Yifeng Gao Rui Zheng ... Yu Qiao Xuanjing Huang Feng Zhao Tao Gui Jing Shao VLM 62 22 0 17 Jun 2024
Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attacks Zonghao Ying Aishan Liu Xianglong Liu Dacheng Tao 54 16 0 10 Jun 2024
Jailbreak Vision Language Models via Bi-Modal Adversarial Prompt Zonghao Ying Aishan Liu Tianyuan Zhang Zhengmin Yu Siyuan Liang Xianglong Liu Dacheng Tao AAML 33 26 0 06 Jun 2024
LanEvil: Benchmarking the Robustness of Lane Detection to Environmental Illusions Tianyuan Zhang Lu Wang Hainan Li Yisong Xiao Siyuan Liang Aishan Liu Xianglong Liu Dacheng Tao 35 10 0 03 Jun 2024
Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attack Mingli Zhu Siyuan Liang Baoyuan Wu AAML 26 14 0 25 May 2024
Towards Robust Physical-world Backdoor Attacks on Lane Detection Xinwei Zhang Aishan Liu Tianyuan Zhang Siyuan Liang Xianglong Liu AAML 27 10 0 09 May 2024
Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in Multimodal Large Language Model Security Yihe Fan Yuxin Cao Ziyu Zhao Ziyao Liu Shaofeng Li 27 11 0 08 Apr 2024
Unlearning Backdoor Threats: Enhancing Backdoor Defense in Multimodal Contrastive Learning via Local Token Unlearning Siyuan Liang Kuanrong Liu Jiajun Gong Jiawei Liang Yuan Xun Ee-Chien Chang Xiaochun Cao AAML MU 16 12 0 24 Mar 2024
Effectiveness Assessment of Recent Large Vision-Language Models Yao Jiang Xinyu Yan Ge-Peng Ji Keren Fu Meijun Sun Huan Xiong Deng-Ping Fan Fahad Shahbaz Khan 19 14 0 07 Mar 2024
Security and Privacy Challenges of Large Language Models: A Survey B. Das M. H. Amini Yanzhao Wu PILM ELM 17 98 0 30 Jan 2024
Pre-trained Trojan Attacks for Visual Recognition Aishan Liu Xinwei Zhang Yisong Xiao Yuguang Zhou Siyuan Liang Jiakai Wang Xianglong Liu Xiaochun Cao Dacheng Tao AAML 61 25 0 23 Dec 2023
Poisoning Language Models During Instruction Tuning Alexander Wan Eric Wallace Sheng Shen Dan Klein SILM 90 124 0 01 May 2023
Exploring Inconsistent Knowledge Distillation for Object Detection with Data Augmentation Jiawei Liang Siyuan Liang Aishan Liu Ke Ma Jingzhi Li Xiaochun Cao VLM 41 15 0 20 Sep 2022
Dual Attention Suppression Attack: Generate Adversarial Camouflage in Physical World Jiakai Wang Aishan Liu Zixin Yin Shunchang Liu Shiyu Tang Xianglong Liu AAML 133 191 0 01 Mar 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 220 450 0 11 Feb 2021
Transferable Adversarial Attacks for Image and Video Object Detection Xingxing Wei Siyuan Liang Ning Chen Xiaochun Cao AAML 69 186 0 30 Nov 2018