A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends

10 July 2024

Xiaoye Qu

Wei Hu

Papers citing "A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends"

28 / 28 papers shown

Title
Adversarial Attacks in Multimodal Systems: A Practitioner's Survey Shashank Kapoor Sanjay Surendranath Girija Lakshit Arora Dipen Pradhan Ankit Shetgaonkar Aman Raj AAML 45 0 0 06 May 2025
From Head to Tail: Towards Balanced Representation in Large Vision-Language Models through Adaptive Data Calibration Mingyang Song Xiaoye Qu Jiawei Zhou Yu-Xi Cheng VLM 35 1 0 17 Mar 2025
First-place Solution for Streetscape Shop Sign Recognition Competition Bin Wang Li Jing 43 0 0 06 Jan 2025
Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models Shicheng Xu Liang Pang Yunchang Zhu Huawei Shen Xueqi Cheng MLLM 33 1 0 16 Oct 2024
From LLMs to MLLMs: Exploring the Landscape of Multimodal Jailbreaking Siyuan Wang Zhuohan Long Zhihao Fan Zhongyu Wei 22 1 0 21 Jun 2024
Dissecting Adversarial Robustness of Multimodal LM Agents Chen Henry Wu Jing Yu Koh Ruslan Salakhutdinov Ruslan Salakhutdinov Aditi Raghunathan Aditi Raghunathan AAML VLM 35 26 0 18 Jun 2024
Unveiling the Safety of GPT-4o: An Empirical Study using Jailbreak Attacks Zonghao Ying Aishan Liu Xianglong Liu Dacheng Tao 46 7 0 10 Jun 2024
JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks Weidi Luo Siyuan Ma Xiaogeng Liu Xiaoyu Guo Chaowei Xiao AAML 58 17 0 03 Apr 2024
An Image Is Worth 1000 Lies: Adversarial Transferability across Prompts on Vision-Language Models Haochen Luo Jindong Gu Fengyuan Liu Philip H. S. Torr VLM VPVLM AAML 34 19 0 14 Mar 2024
One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models Lin Li Haoyan Guan Jianing Qiu Michael W. Spratling AAML VLM VPVLM 23 21 0 04 Mar 2024
Breaking Down the Defenses: A Comparative Survey of Attacks on Large Language Models Arijit Ghosh Chowdhury Md. Mofijul Islam Vaibhav Kumar F. H. Shezan Vaibhav Kumar Vinija Jain Aman Chadha AAML PILM 21 4 0 03 Mar 2024
The Wolf Within: Covert Injection of Malice into MLLM Societies via an MLLM Operative Zhen Tan Chengshuai Zhao Raha Moraffah Yifan Li Yu Kong Tianlong Chen Huan Liu 31 6 0 20 Feb 2024
Test-Time Backdoor Attacks on Multimodal Large Language Models Dong Lu Tianyu Pang Chao Du Qian Liu Xianjun Yang Min-Bin Lin AAML 36 20 0 13 Feb 2024
Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast Xiangming Gu Xiaosen Zheng Tianyu Pang Chao Du Qian Liu Ye Wang Jing Jiang Min-Bin Lin LLMAG LM&Ro 35 47 0 13 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 116 106 0 08 Feb 2024
Red Teaming Visual Language Models Mukai Li Lei Li Yuwei Yin Masood Ahmed Zhenguang Liu Qi Liu VLM 25 11 0 23 Jan 2024
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 127 116 0 09 Nov 2023
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 152 280 0 14 Oct 2023
On the Adversarial Robustness of Multi-Modal Foundation Models Christian Schlarmann Matthias Hein AAML 90 45 0 21 Aug 2023
Robust Contrastive Language-Image Pre-training against Data Poisoning and Backdoor Attacks Wenhan Yang Jingdong Gao Baharan Mirzasoleiman VLM 92 17 0 13 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 380 4,010 0 28 Jan 2022
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 253 3,790 0 24 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 220 450 0 11 Feb 2021
Sign-OPT: A Query-Efficient Hard-label Adversarial Attack Minhao Cheng Simranjit Singh Patrick H. Chen Pin-Yu Chen Sijia Liu Cho-Jui Hsieh AAML 122 197 0 24 Sep 2019
A Survey on Bias and Fairness in Machine Learning Ninareh Mehrabi Fred Morstatter N. Saxena Kristina Lerman Aram Galstyan SyDa FaML 283 4,143 0 23 Aug 2019
Towards A Rigorous Science of Interpretable Machine Learning Finale Doshi-Velez Been Kim XAI FaML 219 2,098 0 28 Feb 2017
Adversarial Machine Learning at Scale Alexey Kurakin Ian Goodfellow Samy Bengio AAML 253 2,899 0 04 Nov 2016