Title
Mapping User Trust in Vision Language Models: Research Landscape, Challenges, and Prospects Agnese Chiatti Sara Bernardini Lara Shibelski Godoy Piccolo Viola Schiaffonati Matteo Matteucci 39 0 0 08 May 2025
Safety in Large Reasoning Models: A Survey Cheng Wang Y. Liu B. Li Duzhen Zhang Z. Li Junfeng Fang LRM 33 1 0 24 Apr 2025
VLMGuard-R1: Proactive Safety Alignment for VLMs via Reasoning-Driven Prompt Optimization Menglan Chen Xianghe Pang Jingjing Dong Wenhao Wang Yaxin Du Siheng Chen LRM 20 0 0 17 Apr 2025
Misaligned Roles, Misplaced Images: Structural Input Perturbations Expose Multimodal Alignment Blind Spots Erfan Shayegani G M Shahariar Sara Abdali Lei Yu Nael B. Abu-Ghazaleh Yue Dong AAML 27 0 0 01 Apr 2025
Safe RLHF-V: Safe Reinforcement Learning from Human Feedback in Multimodal Large Language Models Jiaming Ji X. Chen Rui Pan Han Zhu C. Zhang ... Juntao Dai Chi-Min Chan Sirui Han Yike Guo Y. Yang OffRL 71 2 0 22 Mar 2025
Safety Mirage: How Spurious Correlations Undermine VLM Safety Fine-tuning Yiwei Chen Yuguang Yao Yihua Zhang Bingquan Shen Gaowen Liu Sijia Liu AAML MU 45 1 0 14 Mar 2025
SHAPE : Self-Improved Visual Preference Alignment by Iteratively Generating Holistic Winner Kejia Chen Jiawen Zhang Jiacong Hu Jiazhen Yang Jian Lou Zunlei Feng Mingli Song 48 0 0 06 Mar 2025
Understanding and Rectifying Safety Perception Distortion in VLMs Xiaohan Zou Jian Kang George Kesidis Lu Lin 77 0 0 18 Feb 2025
Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning Di Zhang Jingdi Lei Junxian Li Xunzhi Wang Y. Liu ... S. M. I. Simon X. Yang Jianbo Wu Peng Ye Wanli Ouyang Dongzhan Zhou OffRL LRM 85 6 0 27 Nov 2024
Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization Weiyun Wang Zhe Chen Wenhai Wang Yue Cao Yangzhou Liu ... Jinguo Zhu X. Zhu Lewei Lu Yu Qiao Jifeng Dai LRM 32 45 1 15 Nov 2024
Audio Is the Achilles' Heel: Red Teaming Audio Large Multimodal Models Hao Yang Lizhen Qu Ehsan Shareghi Gholamreza Haffari AAML 31 3 0 31 Oct 2024
Jigsaw Puzzles: Splitting Harmful Questions to Jailbreak Large Language Models Hao Yang Lizhen Qu Ehsan Shareghi Gholamreza Haffari AAML 29 1 0 15 Oct 2024
Have the VLMs Lost Confidence? A Study of Sycophancy in VLMs Shuo Li Tao Ji Xiaoran Fan Linsheng Lu L. Yang ... Y. Wang Xiaohui Zhao Tao Gui Qi Zhang Xuanjing Huang 31 0 0 15 Oct 2024
How Does Vision-Language Adaptation Impact the Safety of Vision Language Models? Seongyun Lee Geewook Kim Jiyeon Kim Hyunji Lee Hoyeon Chang Sue Hyun Park Minjoon Seo 23 0 0 10 Oct 2024
ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time Yi Ding Bolian Li Ruqi Zhang MLLM 36 4 0 09 Oct 2024
Building and better understanding vision-language models: insights and future directions Hugo Laurençon Andrés Marafioti Victor Sanh Léo Tronchon VLM 21 45 0 22 Aug 2024
$$\textit{MMJ-Bench}$: A Comprehensive Study on Jailbreak Attacks and Defenses for Vision Language Models$ $\textit{MMJ-Bench}$ : A Comprehensive Study on Jailbreak Attacks and Defenses for Vision Language Models Fenghua Weng Yue Xu Chengyan Fu Wenjie Wang AAML 30 1 0 16 Aug 2024
A Single Transformer for Scalable Vision-Language Modeling Yangyi Chen Xingyao Wang Hao Peng Heng Ji LRM 27 10 0 08 Jul 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 65 136 0 29 Apr 2024
JailBreakV-28K: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks Weidi Luo Siyuan Ma Xiaogeng Liu Xiaoyu Guo Chaowei Xiao AAML 45 17 0 03 Apr 2024
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models Siddharth Karamcheti Suraj Nair Ashwin Balakrishna Percy Liang Thomas Kollar Dorsa Sadigh MLLM VLM 54 95 0 12 Feb 2024
Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models Yongshuo Zong Ondrej Bohdal Tingyang Yu Yongxin Yang Timothy M. Hospedales VLM MLLM 41 33 0 03 Feb 2024
Harm Amplification in Text-to-Image Models Susan Hao Renee Shelby Yuchi Liu Hansa Srinivasan Mukul Bhutani Burcu Karagol Ayan Ryan Poplin Shivani Poddar Sarah Laszlo 23 4 0 01 Feb 2024
Red Teaming Visual Language Models Mukai Li Lei Li Yuwei Yin Masood Ahmed Zhenguang Liu Qi Liu VLM 23 11 0 23 Jan 2024
FigStep: Jailbreaking Large Vision-Language Models via Typographic Visual Prompts Yichen Gong Delong Ran Jinyuan Liu Conglei Wang Tianshuo Cong Anyu Wang Sisi Duan Xiaoyun Wang MLLM 111 116 0 09 Nov 2023
Improving Generalization of Alignment with Human Preferences through Group Invariant Learning Rui Zheng Wei Shen Yuan Hua Wenbin Lai Shihan Dou ... Xiao Wang Haoran Huang Tao Gui Qi Zhang Xuanjing Huang 41 9 0 18 Oct 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 198 575 0 27 Apr 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 195 1,089 0 20 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 8,441 0 04 Mar 2022
Fine-Tuning Language Models from Human Preferences Daniel M. Ziegler Nisan Stiennon Jeff Wu Tom B. Brown Alec Radford Dario Amodei Paul Christiano G. Irving ALM 273 1,151 0 18 Sep 2019