OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

29 November 2023

Conghui He

Dahua Lin

Papers citing "OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation"

50 / 139 papers shown

Title
Mapping User Trust in Vision Language Models: Research Landscape, Challenges, and Prospects Agnese Chiatti Sara Bernardini Lara Shibelski Godoy Piccolo Viola Schiaffonati Matteo Matteucci 44 0 0 08 May 2025
A Comprehensive Analysis for Visual Object Hallucination in Large Vision-Language Models Liqiang Jing Guiming Hardy Chen Ehsan Aghazadeh Xin Eric Wang Xinya Du 48 0 0 04 May 2025
Localizing Before Answering: A Hallucination Evaluation Benchmark for Grounded Medical Multimodal LLMs Dung Tien Nguyen Minh Khoi Ho Huy Ta T. Nguyen Qi Chen ... Zhibin Liao Minh Nguyen Nhat To Johan W. Verjans Phi Le Nguyen Vu Minh Hieu Phan 19 0 0 30 Apr 2025
Black-Box Visual Prompt Engineering for Mitigating Object Hallucination in Large Vision Language Models Sangmin Woo Kang Zhou Yun Zhou Shuai Wang Sheng Guan Haibo Ding Lin Lee Cheong VPVLM 74 0 0 30 Apr 2025
Antidote: A Unified Framework for Mitigating LVLM Hallucinations in Counterfactual Presupposition and Object Perception Yuanchen Wu Lu Zhang Hang Yao Junlong Du Ke Yan Shouhong Ding Yunsheng Wu X. Li MLLM 68 0 0 29 Apr 2025
AdaViP: Aligning Multi-modal LLMs via Adaptive Vision-enhanced Preference Optimization Jinda Lu Jinghan Li Yuan Gao Junkang Wu Jiancan Wu X. Wang Xiangnan He 25 0 0 22 Apr 2025
Low-hallucination Synthetic Captions for Large-Scale Vision-Language Model Pre-training X. Zhang Yarong Zeng Xinting Huang Hu Hu Runquan Xie Han Hu Zhanhui Kang MLLM VLM 35 0 0 17 Apr 2025
From Token to Line: Enhancing Code Generation with a Long-Term Perspective Tingwei Lu Yangning Li Liyuan Wang Binghuai Lin Jiwei Tang ... Hai-tao Zheng Yinghui Li Bingxu An Zhao Wei Y. Xu LLMAG 55 0 0 10 Apr 2025
Perception in Reflection Yana Wei Liang Zhao Kangheng Lin En Yu Yuang Peng ... Jianjian Sun Haoran Wei Zheng Ge Xiangyu Zhang Vishal M. Patel 31 0 0 09 Apr 2025
Video-Bench: Human-Aligned Video Generation Benchmark Hui Han Siyuan Li Jiaqi Chen Yiwen Yuan Yuling Wu ... Y. Li J. Zhang Chi Zhang Li Li Yongxin Ni EGVM VGen 65 0 0 07 Apr 2025
POPEN: Preference-Based Optimization and Ensemble for LVLM-Based Reasoning Segmentation Lanyun Zhu Tianrun Chen Qianxiong Xu Xuanyi Liu Deyi Ji Haiyang Wu De Wen Soh J. Liu VLM LRM 44 0 0 01 Apr 2025
Towards Trustworthy GUI Agents: A Survey Yucheng Shi Wenhao Yu Wenlin Yao Wenhu Chen Ninghao Liu 37 2 0 30 Mar 2025
Learning to Instruct for Visual Instruction Tuning Zhihan Zhou Feng Hong Jiaan Luo Jiangchao Yao Dongsheng Li Bo Han Y. Zhang Yanfeng Wang VLM 59 0 0 28 Mar 2025
Mitigating Low-Level Visual Hallucinations Requires Self-Awareness: Database, Model and Training Strategy Yinan Sun Xiongkuo Min Zicheng Zhang Yixuan Gao Y. Cao Guangtao Zhai VLM 59 0 0 26 Mar 2025
Mind with Eyes: from Language Reasoning to Multimodal Reasoning Zhiyu Lin Yifei Gao Xian Zhao Yunfan Yang Jitao Sang LRM 42 1 0 23 Mar 2025
Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization Zefeng Zhang Hengzhu Tang Jiawei Sheng Zhenyu Zhang Yiming Ren Zhenyang Li Dawei Yin Duohe Ma Tingwen Liu 41 0 0 23 Mar 2025
Mitigating Object Hallucinations in MLLMs via Multi-Frequency Perturbations Shuo Li Jiajun Sun Guodong Zheng Xiaoran Fan Yujiong Shen ... Wenming Tan Tao Ji Tao Gui Qi Zhang Xuanjing Huang AAML VLM 75 0 0 19 Mar 2025
Where do Large Vision-Language Models Look at when Answering Questions? X. Xing Chia-Wen Kuo Li Fuxin Yulei Niu Fan Chen Ming Li Ying Wu Longyin Wen Sijie Zhu LRM 53 0 0 18 Mar 2025
ClearSight: Visual Signal Enhancement for Object Hallucination Mitigation in Multimodal Large language Models Hao Yin Guangzong Si Zilei Wang 43 0 0 17 Mar 2025
TruthPrInt: Mitigating LVLM Object Hallucination Via Latent Truthful-Guided Pre-Intervention Jinhao Duan Fei Kong Hao-Ran Cheng James Diffenderfer B. Kailkhura Lichao Sun Xiaofeng Zhu Xiaoshuang Shi Kaidi Xu 52 0 0 13 Mar 2025
Through the Magnifying Glass: Adaptive Perception Magnification for Hallucination-Free VLM Decoding Shunqi Mao Chaoyi Zhang Weidong Cai MLLM 53 0 0 13 Mar 2025
TIME: Temporal-sensitive Multi-dimensional Instruction Tuning and Benchmarking for Video-LLMs Yunxiao Wang Meng Liu Rui Shao Haoyu Zhang Bin Wen Fan Yang Tingting Gao Di Zhang Liqiang Nie 56 1 0 13 Mar 2025
Attention Reallocation: Towards Zero-cost and Controllable Hallucination Mitigation of MLLMs Chongjun Tu Peng Ye Dongzhan Zhou Lei Bai Gang Yu Tao Chen Wanli Ouyang 56 0 0 13 Mar 2025
Attention Hijackers: Detect and Disentangle Attention Hijacking in LVLMs for Hallucination Mitigation Beitao Chen Xinyu Lyu Lianli Gao Jingkuan Song H. Shen 57 1 0 11 Mar 2025
EAZY: Eliminating Hallucinations in LVLMs by Zeroing out Hallucinatory Image Tokens Liwei Che Tony Qingze Liu Jing Jia Weiyi Qin Ruixiang Tang Vladimir Pavlovic MLLM VLM 97 1 0 10 Mar 2025
PerturboLLaVA: Reducing Multimodal Hallucinations with Perturbative Visual Training Cong Chen Mingyu Liu Chenchen Jing Y. Zhou Fengyun Rao Hao Chen Bo Zhang Chunhua Shen MLLM AAML VLM 44 2 0 09 Mar 2025
Treble Counterfactual VLMs: A Causal Approach to Hallucination Li Li Jiashu Qu Yuxiao Zhou Yuehan Qin Tiankai Yang Yue Zhao 59 1 0 08 Mar 2025
TPC: Cross-Temporal Prediction Connection for Vision-Language Model Hallucination Reduction Chao Wang Weiwei Fu Yang Zhou MLLM VLM 62 0 0 06 Mar 2025
MedHEval: Benchmarking Hallucinations and Mitigation Strategies in Medical Large Vision-Language Models Aofei Chang Le Huang Parminder Bhatia Taha A. Kass-Hout Fenglong Ma Cao Xiao VLM 64 0 0 04 Mar 2025
Octopus: Alleviating Hallucination via Dynamic Contrastive Decoding Wei Suo Lijun Zhang Mengyang Sun Lin Yuanbo Wu Peng Wang Y. Zhang MLLM VLM 44 1 0 01 Mar 2025
Mitigating Hallucinations in Large Vision-Language Models by Adaptively Constraining Information Flow Jiaqi Bai Hongcheng Guo Zhongyuan Peng Jian Yang Z. Li M. Li Zhihong Tian VLM 52 0 0 28 Feb 2025
Magnifier Prompt: Tackling Multimodal Hallucination via Extremely Simple Instructions Yuhan Fu Ruobing Xie Jiazhen Liu Bangxiang Lan X. Sun Zhanhui Kang Xirong Li VLM LRM MLLM 55 0 0 24 Feb 2025
Exploring Causes and Mitigation of Hallucinations in Large Vision Language Models Yaqi Sun Kyohei Atarashi Koh Takeuchi Hisashi Kashima MLLM 41 0 0 24 Feb 2025
Can Hallucination Correction Improve Video-Language Alignment? Lingjun Zhao Mingyang Xie Paola Cascante-Bonilla Hal Daumé III Kwonjoon Lee HILM VLM 57 0 0 20 Feb 2025
Mitigating Hallucinations in Large Vision-Language Models via Summary-Guided Decoding Kyungmin Min Minbeom Kim Kang-il Lee Dongryeol Lee Kyomin Jung MLLM 60 3 0 20 Feb 2025
Visual Attention Never Fades: Selective Progressive Attention ReCalibration for Detailed Image Captioning in Multimodal Large Language Models Mingi Jung Saehuyng Lee Eunji Kim Sungroh Yoon 62 0 0 03 Feb 2025
PAINT: Paying Attention to INformed Tokens to Mitigate Hallucination in Large Vision-Language Model Kazi Hasan Ibn Arif Sajib Acharjee Dip Khizar Hussain Lang Zhang Chris Thomas 62 0 0 21 Jan 2025
Advancing General Multimodal Capability of Vision-language Models with Pyramid-descent Visual Position Encoding Z. Chen Mingxiao Li Z. Chen Nan Du Xiaolong Li Yuexian Zou 48 0 0 19 Jan 2025
Visual Large Language Models for Generalized and Specialized Applications Yifan Li Zhixin Lai Wentao Bao Zhen Tan Anh Dao Kewei Sui Jiayi Shen Dong Liu Huan Liu Yu Kong VLM 83 10 0 06 Jan 2025
Cracking the Code of Hallucination in LVLMs with Vision-aware Head Divergence Jinghan He Kuan Zhu Haiyun Guo Junfeng Fang Zhenglin Hua Yuheng Jia Ming Tang Tat-Seng Chua J. T. Wang VLM 72 0 0 18 Dec 2024
Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection Le Yang Ziwei Zheng Boxu Chen Zhengyu Zhao Chenhao Lin Chao Shen VLM 132 3 0 18 Dec 2024
Attention with Dependency Parsing Augmentation for Fine-Grained Attribution Qiang Ding Lvzhou Luo Yixuan Cao Ping Luo 74 0 0 16 Dec 2024
Who Brings the Frisbee: Probing Hidden Hallucination Factors in Large Vision-Language Model via Causality Analysis Po-Hsuan Huang Jeng-Lin Li Chin-Po Chen Ming-Ching Chang Wei-Chao Chen LRM 69 1 0 04 Dec 2024
Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Survey Yunkai Dang Kaichen Huang Jiahao Huo Yibo Yan S. Huang ... Kun Wang Yong Liu Jing Shao Hui Xiong Xuming Hu LRM 94 14 0 03 Dec 2024
Beyond Text-Visual Attention: Exploiting Visual Cues for Effective Token Pruning in VLMs Qizhe Zhang Aosong Cheng Ming Lu Zhiyong Zhuo Minqi Wang Jiajun Cao Shaobo Guo Qi She Shanghang Zhang VLM 78 11 0 02 Dec 2024
Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning Di Zhang Jingdi Lei Junxian Li Xunzhi Wang Y. Liu ... S. M. I. Simon X. Yang Jianbo Wu Peng Ye Wanli Ouyang Dongzhan Zhou OffRL LRM 95 6 0 27 Nov 2024
Efficient Self-Improvement in Multimodal Large Language Models: A Model-Level Judge-Free Approach Shijian Deng Wentian Zhao Yu-Jhe Li Kun Wan Daniel Miranda Ajinkya Kale Yapeng Tian LRM 64 0 0 26 Nov 2024
VaLiD: Mitigating the Hallucination of Large Vision Language Models by Visual Layer Fusion Contrastive Decoding Jiaqi Wang Yifei Gao Jitao Sang MLLM 99 2 0 24 Nov 2024
Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens Zhangqi Jiang Junkai Chen Beier Zhu Tingjin Luo Yankun Shen Xu Yang 80 4 0 23 Nov 2024
ICT: Image-Object Cross-Level Trusted Intervention for Mitigating Object Hallucination in Large Vision-Language Models Junzhe Chen Tianshu Zhang S. Huang Yuwei Niu Linfeng Zhang Lijie Wen Xuming Hu MLLM VLM 95 1 0 22 Nov 2024