OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

29 November 2023

Conghui He

Dahua Lin

Papers citing "OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation"

39 / 139 papers shown

Title
Beyond LLaVA-HD: Diving into High-Resolution Large Multimodal Models Yi-Fan Zhang Qingsong Wen Chaoyou Fu Xue Wang Zhang Zhang L. Wang Rong Jin 16 9 0 12 Jun 2024
F-LMM: Grounding Frozen Large Multimodal Models Size Wu Sheng Jin Wenwei Zhang Lumin Xu Wentao Liu Wei Li Chen Change Loy MLLM 62 12 0 09 Jun 2024
3D-GRAND: A Million-Scale Dataset for 3D-LLMs with Better Grounding and Less Hallucination Jianing Yang Xuweiyi Chen Nikhil Madaan Madhavan Iyengar Shengyi Qian David Fouhey Joyce Chai 3DV 61 11 0 07 Jun 2024
From Redundancy to Relevance: Enhancing Explainability in Multimodal Large Language Models Xiaofeng Zhang Chen Shen Xiaosong Yuan Shaotian Yan Liang Xie Wenxiao Wang Chaochen Gu Hao Tang Jieping Ye 35 8 0 04 Jun 2024
CODE: Contrasting Self-generated Description to Combat Hallucination in Large Multi-modal Models Junho Kim Hyunjun Kim Yeonju Kim Yong Man Ro MLLM 31 10 0 04 Jun 2024
NoiseBoost: Alleviating Hallucination with Noise Perturbation for Multimodal Large Language Models Kai Wu Boyuan Jiang Zhengkai Jiang Qingdong He Donghao Luo Shengzhi Wang Qingwen Liu Chengjie Wang VLM MLLM 17 3 0 30 May 2024
RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in LVLMs Sangmin Woo Jaehyuk Jang Donguk Kim Yubin Choi Changick Kim 21 0 0 28 May 2024
Don't Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models Sangmin Woo Donguk Kim Jaehyuk Jang Yubin Choi Changick Kim 27 12 0 28 May 2024
Alleviating Hallucinations in Large Vision-Language Models through Hallucination-Induced Optimization Beitao Chen Xinyu Lyu Lianli Gao Jingkuan Song Hengtao Shen MLLM 49 10 0 24 May 2024
Calibrated Self-Rewarding Vision Language Models Yiyang Zhou Zhiyuan Fan Dongjie Cheng Sihan Yang Zhaorun Chen Chenhang Cui Xiyao Wang Yun-Qing Li Linjun Zhang Huaxiu Yao VLM 66 26 0 23 May 2024
Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models Yuchen Hu Chen Chen Chao-Han Huck Yang Chengwei Qin Pin-Yu Chen Chng Eng Siong Chao Zhang VLM 25 3 0 23 May 2024
Automated Multi-level Preference for MLLMs Mengxi Zhang Wenhao Wu Yu Lu Yuxin Song Kang Rong ... Jianbo Zhao Fanglong Liu Yifan Sun Haocheng Feng Jingdong Wang MLLM 56 10 0 18 May 2024
MileBench: Benchmarking MLLMs in Long Context Dingjie Song Shunian Chen Guiming Hardy Chen Fei Yu Xiang Wan Benyou Wang VLM 56 34 0 29 Apr 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 68 136 0 29 Apr 2024
TinyChart: Efficient Chart Understanding with Visual Token Merging and Program-of-Thoughts Learning Liang Zhang Anwen Hu Haiyang Xu Mingshi Yan Yichen Xu Qin Jin Ji Zhang Fei Huang 33 15 0 25 Apr 2024
Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunities Xiaomin Yu Yezhaohui Wang Yanfang Chen Zhen Tao Dinghao Xi Shichao Song Simin Niu Zhiyu Li 62 7 0 25 Apr 2024
TextCoT: Zoom In for Enhanced Multimodal Text-Rich Image Understanding Bozhi Luan Hao Feng Hong Chen Yonghui Wang Wen-gang Zhou Houqiang Li MLLM 21 10 0 15 Apr 2024
FGAIF: Aligning Large Vision-Language Models with Fine-grained AI Feedback Liqiang Jing Xinya Du 68 17 0 07 Apr 2024
Are We on the Right Way for Evaluating Large Vision-Language Models? Lin Chen Jinsong Li Xiao-wen Dong Pan Zhang Yuhang Zang ... Haodong Duan Jiaqi Wang Yu Qiao Dahua Lin Feng Zhao VLM 55 216 0 29 Mar 2024
Hallucination Detection in Foundation Models for Decision-Making: A Flexible Definition and Review of the State of the Art Neeloy Chakraborty Melkior Ornik Katherine Driggs-Campbell LRM 50 9 0 25 Mar 2024
Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination Dingchen Yang Bowen Cao Guang Chen Changjun Jiang 46 7 0 21 Mar 2024
HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding Zhaorun Chen Zhuokai Zhao Hongyin Luo Huaxiu Yao Bo Li Jiawei Zhou MLLM 38 57 0 01 Mar 2024
IBD: Alleviating Hallucinations in Large Vision-Language Models via Image-Biased Decoding Lanyun Zhu Deyi Ji Tianrun Chen Peng Xu Jieping Ye Jun Liu MLLM 31 41 0 28 Feb 2024
DualFocus: Integrating Macro and Micro Perspectives in Multi-modal Large Language Models Yuhang Cao Pan Zhang Xiao-wen Dong Dahua Lin Jiaqi Wang 29 10 0 22 Feb 2024
Logical Closed Loop: Uncovering Object Hallucinations in Large Vision-Language Models Jun Wu Qiang Liu Ding Wang Jinghao Zhang Shu Wu Liang Wang Tien-Ping Tan LRM 19 20 0 18 Feb 2024
Aligning Modalities in Vision Large Language Models via Preference Fine-tuning Yiyang Zhou Chenhang Cui Rafael Rafailov Chelsea Finn Huaxiu Yao VLM MLLM 16 89 0 18 Feb 2024
Unified Hallucination Detection for Multimodal Large Language Models Xiang Chen Chenxi Wang Yida Xue Ningyu Zhang Xiaoyan Yang Qian Li Yue Shen Lei Liang Jinjie Gu Huajun Chen HILM 17 38 0 05 Feb 2024
$Skip \n: A Simple Method to Reduce Hallucination in Large Vision-Language Models$ Skip \n: A Simple Method to Reduce Hallucination in Large Vision-Language Models Zongbo Han Zechen Bai Haiyang Mei Qianli Xu Changqing Zhang Mike Zheng Shou VLM 16 7 0 02 Feb 2024
A Survey on Hallucination in Large Vision-Language Models Hanchao Liu Wenyuan Xue Yifei Chen Dapeng Chen Xiutian Zhao Ke Wang Liping Hou Rong-Zhi Li Wei Peng LRM MLLM 14 110 0 01 Feb 2024
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model Xiao-wen Dong Pan Zhang Yuhang Zang Yuhang Cao Bin Wang ... Conghui He Xingcheng Zhang Yu Qiao Dahua Lin Jiaqi Wang VLM MLLM 73 89 0 29 Jan 2024
Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences Xiyao Wang Yuhang Zhou Xiaoyu Liu Hongjin Lu Yuancheng Xu ... Taixi Lu Gedas Bertasius Mohit Bansal Huaxiu Yao Furong Huang LRM VLM 70 65 0 19 Jan 2024
MM-SAP: A Comprehensive Benchmark for Assessing Self-Awareness of Multimodal Large Language Models in Perception Yuhao Wang Yusheng Liao Heyang Liu Hongcheng Liu Yu Wang Yanfeng Wang LRM VLM 19 13 0 15 Jan 2024
SimAC: A Simple Anti-Customization Method for Protecting Face Privacy against Text-to-Image Synthesis of Diffusion Models Feifei Wang Zhentao Tan Tianyi Wei Yue-bo Wu Qidong Huang DiffM 25 9 0 13 Dec 2023
Woodpecker: Hallucination Correction for Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Tong Bill Xu Hao Wang Dianbo Sui Yunhang Shen Ke Li Xingguo Sun Enhong Chen VLM MLLM 22 112 0 24 Oct 2023
VIGC: Visual Instruction Generation and Correction Bin Wang Fan Wu Xiao Han Jiahui Peng Huaping Zhong ... Xiao-wen Dong Weijia Li Wei Li Jiaqi Wang Conghui He MLLM 19 61 0 24 Aug 2023
How Language Model Hallucinations Can Snowball Muru Zhang Ofir Press William Merrill Alisa Liu Noah A. Smith HILM LRM 75 246 0 22 May 2023
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models Potsawee Manakul Adian Liusie Mark J. F. Gales HILM LRM 145 386 0 15 Mar 2023
Diversity-Aware Meta Visual Prompting Qidong Huang Xiaoyi Dong Dongdong Chen Weiming Zhang Feifei Wang Gang Hua Neng H. Yu VLM VPVLM 33 52 0 14 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023