Communities
Connect sessions
AI calendar
Organizations
Join Slack
Contact Sales
Search
Open menu
Home
Papers
2311.17911
Cited By
v1
v2
v3 (latest)
OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation
Computer Vision and Pattern Recognition (CVPR), 2023
29 November 2023
Qidong Huang
Xiao-wen Dong
Pan Zhang
Sijin Yu
Conghui He
Yuan Liu
Dahua Lin
Weiming Zhang
Neng H. Yu
MLLM
Re-assign community
ArXiv (abs)
PDF
HTML
HuggingFace (2 upvotes)
Github (341★)
Papers citing
"OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation"
50 / 238 papers shown
Looking Beyond Text: Reducing Language bias in Large Vision-Language Models via Multimodal Dual-Attention and Soft-Image Guidance
Haozhe Zhao
Shuzheng Si
L. Chen
Yichi Zhang
Maosong Sun
Mingjia Zhang
Baobao Chang
VLM
185
15
0
21 Nov 2024
Mitigating Hallucination in Multimodal Large Language Model via Hallucination-targeted Direct Preference Optimization
Annual Meeting of the Association for Computational Linguistics (ACL), 2024
Yuhan Fu
Ruobing Xie
Xingwu Sun
Zhanhui Kang
Xirong Li
MLLM
236
11
0
15 Nov 2024
DDFAV: Remote Sensing Large Vision Language Models Dataset and Evaluation Benchmark
Haodong Li
Haicheng Qu
Xiaofeng Zhang
182
8
0
05 Nov 2024
V-DPO: Mitigating Hallucination in Large Vision Language Models via Vision-Guided Direct Preference Optimization
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Yuxi Xie
Guanzhen Li
Xiao Xu
Min-Yen Kan
MLLM
VLM
213
46
0
05 Nov 2024
Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models
J. Wu
Tsz Ting Chung
Kai Chen
Dit-Yan Yeung
LRM
VLM
721
7
0
30 Oct 2024
Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluation
North American Chapter of the Association for Computational Linguistics (NAACL), 2024
Dongryeol Lee
Yerin Hwang
Yongil Kim
Joonsuk Park
Kyomin Jung
ELM
412
18
0
28 Oct 2024
LLMScan: Causal Scan for LLM Misbehavior Detection
Mengdi Zhang
Kai Kiat Goh
Peixin Zhang
Jun Sun
Rose Lin Xin
Hongyu Zhang
627
5
0
22 Oct 2024
PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction
Long Xing
Qidong Huang
Xiaoyi Dong
Jiajie Lu
Pan Zhang
...
Yuhang Cao
Bin Wang
Jiaqi Wang
Feng Wu
Dahua Lin
VLM
334
133
0
22 Oct 2024
Mitigating Object Hallucination via Concentric Causal Attention
Neural Information Processing Systems (NeurIPS), 2024
Yun Xing
Yiheng Li
Ivan Laptev
Shijian Lu
277
40
0
21 Oct 2024
Reducing Hallucinations in Vision-Language Models via Latent Space Steering
Sheng Liu
Haotian Ye
Lei Xing
James Zou
VLM
LLMSV
379
37
0
21 Oct 2024
A Survey of Hallucination in Large Visual Language Models
Wei Lan
Wenyi Chen
Qingfeng Chen
Shirui Pan
Huiyu Zhou
Yi-Lun Pan
LRM
315
12
0
20 Oct 2024
Modality-Fair Preference Optimization for Trustworthy MLLM Alignment
International Joint Conference on Artificial Intelligence (IJCAI), 2024
Songtao Jiang
Yan Zhang
Ruizhe Chen
Yeying Jin
Zuozhu Liu
Qinglin He
Yang Feng
Jian Wu
Zuozhu Liu
MoE
MLLM
316
18
0
20 Oct 2024
Trust but Verify: Programmatic VLM Evaluation in the Wild
Viraj Prabhu
Senthil Purushwalkam
An Yan
Caiming Xiong
Ran Xu
MLLM
163
2
0
17 Oct 2024
The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audio
Sicong Leng
Yun Xing
Zesen Cheng
Yang Zhou
Hang Zhang
Xin Li
Deli Zhao
Shijian Lu
Chunyan Miao
Lidong Bing
324
25
0
16 Oct 2024
A Theoretical Survey on Foundation Models
Shi Fu
Yuzhu Chen
Yingjie Wang
Dacheng Tao
304
0
0
15 Oct 2024
MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation
International Conference on Learning Representations (ICLR), 2024
Chenxi Wang
Xiang Chen
Ningyu Zhang
Bozhong Tian
Haoming Xu
Shumin Deng
Ningyu Zhang
MLLM
LRM
788
49
0
15 Oct 2024
LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models
Han Qiu
Jiaxing Huang
Peng Gao
Qin Qi
Xiaoqin Zhang
Ling Shao
Shijian Lu
HILM
291
6
0
13 Oct 2024
Insight Over Sight: Exploring the Vision-Knowledge Conflicts in Multimodal LLMs
Annual Meeting of the Association for Computational Linguistics (ACL), 2024
Xiaoyuan Liu
Wenxuan Wang
Youliang Yuan
Shu Yang
Qiuzhi Liu
Pinjia He
Zhaopeng Tu
951
2
0
10 Oct 2024
From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models
Yuying Shang
Xinyi Zeng
Yutao Zhu
Xiao Yang
Zhengwei Fang
Jingyuan Zhang
Jiawei Chen
Zinan Liu
Yu Tian
VLM
MLLM
818
2
0
09 Oct 2024
Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality
International Conference on Learning Representations (ICLR), 2024
Guanyu Zhou
Yibo Yan
Xin Zou
Kun Wang
Aiwei Liu
Xuming Hu
230
22
0
07 Oct 2024
Differential Transformer
International Conference on Learning Representations (ICLR), 2024
Tianzhu Ye
Li Dong
Yuqing Xia
Yutao Sun
Yi Zhu
Gao Huang
Furu Wei
1.2K
0
0
07 Oct 2024
DAMRO: Dive into the Attention Mechanism of LVLM to Reduce Object Hallucination
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Xuan Gong
Tianshi Ming
Xinpeng Wang
Zhihua Wei
MLLM
398
36
0
06 Oct 2024
SELU: Self-Learning Embodied MLLMs in Unknown Environments
Boyu Li
Haobin Jiang
Haobin Jiang
Weishuai Zeng
Haoran Li
Dongbin Zhao
Zongqing Lu
LRM
188
6
0
04 Oct 2024
Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models
Xin Zou
Yizhou Wang
Yibo Yan
Yuanhuiyi Lyu
Kening Zheng
...
Junkai Chen
Peijie Jiang
Qingbin Liu
Chang Tang
Xuming Hu
461
28
0
04 Oct 2024
Interpreting and Editing Vision-Language Representations to Mitigate Hallucinations
International Conference on Learning Representations (ICLR), 2024
Nick Jiang
Anish Kachinthaya
Suzie Petryk
Yossi Gandelsman
VLM
412
62
0
03 Oct 2024
HELPD: Mitigating Hallucination of LVLMs by Hierarchical Feedback Learning with Vision-enhanced Penalty Decoding
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Fan Yuan
Chi Qin
Xiaogang Xu
Piji Li
VLM
MLLM
162
9
0
30 Sep 2024
A Unified Hallucination Mitigation Framework for Large Vision-Language Models
Yue Chang
Liqiang Jing
Xiaopeng Zhang
Yue Zhang
VLM
MLLM
223
5
0
24 Sep 2024
Mitigating Hallucination in Visual-Language Models via Re-Balancing Contrastive Decoding
Chinese Conference on Pattern Recognition and Computer Vision (CPRCV), 2024
Xiaoyu Liang
Jiayuan Yu
Lianrui Mu
Jiedong Zhuang
Jiaqi Hu
Yuchen Yang
Jiangnan Ye
Lu Lu
Jian Chen
Haoji Hu
VLM
135
7
0
10 Sep 2024
Understanding Multimodal Hallucination with Parameter-Free Representation Alignment
Yueqian Wang
Jianxin Liang
Yuxuan Wang
Huishuai Zhang
Dongyan Zhao
237
2
0
02 Sep 2024
Pre-Training Multimodal Hallucination Detectors with Corrupted Grounding Data
Spencer Whitehead
Jacob Phillips
Sean Hendryx
183
0
0
30 Aug 2024
Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning
International Conference on Computational Linguistics (COLING), 2024
Xiaoye Qu
Jiashuo Sun
Wei Wei
Yu Cheng
MLLM
LRM
277
21
0
30 Aug 2024
LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillation
International Conference on Learning Representations (ICLR), 2024
Fangxun Shu
Yue Liao
Le Zhuo
Chenning Xu
Guanghao Zhang
...
Bolin Li
Zhelun Yu
Si Liu
Hongsheng Li
Hao Jiang
VLM
MoE
210
33
0
28 Aug 2024
ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models
AAAI Conference on Artificial Intelligence (AAAI), 2024
Yeji Park
Deokyeong Lee
Junsuk Choe
Buru Chang
MLLM
VLM
240
9
0
25 Aug 2024
RoVRM: A Robust Visual Reward Model Optimized via Auxiliary Textual Preference Data
Chenglong Wang
Yang Gan
Yifu Huo
Yongyu Mu
Murun Yang
...
Chunliang Zhang
Tongran Liu
Quan Du
Di Yang
Jingbo Zhu
VLM
414
11
0
22 Aug 2024
Sycophancy in Vision-Language Models: A Systematic Analysis and an Inference-Time Mitigation Framework
Yunpu Zhao
Rui Zhang
Junbin Xiao
Changxin Ke
Ruibo Hou
Yifan Hao
Qi Guo
237
4
0
21 Aug 2024
Reefknot: A Comprehensive Benchmark for Relation Hallucination Evaluation, Analysis and Mitigation in Multimodal Large Language Models
Annual Meeting of the Association for Computational Linguistics (ACL), 2024
Kening Zheng
Junkai Chen
Yibo Yan
Xin Zou
Xuming Hu
693
17
0
18 Aug 2024
Reference-free Hallucination Detection for Large Vision-Language Models
Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024
Qing Li
Chenyang Lyu
Fauzan Farooqui
Derui Zhu
Maxim Panov
Fakhri Karray
175
11
0
11 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models
International Conference on Learning Representations (ICLR), 2024
Fushuo Huo
Wenchao Xu
Zhong Zhang
Yining Qi
Zhicheng Chen
Peilin Zhao
VLM
MLLM
580
59
0
04 Aug 2024
Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs
ACM Multimedia (MM), 2024
Peng Ding
Jingyu Wu
Mark Girolami
Dan Ma
Xuezhi Cao
Xunliang Cai
Shi Chen
T. J. Sullivan
Shujian Huang
AAML
VLM
MLLM
206
17
0
02 Aug 2024
Alleviating Hallucination in Large Vision-Language Models with Active Retrieval Augmentation
Cephas Mpungu
Qiyuan Chen
Xiaoye Qu
Jiashuo Sun
G. Mapp
VLM
RALM
LRM
230
30
0
01 Aug 2024
Mitigating Multilingual Hallucination in Large Vision-Language Models
Xiaoye Qu
Mingyang Song
Xiaoye Qu
Jianfeng Dong
Yu Cheng
VLM
LRM
267
5
0
01 Aug 2024
Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs
Shiping Liu
Kecheng Zheng
Wei Chen
MLLM
253
121
0
31 Jul 2024
DOPRA: Decoding Over-accumulation Penalization and Re-allocation in Specific Weighting Layer
Jinfeng Wei
Xiaofeng Zhang
365
32
0
21 Jul 2024
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective
Zhen Qin
Daoyuan Chen
Wenhao Zhang
Liuyi Yao
Yilun Huang
Bolin Ding
Yaliang Li
Shuiguang Deng
347
12
0
11 Jul 2024
Multi-Object Hallucination in Vision-Language Models
Xuweiyi Chen
Ziqiao Ma
Xuejun Zhang
Sihan Xu
Shengyi Qian
Jianing Yang
David Fouhey
Joyce Chai
304
43
0
08 Jul 2024
RULE: Reliable Multimodal RAG for Factuality in Medical Vision Language Models
Peng Xia
Kangyu Zhu
Haoran Li
Hongtu Zhu
Yun Li
Gang Li
Linjun Zhang
Huaxiu Yao
MedIm
219
79
0
06 Jul 2024
MJ-Bench: Is Your Multimodal Reward Model Really a Good Judge for Text-to-Image Generation?
Zhaorun Chen
Yichao Du
Zichen Wen
Yiyang Zhou
Chenhang Cui
...
Jiawei Zhou
Zhuokai Zhao
Rafael Rafailov
Chelsea Finn
Huaxiu Yao
EGVM
MLLM
327
56
0
05 Jul 2024
MM-SpuBench: Towards Better Understanding of Spurious Biases in Multimodal LLMs
Wenqian Ye
Bohan Liu
Yunsheng Ma
Xu Cao
Bolin Lai
James M. Rehg
Bolin Lai
James M. Rehg
Aidong Zhang
199
27
0
24 Jun 2024
Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models?
Gregor Geigle
Radu Timofte
Goran Glavaš
260
2
0
20 Jun 2024
Do More Details Always Introduce More Hallucinations in LVLM-based Image Captioning?
Mingqian Feng
Yunlong Tang
Zeliang Zhang
Chenliang Xu
228
6
0
18 Jun 2024
Previous
1
2
3
4
5
Next
