Evaluating Object Hallucination in Large Vision-Language Models

17 May 2023

Papers citing "Evaluating Object Hallucination in Large Vision-Language Models"

50 / 577 papers shown

Title
LIBMoE: A Library for comprehensive benchmarking Mixture of Experts in Large Language Models Nam V. Nguyen Thong T. Doan Luong Tran Van Nguyen Quang Pham MoE 55 1 0 01 Nov 2024
Unified Triplet-Level Hallucination Evaluation for Large Vision-Language Models J. Wu Tsz Ting Chung Kai Chen Dit-Yan Yeung VLM LRM 53 3 0 30 Oct 2024
MMDocBench: Benchmarking Large Vision-Language Models for Fine-Grained Visual Document Understanding Fengbin Zhu Ziyang Liu Xiang Yao Ng Haohui Wu W. Wang Fuli Feng Chao Wang Huanbo Luan Tat-Seng Chua VLM 35 3 0 25 Oct 2024
COAT: Compressing Optimizer states and Activation for Memory-Efficient FP8 Training Haocheng Xi Han Cai Ligeng Zhu Y. Lu Kurt Keutzer Jianfei Chen Song Han MQ 57 9 0 25 Oct 2024
CAMEL-Bench: A Comprehensive Arabic LMM Benchmark Sara Ghaboura Ahmed Heakl Omkar Thawakar Ali Alharthi Ines Riahi Abduljalil Saif Jorma T. Laaksonen F. Khan Salman Khan Rao Muhammad Anwer 45 0 0 24 Oct 2024
Distill Visual Chart Reasoning Ability from LLMs to MLLMs Wei He Zhiheng Xi Wanxu Zhao Xiaoran Fan Yiwen Ding Zifei Shan Tao Gui Qi Zhang Xuanjing Huang LRM 51 5 0 24 Oct 2024
MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models Ziyu Liu Yuhang Zang Xiaoyi Dong Pan Zhang Yuhang Cao Haodong Duan Conghui He Yuanjun Xiong Dahua Lin Jiaqi Wang 29 7 0 23 Oct 2024
AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Models Kim Sung-Bin Oh Hyun-Bin JungMok Lee Arda Senocak Joon Son Chung Tae-Hyun Oh MLLM VLM 36 3 0 23 Oct 2024
Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities Zheyuan Zhang Fengyuan Hu Jayjun Lee Freda Shi Parisa Kordjamshidi Joyce Chai Ziqiao Ma 48 11 0 22 Oct 2024
LLaVA-KD: A Framework of Distilling Multimodal Large Language Models Y. Cai Jiangning Zhang Haoyang He Xinwei He Ao Tong Zhenye Gan Chengjie Wang X. Bai VLM 19 2 0 21 Oct 2024
Griffon-G: Bridging Vision-Language and Vision-Centric Tasks via Large Multimodal Models Yufei Zhan Hongyin Zhao Yousong Zhu Fan Yang Ming Tang Jinqiao Wang MLLM 43 1 0 21 Oct 2024
Mitigating Object Hallucination via Concentric Causal Attention Yun Xing Yiheng Li Ivan Laptev Shijian Lu 40 17 0 21 Oct 2024
Reducing Hallucinations in Vision-Language Models via Latent Space Steering Sheng Liu Haotian Ye Lei Xing James Zou VLM LLMSV 37 5 0 21 Oct 2024
A Survey of Hallucination in Large Visual Language Models Wei Lan Wenyi Chen Qingfeng Chen Shirui Pan Huiyu Zhou Yi-Lun Pan LRM 28 4 0 20 Oct 2024
Boardwalk Empire: How Generative AI is Revolutionizing Economic Paradigms Subramanyam Sahoo Kamlesh Dutta 28 1 0 19 Oct 2024
Croc: Pretraining Large Multimodal Models with Cross-Modal Comprehension Yin Xie Kaicheng Yang Ninghua Yang Weimo Deng Xiangzi Dai ... Yumeng Wang Xiang An Yongle Zhao Ziyong Feng Jiankang Deng MLLM VLM 35 1 0 18 Oct 2024
Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment Chenhang Cui An Zhang Yiyang Zhou Zhaorun Chen Gelei Deng Huaxiu Yao Tat-Seng Chua 60 4 0 18 Oct 2024
Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers Yuxin Wen Qingqing Cao Qichen Fu Sachin Mehta Mahyar Najibi VLM 25 4 0 17 Oct 2024
PUMA: Empowering Unified MLLM with Multi-granular Visual Generation Rongyao Fang Chengqi Duan Kun Wang Hao Li H. Tian Xingyu Zeng Rui Zhao Jifeng Dai Hongsheng Li Xihui Liu MLLM 34 11 0 17 Oct 2024
$γ-$ MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models Yaxin Luo Gen Luo Jiayi Ji Yiyi Zhou Xiaoshuai Sun Zhiqiang Shen Rongrong Ji VLM MoE 37 1 0 17 Oct 2024
Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation Chengyue Wu Xiaokang Chen Z. F. Wu Yiyang Ma Xingchao Liu ... Wen Liu Zhenda Xie Xingkai Yu Chong Ruan Ping Luo AI4TS 52 70 0 17 Oct 2024
Improving Multi-modal Large Language Model through Boosting Vision Capabilities Yanpeng Sun H. Zhang Qiang Chen Xinyu Zhang Nong Sang Gang Zhang Jingdong Wang Zechao Li 26 5 0 17 Oct 2024
Trust but Verify: Programmatic VLM Evaluation in the Wild Viraj Prabhu Senthil Purushwalkam An Yan Caiming Xiong R. Xu MLLM 26 1 0 17 Oct 2024
The Curse of Multi-Modalities: Evaluating Hallucinations of Large Multimodal Models across Language, Visual, and Audio Sicong Leng Yun Xing Zesen Cheng Yang Zhou Hang Zhang Xin Li Deli Zhao Shijian Lu Chunyan Miao Lidong Bing 25 7 0 16 Oct 2024
Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models Shicheng Xu Liang Pang Yunchang Zhu Huawei Shen Xueqi Cheng MLLM 33 1 0 16 Oct 2024
MMFuser: Multimodal Multi-Layer Feature Fuser for Fine-Grained Vision-Language Understanding Yue Cao Yangzhou Liu Zhe Chen Guangchen Shi Wenhai Wang Danhuai Zhao Tong Lu 41 5 0 15 Oct 2024
Automatically Generating Visual Hallucination Test Cases for Multimodal Large Language Models Zhongye Liu Hongbin Liu Yuepeng Hu Zedian Shao Neil Zhenqiang Gong VLM MLLM 21 0 0 15 Oct 2024
MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation Chenxi Wang Xiang Chen N. Zhang Bozhong Tian Haoming Xu Shumin Deng H. Chen MLLM LRM 29 4 0 15 Oct 2024
When Does Perceptual Alignment Benefit Vision Representations? Shobhita Sundaram Stephanie Fu Lukas Muttenthaler Netanel Y. Tamir Lucy Chai Simon Kornblith Trevor Darrell Phillip Isola 49 12 1 14 Oct 2024
MEV Capture Through Time-Advantaged Arbitrage Robin Fritsch Maria Ines Silva A. Mamageishvili Benjamin Livshits E. Felten 26 5 0 14 Oct 2024
Can We Predict Performance of Large Models across Vision-Language Tasks? Qinyu Zhao Ming Xu Kartik Gupta Akshay Asthana Liang Zheng Stephen Gould 37 0 0 14 Oct 2024
Locality Alignment Improves Vision-Language Models Ian Covert Tony Sun James Y. Zou Tatsunori Hashimoto VLM 64 3 0 14 Oct 2024
LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models Han Qiu Jiaxing Huang Peng Gao Qin Qi Xiaoqin Zhang Ling Shao Shijian Lu HILM 19 1 0 13 Oct 2024
Skipping Computations in Multimodal LLMs Mustafa Shukor Matthieu Cord 24 2 0 12 Oct 2024
Emerging Pixel Grounding in Large Multimodal Models Without Grounding Supervision Shengcao Cao Liang-Yan Gui Yu-Xiong Wang 41 3 0 10 Oct 2024
Insight Over Sight? Exploring the Vision-Knowledge Conflicts in Multimodal LLMs Xiaoyuan Liu Wenxuan Wang Youliang Yuan Jen-tse Huang Qiuzhi Liu Pinjia He Zhaopeng Tu 40 0 0 10 Oct 2024
ElasticTok: Adaptive Tokenization for Image and Video Wilson Yan Matei A. Zaharia Volodymyr Mnih Pieter Abbeel Aleksandra Faust Hao Liu VGen 36 6 0 10 Oct 2024
Deciphering Cross-Modal Alignment in Large Vision-Language Models with Modality Integration Rate Qidong Huang Xiaoyi Dong Pan Zhang Yuhang Zang Yuhang Cao Jiaqi Wang Dahua Lin Weiming Zhang Nenghai Yu 46 5 0 09 Oct 2024
VHELM: A Holistic Evaluation of Vision Language Models Tony Lee Haoqin Tu Chi Heem Wong Wenhao Zheng Yiyang Zhou ... Josselin Somerville Roberts Michihiro Yasunaga Huaxiu Yao Cihang Xie Percy Liang VLM 37 10 0 09 Oct 2024
From Pixels to Tokens: Revisiting Object Hallucinations in Large Vision-Language Models Yuying Shang Xinyi Zeng Yutao Zhu Xiao Yang Zhengwei Fang Jingyuan Zhang Jiawei Chen Zinan Liu Yu Tian VLM MLLM 41 1 0 09 Oct 2024
LLM Self-Correction with DeCRIM: Decompose, Critique, and Refine for Enhanced Following of Instructions with Multiple Constraints Thomas Palmeira Ferraz Kartik Mehta Yu-Hsiang Lin Haw-Shiuan Chang Shereen Oraby Sijia Liu Vivek Subramanian Tagyoung Chung Mohit Bansal Nanyun Peng 48 7 0 09 Oct 2024
Treat Visual Tokens as Text? But Your MLLM Only Needs Fewer Efforts to See Phu Pham Phu Pham Kun Wan Yu-Jhe Li Zeliang Zhang Daniel Miranda Ajinkya Kale Ajinkya Kale Chenliang Xu 22 5 0 08 Oct 2024
EMMA: Empowering Multi-modal Mamba with Structural and Hierarchical Alignment Yifei Xing Xiangyuan Lan Ruiping Wang D. Jiang Wenjun Huang Qingfang Zheng Yaowei Wang Mamba 33 0 0 08 Oct 2024
On the Modeling Capabilities of Large Language Models for Sequential Decision Making Martin Klissarov Devon Hjelm Alexander Toshev Bogdan Mazoure LM&Ro ELM OffRL LRM 29 2 0 08 Oct 2024
HyperINF: Unleashing the HyperPower of the Schulz's Method for Data Influence Estimation Xinyu Zhou Simin Fan Martin Jaggi TDI 20 0 0 07 Oct 2024
Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality Guanyu Zhou Yibo Yan Xin Zou Kun Wang Aiwei Liu Xuming Hu 36 7 0 07 Oct 2024
TLDR: Token-Level Detective Reward Model for Large Vision Language Models Deqing Fu Tong Xiao Rui Wang Wang Zhu Pengchuan Zhang Guan Pang Robin Jia Lawrence Chen 58 5 0 07 Oct 2024
Geometric Analysis of Reasoning Trajectories: A Phase Space Approach to Understanding Valid and Invalid Multi-Hop Reasoning in LLMs Javier Marin LRM 56 0 0 06 Oct 2024
Gamified crowd-sourcing of high-quality data for visual fine-tuning Shashank Yadav Rohan Tomar Garvit Jain Chirag Ahooja Shubham Chaudhary Charles Elkan 28 0 0 05 Oct 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 74 25 0 04 Oct 2024