Evaluating Object Hallucination in Large Vision-Language Models

17 May 2023

Papers citing "Evaluating Object Hallucination in Large Vision-Language Models"

50 / 577 papers shown

Title
Aligning Modalities in Vision Large Language Models via Preference Fine-tuning Yiyang Zhou Chenhang Cui Rafael Rafailov Chelsea Finn Huaxiu Yao VLM MLLM 27 89 0 18 Feb 2024
Multi-modal preference alignment remedies regression of visual instruction tuning on language model Shengzhi Li Rongyu Lin Shichao Pei 34 20 0 16 Feb 2024
DoRA: Weight-Decomposed Low-Rank Adaptation Shih-yang Liu Chien-Yi Wang Hongxu Yin Pavlo Molchanov Yu-Chiang Frank Wang Kwang-Ting Cheng Min-Hung Chen 19 335 0 14 Feb 2024
Mitigating Object Hallucination in Large Vision-Language Models via Classifier-Free Guidance Linxi Zhao Yihe Deng Weitong Zhang Quanquan Gu MLLM 15 30 0 13 Feb 2024
Visually Dehallucinative Instruction Generation Sungguk Cha Jusung Lee Younghyun Lee Cheoljong Yang MLLM 8 5 0 13 Feb 2024
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models Siddharth Karamcheti Suraj Nair Ashwin Balakrishna Percy Liang Thomas Kollar Dorsa Sadigh MLLM VLM 57 95 0 12 Feb 2024
Exploring Perceptual Limitation of Multimodal Large Language Models Jiarui Zhang Jinyi Hu Mahyar Khayatkhoei Filip Ilievski Maosong Sun LRM 21 10 0 12 Feb 2024
ViGoR: Improving Visual Grounding of Large Vision Language Models with Fine-Grained Reward Modeling Siming Yan Min Bai Weifeng Chen Xiong Zhou Qixing Huang Erran L. Li VLM 19 18 0 09 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 120 106 0 08 Feb 2024
MobileVLM V2: Faster and Stronger Baseline for Vision Language Model Xiangxiang Chu Limeng Qiao Xinyu Zhang Shuang Xu Fei Wei ... Xiaofei Sun Yiming Hu Xinyang Lin Bo-Wen Zhang Chunhua Shen VLM MLLM 17 94 0 06 Feb 2024
Instruction Makes a Difference Tosin P. Adewumi Nudrat Habib Lama Alkhaled Elisa Barney VLM MLLM 8 1 0 01 Feb 2024
A Survey on Hallucination in Large Vision-Language Models Hanchao Liu Wenyuan Xue Yifei Chen Dapeng Chen Xiutian Zhao Ke Wang Liping Hou Rong-Zhi Li Wei Peng LRM MLLM 14 110 0 01 Feb 2024
Enhancing Multimodal Large Language Models with Vision Detection Models: An Empirical Study Qirui Jiao Daoyuan Chen Yilun Huang Yaliang Li Ying Shen 15 15 0 31 Jan 2024
MouSi: Poly-Visual-Expert Vision-Language Models Xiaoran Fan Tao Ji Changhao Jiang Shuo Li Senjie Jin ... Qi Zhang Xipeng Qiu Xuanjing Huang Zuxuan Wu Yunchun Jiang VLM 18 16 0 30 Jan 2024
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model Xiao-wen Dong Pan Zhang Yuhang Zang Yuhang Cao Bin Wang ... Conghui He Xingcheng Zhang Yu Qiao Dahua Lin Jiaqi Wang VLM MLLM 76 242 0 29 Jan 2024
MoE-LLaVA: Mixture of Experts for Large Vision-Language Models Bin Lin Zhenyu Tang Yang Ye Jiaxi Cui Bin Zhu ... Jinfa Huang Junwu Zhang Yatian Pang Munan Ning Li-ming Yuan VLM MLLM MoE 33 151 0 29 Jan 2024
Temporal Insight Enhancement: Mitigating Temporal Hallucination in Multimodal Large Language Models Li Sun Liuan Wang Jun Sun Takayuki Okatani MLLM 14 0 0 18 Jan 2024
Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs Shengbang Tong Zhuang Liu Yuexiang Zhai Yi-An Ma Yann LeCun Saining Xie VLM MLLM 16 281 0 11 Jan 2024
CaMML: Context-Aware Multimodal Learner for Large Models Yixin Chen Shuai Zhang Boran Han Tong He Bo Li VLM 16 4 0 06 Jan 2024
LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model Yichen Zhu Minjie Zhu Ning Liu Zhicai Ou Xiaofeng Mou Jian Tang 63 89 0 04 Jan 2024
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action Jiasen Lu Christopher Clark Sangho Lee Zichen Zhang Savya Khosla Ryan Marten Derek Hoiem Aniruddha Kembhavi VLM MLLM 27 143 0 28 Dec 2023
MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices Xiangxiang Chu Limeng Qiao Xinyang Lin Shuang Xu Yang Yang ... Fei Wei Xinyu Zhang Bo-Wen Zhang Xiaolin Wei Chunhua Shen MLLM 26 32 0 28 Dec 2023
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Zhe Chen Jiannan Wu Wenhai Wang Weijie Su Guo Chen ... Bin Li Ping Luo Tong Lu Yu Qiao Jifeng Dai VLM MLLM 156 895 0 21 Dec 2023
V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs Penghao Wu Saining Xie LRM 49 120 0 21 Dec 2023
VCoder: Versatile Vision Encoders for Multimodal Large Language Models Jitesh Jain Jianwei Yang Humphrey Shi MLLM 11 24 0 21 Dec 2023
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model Shraman Pramanick Guangxing Han Rui Hou Sayan Nag Ser-Nam Lim Nicolas Ballas Qifan Wang Rama Chellappa Amjad Almahairi VLM MLLM 38 29 0 19 Dec 2023
Mixture of Cluster-conditional LoRA Experts for Vision-language Instruction Tuning Yunhao Gou Zhili Liu Kai Chen Lanqing Hong Hang Xu Aoxue Li Dit-Yan Yeung James T. Kwok Yu Zhang MoE MLLM VLM 34 60 0 19 Dec 2023
Tuning LayerNorm in Attention: Towards Efficient Multi-Modal LLM Finetuning Bingchen Zhao Haoqin Tu Chen Wei Jieru Mei Cihang Xie 6 31 0 18 Dec 2023
Silkie: Preference Distillation for Large Visual Language Models Lei Li Zhihui Xie Mukai Li Shunian Chen Peiyi Wang Liang Chen Yazheng Yang Benyou Wang Lingpeng Kong MLLM 99 67 0 17 Dec 2023
M^2ConceptBase: A Fine-Grained Aligned Concept-Centric Multimodal Knowledge Base Zhiwei Zha Jiaan Wang Zhixu Li Xiangru Zhu Wei Song Yanghua Xiao VLM 18 2 0 16 Dec 2023
Osprey: Pixel Understanding with Visual Instruction Tuning Yuqian Yuan Wentong Li Jian Liu Dongqi Tang Xinjie Luo Chi Qin Lei Zhang Jianke Zhu MLLM VLM 40 76 0 15 Dec 2023
VILA: On Pre-training for Visual Language Models Ji Lin Hongxu Yin Wei Ping Yao Lu Pavlo Molchanov Andrew Tao Huizi Mao Jan Kautz M. Shoeybi Song Han MLLM VLM 15 344 0 12 Dec 2023
Hallucination Augmented Contrastive Learning for Multimodal Large Language Model Chaoya Jiang Haiyang Xu Mengfan Dong Jiaxing Chen Wei Ye Mingshi Yan Qinghao Ye Ji Zhang Fei Huang Shikun Zhang VLM 13 51 0 12 Dec 2023
Honeybee: Locality-enhanced Projector for Multimodal LLM Junbum Cha Wooyoung Kang Jonghwan Mun Byungseok Roh MLLM 18 111 0 11 Dec 2023
Genixer: Empowering Multimodal Large Language Models as a Powerful Data Generator Henry Hengyuan Zhao Pan Zhou Mike Zheng Shou MLLM SyDa 33 7 0 11 Dec 2023
AM-RADIO: Agglomerative Vision Foundation Model -- Reduce All Domains Into One Michael Ranzinger Greg Heinrich Jan Kautz Pavlo Molchanov VLM 26 42 0 10 Dec 2023
Causal-CoG: A Causal-Effect Look at Context Generation for Boosting Multi-modal Language Models Shitian Zhao Zhuowan Li Yadong Lu Alan L. Yuille Yan Wang LRM 60 5 0 09 Dec 2023
GlitchBench: Can large multimodal models detect video game glitches? Mohammad Reza Taesiri Tianjun Feng Anh Nguyen C. Bezemer MLLM VLM LRM 22 9 0 08 Dec 2023
On the Robustness of Large Multimodal Models Against Image Adversarial Attacks Xuanimng Cui Alejandro Aparcedo Young Kyun Jang Ser-Nam Lim AAML VLM 8 38 0 06 Dec 2023
Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models Yushi Hu Otilia Stretcu Chun-Ta Lu Krishnamurthy Viswanathan Kenji Hata Enming Luo Ranjay Krishna Ariel Fuxman VLM LRM MLLM 32 26 0 05 Dec 2023
BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Models Rizhao Cai Zirui Song Dayan Guan Zhenhao Chen Xing Luo Chenyu Yi Alex C. Kot MLLM VLM 12 31 0 05 Dec 2023
TimeChat: A Time-sensitive Multimodal Large Language Model for Long Video Understanding Shuhuai Ren Linli Yao Shicheng Li Xu Sun Lu Hou VLM MLLM 10 174 0 04 Dec 2023
Behind the Magic, MERLIM: Multi-modal Evaluation Benchmark for Large Image-Language Models Andrés Villa Juan Carlos León Alcázar Alvaro Soto Bernard Ghanem MLLM VLM 15 9 0 03 Dec 2023
ViP-LLaVA: Making Large Multimodal Models Understand Arbitrary Visual Prompts Mu Cai Haotian Liu Dennis Park Siva Karthik Mustikovela Gregory P. Meyer Yuning Chai Yong Jae Lee VLM LRM MLLM 32 85 0 01 Dec 2023
RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback M. Steyvers Yuan Yao Haoye Zhang Taiwen He Yifeng Han ... Xinyue Hu Zhiyuan Liu Hai-Tao Zheng Maosong Sun Tat-Seng Chua MLLM VLM 130 176 0 01 Dec 2023
Merlin:Empowering Multimodal LLMs with Foresight Minds En Yu Liang Zhao Yana Wei Jinrong Yang Dongming Wu ... Haoran Wei Tiancai Wang Zheng Ge Xiangyu Zhang Wenbing Tao LRM 10 24 0 30 Nov 2023
OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation Qidong Huang Xiao-wen Dong Pan Zhang Bin Wang Conghui He Jiaqi Wang Dahua Lin Weiming Zhang Neng H. Yu MLLM 26 165 0 29 Nov 2023
Contrastive Vision-Language Alignment Makes Efficient Instruction Learner Lizhao Liu Xinyu Sun Tianhang Xiang Zhuangwei Zhuang Liuren Yin Mingkui Tan VLM 9 2 0 29 Nov 2023
LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models Yanwei Li Chengyao Wang Jiaya Jia VLM MLLM 26 259 0 28 Nov 2023
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark Kunchang Li Yali Wang Yinan He Yizhuo Li Yi Wang ... Jilan Xu Guo Chen Ping Luo Limin Wang Yu Qiao VLM MLLM 46 398 0 28 Nov 2023