Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQA

v1v2v3v4v5 (latest)

Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQA

30 May 2024

ArXiv (abs)PDF HTML

Papers citing "Worse than Random? An Embarrassingly Simple Probing Evaluation of Large Multimodal Models in Medical VQA"

12 / 12 papers shown

Title
Toward Automatic Safe Driving Instruction: A Large-Scale Vision Language Model Approach Haruki Sakajo Hiroshi Takato Hiroshi Tsutsui Komei Soda Hidetaka Kamigaito Taro Watanabe MLLM 104 0 0 28 Nov 2025
AMANDA: Agentic Medical Knowledge Augmentation for Data-Efficient Medical Visual Question Answering Ziqing Wang Chengsheng Mao Xiaole Wen Yuan Luo Kaize Ding MedIm 88 0 0 26 Sep 2025
MultiMedEdit: A Scenario-Aware Benchmark for Evaluating Knowledge Editing in Medical VQA Shengtao Wen Haodong Chen Yadong Wang Zhongying Pan Xiang Chen Yu Tian Bo Qian Dong Liang Sheng-Jun Huang KELM 114 0 0 09 Aug 2025
Bringing CLIP to the Clinic: Dynamic Soft Labels and Negation-Aware Learning for Medical AnalysisComputer Vision and Pattern Recognition (CVPR), 2025 Hanbin Ko Chang-Min Park 158 5 0 28 May 2025
MedFrameQA: A Multi-Image Medical VQA Benchmark for Clinical Reasoning Suhao Yu Haojin Wang Juncheng Wu Cihang Xie Yuyin Zhou 183 10 0 22 May 2025
Structure Causal Models and LLMs Integration in Medical Visual Question AnsweringIEEE Transactions on Medical Imaging (IEEE TMI), 2025 Zibo Xu Qiang Li Weizhi Nie Weijie Wang Anan Liu CML MedIm 259 1 0 05 May 2025
Localizing Before Answering: A Hallucination Evaluation Benchmark for Grounded Medical Multimodal LLMs Dung Nguyen Minh Khoi Ho Huy Ta T. Nguyen Qi Chen ... Zhibin Liao Minh-Son To Johan Verjans Phi Le Nguyen Vu Minh Hieu Phan 424 0 0 30 Apr 2025
Tonguescape: Exploring Language Models Understanding of Vowel ArticulationNorth American Chapter of the Association for Computational Linguistics (NAACL), 2025 Haruki Sakajo Yusuke Sakai Hidetaka Kamigaito Taro Watanabe 192 2 0 29 Jan 2025
Uni-Med: A Unified Medical Generalist Foundation Model For Multi-Task Learning Via Connector-MoENeural Information Processing Systems (NeurIPS), 2024 Xun Zhu Ying Hu Fanbin Mo Chenyi Guo Ji Wu 265 15 0 26 Sep 2024
WildHallucinations: Evaluating Long-form Factuality in LLMs with Real-World Entity Queries Wenting Zhao Tanya Goyal Yu Ying Chiu Liwei Jiang Benjamin Newman ... Khyathi Chandu Ronan Le Bras Claire Cardie Yuntian Deng Yejin Choi HILM 167 19 0 24 Jul 2024
A Survey on Trustworthiness in Foundation Models for Medical Image Analysis Congzhen Shi Ryan Rezai Jiaxi Yang Qi Dou Xiaoxiao Li MedIm 191 15 0 03 Jul 2024
Language Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks Jack Gallifant Shan Chen Pedro Moreira Nikolaj Munch Mingye Gao Jackson Pond Leo Anthony Celi Hugo J. W. L. Aerts Thomas Hartvigsen Danielle S. Bitterman 235 23 0 17 Jun 2024