Title
Benchmarking Vision Language Models on German Factual Data René Peinl Vincent Tischler CoGe 55 0 0 15 Apr 2025
LiveVQA: Live Visual Knowledge Seeking Mingyang Fu Yuyang Peng Benlin Liu Yao Wan D. Z. Chen 26 0 0 07 Apr 2025
M2IV: Towards Efficient and Fine-grained Multimodal In-Context Learning in Large Vision-Language Models Yanshu Li Hongyang He Yi Cao Qisen Cheng Xiang Fu Ruixiang Tang VLM 35 0 0 06 Apr 2025
JEEM: Vision-Language Understanding in Four Arabic Dialects Karima Kadaoui Hanin Atwany Hamdan Al-Ali Abdelrahman Mohamed Ali Mekky Sergei Tilga Natalia Fedorova Ekaterina Artemova Hanan Aldarmaki Yova Kementchedjhieva VLM 32 1 0 27 Mar 2025
PM4Bench: A Parallel Multilingual Multi-Modal Multi-task Benchmark for Large Vision Language Model Junyuan Gao Jiahe Song J. Wu Runchuan Zhu Guanlin Shen ... Weijia Li Bin Wang D. Lin Lijun Wu Conghui He 79 0 0 24 Mar 2025
Multi-Agent Multimodal Models for Multicultural Text to Image Generation Parth Bhalerao Mounika Yalamarty Brian Trinh Oana Ignat 32 0 0 21 Feb 2025
Beyond Words: Exploring Cultural Value Sensitivity in Multimodal Models Srishti Yadav Zhi Zhang Daniel Hershcovich Ekaterina Shutova 31 0 0 18 Feb 2025
All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages Ashmal Vayani Dinura Dissanayake Hasindri Watawana Noor Ahsan Nevasini Sasikumar ... Monojit Choudhury Ivan Laptev Mubarak Shah Salman Khan Fahad A Khan 124 8 0 25 Nov 2024
CAMEL-Bench: A Comprehensive Arabic LMM Benchmark Sara Ghaboura Ahmed Heakl Omkar Thawakar Ali Alharthi Ines Riahi Abduljalil Saif Jorma T. Laaksonen F. Khan Salman Khan Rao Muhammad Anwer 34 0 0 24 Oct 2024
JMMMU: A Japanese Massive Multi-discipline Multimodal Understanding Benchmark for Culture-aware Evaluation Shota Onohara Atsuyuki Miyai Yuki Imajuku Kazuki Egashira Jeonghun Baek Xiang Yue Graham Neubig Kiyoharu Aizawa OSLM 69 1 0 22 Oct 2024
Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities Zheyuan Zhang Fengyuan Hu Jayjun Lee Freda Shi Parisa Kordjamshidi Joyce Chai Ziqiao Ma 39 11 0 22 Oct 2024
WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines Genta Indra Winata Frederikus Hudi Patrick Amadeus Irawan David Anugraha Rifki Afina Putri ... Alham Fikri Aji Taro Watanabe Derry Wijaya Alice H. Oh Chong-Wah Ngo CoGe 92 9 0 16 Oct 2024
DARE: Diverse Visual Question Answering with Robustness Evaluation Hannah Sterz Jonas Pfeiffer Ivan Vulić OOD VLM 11 0 0 26 Sep 2024
A Survey on Multimodal Benchmarks: In the Era of Large AI Models Lin Li Guikun Chen Hanrong Shi Jun Xiao Long Chen 34 8 0 21 Sep 2024
A Survey on Evaluation of Multimodal Large Language Models Jiaxing Huang Jingyi Zhang LM&MA ELM LRM 43 20 0 28 Aug 2024
Benchmarking Vision Language Models for Cultural Understanding Shravan Nayak Kanishk Jain Rabiul Awal Siva Reddy Sjoerd van Steenkiste Lisa Anne Hendricks Karolina Stañczak Aishwarya Agrawal VLM CoGe 44 1 0 15 Jul 2024
Vision-Language Models under Cultural and Inclusive Considerations Antonia Karamolegkou Phillip Rust Yong Cao Ruixiang Cui Anders Søgaard Daniel Hershcovich VLM 45 7 0 08 Jul 2024
CVLUE: A New Benchmark Dataset for Chinese Vision-Language Understanding Evaluation Yuxuan Wang Yijun Liu Fei Yu Chen Huang Kexin Li Zhiguo Wan Wanxiang Che VLM CoGe 19 5 0 01 Jul 2024
Towards Measuring and Modeling "Culture" in LLMs: A Survey Muhammad Farid Adilazuarda Sagnik Mukherjee Pradhyumna Lavania Siddhant Singh Alham Fikri Aji Jacki OÑeill Ashutosh Modi Monojit Choudhury 43 23 0 05 Mar 2024
Multimodal Foundation Models: From Specialists to General-Purpose Assistants Chunyuan Li Zhe Gan Zhengyuan Yang Jianwei Yang Linjie Li Lijuan Wang Jianfeng Gao MLLM 107 221 0 18 Sep 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023
Leveraging Large Language Models for Multiple Choice Question Answering Joshua Robinson Christopher Rytting David Wingate ELM 123 181 0 22 Oct 2022
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 198 1,089 0 20 Sep 2022
Visually Grounded Reasoning across Languages and Cultures Fangyu Liu Emanuele Bugliarello E. Ponti Siva Reddy Nigel Collier Desmond Elliott VLM LRM 92 167 0 28 Sep 2021