Title
M $^3$ CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought Qiguang Chen Libo Qin Jin Zhang Zhi Chen Xiao Xu Wanxiang Che LRM 27 34 0 26 May 2024
ConvLLaVA: Hierarchical Backbones as Visual Encoder for Large Multimodal Models Chunjiang Ge Sijie Cheng Ziming Wang Jiale Yuan Yuan Gao Jun Song Shiji Song Gao Huang Bo Zheng MLLM VLM 18 17 0 24 May 2024
M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models Hongyu Wang Jiayu Xu Senwei Xie Ruiping Wang Jialin Li Zhaojie Xie Bin Zhang Chuyan Xiong Xilin Chen ELM VLM LRM 83 7 0 24 May 2024
Calibrated Self-Rewarding Vision Language Models Yiyang Zhou Zhiyuan Fan Dongjie Cheng Sihan Yang Zhaorun Chen Chenhang Cui Xiyao Wang Yun-Qing Li Linjun Zhang Huaxiu Yao VLM 74 26 0 23 May 2024
Unveiling the Tapestry of Consistency in Large Vision-Language Models Yuan Zhang Fei Xiao Tao Huang Chun-Kai Fan Hongyuan Dong Jiawen Li Jiacong Wang Kuan Cheng Shanghang Zhang Haoyuan Guo 37 7 0 23 May 2024
Dense Connector for MLLMs Huanjin Yao Wenhao Wu Taojiannan Yang Yuxin Song Mengxi Zhang Haocheng Feng Yifan Sun Zhiheng Li Wanli Ouyang Jingdong Wang MLLM VLM 32 16 0 22 May 2024
TinyLLaVA Factory: A Modularized Codebase for Small-scale Large Multimodal Models Junlong Jia Ying Hu Xi Weng Yiming Shi Miao Li ... Baichuan Zhou Ziyu Liu Jie Luo Lei Huang Ji Wu 28 9 0 20 May 2024
Efficient Multimodal Large Language Models: A Survey Yizhang Jin Jian Li Yexin Liu Tianjun Gu Kai Wu ... Xin Tan Zhenye Gan Yabiao Wang Chengjie Wang Lizhuang Ma LRM 39 44 0 17 May 2024
SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretation Jonathan Roberts Kai Han N. Houlsby Samuel Albanie 40 12 0 14 May 2024
Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots Chengyue Wu Yixiao Ge Qiushan Guo Jiahao Wang Zhixuan Liang Zeyu Lu Ying Shan Ping Luo MLLM VLM 27 13 0 13 May 2024
CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts Jiachen Li Xinyao Wang Sijie Zhu Chia-Wen Kuo Lu Xu Fan Chen Jitesh Jain Humphrey Shi Longyin Wen MLLM MoE 28 26 0 09 May 2024
MAmmoTH2: Scaling Instructions from the Web Xiang Yue Tuney Zheng Ge Zhang Wenhu Chen ALM LRM 38 77 0 06 May 2024
Vibe-Eval: A hard evaluation suite for measuring progress of multimodal language models Piotr Padlewski Max Bain Matthew Henderson Zhongkai Zhu Nishant Relan ... Che Zheng Cyprien de Masson dÁutume Dani Yogatama Mikel Artetxe Yi Tay VLM 82 26 0 03 May 2024
What matters when building vision-language models? Hugo Laurençon Léo Tronchon Matthieu Cord Victor Sanh VLM 30 155 0 03 May 2024
MANTIS: Interleaved Multi-Image Instruction Tuning Dongfu Jiang Xuan He Huaye Zeng Cong Wei Max W.F. Ku Qian Liu Wenhu Chen VLM MLLM 28 32 0 02 May 2024
MileBench: Benchmarking MLLMs in Long Context Dingjie Song Shunian Chen Guiming Hardy Chen Fei Yu Xiang Wan Benyou Wang VLM 58 34 0 29 Apr 2024
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites Zhe Chen Weiyun Wang Hao Tian Shenglong Ye Zhangwei Gao ... Tong Lu Dahua Lin Yu Qiao Jifeng Dai Wenhai Wang MLLM VLM 49 513 0 25 Apr 2024
SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension Bohao Li Yuying Ge Yi Chen Yixiao Ge Ruimao Zhang Ying Shan VLM 30 39 0 25 Apr 2024
What Makes Multimodal In-Context Learning Work? Folco Bertini Baldassini Mustafa Shukor Matthieu Cord Laure Soulier Benjamin Piwowarski 32 18 0 24 Apr 2024
Wiki-LLaVA: Hierarchical Retrieval-Augmented Generation for Multimodal LLMs Davide Caffagni Federico Cocchi Nicholas Moratelli Sara Sarto Marcella Cornia Lorenzo Baraldi Rita Cucchiara KELM 24 34 0 23 Apr 2024
DesignProbe: A Graphic Design Benchmark for Multimodal Large Language Models Jieru Lin Danqing Huang Tiejun Zhao Dechen Zhan Chin-Yew Lin VLM MLLM 19 3 0 23 Apr 2024
SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation Yuying Ge Sijie Zhao Jinguo Zhu Yixiao Ge Kun Yi Lin Song Chen Li Xiaohan Ding Ying Shan VLM 54 103 0 22 Apr 2024
MoVA: Adapting Mixture of Vision Experts to Multimodal Context Zhuofan Zong Bingqi Ma Dazhong Shen Guanglu Song Hao Shao Dongzhi Jiang Hongsheng Li Yu Liu MoE 37 40 0 19 Apr 2024
BLINK: Multimodal Large Language Models Can See but Not Perceive Xingyu Fu Yushi Hu Bangzheng Li Yu Feng Haoyu Wang Xudong Lin Dan Roth Noah A. Smith Wei-Chiu Ma Ranjay Krishna VLM LRM MLLM 41 107 0 18 Apr 2024
Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models Aitor Ormazabal Che Zheng Cyprien de Masson dÁutume Dani Yogatama Deyu Fu ... Yazheng Yang Yi Tay Yuqi Wang Zhongkai Zhu Zhihui Xie LRM VLM ReLM 22 22 0 18 Apr 2024
Private Attribute Inference from Images with Vision-Language Models Batuhan Tömekçe Mark Vero Robin Staab Martin Vechev VLM PILM 44 6 0 16 Apr 2024
DesignQA: A Multimodal Benchmark for Evaluating Large Language Models' Understanding of Engineering Documentation Anna C. Doris Daniele Grandi Ryan Tomich Md Ferdous Alam Hyunmin Cheong Faez Ahmed 28 14 0 11 Apr 2024
CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledge Yu Ying Chiu Amirhossein Ajalloeian Maria Antoniak Chan Young Park Shuyue Stella Li Mehar Bhatia Sahithya Ravi Yulia Tsvetkov Vered Shwartz Yejin Choi 34 20 0 10 Apr 2024
InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD Xiao-wen Dong Pan Zhang Yuhang Zang Yuhang Cao Bin Wang ... Xingcheng Zhang Jifeng Dai Yuxin Qiao Dahua Lin Jiaqi Wang VLM MLLM 31 107 0 09 Apr 2024
Rethinking How to Evaluate Language Model Jailbreak Hongyu Cai Arjun Arunasalam Leo Y. Lin Antonio Bianchi Z. Berkay Celik ALM 24 5 0 09 Apr 2024
VisualWebBench: How Far Have Multimodal LLMs Evolved in Web Page Understanding and Grounding? Junpeng Liu Yifan Song Bill Yuchen Lin Wai Lam Graham Neubig Yuanzhi Li Xiang Yue VLM 62 39 0 09 Apr 2024
MM-MATH: Advancing Multimodal Math Evaluation with Process Evaluation and Fine-grained Classification Kai Sun Yushi Bai Ji Qi Lei Hou Juanzi Li LRM 24 12 0 07 Apr 2024
IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representations Deqing Fu Ghazal Khalighinejad Ollie Liu Bhuwan Dhingra Dani Yogatama Robin Jia W. Neiswanger 23 14 0 01 Apr 2024
Are We on the Right Way for Evaluating Large Vision-Language Models? Lin Chen Jinsong Li Xiao-wen Dong Pan Zhang Yuhang Zang ... Haodong Duan Jiaqi Wang Yu Qiao Dahua Lin Feng Zhao VLM 61 216 0 29 Mar 2024
Constructing Multilingual Visual-Text Datasets Revealing Visual Multilingual Ability of Vision Language Models Jesse Atuhurra Iqra Ali Tatsuya Hiraoka Hidetaka Kamigaito Tomoya Iwakura Taro Watanabe 38 1 0 29 Mar 2024
Croissant: A Metadata Format for ML-Ready Datasets Mubashara Akhtar Omar Benjelloun Costanza Conforti Pieter Gijsbers Joan Giner-Miguelez ... Slava Tykhonov Joaquin Vanschoren Jos van der Velde Steffen Vogler Carole-Jean Wu 16 28 0 28 Mar 2024
A Review of Multi-Modal Large Language and Vision Models Kilian Carolan Laura Fennelly A. Smeaton VLM 19 22 0 28 Mar 2024
Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models Yanwei Li Yuechen Zhang Chengyao Wang Zhisheng Zhong Yixin Chen Ruihang Chu Shaoteng Liu Jiaya Jia VLM MLLM MoE 32 210 0 27 Mar 2024
ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object Chenshuang Zhang Fei Pan Junmo Kim In So Kweon Chengzhi Mao 36 9 1 27 Mar 2024
Beyond Embeddings: The Promise of Visual Table in Visual Reasoning Yiwu Zhong Zi-Yuan Hu Michael R. Lyu Liwei Wang 21 1 0 27 Mar 2024
Assessment of Multimodal Large Language Models in Alignment with Human Values Zhelun Shi Zhipin Wang Hongxing Fan Zaibin Zhang Lijun Li Yongting Zhang Zhen-fei Yin Lu Sheng Yu Qiao Jing Shao 27 14 0 26 Mar 2024
MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems? Renrui Zhang Dongzhi Jiang Yichi Zhang Haokun Lin Ziyu Guo ... Aojun Zhou Pan Lu Kai-Wei Chang Peng Gao Hongsheng Li 27 165 0 21 Mar 2024
PuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Abstract Visual Patterns Yew Ken Chia Vernon Toh Yan Han Deepanway Ghosal Lidong Bing Soujanya Poria LRM ReLM 33 13 0 20 Mar 2024
When Do We Not Need Larger Vision Models? Baifeng Shi Ziyang Wu Maolin Mao Xin Wang Trevor Darrell VLM LRM 44 23 0 19 Mar 2024
From Pixels to Insights: A Survey on Automatic Chart Understanding in the Era of Large Foundation Models Kung-Hsiang Huang Hou Pong Chan Yi Ren Fung Haoyi Qiu Mingyang Zhou Shafiq R. Joty Shih-Fu Chang Heng Ji AI4TS 62 14 0 18 Mar 2024
EXAMS-V: A Multi-Discipline Multilingual Multimodal Exam Benchmark for Evaluating Vision Language Models Rocktim Jyoti Das Simeon Emilov Hristov Haonan Li Dimitar Iliyanov Dimitrov Ivan Koychev Preslav Nakov CoGe ELM 67 13 0 15 Mar 2024
MM1: Methods, Analysis & Insights from Multimodal LLM Pre-training Brandon McKinzie Zhe Gan J. Fauconnier Sam Dodge Bowen Zhang ... Zirui Wang Ruoming Pang Peter Grasch Alexander Toshev Yinfei Yang MLLM 27 185 0 14 Mar 2024
MoleculeQA: A Dataset to Evaluate Factual Accuracy in Molecular Comprehension Xingyu Lu He Cao Zijing Liu Shengyuan Bai Leqing Chen Yuan Yao Hai-Tao Zheng Yu Li HILM 19 6 0 13 Mar 2024
Lumen: Unleashing Versatile Vision-Centric Capabilities of Large Multimodal Models Yang Jiao Shaoxiang Chen Zequn Jie Jing Chen Lin Ma Yueping Jiang MLLM 29 18 0 12 Mar 2024
An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models Liang Chen Haozhe Zhao Tianyu Liu Shuai Bai Junyang Lin Chang Zhou Baobao Chang MLLM VLM 45 22 0 11 Mar 2024