Title
A Benchmark for Multi-modal Foundation Models on Low-level Vision: from Single Images to Pairs Zicheng Zhang Haoning Wu Erli Zhang Guangtao Zhai Weisi Lin VLM 24 8 0 11 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 126 106 0 08 Feb 2024
Enhancing Multimodal Large Language Models with Vision Detection Models: An Empirical Study Qirui Jiao Daoyuan Chen Yilun Huang Yaliang Li Ying Shen 23 12 0 31 Jan 2024
Proximity QA: Unleashing the Power of Multi-Modal Large Language Models for Spatial Proximity Analysis Jianing Li Xi Nan Ming Lu Li Du Shanghang Zhang 40 1 0 31 Jan 2024
EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain Wei Zhang Miaoxin Cai Tong Zhang Zhuang Yin Xuerui Mao 19 87 0 30 Jan 2024
Exploring the Reasoning Abilities of Multimodal Large Language Models (MLLMs): A Comprehensive Survey on Emerging Trends in Multimodal Reasoning Yiqi Wang Wentao Chen Xiaotian Han Xudong Lin Haiteng Zhao Yongfei Liu Bohan Zhai Jianbo Yuan Quanzeng You Hongxia Yang LRM 33 66 0 10 Jan 2024
Uni3D-LLM: Unifying Point Cloud Perception, Generation and Editing with Large Language Models Dingning Liu Xiaoshui Huang Yuenan Hou Zhihui Wang Zhen-fei Yin Yongshun Gong Peng Gao Wanli Ouyang 11 8 0 09 Jan 2024
CaMML: Context-Aware Multimodal Learner for Large Models Yixin Chen Shuai Zhang Boran Han Tong He Bo Li VLM 19 4 0 06 Jan 2024
ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning Fanqing Meng Wenqi Shao Quanfeng Lu Peng Gao Kaipeng Zhang Yu Qiao Ping Luo 27 45 0 04 Jan 2024
ChartBench: A Benchmark for Complex Visual Reasoning in Charts Zhengzhuo Xu Sinan Du Yiyan Qi Chengjin Xu Chun Yuan Jian Guo 33 34 0 26 Dec 2023
GSVA: Generalized Segmentation via Multimodal Large Language Models Zhuofan Xia Dongchen Han Yizeng Han Xuran Pan Shiji Song Gao Huang VLM 23 54 0 15 Dec 2023
Assessing GPT4-V on Structured Reasoning Tasks Mukul Singh J. Cambronero Sumit Gulwani Vu Le Gust Verbruggen LRM 35 10 0 13 Dec 2023
GlitchBench: Can large multimodal models detect video game glitches? Mohammad Reza Taesiri Tianjun Feng Anh Nguyen C. Bezemer MLLM VLM LRM 25 9 0 08 Dec 2023
Prompt Highlighter: Interactive Control for Multi-Modal LLMs Yuechen Zhang Shengju Qian Bohao Peng Shu-Lin Liu Jiaya Jia MLLM 32 21 0 07 Dec 2023
$M$^{2}$Chat: Empowering VLM for Multimodal LLM Interleaved Text-Image Generation$ M $^{2}$ Chat: Empowering VLM for Multimodal LLM Interleaved Text-Image Generation Xiaowei Chi Rongyu Zhang Zhengkai Jiang Yijiang Liu Ziyi Lin ... Chaoyou Fu Peng Gao Shanghang Zhang Qi-fei Liu Yi-Ting Guo MLLM 33 1 0 29 Nov 2023
Compositional Chain-of-Thought Prompting for Large Multimodal Models Chancharik Mitra Brandon Huang Trevor Darrell Roei Herzig MLLM LRM 26 80 0 27 Nov 2023
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI Xiang Yue Yuansheng Ni Kai Zhang Tianyu Zheng Ruoqi Liu ... Yibo Liu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen OSLM ELM VLM 66 729 0 27 Nov 2023
Enhancing Visual Grounding and Generalization: A Multi-Task Cycle Training Approach for Vision-Language Models Xiaoyu Yang Lijian Xu Hao Sun Hongsheng Li Shaoting Zhang ObjD 22 4 0 21 Nov 2023
How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model Shezheng Song Xiaopeng Li Shasha Li Shan Zhao Jie Yu Jun Ma Xiaoguang Mao Weimin Zhang 66 4 0 10 Nov 2023
CogVLM: Visual Expert for Pretrained Language Models Weihan Wang Qingsong Lv Wenmeng Yu Wenyi Hong Ji Qi ... Bin Xu Juanzi Li Yuxiao Dong Ming Ding Jie Tang VLM MLLM 17 445 0 06 Nov 2023
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 160 440 0 14 Oct 2023
SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models Xiaoxuan Wang Ziniu Hu Pan Lu Yanqiao Zhu Jieyu Zhang Satyen Subramaniam Arjun R. Loomba Shichang Zhang Yizhou Sun Wei Wang ELM LRM 23 84 0 20 Jul 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Bill Xu Enhong Chen MLLM LRM 33 551 0 23 Jun 2023
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models Chaoyou Fu Peixian Chen Yunhang Shen Yulei Qin Mengdan Zhang ... Xiawu Zheng Ke Li Xing Sun Zhenyu Qiu Rongrong Ji ELM MLLM 32 756 0 23 Jun 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 206 899 0 27 Apr 2023
Instruction Tuning with GPT-4 Baolin Peng Chunyuan Li Pengcheng He Michel Galley Jianfeng Gao SyDa ALM LM&MA 157 579 0 06 Apr 2023
LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention Renrui Zhang Jiaming Han Chris Liu Peng Gao Aojun Zhou Xiangfei Hu Shilin Yan Pan Lu Hongsheng Li Yu Qiao MLLM 23 737 0 28 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 259 4,223 0 30 Jan 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 207 1,101 0 20 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 303 11,881 0 04 Mar 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 388 4,110 0 28 Jan 2022
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 282 39,170 0 01 Sep 2014