Title
MCTBench: Multimodal Cognition towards Text-Rich Visual Scenes Benchmark Bin Shan Xiang Fei Wei Shi An-Lan Wang Guozhi Tang Lei Liao Jingqun Tang Xiang Bai Can Huang VLM 25 5 0 15 Oct 2024
VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents S. Yu C. Tang Bokai Xu Junbo Cui Junhao Ran ... Zhenghao Liu Shuo Wang Xu Han Zhiyuan Liu Maosong Sun VLM 37 22 0 14 Oct 2024
CoPESD: A Multi-Level Surgical Motion Dataset for Training Large Vision-Language Models to Co-Pilot Endoscopic Submucosal Dissection Guankun Wang Han Xiao Huxin Gao Renrui Zhang Long Bai Xiaoxiao Yang Zhen Li Hongsheng Li Hongliang Ren 31 4 0 10 Oct 2024
To Preserve or To Compress: An In-Depth Study of Connector Selection in Multimodal Large Language Models Junyan Lin Haoran Chen Dawei Zhu Xiaoyu Shen 20 2 0 09 Oct 2024
Training-Free Open-Ended Object Detection and Segmentation via Attention as Prompts Zhiwei Lin Yongtao Wang Zhi Tang ObjD VLM 28 2 0 08 Oct 2024
Polymath: A Challenging Multi-modal Mathematical Reasoning Benchmark Himanshu Gupta Shreyas Verma Ujjwala Anantheswaran Kevin Scaria Mihir Parmar Swaroop Mishra Chitta Baral ReLM LRM 24 4 0 06 Oct 2024
Unleashing the Potentials of Likelihood Composition for Multi-modal Language Models Shitian Zhao Renrui Zhang Xu Luo Yan Wang Shanghang Zhang Peng Gao 18 0 0 01 Oct 2024
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning Haotian Zhang Mingfei Gao Zhe Gan Philipp Dufter Nina Wenzel ... Haoxuan You Zirui Wang Afshin Dehghan Peter Grasch Yinfei Yang VLM MLLM 36 32 1 30 Sep 2024
UniAff: A Unified Representation of Affordances for Tool Usage and Articulation with Vision-Language Models Qiaojun Yu Siyuan Huang Xibin Yuan Zhengkai Jiang Ce Hao ... Junbo Wang Liu Liu Hongsheng Li Peng Gao Cewu Lu 68 3 0 30 Sep 2024
Attention Prompting on Image for Large Vision-Language Models Runpeng Yu Weihao Yu Xinchao Wang VLM 30 6 0 25 Sep 2024
Phantom of Latent for Large Language and Vision Models Byung-Kwan Lee Sangyun Chung Chae Won Kim Beomchan Park Yong Man Ro VLM LRM 39 6 0 23 Sep 2024
AVG-LLaVA: A Large Multimodal Model with Adaptive Visual Granularity Zhibin Lan Liqiang Niu Fandong Meng Wenbo Li Jie Zhou Jinsong Su VLM 30 2 0 20 Sep 2024
MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Engines Dongzhi Jiang Renrui Zhang Ziyu Guo Yanmin Wu Jiayi Lei ... Guanglu Song Peng Gao Yu Liu Chunyuan Li Hongsheng Li MLLM 27 16 0 19 Sep 2024
TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation Junjie Wen Y. X. Zhu Jinming Li Minjie Zhu Kun Wu ... Ran Cheng Chaomin Shen Yaxin Peng Feifei Feng Jian Tang LM&Ro 56 41 0 19 Sep 2024
POINTS: Improving Your Vision-language Model with Affordable Strategies Yuan Liu Zhongyin Zhao Ziyuan Zhuang Le Tian Xiao Zhou Jie Zhou VLM 35 5 0 07 Sep 2024
EvoChart: A Benchmark and a Self-Training Approach Towards Real-World Chart Understanding Muye Huang Han Lai Xinyu Zhang Wenjun Wu Jie Ma Lingling Zhang Jun Liu 33 4 0 03 Sep 2024
Understanding Multimodal Hallucination with Parameter-Free Representation Alignment Yueqian Wang Jianxin Liang Yuxuan Wang Huishuai Zhang Dongyan Zhao 39 1 0 02 Sep 2024
SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentation Yi-Chia Chen Wei-Hua Li Cheng Sun Yu-Chiang Frank Wang Chu-Song Chen VLM 30 10 0 01 Sep 2024
Law of Vision Representation in MLLMs Shijia Yang Bohan Zhai Quanzeng You Jianbo Yuan Hongxia Yang Chenfeng Xu 40 9 0 29 Aug 2024
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders Min Shi Fuxiao Liu Shihao Wang Shijia Liao Subhashree Radhakrishnan ... Andrew Tao Andrew Tao Zhiding Yu Guilin Liu Guilin Liu MLLM 23 53 0 28 Aug 2024
Building and better understanding vision-language models: insights and future directions Hugo Laurençon Andrés Marafioti Victor Sanh Léo Tronchon VLM 34 60 0 22 Aug 2024
Semantic Communication based on Large Language Model for Underwater Image Transmission Weilong Chen Wenxuan Xu Haoran Chen Xinran Zhang Zhijin Qin Yanru Zhang Zhu Han 21 2 0 08 Aug 2024
Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining Dongyang Liu Shitian Zhao Le Zhuo Weifeng Lin Yu Qiao Xinyue Li Qi Qin Yu Qiao Hongsheng Li Peng Gao MLLM 62 48 0 05 Aug 2024
Dissecting Dissonance: Benchmarking Large Multimodal Models Against Self-Contradictory Instructions Jin Gao Lei Gan Yuankai Li Yixin Ye Dequan Wang 21 2 0 02 Aug 2024
INF-LLaVA: Dual-perspective Perception for High-Resolution Multimodal Large Language Model Yiwei Ma Zhibin Wang Xiaoshuai Sun Weihuang Lin Qiang-feng Zhou Jiayi Ji Rongrong Ji MLLM VLM 39 1 0 23 Jul 2024
Accelerating Pre-training of Multimodal LLMs via Chain-of-Sight Ziyuan Huang Kaixiang Ji Biao Gong Zhiwu Qing Qinglong Zhang Kecheng Zheng Jian Wang Jingdong Chen Ming Yang LRM 34 1 0 22 Jul 2024
SegPoint: Segment Any Point Cloud via Large Language Model Shuting He Henghui Ding Xudong Jiang Bihan Wen 3DV MLLM 3DPC 35 17 0 18 Jul 2024
EarthMarker: Visual Prompt Learning for Region-level and Point-level Remote Sensing Imagery Comprehension Wei Zhang Miaoxin Cai Tong Zhang Jun Li Zhuang Yin Xuerui Mao 61 5 0 18 Jul 2024
MoME: Mixture of Multimodal Experts for Generalist Multimodal Large Language Models Leyang Shen Gongwei Chen Rui Shao Weili Guan Liqiang Nie MoE 35 6 0 17 Jul 2024
MAVIS: Mathematical Visual Instruction Tuning Renrui Zhang Xinyu Wei Dongzhi Jiang Yichi Zhang Ziyu Guo ... Aojun Zhou Bin Wei Shanghang Zhang Peng Gao Hongsheng Li MLLM 27 25 0 11 Jul 2024
HiRes-LLaVA: Restoring Fragmentation Input in High-Resolution Large Vision-Language Models Runhui Huang Xinpeng Ding Chunwei Wang J. N. Han Yulong Liu Hengshuang Zhao Hang Xu Lu Hou Wei Zhang Xiaodan Liang VLM 23 8 0 11 Jul 2024
SEED-Story: Multimodal Long Story Generation with Large Language Model Shuai Yang Yuying Ge Yang Li Yukang Chen Yixiao Ge Ying Shan Yingcong Chen VGen DiffM 73 26 0 11 Jul 2024
LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models Feng Li Renrui Zhang Hao Zhang Yuanhan Zhang Bo Li Wei Li Zejun Ma Chunyuan Li MLLM VLM 36 193 0 10 Jul 2024
A Single Transformer for Scalable Vision-Language Modeling Yangyi Chen Xingyao Wang Hao Peng Heng Ji LRM 40 13 0 08 Jul 2024
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output Pan Zhang Xiaoyi Dong Yuhang Zang Yuhang Cao Rui Qian ... Kai Chen Jifeng Dai Yu Qiao Dahua Lin Jiaqi Wang 43 99 0 03 Jul 2024
TokenPacker: Efficient Visual Projector for Multimodal LLM Wentong Li Yuqian Yuan Jian Liu Dongqi Tang Song Wang Jie Qin Jianke Zhu Lei Zhang MLLM 37 50 0 02 Jul 2024
VSP: Assessing the dual challenges of perception and reasoning in spatial planning tasks for VLMs Qiucheng Wu Handong Zhao Michael Stephen Saxon T. Bui William Yang Wang Yang Zhang Shiyu Chang CoGe 36 4 0 02 Jul 2024
MIA-Bench: Towards Better Instruction Following Evaluation of Multimodal LLMs Yusu Qian Hanrong Ye J. Fauconnier Peter Grasch Yinfei Yang Zhe Gan 108 13 0 01 Jul 2024
MMRo: Are Multimodal LLMs Eligible as the Brain for In-Home Robotics? Jinming Li Yichen Zhu Zhiyuan Xu Jindong Gu Minjie Zhu Xin Liu Ning Liu Yaxin Peng Feifei Feng Jian Tang LRM LM&Ro 26 6 0 28 Jun 2024
DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming Jiaxin Zhang Wentao Yang Songxuan Lai Zecheng Xie Lianwen Jin 32 15 0 27 Jun 2024
S3: A Simple Strong Sample-effective Multimodal Dialog System Elisei Rykov Egor Malkershin Alexander Panchenko 20 0 0 26 Jun 2024
MG-LLaVA: Towards Multi-Granularity Visual Instruction Tuning Xiangyu Zhao Xiangtai Li Haodong Duan Haian Huang Yining Li Kai Chen Hua Yang VLM MLLM 37 10 0 25 Jun 2024
Revisiting Referring Expression Comprehension Evaluation in the Era of Large Multimodal Models Jierun Chen Fangyun Wei Jinjing Zhao Sizhe Song Bohuai Wu Zhuoxuan Peng S.-H. Gary Chan Hongyang R. Zhang 33 8 0 24 Jun 2024
MR-MLLM: Mutual Reinforcement of Multimodal Comprehension and Vision Perception Guanqun Wang Xinyu Wei Jiaming Liu Ray Zhang Yichi Zhang Kevin Zhang Maurice Chong Shanghang Zhang VLM LRM 33 0 0 22 Jun 2024
A3VLM: Actionable Articulation-Aware Vision Language Model Siyuan Huang Haonan Chang Yuhan Liu Yimeng Zhu Hao Dong Peng Gao Abdeslam Boularias Hongsheng Li 36 10 0 11 Jun 2024
RoboMamba: Multimodal State Space Model for Efficient Robot Reasoning and Manipulation Jiaming Liu Mengzhen Liu Zhenyu Wang Lily Lee Kaichen Zhou Pengju An Senqiao Yang Renrui Zhang Yandong Guo Shanghang Zhang LM&Ro LRM Mamba 32 5 0 06 Jun 2024
DeepStack: Deeply Stacking Visual Tokens is Surprisingly Simple and Effective for LMMs Lingchen Meng Jianwei Yang Rui Tian Xiyang Dai Zuxuan Wu Jianfeng Gao Yu-Gang Jiang VLM 22 8 0 06 Jun 2024
Wings: Learning Multimodal LLMs without Text-only Forgetting Yi-Kai Zhang Shiyin Lu Yang Li Yanqing Ma Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang De-Chuan Zhan Han-Jia Ye VLM 33 6 0 05 Jun 2024
SpatialRGPT: Grounded Spatial Reasoning in Vision Language Model An-Chieh Cheng Hongxu Yin Yang Fu Qiushan Guo Ruihan Yang Jan Kautz Xiaolong Wang Sifei Liu LRM 48 43 0 03 Jun 2024
UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment Hantao Zhou Longxiang Tang Rui Yang Guanyi Qin Yan Zhang Runze Hu Xiu Li 29 5 0 03 Jun 2024