Title
Why context matters in VQA and Reasoning: Semantic interventions for VLM input modalities Kenza Amara Lukas Klein Carsten T. Lüth Paul Jäger Hendrik Strobelt Mennatallah El-Assady 25 1 0 02 Oct 2024
LMOD: A Large Multimodal Ophthalmology Dataset and Benchmark for Large Vision-Language Models Zhenyue Qin Yu Yin Dylan Campbell Xuansheng Wu Ke Zou Yih-Chung Tham Ninghao Liu Xiuzhen Zhang Qingyu Chen 36 1 0 02 Oct 2024
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning Haotian Zhang Mingfei Gao Zhe Gan Philipp Dufter Nina Wenzel ... Haoxuan You Zirui Wang Afshin Dehghan Peter Grasch Yinfei Yang VLM MLLM 36 32 1 30 Sep 2024
Emu3: Next-Token Prediction is All You Need Xinlong Wang Xiaosong Zhang Zhengxiong Luo Quan-Sen Sun Yufeng Cui ... Xi Yang Jingjing Liu Yonghua Lin Tiejun Huang Zhongyuan Wang MLLM 34 147 0 27 Sep 2024
E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding Ye Liu Zongyang Ma Zhongang Qi Yang Wu Ying Shan Chang Wen Chen 31 15 0 26 Sep 2024
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions Kai Chen Yunhao Gou Runhui Huang Zhili Liu Daxin Tan ... Qun Liu Jun Yao Lu Hou Hang Xu Hang Xu AuLLM MLLM VLM 62 21 0 26 Sep 2024
Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Multimodal Models Matt Deitke Christopher Clark Sangho Lee Rohun Tripathi Yue Yang ... Noah A. Smith Hannaneh Hajishirzi Ross Girshick Ali Farhadi Aniruddha Kembhavi OSLM VLM 18 8 0 25 Sep 2024
Attention Prompting on Image for Large Vision-Language Models Runpeng Yu Weihao Yu Xinchao Wang VLM 30 5 0 25 Sep 2024
OmniBench: Towards The Future of Universal Omni-Language Models Yizhi Li Ge Zhang Yinghao Ma Ruibin Yuan Kang Zhu ... Zhaoxiang Zhang Zachary Liu Emmanouil Benetos Wenhao Huang Chenghua Lin LRM 44 11 0 23 Sep 2024
Can-Do! A Dataset and Neuro-Symbolic Grounded Framework for Embodied Planning with Large Multimodal Models Yew Ken Chia Qi Sun Lidong Bing Soujanya Poria LM&Ro 29 1 0 22 Sep 2024
A Survey on Multimodal Benchmarks: In the Era of Large AI Models Lin Li Guikun Chen Hanrong Shi Jun Xiao Long Chen 34 9 0 21 Sep 2024
JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images Zhecan Wang Junzhang Liu Chia-Wei Tang Hani Alomari Anushka Sivakumar ... Haoxuan You A. Ishmam Kai-Wei Chang Shih-Fu Chang Chris Thomas CoGe VLM 59 2 0 19 Sep 2024
Oryx MLLM: On-Demand Spatial-Temporal Understanding at Arbitrary Resolution Zuyan Liu Yuhao Dong Ziwei Liu Winston Hu Jiwen Lu Yongming Rao ObjD 74 54 0 19 Sep 2024
NVLM: Open Frontier-Class Multimodal LLMs Wenliang Dai Nayeon Lee Boxin Wang Zhuoling Yang Zihan Liu Jon Barker Tuomas Rintamaki M. Shoeybi Bryan Catanzaro Wei Ping MLLM VLM LRM 40 50 0 17 Sep 2024
CAST: Cross-modal Alignment Similarity Test for Vision Language Models Gautier Dagan Olga Loginova Anil Batra CoGe 72 1 0 17 Sep 2024
TG-LLaVA: Text Guided LLaVA via Learnable Latent Embeddings Dawei Yan Pengcheng Li Yang Li Hao Chen Qingguo Chen Weihua Luo Wei Dong Qingsen Yan Haokui Zhang Chunhua Shen 3DV VLM 37 4 0 15 Sep 2024
One missing piece in Vision and Language: A Survey on Comics Understanding Emanuele Vivoli Andrey Barsky Mohamed Ali Souibgui Artemis LLabres Marco Bertini Dimosthenis Karatzas 34 3 0 14 Sep 2024
SimulBench: Evaluating Language Models with Creative Simulation Tasks Qi Jia Xiang Yue Tianyu Zheng Jie Huang Bill Yuchen Lin LM&MA 31 3 0 11 Sep 2024
MathGLM-Vision: Solving Mathematical Problems with Multi-Modal Large Language Model Zhen Yang Jinhao Chen Zhengxiao Du Wenmeng Yu Weihan Wang Wenyi Hong Zhihuan Jiang Bin Xu Yuxiao Dong Jie Tang VLM LRM 32 8 0 10 Sep 2024
POINTS: Improving Your Vision-language Model with Affordable Strategies Yuan Liu Zhongyin Zhao Ziyuan Zhuang Le Tian Xiao Zhou Jie Zhou VLM 35 5 0 07 Sep 2024
Blocks as Probes: Dissecting Categorization Ability of Large Multimodal Models Bin Fu Qiyang Wan Jialin Li Ruiping Wang Xilin Chen 34 0 0 03 Sep 2024
Training on the Benchmark Is Not All You Need Shiwen Ni Xiangtao Kong Chengming Li Xiping Hu Ruifeng Xu Jia Zhu Min Yang 56 5 0 03 Sep 2024
Seeing Through Their Eyes: Evaluating Visual Perspective Taking in Vision Language Models Gracjan Góral Alicja Ziarko Michal Nauman Maciej Wołczyk LRM 28 1 0 02 Sep 2024
UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarios Baichuan Zhou Haote Yang Dairong Chen Junyan Ye Tianyi Bai Jinhua Yu Songyang Zhang Dahua Lin Conghui He Weijia Li VLM 53 3 0 30 Aug 2024
CogVLM2: Visual Language Models for Image and Video Understanding Wenyi Hong Weihan Wang Ming Ding Wenmeng Yu Qingsong Lv ... Debing Liu Bin Xu Juanzi Li Yuxiao Dong Jie Tang VLM MLLM 45 87 0 29 Aug 2024
Law of Vision Representation in MLLMs Shijia Yang Bohan Zhai Quanzeng You Jianbo Yuan Hongxia Yang Chenfeng Xu 40 9 0 29 Aug 2024
VLM4Bio: A Benchmark Dataset to Evaluate Pretrained Vision-Language Models for Trait Discovery from Biological Images M. Maruf Arka Daw Kazi Sajeed Mehrab Harish Babu Manogaran Abhilash Neog ... Wei-Lun Chao Charles V. Stewart T. Berger-Wolf Wasila Dahdul Anuj Karpatne CoGe 27 3 0 28 Aug 2024
A Survey on Evaluation of Multimodal Large Language Models Jiaxing Huang Jingyi Zhang LM&MA ELM LRM 43 20 0 28 Aug 2024
GlaLSTM: A Concurrent LSTM Stream Framework for Glaucoma Detection via Biomarker Mining Cheng Huang Weizheng Xie Jian Zhou Karanjit S Kooner Karanjit Kooner Yishen Liu 33 1 0 28 Aug 2024
Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders Min Shi Fuxiao Liu Shihao Wang Shijia Liao Subhashree Radhakrishnan ... Andrew Tao Andrew Tao Zhiding Yu Guilin Liu Guilin Liu MLLM 23 53 0 28 Aug 2024
Grounded Multi-Hop VideoQA in Long-Form Egocentric Videos Qirui Chen Shangzhe Di Weidi Xie 19 12 0 26 Aug 2024
Multimodal Contrastive In-Context Learning Yosuke Miyanishi Minh Le Nguyen 32 2 0 23 Aug 2024
IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities Bin Wang Chunyu Xie Dawei Leng Yuhui Yin MLLM 45 1 0 23 Aug 2024
Building and better understanding vision-language models: insights and future directions Hugo Laurençon Andrés Marafioti Victor Sanh Léo Tronchon VLM 34 60 0 22 Aug 2024
EE-MLLM: A Data-Efficient and Compute-Efficient Multimodal Large Language Model Feipeng Ma Yizhou Zhou Hebei Li Zilong He Siying Wu Fengyun Rao Siying Wu Fengyun Rao Yueyi Zhang Xiaoyan Sun 29 3 0 21 Aug 2024
Open-FinLLMs: Open Multimodal Large Language Models for Financial Applications Qianqian Xie Dong Li Mengxi Xiao Zihao Jiang Ruoyu Xiang ... Benyou Wang Alejandro Lopez-Lira Qianqian Xie Sophia Ananiadou Junichi Tsujii AIFin AI4TS 30 14 0 20 Aug 2024
Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation Yuyang Ye Zhi Zheng Yishan Shen Tianshu Wang Hengruo Zhang Peijun Zhu Runlong Yu Kai Zhang Hui Xiong 35 7 0 19 Aug 2024
Quality Assessment in the Era of Large Models: A Survey Zicheng Zhang Yingjie Zhou Chunyi Li Baixuan Zhao Xiaohong Liu Guangtao Zhai 40 10 0 17 Aug 2024
xGen-MM (BLIP-3): A Family of Open Large Multimodal Models Le Xue Manli Shu Anas Awadalla Jun Wang An Yan ... Zeyuan Chen Silvio Savarese Juan Carlos Niebles Caiming Xiong Ran Xu VLM 41 91 0 16 Aug 2024
Level Up Your Tutorials: VLMs for Game Tutorials Quality Assessment Daniele Rege Cambrin Gabriele Scaffidi Militone Luca Colomba Giovanni Malnati D. Apiletti Paolo Garza 21 0 0 15 Aug 2024
CROME: Cross-Modal Adapters for Efficient Multimodal LLM Sayna Ebrahimi Sercan Ö. Arik Tejas Nama Tomas Pfister 37 1 0 13 Aug 2024
VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agents Xiao-Yang Liu Tianjie Zhang Yu Gu Iat Long Iong Yifan Xu ... Zhengxiao Du Chan Hee Song Yu Su Yuxiao Dong Jie Tang VLM LLMAG 33 22 0 12 Aug 2024
VITA: Towards Open-Source Interactive Omni Multimodal LLM Chaoyou Fu Haojia Lin Zuwei Long Yunhang Shen Meng Zhao ... Ran He Rongrong Ji Yunsheng Wu Caifeng Shan Xing Sun MLLM 34 79 0 09 Aug 2024
Openstory++: A Large-scale Dataset and Benchmark for Instance-aware Open-domain Visual Storytelling Zilyu Ye Jinxiu Liu Ruotian Peng Jinjin Cao Zhiyang Chen ... Mingyuan Zhou Xiaoqian Shen Mohamed Elhoseiny Qi Liu Guo-Jun Qi VGen VLM 32 1 0 07 Aug 2024
LLaVA-OneVision: Easy Visual Task Transfer Bo Li Yuanhan Zhang Dong Guo Renrui Zhang Feng Li Hao Zhang Kaichen Zhang Yanwei Li Ziwei Liu Chunyuan Li MLLM SyDa VLM 56 538 0 06 Aug 2024
MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models Fanqing Meng J. Wang Chuanhao Li Quanfeng Lu Hao Tian ... Jifeng Dai Yu Qiao Ping Luo Kaipeng Zhang Wenqi Shao VLM 50 17 0 05 Aug 2024
Lumina-mGPT: Illuminate Flexible Photorealistic Text-to-Image Generation with Multimodal Generative Pretraining Dongyang Liu Shitian Zhao Le Zhuo Weifeng Lin Yu Qiao Xinyue Li Qi Qin Yu Qiao Hongsheng Li Peng Gao MLLM 62 48 0 05 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Haozhao Wang Zhicheng Chen Peilin Zhao VLM MLLM 61 18 0 04 Aug 2024
Dissecting Dissonance: Benchmarking Large Multimodal Models Against Self-Contradictory Instructions Jin Gao Lei Gan Yuankai Li Yixin Ye Dequan Wang 16 2 0 02 Aug 2024
MM-Vet v2: A Challenging Benchmark to Evaluate Large Multimodal Models for Integrated Capabilities Weihao Yu Zhengyuan Yang Linfeng Ren Linjie Li Jianfeng Wang K. Lin Chung-Ching Lin Zicheng Liu Lijuan Wang Xinchao Wang VLM MLLM 31 17 0 01 Aug 2024