Title
Vision Model Pre-training on Interleaved Image-Text Data via Latent Compression Learning Chenyu Yang Xizhou Zhu Jinguo Zhu Weijie Su Junjie Wang ... Lewei Lu Bin Li Jie Zhou Yu Qiao Jifeng Dai VLM CLIP 34 4 0 11 Jun 2024
RoboMamba: Multimodal State Space Model for Efficient Robot Reasoning and Manipulation Jiaming Liu Mengzhen Liu Zhenyu Wang Lily Lee Kaichen Zhou Pengju An Senqiao Yang Renrui Zhang Yandong Guo Shanghang Zhang LM&Ro LRM Mamba 32 5 0 06 Jun 2024
Wings: Learning Multimodal LLMs without Text-only Forgetting Yi-Kai Zhang Shiyin Lu Yang Li Yanqing Ma Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang De-Chuan Zhan Han-Jia Ye VLM 33 6 0 05 Jun 2024
A-Bench: Are LMMs Masters at Evaluating AI-generated Images? Zicheng Zhang H. Wu Chunyi Li Yingjie Zhou Wei Sun Xiongkuo Min Zijian Chen Xiaohong Liu Weisi Lin Guangtao Zhai EGVM 59 15 0 05 Jun 2024
Leveraging Visual Tokens for Extended Text Contexts in Multi-Modal Learning Alex Jinpeng Wang Linjie Li Yiqi Lin Min Li Lijuan Wang Mike Zheng Shou VLM 18 3 0 04 Jun 2024
Parrot: Multilingual Visual Instruction Tuning Hai-Long Sun Da-Wei Zhou Y. Li Shiyin Lu Chao Yi ... Zhao Xu Weihua Luo Kaifu Zhang De-Chuan Zhan Han-Jia Ye MLLM 23 9 0 04 Jun 2024
Dragonfly: Multi-Resolution Zoom Supercharges Large Visual-Language Model Kezhen Chen Rahul Thapa Rahul Chalamala Ben Athiwaratkun S. Song James Y. Zou VLM 50 5 0 03 Jun 2024
Bootstrap3D: Improving 3D Content Creation with Synthetic Data Zeyi Sun Tong Wu Pan Zhang Yuhang Zang Xiao-wen Dong Yuanjun Xiong Dahua Lin Jiaqi Wang 34 0 0 31 May 2024
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis Chaoyou Fu Yuhan Dai Yondong Luo Lei Li Shuhuai Ren ... Tong Bill Xu Xiawu Zheng Enhong Chen Rongrong Ji Xing Sun VLM MLLM 46 297 0 31 May 2024
DeCo: Decoupling Token Compression from Semantic Abstraction in Multimodal Large Language Models Linli Yao Lei Li Shuhuai Ren Lean Wang Yuanxin Liu Xu Sun Lu Hou 35 28 0 31 May 2024
Visual Perception by Large Language Model's Weights Feipeng Ma Hongwei Xue Guangting Wang Yizhou Zhou Fengyun Rao Shilin Yan Yueyi Zhang Siying Wu Mike Zheng Shou Xiaoyan Sun VLM 25 5 0 30 May 2024
Text Guided Image Editing with Automatic Concept Locating and Forgetting Jia Li Lijie Hu Zhixian He Jingfeng Zhang Tianhang Zheng Di Wang DiffM 33 8 0 30 May 2024
Source Code Foundation Models are Transferable Binary Analysis Knowledge Bases Zian Su Xiangzhe Xu Ziyang Huang Kaiyuan Zhang Xiangyu Zhang 32 5 0 30 May 2024
Dr-LLaVA: Visual Instruction Tuning with Symbolic Clinical Grounding Shenghuan Sun Gregory M. Goldgof Alexander Schubert Zhiqing Sun Thomas Hartvigsen A. Butte Ahmed Alaa LM&MA 27 4 0 29 May 2024
Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare Hanwei Zhu Haoning Wu Yixuan Li Zicheng Zhang Baoliang Chen Lingyu Zhu Yuming Fang Guangtao Zhai Weisi Lin Shiqi Wang 38 18 0 29 May 2024
Why are Visually-Grounded Language Models Bad at Image Classification? Yuhui Zhang Alyssa Unell Xiaohan Wang Dhruba Ghosh Yuchang Su Ludwig Schmidt Serena Yeung-Levy VLM 35 27 0 28 May 2024
The Evolution of Multimodal Model Architectures S. Wadekar Abhishek Chaurasia Aman Chadha Eugenio Culurciello 41 14 0 28 May 2024
Seeing the Image: Prioritizing Visual Correlation by Contrastive Alignment Xin Xiao Bohong Wu Jiacong Wang Chunyuan Li Xun Zhou Haoyuan Guo VLM 34 7 0 28 May 2024
Visual Anchors Are Strong Information Aggregators For Multimodal Large Language Model Haogeng Liu Quanzeng You Xiaotian Han Yongfei Liu Huaibo Huang Ran He Hongxia Yang 31 2 0 28 May 2024
Cross-Modal Safety Alignment: Is textual unlearning all you need? Trishna Chakraborty Erfan Shayegani Zikui Cai Nael B. Abu-Ghazaleh M. Salman Asif Yue Dong A. Roy-Chowdhury Chengyu Song 39 15 0 27 May 2024
PromptFix: You Prompt and We Fix the Photo Yongsheng Yu Ziyun Zeng Hang Hua Jianlong Fu Jiebo Luo MLLM DiffM VLM 33 20 0 27 May 2024
Reason3D: Searching and Reasoning 3D Segmentation via Large Language Model Kuan-Chih Huang Xiangtai Li Lu Qi Shuicheng Yan Ming-Hsuan Yang LRM 66 9 0 27 May 2024
Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs Mustafa Shukor Matthieu Cord 64 5 0 26 May 2024
Streaming Long Video Understanding with Large Language Models Rui Qian Xiao-wen Dong Pan Zhang Yuhang Zang Shuangrui Ding Dahua Lin Jiaqi Wang VLM 29 40 0 25 May 2024
How Culturally Aware are Vision-Language Models? Olena Burda-Lassen Aman Chadha Shashank Goswami Vinija Jain VLM 34 0 0 24 May 2024
A Survey on Vision-Language-Action Models for Embodied AI Yueen Ma Zixing Song Yuzheng Zhuang Jianye Hao Irwin King LM&Ro 67 41 0 23 May 2024
TinyLLaVA Factory: A Modularized Codebase for Small-scale Large Multimodal Models Junlong Jia Ying Hu Xi Weng Yiming Shi Miao Li ... Baichuan Zhou Ziyu Liu Jie Luo Lei Huang Ji Wu 30 9 0 20 May 2024
MemeMQA: Multimodal Question Answering for Memes via Rationale-Based Inferencing Siddhant Agarwal Shivam Sharma Preslav Nakov Tanmoy Chakraborty 19 4 0 18 May 2024
SIGMA: An Open-Source Interactive System for Mixed-Reality Task Assistance Research D. Bohus Sean Andrist Nick Saw Ann Paradiso Ishani Chakraborty Mahdi Rad 38 9 0 16 May 2024
ROCOv2: Radiology Objects in COntext Version 2, an Updated Multimodal Image Dataset Johannes Ruckert Louise Bloch Raphael Brüngel Ahmad Idrissi-Yaghir Henning Schafer ... A. G. S. D. Herrera Henning Müller Peter A. Horn F. Nensa Christoph M. Friedrich 37 24 0 16 May 2024
Xmodel-VLM: A Simple Baseline for Multimodal Vision Language Model Wanting Xu Yang Liu Langping He Xucheng Huang Ling Jiang VLM MLLM 26 2 0 15 May 2024
From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control Yide Shentu Philipp Wu Aravind Rajeswaran Pieter Abbeel 32 9 0 08 May 2024
Language-Image Models with 3D Understanding Jang Hyun Cho B. Ivanovic Yulong Cao Edward Schmerling Yue Wang ... Boyi Li Yurong You Philipp Krahenbuhl Yan Wang Marco Pavone LRM 40 16 0 06 May 2024
WorldQA: Multimodal World Knowledge in Videos through Long-Chain Reasoning Yuanhan Zhang Kaichen Zhang Bo-wen Li Fanyi Pu Christopher Arif Setiadharma Jingkang Yang Ziwei Liu VGen 47 7 0 06 May 2024
What matters when building vision-language models? Hugo Laurençon Léo Tronchon Matthieu Cord Victor Sanh VLM 30 156 0 03 May 2024
MANTIS: Interleaved Multi-Image Instruction Tuning Dongfu Jiang Xuan He Huaye Zeng Cong Wei Max W.F. Ku Qian Liu Wenhu Chen VLM MLLM 28 100 0 02 May 2024
Simplifying Multimodality: Unimodal Approach to Multimodal Challenges in Radiology with General-Domain Large Language Model Seonhee Cho Choonghan Kim Jiho Lee Chetan Chilkunda Sujin Choi Joo Heung Yoon 44 0 0 29 Apr 2024
MileBench: Benchmarking MLLMs in Long Context Dingjie Song Shunian Chen Guiming Hardy Chen Fei Yu Xiang Wan Benyou Wang VLM 76 34 0 29 Apr 2024
Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations? Letitia Parcalabescu Anette Frank MLLM CoGe VLM 82 3 0 29 Apr 2024
What Foundation Models can Bring for Robot Learning in Manipulation : A Survey Dingzhe Li Yixiang Jin A. Yong Hongze Yu Jun Shi Xiaoshuai Hao Peng Hao Huaping Liu Fuchun Sun Bin Fang AI4CE LM&Ro 64 13 0 28 Apr 2024
SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension Bohao Li Yuying Ge Yi Chen Yixiao Ge Ruimao Zhang Ying Shan VLM 35 40 0 25 Apr 2024
What Makes Multimodal In-Context Learning Work? Folco Bertini Baldassini Mustafa Shukor Matthieu Cord Laure Soulier Benjamin Piwowarski 32 18 0 24 Apr 2024
DesignProbe: A Graphic Design Benchmark for Multimodal Large Language Models Jieru Lin Danqing Huang Tiejun Zhao Dechen Zhan Chin-Yew Lin VLM MLLM 27 3 0 23 Apr 2024
FINEMATCH: Aspect-based Fine-grained Image and Text Mismatch Detection and Correction Hang Hua Jing Shi Kushal Kafle Simon Jenni Daoan Zhang John Collomosse Scott D. Cohen Jiebo Luo CoGe VLM 42 9 0 23 Apr 2024
From Matching to Generation: A Survey on Generative Information Retrieval Xiaoxi Li Jiajie Jin Yujia Zhou Yuyao Zhang Peitian Zhang Yutao Zhu Zhicheng Dou 3DV 67 45 0 23 Apr 2024
SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generation Yuying Ge Sijie Zhao Jinguo Zhu Yixiao Ge Kun Yi Lin Song Chen Li Xiaohan Ding Ying Shan VLM 60 105 0 22 Apr 2024
BLINK: Multimodal Large Language Models Can See but Not Perceive Xingyu Fu Yushi Hu Bangzheng Li Yu Feng Haoyu Wang Xudong Lin Dan Roth Noah A. Smith Wei-Chiu Ma Ranjay Krishna VLM LRM MLLM 41 108 0 18 Apr 2024
Omniview-Tuning: Boosting Viewpoint Invariance of Vision-Language Pre-training Models Shouwei Ruan Yinpeng Dong Hanqing Liu Yao Huang Hang Su Xingxing Wei VLM 45 1 0 18 Apr 2024
Empowering Large Language Models on Robotic Manipulation with Affordance Prompting Guangran Cheng Chuheng Zhang Wenzhe Cai Li Zhao Changyin Sun Jiang Bian LM&Ro LLMAG 182 9 0 17 Apr 2024
HOI-Ref: Hand-Object Interaction Referral in Egocentric Vision Siddhant Bansal Michael Wray Dima Damen 31 3 0 15 Apr 2024