Title
Unified Multimodal Understanding and Generation Models: Advances, Challenges, and Opportunities X. Zhang Jintao Guo Shanshan Zhao Minghao Fu Lunhao Duan Guo-Hua Wang Qing-Guo Chen Zhao Xu Weihua Luo Kaifu Zhang DiffM 57 0 0 05 May 2025
Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokens Kaihang Pan Wang Lin Zhongqi Yue Tenglong Ao Liyu Jia Wei Zhao Juncheng Billy Li Siliang Tang Hanwang Zhang 39 1 0 20 Apr 2025
MM-IFEngine: Towards Multimodal Instruction Following Shengyuan Ding Shenxi Wu Xiangyu Zhao Yuhang Zang Haodong Duan Xiaoyi Dong Pan Zhang Y. Cao D. Lin Jiaqi Wang OffRL 56 1 0 10 Apr 2025
From Head to Tail: Towards Balanced Representation in Large Vision-Language Models through Adaptive Data Calibration Mingyang Song Xiaoye Qu Jiawei Zhou Yu-Xi Cheng VLM 50 1 0 17 Mar 2025
BREEN: Bridge Data-Efficient Encoder-Free Multimodal Learning with Learnable Queries Tianle Li Yongming Rao Winston Hu Yu Cheng MLLM 61 0 0 16 Mar 2025
Oasis: One Image is All You Need for Multimodal Instruction Data Synthesis Letian Zhang Quan Cui Bingchen Zhao Cheng Yang MLLM SyDa 49 0 0 11 Mar 2025
Referring to Any Person Qing Jiang Lin Wu Zhaoyang Zeng Tianhe Ren Yuda Xiong Yihao Chen Qin Liu Lei Zhang 77 0 0 11 Mar 2025
RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete Yuheng Ji Huajie Tan Jiayu Shi Xiaoshuai Hao Yuan Zhang ... Huaihai Lyu Xiaolong Zheng Jiaming Liu Zhongyuan Wang Shanghang Zhang 84 6 0 28 Feb 2025
M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performance Qingpei Guo Kaiyou Song Zipeng Feng Ziping Ma Qinglong Zhang ... Yunxiao Sun Tai-WeiChang Jingdong Chen Ming Yang Jun Zhou MLLM VLM 77 3 0 26 Feb 2025
Vision-Language Models for Edge Networks: A Comprehensive Survey Ahmed Sharshar Latif U. Khan Waseem Ullah Mohsen Guizani VLM 62 3 0 11 Feb 2025
InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling Yi Wang Xinhao Li Ziang Yan Yinan He Jiashuo Yu ... Kai Chen Wenhai Wang Yu Qiao Yali Wang Limin Wang 73 19 0 21 Jan 2025
Social-LLaVA: Enhancing Robot Navigation through Human-Language Reasoning in Social Spaces Amirreza Payandeh Daeun Song Mohammad Nazeri Jing Liang Praneel Mukherjee Amir Hossain Raj Yangzhe Kong Dinesh Manocha Xuesu Xiao LM&Ro LRM 70 5 0 17 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 91 45 0 03 Jan 2025
On Domain-Specific Post-Training for Multimodal Large Language Models Daixuan Cheng Shaohan Huang Ziyu Zhu Xintong Zhang Wayne Xin Zhao Zhongzhi Luan Bo Dai Zhenliang Zhang VLM 87 2 0 29 Nov 2024
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding Qing Jiang Gen Luo Yuqin Yang Yuda Xiong Yihao Chen Zhaoyang Zeng Tianhe Ren Lei Zhang VLM LRM 105 6 0 27 Nov 2024
VisRAG: Vision-based Retrieval-augmented Generation on Multi-modality Documents S. Yu C. Tang Bokai Xu Junbo Cui Junhao Ran ... Zhenghao Liu Shuo Wang Xu Han Zhiyuan Liu Maosong Sun VLM 34 22 0 14 Oct 2024
Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training Gen Luo Xue Yang Wenhan Dou Zhaokai Wang Jifeng Dai Jifeng Dai Yu Qiao Xizhou Zhu VLM MLLM 59 25 0 10 Oct 2024
MM-Ego: Towards Building Egocentric Multimodal LLMs for Video QA Hanrong Ye Haotian Zhang Erik Daxberger Lin Chen Zongyu Lin ... Haoxuan You Dan Xu Zhe Gan Jiasen Lu Yinfei Yang EgoV MLLM 64 12 0 09 Oct 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 74 25 0 04 Oct 2024
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions Kai Chen Yunhao Gou Runhui Huang Zhili Liu Daxin Tan ... Qun Liu Jun Yao Lu Hou Hang Xu Hang Xu AuLLM MLLM VLM 62 21 0 26 Sep 2024
FullAnno: A Data Engine for Enhancing Image Comprehension of MLLMs Jing Hao Yuxiang Zhao Song Chen Yanpeng Sun Qiang Chen Gang Zhang Kun Yao Errui Ding Jingdong Wang VLM VGen MLLM 23 5 0 20 Sep 2024
Towards Semantic Equivalence of Tokenization in Multimodal LLM Shengqiong Wu Hao Fei Xiangtai Li Jiayi Ji Hanwang Zhang Tat-Seng Chua Shuicheng Yan MLLM 59 31 0 07 Jun 2024
VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Models Zejun Li Ruipu Luo Jiwen Zhang Minghui Qiu Zhongyu Wei Zhongyu Wei LRM MLLM 52 7 0 27 May 2024
Meteor: Mamba-based Traversal of Rationale for Large Language and Vision Models Byung-Kwan Lee Chae Won Kim Beomchan Park Yonghyun Ro MLLM LRM 22 17 0 24 May 2024
TextSquare: Scaling up Text-Centric Visual Instruction Tuning Jingqun Tang Chunhui Lin Zhen Zhao Shubo Wei Binghong Wu ... Yuliang Liu Hao Liu Yuan Xie Xiang Bai Can Huang LRM VLM MLLM 61 28 0 19 Apr 2024
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward Ruohong Zhang Liangke Gui Zhiqing Sun Yihao Feng Keyang Xu ... Di Fu Chunyuan Li Alexander G. Hauptmann Yonatan Bisk Yiming Yang MLLM 43 57 0 01 Apr 2024
Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want Weifeng Lin Xinyu Wei Ruichuan An Peng Gao Bocheng Zou Yulin Luo Siyuan Huang Shanghang Zhang Hongsheng Li VLM 55 32 0 29 Mar 2024
LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model Yichen Zhu Minjie Zhu Ning Liu Zhicai Ou Xiaofeng Mou Jian Tang 63 89 0 04 Jan 2024
Silkie: Preference Distillation for Large Visual Language Models Lei Li Zhihui Xie Mukai Li Shunian Chen Peiyi Wang Liang Chen Yazheng Yang Benyou Wang Lingpeng Kong MLLM 99 67 0 17 Dec 2023
u-LLaVA: Unifying Multi-Modal Tasks via Large Language Model Jinjin Xu Liwu Xu Yuzhe Yang Xiang Li Fanyi Wang Yanchun Xie Yi-Jie Huang Yaqian Li MoE MLLM VLM 24 12 0 09 Nov 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 207 1,089 0 20 Sep 2022
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 273 1,077 0 17 Feb 2021