MM-LLMs: Recent Advances in MultiModal Large Language Models

24 January 2024

Papers citing "MM-LLMs: Recent Advances in MultiModal Large Language Models"

41 / 41 papers shown

Title
Looking Beyond Language Priors: Enhancing Visual Comprehension and Attention in Multimodal Models Aarti Ghatkesar Uddeshya Upadhyay Ganesh Venkatesh VLM 20 0 0 08 May 2025
Retrieval-augmented in-context learning for multimodal large language models in disease classification Zaifu Zhan Shuang Zhou Xiaoshan Zhou Yongkang Xiao J. Wang Jiawen Deng He Zhu Yu Hou Rui Zhang 29 0 0 04 May 2025
Nexus-O: An Omni-Perceptive And -Interactive Model for Language, Audio, And Vision Che Liu Yingji Zhang D. Zhang Weijie Zhang Chenggong Gong ... André Freitas Qifan Wang Z. Xu Rongjuncheng Zhang Yong Dai AuLLM 56 0 0 26 Feb 2025
TinyEmo: Scaling down Emotional Reasoning via Metric Projection Cristian Gutierrez LRM 52 0 0 17 Feb 2025
CE-SDWV: Effective and Efficient Concept Erasure for Text-to-Image Diffusion Models via a Semantic-Driven Word Vocabulary Jiahang Tu Qian Feng Chufan Chen Jiahua Dong Hanbin Zhao Chao Zhang Hui Qian 41 2 0 28 Jan 2025
Understanding Multimodal LLMs: the Mechanistic Interpretability of Llava in Visual Question Answering Zeping Yu Sophia Ananiadou 35 0 0 17 Nov 2024
Multi-modal Speech Transformer Decoders: When Do Multiple Modalities Improve Accuracy? Yiwen Guan V. Trinh Vivek Voleti Jacob Whitehill 21 1 0 13 Sep 2024
VANE-Bench: Video Anomaly Evaluation Benchmark for Conversational LMMs Rohit K Bharadwaj Hanan Gani Muzammal Naseer F. Khan Salman Khan 35 3 0 14 Jun 2024
Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models Jiaqi Li Qianshan Wei Chuanyi Zhang Guilin Qi Miaozeng Du Yongrui Chen Sheng Bi Fan Liu VLM MU 42 12 0 21 May 2024
VisLingInstruct: Elevating Zero-Shot Learning in Multi-Modal Language Models with Autonomous Instruction Optimization Dongsheng Zhu Xunzhu Tang Weidong Han Jinghui Lu Yukun Zhao Guoliang Xing Junfeng Wang Dawei Yin VLM MLLM 38 5 0 12 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 110 106 0 08 Feb 2024
C-RAG: Certified Generation Risks for Retrieval-Augmented Language Models Mintong Kang Nezihe Merve Gürel Ning Yu D. Song Bo-wen Li 71 15 0 05 Feb 2024
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization Yang Jin Zhicheng Sun Kun Xu Kun Xu Liwei Chen ... Yuliang Liu Di Zhang Yang Song Kun Gai Yadong Mu VGen 36 42 0 05 Feb 2024
InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model Xiao-wen Dong Pan Zhang Yuhang Zang Yuhang Cao Bin Wang ... Conghui He Xingcheng Zhang Yu Qiao Dahua Lin Jiaqi Wang VLM MLLM 67 89 0 29 Jan 2024
Small Language Model Meets with Reinforced Vision Vocabulary Haoran Wei Lingyu Kong Jinyue Chen Liang Zhao Zheng Ge En Yu Jian‐Yuan Sun Chunrui Han Xiangyu Zhang VLM 49 14 0 23 Jan 2024
MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer Changyao Tian Xizhou Zhu Yuwen Xiong Weiyun Wang Zhe Chen ... Tong Lu Jie Zhou Hongsheng Li Yu Qiao Jifeng Dai AuLLM 71 40 0 18 Jan 2024
DiffusionGPT: LLM-Driven Text-to-Image Generation System Jie Qin Jie Wu Weifeng Chen Yuxi Ren Huixian Li Hefeng Wu Xuefeng Xiao Rui Wang S. Wen DiffM 46 22 0 18 Jan 2024
LLaVA-Phi: Efficient Multi-Modal Assistant with Small Language Model Yichen Zhu Minjie Zhu Ning Liu Zhicai Ou Xiaofeng Mou Jian Tang 57 48 0 04 Jan 2024
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Zhe Chen Jiannan Wu Wenhai Wang Weijie Su Guo Chen ... Bin Li Ping Luo Tong Lu Yu Qiao Jifeng Dai VLM MLLM 122 149 0 21 Dec 2023
V*: Guided Visual Search as a Core Mechanism in Multimodal LLMs Penghao Wu Saining Xie LRM 34 105 0 21 Dec 2023
CogAgent: A Visual Language Model for GUI Agents Wenyi Hong Weihan Wang Qingsong Lv Jiazheng Xu Wenmeng Yu ... Juanzi Li Bin Xu Yuxiao Dong Ming Ding Jie Tang MLLM 126 122 0 14 Dec 2023
RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-grained Correctional Human Feedback M. Steyvers Yuan Yao Haoye Zhang Taiwen He Yifeng Han ... Xinyue Hu Zhiyuan Liu Hai-Tao Zheng Maosong Sun Tat-Seng Chua MLLM VLM 122 176 0 01 Dec 2023
CoDi-2: In-Context, Interleaved, and Interactive Any-to-Any Generation Zineng Tang Ziyi Yang Mahmoud Khademi Yang Liu Chenguang Zhu Mohit Bansal LRM MLLM AuLLM 41 12 0 30 Nov 2023
mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration Qinghao Ye Haiyang Xu Jiabo Ye Mingshi Yan Anwen Hu Haowei Liu Qi Qian Ji Zhang Fei Huang Jingren Zhou MLLM VLM 114 367 0 07 Nov 2023
Advances in Embodied Navigation Using Large Language Models: A Survey Jinzhou Lin Han Gao Xuxiang Feng Rongtao Xu Changwei Wang Man Zhang Li Guo Shibiao Xu LM&Ro LLMAG 30 9 0 01 Nov 2023
ControlLLM: Augment Language Models with Tools by Searching on Graphs Zhaoyang Liu Zeqiang Lai Zhangwei Gao Erfei Cui Ziheng Li ... Lewei Lu Qifeng Chen Yu Qiao Jifeng Dai Wenhai Wang MLLM 123 20 0 26 Oct 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 198 883 0 27 Apr 2023
Knowledge Transfer from Pre-trained Language Models to Cif-based Speech Recognizers via Hierarchical Distillation Minglun Han Feilong Chen Jing Shi Shuang Xu Bo Xu VLM 17 8 0 30 Jan 2023
GLM-130B: An Open Bilingual Pre-trained Model Aohan Zeng Xiao Liu Zhengxiao Du Zihan Wang Hanyu Lai ... Jidong Zhai Wenguang Chen Peng-Zhen Zhang Yuxiao Dong Jie Tang BDL LRM 237 840 0 05 Oct 2022
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 198 1,089 0 20 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 8,441 0 04 Mar 2022
VLP: A Survey on Vision-Language Pre-training Feilong Chen Duzhen Zhang Minglun Han Xiuyi Chen Jing Shi Shuang Xu Bo Xu VLM 68 208 0 18 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 375 4,010 0 28 Jan 2022
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 228 604 0 14 Oct 2021
Survey: Transformer based Video-Language Pre-training Ludan Ruan Qin Jin VLM ViT 48 44 0 21 Sep 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Yin Cui Boqing Gong ViT 229 499 0 22 Apr 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 272 3,784 0 18 Apr 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 273 845 0 17 Feb 2021
High-Performance Large-Scale Image Recognition Without Normalization Andrew Brock Soham De Samuel L. Smith Karen Simonyan VLM 217 450 0 11 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 288 2,875 0 11 Feb 2021
U-Net: Convolutional Networks for Biomedical Image Segmentation Olaf Ronneberger Philipp Fischer Thomas Brox SSeg 3DV 226 9,999 0 18 May 2015