Title
Visual Data-Type Understanding does not emerge from Scaling Vision-Language Models Vishaal Udandarao Max F. Burg Samuel Albanie Matthias Bethge VLM 24 8 0 12 Oct 2023
MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts Pan Lu Hritik Bansal Tony Xia Jiacheng Liu Chun-yue Li Hannaneh Hajishirzi Hao Cheng Kai-Wei Chang Michel Galley Jianfeng Gao LRM MLLM 36 494 0 03 Oct 2023
HallE-Control: Controlling Object Hallucination in Large Multimodal Models Bohan Zhai Shijia Yang Chenfeng Xu Sheng Shen Kurt Keutzer Chunyuan Li Manling Li MLLM 18 12 0 03 Oct 2023
Fool Your (Vision and) Language Model With Embarrassingly Simple Permutations Yongshuo Zong Tingyang Yu Ruchika Chavhan Bingchen Zhao Timothy M. Hospedales MLLM AAML LRM 24 18 0 02 Oct 2023
Application of frozen large-scale models to multimodal task-oriented dialogue Tatsuki Kawamoto Takuma Suzuki Ko Miyama Takumi Meguro Tomohiro Takagi 19 0 0 02 Oct 2023
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning Mustafa Shukor Alexandre Ramé Corentin Dancette Matthieu Cord LRM MLLM 33 20 0 01 Oct 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 24 92 0 27 Sep 2023
InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition Pan Zhang Xiaoyi Wang Bin Wang Yuhang Cao Chao Xu ... Conghui He Xingcheng Zhang Yu Qiao Da Lin Jiaqi Wang MLLM 61 222 0 26 Sep 2023
Aligning Large Multimodal Models with Factually Augmented RLHF Zhiqing Sun Sheng Shen Shengcao Cao Haotian Liu Chunyuan Li ... Liangyan Gui Yu-xiong Wang Yiming Yang Kurt Keutzer Trevor Darrell VLM 39 311 0 25 Sep 2023
DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention Z. Yao Xiaoxia Wu Conglong Li Minjia Zhang Heyang Qi Olatunji Ruwase A. A. Awan Samyam Rajbhandari Yuxiong He 26 11 0 25 Sep 2023
DreamLLM: Synergistic Multimodal Comprehension and Creation Runpei Dong Chunrui Han Yuang Peng Zekun Qi Zheng Ge ... Hao-Ran Wei Xiangwen Kong Xiangyu Zhang Kaisheng Ma Li Yi MLLM 34 168 0 20 Sep 2023
MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learning Haozhe Zhao Zefan Cai Shuzheng Si Xiaojian Ma Kaikai An Liang Chen Zixuan Liu Sheng Wang Wenjuan Han Baobao Chang MLLM VLM 24 133 0 14 Sep 2023
Language Models as Black-Box Optimizers for Vision-Language Models Shihong Liu Zhiqiu Lin Samuel Yu Ryan Lee Tiffany Ling Deepak Pathak Deva Ramanan VLM 22 28 0 12 Sep 2023
Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization Yang Jin Kun Xu Kun Xu Liwei Chen Chao Liao ... Xiaoqiang Lei Di Zhang Wenwu Ou Kun Gai Yadong Mu MLLM VLM 14 41 0 09 Sep 2023
A Joint Study of Phrase Grounding and Task Performance in Vision and Language Models Noriyuki Kojima Hadar Averbuch-Elor Yoav Artzi 21 2 0 06 Sep 2023
PointLLM: Empowering Large Language Models to Understand Point Clouds Runsen Xu Xiaolong Wang Tai Wang Yilun Chen Jiangmiao Pang Dahua Lin MLLM 51 148 0 31 Aug 2023
Sparkles: Unlocking Chats Across Multiple Images for Multimodal Instruction-Following Models Yupan Huang Zaiqiao Meng Fangyu Liu Yixuan Su Nigel Collier Yutong Lu MLLM 28 22 0 31 Aug 2023
On the Adversarial Robustness of Multi-Modal Foundation Models Christian Schlarmann Matthias Hein AAML 105 84 0 21 Aug 2023
VisIT-Bench: A Benchmark for Vision-Language Instruction Following Inspired by Real-World Use Yonatan Bitton Hritik Bansal Jack Hessel Rulin Shao Wanrong Zhu Anas Awadalla Josh Gardner Rohan Taori L. Schimdt VLM 29 77 0 12 Aug 2023
MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities Weihao Yu Zhengyuan Yang Linjie Li Jianfeng Wang Kevin Qinghong Lin Zicheng Liu Xinchao Wang Lijuan Wang MLLM 36 605 0 04 Aug 2023
Look, Remember and Reason: Grounded reasoning in videos with language models Apratim Bhattacharyya Sunny Panchal Mingu Lee Reza Pourreza Pulkit Madan Roland Memisevic LRM 33 7 0 30 Jun 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Bill Xu Enhong Chen MLLM LRM 33 551 0 23 Jun 2023
OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documents Hugo Laurenccon Lucile Saulnier Léo Tronchon Stas Bekman Amanpreet Singh ... Siddharth Karamcheti Alexander M. Rush Douwe Kiela Matthieu Cord Victor Sanh 25 227 0 21 Jun 2023
Investigating Prompting Techniques for Zero- and Few-Shot Visual Question Answering Rabiul Awal Le Zhang Aishwarya Agrawal LRM 38 12 0 16 Jun 2023
ArtWhisperer: A Dataset for Characterizing Human-AI Interactions in Artistic Creations Kailas Vodrahalli James Y. Zou 25 5 0 13 Jun 2023
Exploring Diverse In-Context Configurations for Image Captioning Xu Yang Yongliang Wu Mingzhuo Yang Haokun Chen Xin Geng MLLM 16 51 0 24 May 2023
On the Hidden Mystery of OCR in Large Multimodal Models Yuliang Liu Zhang Li Mingxin Huang Chunyuan Li Dezhi Peng Mingyu Liu Lianwen Jin Xiang Bai VLM MLLM 18 48 0 13 May 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 206 899 0 27 Apr 2023
MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models Deyao Zhu Jun Chen Xiaoqian Shen Xiang Li Mohamed Elhoseiny VLM MLLM 23 1,896 0 20 Apr 2023
Multimodal Chain-of-Thought Reasoning in Language Models Zhuosheng Zhang Aston Zhang Mu Li Hai Zhao George Karypis Alexander J. Smola LRM 22 400 0 02 Feb 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 259 4,223 0 30 Jan 2023
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 388 4,110 0 28 Jan 2022
An Empirical Study of GPT-3 for Few-Shot Knowledge-Based VQA Zhengyuan Yang Zhe Gan Jianfeng Wang Xiaowei Hu Yumao Lu Zicheng Liu Lijuan Wang 169 402 0 10 Sep 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 3,683 0 11 Feb 2021
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 250 926 0 24 Sep 2019