Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs

Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs

11 January 2024

Shengbang Tong

Papers citing "Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs"

13 / 63 papers shown

Title
Comparing Apples to Oranges: LLM-powered Multimodal Intention Prediction in an Object Categorization Task Hassan Ali Philipp Allgeuer Stefan Wermter 44 1 0 12 Apr 2024
Dialogue with Robots: Proposals for Broadening Participation and Research in the SLIVAR Community Casey Kennington Malihe Alikhani Heather Pon-Barry Katherine Atwell Yonatan Bisk ... Jivko Sinapov Angela Stewart Matthew Stone Stefanie Tellex Tom Williams 44 0 0 01 Apr 2024
Visual Hallucinations of Multi-modal Large Language Models Wen Huang Hongbin Liu Minxin Guo Neil Zhenqiang Gong MLLM VLM 32 24 0 22 Feb 2024
Subobject-level Image Tokenization Delong Chen Samuel Cahyawijaya Jianfeng Liu Baoyuan Wang Pascale Fung VLM OCL 46 6 0 22 Feb 2024
Exploring Perceptual Limitation of Multimodal Large Language Models Jiarui Zhang Jinyi Hu Mahyar Khayatkhoei Filip Ilievski Maosong Sun LRM 24 10 0 12 Feb 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 126 106 0 08 Feb 2024
Investigating the Catastrophic Forgetting in Multimodal Large Language Models Yuexiang Zhai Shengbang Tong Xiao Li Mu Cai Qing Qu Yong Jae Lee Y. Ma VLM MLLM CLL 72 75 0 19 Sep 2023
The effectiveness of MAE pre-pretraining for billion-scale pretraining Mannat Singh Quentin Duval Kalyan Vasudev Alwala Haoqi Fan Vaibhav Aggarwal ... Piotr Dollár Christoph Feichtenhofer Ross B. Girshick Rohit Girdhar Ishan Misra LRM 102 62 0 23 Mar 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 218 2,232 0 22 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 247 4,186 0 30 Jan 2023
Masked Autoencoders Are Scalable Vision Learners Kaiming He Xinlei Chen Saining Xie Yanghao Li Piotr Dollár Ross B. Girshick ViT TPM 258 7,337 0 11 Nov 2021
ImageNet-21K Pretraining for the Masses T. Ridnik Emanuel Ben-Baruch Asaf Noy Lihi Zelnik-Manor SSeg VLM CLIP 166 676 0 22 Apr 2021
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 279 39,083 0 01 Sep 2014