Why are Visually-Grounded Language Models Bad at Image Classification?

Why are Visually-Grounded Language Models Bad at Image Classification?

28 May 2024

Serena Yeung-Levy

Papers citing "Why are Visually-Grounded Language Models Bad at Image Classification?"

16 / 16 papers shown

Title
Rethinking Visual Layer Selection in Multimodal LLMs H. Chen Junyan Lin Xinhao Chen Yue Fan Xin Jin Hui Su Jianfeng Dong Jinlan Fu Xiaoyu Shen VLM 93 0 0 30 Apr 2025
Enhancing Multimodal In-Context Learning for Image Classification through Coreset Optimization Huiyi Chen Jiawei Peng Kaihua Tang Xin Geng Xu Yang 22 0 0 19 Apr 2025
Why Vision Language Models Struggle with Visual Arithmetic? Towards Enhanced Chart and Geometry Understanding Kung-Hsiang Huang Can Qin Haoyi Qiu Philippe Laban Shafiq R. Joty Caiming Xiong C. Wu VLM 73 1 0 17 Feb 2025
TinyEmo: Scaling down Emotional Reasoning via Metric Projection Cristian Gutierrez LRM 60 0 0 17 Feb 2025
Sparse Attention Vectors: Generative Multimodal Model Features Are Discriminative Vision-Language Classifiers Chancharik Mitra Brandon Huang Tianning Chai Zhiqiu Lin Assaf Arbelle Rogerio Feris Leonid Karlinsky Trevor Darrell Deva Ramanan Roei Herzig VLM 116 4 0 28 Nov 2024
MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models Wenbo Hu Jia-Chen Gu Zi-Yi Dou Mohsen Fayyaz Pan Lu Kai-Wei Chang Nanyun Peng VLM 54 4 0 10 Oct 2024
Towards Interpreting Visual Information Processing in Vision-Language Models Clement Neo Luke Ong Philip H. S. Torr Mor Geva David M. Krueger Fazl Barez 84 6 0 09 Oct 2024
GLOV: Guided Large Language Models as Implicit Optimizers for Vision Language Models Muhammad Jehanzeb Mirza Mengjie Zhao Zhuoyuan Mao Sivan Doveh Wei Lin ... Yuki Mitsufuji Horst Possegger Rogerio Feris Leonid Karlinsky James Glass VLM 74 1 0 08 Oct 2024
An Efficient Sign Language Translation Using Spatial Configuration and Motion Dynamics with LLMs Eui Jun Hwang Sukmin Cho Junmyeong Lee Jong C. Park SLR 59 4 0 20 Aug 2024
MAC: A Benchmark for Multiple Attributes Compositional Zero-Shot Learning Shuo Xu Sai Wang Xinyue Hu Yutian Lin Bo Du Yu Wu CoGe 46 0 0 18 Jun 2024
No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance Vishaal Udandarao Ameya Prabhu Adhiraj Ghosh Yash Sharma Philip H. S. Torr Adel Bibi Samuel Albanie Matthias Bethge VLM 115 43 0 04 Apr 2024
When Do We Not Need Larger Vision Models? Baifeng Shi Ziyang Wu Maolin Mao Xin Wang Trevor Darrell VLM LRM 44 23 0 19 Mar 2024
Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models Siddharth Karamcheti Suraj Nair Ashwin Balakrishna Percy Liang Thomas Kollar Dorsa Sadigh MLLM VLM 57 95 0 12 Feb 2024
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023
Large Language Models are Zero-Shot Reasoners Takeshi Kojima S. Gu Machel Reid Yutaka Matsuo Yusuke Iwasawa ReLM LRM 291 2,712 0 24 May 2022
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022