Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models

Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models

11 April 2024

David T. Hoffmann

Max Argus

Thomas Brox

Papers citing "Two Effects, One Trigger: On the Modality Gap, Object Bias, and Information Imbalance in Contrastive Vision-Language Models"

7 / 7 papers shown

Title
R2-T2: Re-Routing in Test-Time for Multimodal Mixture-of-Experts Zhongyang Li Ziyue Li Tianyi Zhou MoE 44 0 0 27 Feb 2025
Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion Marco Mistretta Alberto Baldrati Lorenzo Agnolucci Marco Bertini Andrew D. Bagdanov CLIP VLM 99 2 0 06 Feb 2025
jina-clip-v2: Multilingual Multimodal Embeddings for Text and Images Andreas Koukounas Georgios Mastrapas Bo Wang Mohammad Kalim Akram Sedigheh Eslami Michael Gunther Isabelle Mohr Saba Sturua Scott Martens Nan Wang VLM 94 6 0 11 Dec 2024
CLIP-PAE: Projection-Augmentation Embedding to Extract Relevant Features for a Disentangled, Interpretable, and Controllable Text-Guided Face Manipulation Chenliang Zhou Fangcheng Zhong Cengiz Öztireli CLIP 40 19 0 08 Oct 2022
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 273 1,077 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 3,683 0 11 Feb 2021
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 279 39,083 0 01 Sep 2014