Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive
Representation Learning

Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning

3 March 2022

Papers citing "Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning"

11 / 61 papers shown

Title
Expectation-Maximization Contrastive Learning for Compact Video-and-Language Representations Peng Jin Jinfa Huang Fenglin Liu Xian Wu Shen Ge Guoli Song David A. Clifton Jing Chen VLM 16 63 0 21 Nov 2022
A Law of Data Separation in Deep Learning Hangfeng He Weijie J. Su OOD 19 36 0 31 Oct 2022
clip2latent: Text driven sampling of a pre-trained StyleGAN using denoising diffusion and CLIP Justin N. M. Pinkney Chuan Li CLIP VLM 40 20 0 05 Oct 2022
Multimodal Frame-Scoring Transformer for Video Summarization Jeiyoon Park Kiho Kwoun Chanhee Lee Heuiseok Lim ViT 25 6 0 05 Jul 2022
VQA-GNN: Reasoning with Multimodal Knowledge via Graph Neural Networks for Visual Question Answering Yanan Wang Michihiro Yasunaga Hongyu Ren Shinya Wada J. Leskovec 8 17 0 23 May 2022
Omnivore: A Single Model for Many Visual Modalities Rohit Girdhar Mannat Singh Nikhil Ravi L. V. D. van der Maaten Armand Joulin Ishan Misra 211 225 0 20 Jan 2022
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 245 557 0 28 Sep 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 3,683 0 11 Feb 2021
Word Translation Without Parallel Data Alexis Conneau Guillaume Lample MarcÁurelio Ranzato Ludovic Denoyer Hervé Jégou 165 1,634 0 11 Oct 2017
Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results Antti Tarvainen Harri Valpola OOD MoMe 244 1,276 0 06 Mar 2017
On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima N. Keskar Dheevatsa Mudigere J. Nocedal M. Smelyanskiy P. T. P. Tang ODL 273 2,878 0 15 Sep 2016