Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

16 July 2021

Junnan Li

Ramprasaath R. Selvaraju

Akhilesh Deepak Gotmare

Papers citing "Align before Fuse: Vision and Language Representation Learning with Momentum Distillation"

50 / 1,192 papers shown

Title
Knowledge from Large-Scale Protein Contact Prediction Models Can Be Transferred to the Data-Scarce RNA Contact Prediction Task Yiren Jian Chongyang Gao Chen Zeng Yunjie Zhao Soroush Vosoughi 19 0 0 13 Feb 2023
Differentiable Outlier Detection Enable Robust Deep Multimodal Analysis Zhu Wang Sourav Medya Sathya Ravi VLM 25 0 0 11 Feb 2023
Analyzing Multimodal Objectives Through the Lens of Generative Diffusion Guidance Chaerin Kong Nojun Kwak DiffM 26 2 0 10 Feb 2023
Unified Vision-Language Representation Modeling for E-Commerce Same-Style Products Retrieval Ben Chen Linbo Jin Xinxin Wang D. Gao Wen Jiang Wei Ning 6 3 0 10 Feb 2023
Is Multimodal Vision Supervision Beneficial to Language? Avinash Madasu Vasudev Lal 29 4 0 10 Feb 2023
SimCon Loss with Multiple Views for Text Supervised Semantic Segmentation Yash J. Patel Yusheng Xie Yi Zhu Srikar Appalaraju R. Manmatha 19 4 0 07 Feb 2023
Pic2Word: Mapping Pictures to Words for Zero-shot Composed Image Retrieval Kuniaki Saito Kihyuk Sohn Xiang Zhang Chun-Liang Li Chen-Yu Lee Kate Saenko Tomas Pfister 19 108 0 06 Feb 2023
Contrast with Reconstruct: Contrastive 3D Representation Learning Guided by Generative Pretraining Zekun Qi Runpei Dong Guo Fan Zheng Ge Xiangyu Zhang Kaisheng Ma Li Yi 28 117 0 05 Feb 2023
Learning to Agree on Vision Attention for Visual Commonsense Reasoning Zhenyang Li Yangyang Guo Ke-Jyun Wang Fan Liu Liqiang Nie Mohan S. Kankanhalli 27 10 0 04 Feb 2023
mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video Haiyang Xu Qinghao Ye Mingshi Yan Yaya Shi Jiabo Ye ... Guohai Xu Ji Zhang Songfang Huang Feiran Huang Jingren Zhou MLLM VLM MoE 25 158 0 01 Feb 2023
Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applications Muhammad Arslan Manzoor S. Albarri Ziting Xian Zaiqiao Meng Preslav Nakov Shangsong Liang AI4TS 21 26 0 01 Feb 2023
ViewCo: Discovering Text-Supervised Segmentation Masks via Multi-View Semantic Consistency Pengzhen Ren Changlin Li Hang Xu Yi Zhu Guangrun Wang Jian-zhuo Liu Xiaojun Chang Xiaodan Liang 26 43 0 31 Jan 2023
Understanding Self-Distillation in the Presence of Label Noise Rudrajit Das Sujay Sanghavi 25 13 0 30 Jan 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 265 4,223 0 30 Jan 2023
OvarNet: Towards Open-vocabulary Object Attribute Recognition Keyan Chen Xiaolong Jiang Yao Hu Xu Tang Yan Gao Jianqi Chen Weidi Xie VLM ObjD 27 40 0 23 Jan 2023
Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervision Jilan Xu Junlin Hou Yuejie Zhang Rui Feng Yi Wang Yu Qiao Weidi Xie VLM 11 81 0 22 Jan 2023
Masked Autoencoding Does Not Help Natural Language Supervision at Scale Floris Weers Vaishaal Shankar Angelos Katharopoulos Yinfei Yang Tom Gunter CLIP 18 4 0 19 Jan 2023
CLIPTER: Looking at the Bigger Picture in Scene Text Recognition Aviad Aberdam David Bensaid Alona Golts Roy Ganz Oren Nuriel Royee Tichauer Shai Mazor Ron Litman VLM CLIP 24 12 0 18 Jan 2023
Towards Models that Can See and Read Roy Ganz Oren Nuriel Aviad Aberdam Yair Kittenplon Shai Mazor Ron Litman 16 13 0 18 Jan 2023
GLIGEN: Open-Set Grounded Text-to-Image Generation Yuheng Li Haotian Liu Qingyang Wu Fangzhou Mu Jianwei Yang Jianfeng Gao Chunyuan Li Yong Jae Lee VLM 49 568 1 17 Jan 2023
USER: Unified Semantic Enhancement with Momentum Contrast for Image-Text Retrieval Yan Zhang Zhong Ji Dingrong Wang Yanwei Pang Xuelong Li VLM 16 21 0 17 Jan 2023
UATVR: Uncertainty-Adaptive Text-Video Retrieval Bo Fang Wenhao Wu Chang-rui Liu Yu Zhou Yuxin Song Weiping Wang Min Yang Xiang Ji Jingdong Wang 24 45 0 16 Jan 2023
Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study Mariya Hendriksen Svitlana Vakulenko E. Kuiper Maarten de Rijke 21 5 0 12 Jan 2023
Toward Building General Foundation Models for Language, Vision, and Vision-Language Understanding Tasks Xinsong Zhang Yan Zeng Jipeng Zhang Hang Li VLM AI4CE LRM 14 17 0 12 Jan 2023
HADA: A Graph-based Amalgamation Framework in Image-text Retrieval Manh-Duy Nguyen Binh T. Nguyen C. Gurrin VLM 20 4 0 11 Jan 2023
Learning to Exploit Temporal Structure for Biomedical Vision-Language Processing Shruthi Bannur Stephanie L. Hyland Qianchu Liu Fernando Pérez-García Maximilian Ilse ... Maria T. A. Wetscherek M. Lungren A. Nori Javier Alvarez-Valle Ozan Oktay 27 109 0 11 Jan 2023
Filtering, Distillation, and Hard Negatives for Vision-Language Pre-Training Filip Radenovic Abhimanyu Dubey Abhishek Kadian Todor Mihaylov Simon Vandenhende Yash J. Patel Y. Wen Vignesh Ramanathan D. Mahajan VLM 32 81 0 05 Jan 2023
MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiology Chaoyi Wu Xiaoman Zhang Ya-Qin Zhang Yanfeng Wang Weidi Xie LM&MA VLM 22 109 0 05 Jan 2023
Learning Trajectory-Word Alignments for Video-Language Tasks Xu Yang Zhang Li Haiyang Xu Hanwang Zhang Qinghao Ye Chenliang Li Ming Yan Yu Zhang Fei Huang Songfang Huang 28 7 0 05 Jan 2023
GIVL: Improving Geographical Inclusivity of Vision-Language Models with Pre-Training Methods Da Yin Feng Gao Govind Thattai Michael F. Johnston Kai-Wei Chang VLM 27 15 0 05 Jan 2023
Unlearnable Clusters: Towards Label-agnostic Unlearnable Examples Jiaming Zhang Xingjun Ma Qiaomin Yi Jitao Sang Yugang Jiang Yaowei Wang Changsheng Xu 13 24 0 31 Dec 2022
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training Qinghao Ye Guohai Xu Ming Yan Haiyang Xu Qi Qian Ji Zhang Fei Huang VLM AI4TS 163 69 0 30 Dec 2022
Learning Multimodal Data Augmentation in Feature Space Zichang Liu Zhiqiang Tang Xingjian Shi Aston Zhang Mu Li Anshumali Shrivastava A. Wilson 37 19 0 29 Dec 2022
BagFormer: Better Cross-Modal Retrieval via bag-wise interaction Haowen Hou Xiaopeng Yan Yigeng Zhang Fengzong Lian Zhanhui Kang BDL 24 0 0 29 Dec 2022
When are Lemons Purple? The Concept Association Bias of Vision-Language Models Yutaro Yamada Yingtian Tang Yoyo Zhang Ilker Yildirim CoGe 6 14 0 22 Dec 2022
Multi-queue Momentum Contrast for Microvideo-Product Retrieval Yali Du Yin-wei Wei Wei Ji Fan Liu Xin Luo Liqiang Nie 32 16 0 22 Dec 2022
Generalized Decoding for Pixel, Image, and Language Xueyan Zou Zi-Yi Dou Jianwei Yang Zhe Gan Linjie Li ... Lu Yuan Nanyun Peng Lijuan Wang Yong Jae Lee Jianfeng Gao VLM MLLM ObjD 13 238 0 21 Dec 2022
ALCAP: Alignment-Augmented Music Captioner Zihao He Weituo Hao Weiyi Lu Changyou Chen Kristina Lerman Xuchen Song 19 1 0 21 Dec 2022
From Images to Textual Prompts: Zero-shot VQA with Frozen Large Language Models Jiaxian Guo Junnan Li Dongxu Li A. M. H. Tiong Boyang Albert Li Dacheng Tao Steven C. H. Hoi VLM MLLM 24 106 0 21 Dec 2022
UnICLAM:Contrastive Representation Learning with Adversarial Masking for Unified and Interpretable Medical Vision Question Answering Chenlu Zhan Peng Peng Hongsen Wang Tao Chen Hongwei Wang MedIm 18 3 0 21 Dec 2022
MetaCLUE: Towards Comprehensive Visual Metaphors Research Arjun Reddy Akula Brenda S. Driscoll P. Narayana Soravit Changpinyo Zhi-xuan Jia ... Sugato Basu Leonidas J. Guibas William T. Freeman Yuanzhen Li Varun Jampani CLIP VLM 13 22 0 19 Dec 2022
Position-guided Text Prompt for Vision-Language Pre-training Alex Jinpeng Wang Pan Zhou Mike Zheng Shou Shuicheng Yan VLM 19 37 0 19 Dec 2022
Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understanding Haoli Bai Zhiguang Liu Xiaojun Meng Wentao Li Shuangning Liu ... Liangwei Wang Lu Hou Jiansheng Wei Xin Jiang Qun Liu ViT 14 11 0 19 Dec 2022
Attentive Mask CLIP Yifan Yang Weiquan Huang Yixuan Wei Houwen Peng Xinyang Jiang ... Fangyun Wei Yin Wang Han Hu Lili Qiu Yuqing Yang CLIP VLM 32 26 0 16 Dec 2022
MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks Letitia Parcalabescu Anette Frank 16 22 0 15 Dec 2022
Visually-augmented pretrained language models for NLP tasks without images Hangyu Guo Kun Zhou Wayne Xin Zhao Qinyu Zhang Ji-Rong Wen VLM 13 10 0 15 Dec 2022
NLIP: Noise-robust Language-Image Pre-training Runhu Huang Yanxin Long Jianhua Han Hang Xu Xiwen Liang Chunjing Xu Xiaodan Liang VLM 26 30 0 14 Dec 2022
CREPE: Can Vision-Language Foundation Models Reason Compositionally? Zixian Ma Jerry Hong Mustafa Omer Gul Mona Gandhi Irena Gao Ranjay Krishna CoGe 18 125 0 13 Dec 2022
Uniform Masking Prevails in Vision-Language Pretraining Siddharth Verma Yuchen Lu Rui Hou Hanchao Yu Nicolas Ballas Madian Khabsa Amjad Almahairi VLM 21 0 0 10 Dec 2022
VindLU: A Recipe for Effective Video-and-Language Pretraining Feng Cheng Xizi Wang Jie Lei David J. Crandall Mohit Bansal Gedas Bertasius VLM 27 78 0 09 Dec 2022