Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

16 July 2021

Junnan Li

Ramprasaath R. Selvaraju

Akhilesh Deepak Gotmare

Papers citing "Align before Fuse: Vision and Language Representation Learning with Momentum Distillation"

50 / 1,191 papers shown

Title
Dissecting Deep Metric Learning Losses for Image-Text Retrieval Hong Xuan Xi Chen 18 2 0 21 Oct 2022
Image-Text Retrieval with Binary and Continuous Label Supervision Zheng Li Caili Guo Zerun Feng Jenq-Neng Hwang Ying Jin Yufeng Zhang VLM 9 4 0 20 Oct 2022
CLIP-Driven Fine-grained Text-Image Person Re-identification Shuanglin Yan Neng Dong Liyan Zhang Jinhui Tang 19 86 0 19 Oct 2022
MMGA: Multimodal Learning with Graph Alignment Xuanqi Yang Quanjin Tao Xiaojuan Feng Donghong Cai Xiang Ren Yang Yang 13 0 0 18 Oct 2022
Probing Cross-modal Semantics Alignment Capability from the Textual Perspective Zheng Ma Shi Zong Mianzhi Pan Jianbing Zhang Shujian Huang Xinyu Dai Jiajun Chen 22 4 0 18 Oct 2022
Non-Contrastive Learning Meets Language-Image Pre-Training Jinghao Zhou Li Dong Zhe Gan Lijuan Wang Furu Wei VLM CLIP 17 25 0 17 Oct 2022
Contrastive Language-Image Pre-Training with Knowledge Graphs Xuran Pan Tianzhu Ye Dongchen Han S. Song Gao Huang VLM CLIP 14 42 0 17 Oct 2022
Plug-and-Play VQA: Zero-shot VQA by Conjoining Large Pretrained Models with Zero Training A. M. H. Tiong Junnan Li Boyang Albert Li Silvio Savarese S. Hoi MLLM 24 101 0 17 Oct 2022
EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning Tiannan Wang Wangchunshu Zhou Yan Zeng Xinsong Zhang VLM 17 35 0 14 Oct 2022
Plausible May Not Be Faithful: Probing Object Hallucination in Vision-Language Pre-training Wenliang Dai Zihan Liu Ziwei Ji Dan Su Pascale Fung MLLM VLM 19 61 0 14 Oct 2022
FedFM: Anchor-based Feature Matching for Data Heterogeneity in Federated Learning Rui Ye Zhenyang Ni Chenxin Xu Jianyu Wang Siheng Chen Yonina C. Eldar FedML 14 31 0 14 Oct 2022
MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Prompting Oscar Manas Pau Rodríguez López Saba Ahmadi Aida Nematzadeh Yash Goyal Aishwarya Agrawal VLM VPVLM 10 48 0 13 Oct 2022
One does not fit all! On the Complementarity of Vision Encoders for Vision and Language Tasks Gregor Geigle Chen Cecilia Liu Jonas Pfeiffer Iryna Gurevych VLM 24 1 0 12 Oct 2022
Long-Form Video-Language Pre-Training with Multimodal Temporal Contrastive Learning Yuchong Sun Hongwei Xue Ruihua Song Bei Liu Huan Yang Jianlong Fu AI4TS VLM 16 68 0 12 Oct 2022
Stochastic Constrained DRO with a Complexity Independent of Sample Size Q. Qi Jiameng Lyu Kung-Sik Chan E. Bai Tianbao Yang 50 15 0 11 Oct 2022
MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model Yatai Ji Junjie Wang Yuan Gong Lin Zhang Yan Zhu Hongfa Wang Jiaxing Zhang Tetsuya Sakai Yujiu Yang MLLM 14 29 0 11 Oct 2022
Contrastive Video-Language Learning with Fine-grained Frame Sampling Zixu Wang Yujie Zhong Yishu Miao Lin Ma Lucia Specia 30 11 0 10 Oct 2022
MAMO: Masked Multimodal Modeling for Fine-Grained Vision-Language Representation Learning Zijia Zhao Longteng Guo Xingjian He Shuai Shao Zehuan Yuan Jing Liu 16 8 0 09 Oct 2022
Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment Siyu Lai Zhen Yang Fandong Meng Yufeng Chen Jinan Xu Jie Zhou 31 6 0 09 Oct 2022
VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignment Shraman Pramanick Li Jing Sayan Nag Jiachen Zhu Hardik Shah Yann LeCun Ramalingam Chellappa 24 21 0 09 Oct 2022
Learning Fine-Grained Visual Understanding for Video Question Answering via Decoupling Spatial-Temporal Modeling Hsin-Ying Lee Hung-Ting Su Bing-Chen Tsai Tsung-Han Wu Jia-Fong Yeh Winston H. Hsu 25 2 0 08 Oct 2022
CLIP model is an Efficient Continual Learner Vishal G. Thengane Salman Khan Munawar Hayat F. Khan BDL VLM CLL 104 46 0 06 Oct 2022
Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture M. Jung He Zhao Joanna Dipnall Belinda Gabbe Lan Du UQCV EDL 42 12 0 06 Oct 2022
Progressive Text-to-Image Generation Zhengcong Fei Mingyuan Fan Li Zhu Junshi Huang 70 4 0 05 Oct 2022
When and why vision-language models behave like bags-of-words, and what to do about it? Mert Yuksekgonul Federico Bianchi Pratyusha Kalluri Dan Jurafsky James Y. Zou VLM CoGe 11 357 0 04 Oct 2022
PLOT: Prompt Learning with Optimal Transport for Vision-Language Models Guangyi Chen Weiran Yao Xiangchen Song Xinyue Li Yongming Rao Kun Zhang VPVLM VLM 6 62 0 03 Oct 2022
Enhancing Interpretability and Interactivity in Robot Manipulation: A Neurosymbolic Approach Georgios Tziafas H. Kasaei LM&Ro 20 3 0 03 Oct 2022
F-VLM: Open-Vocabulary Object Detection upon Frozen Vision and Language Models Weicheng Kuo Yin Cui Xiuye Gu A. Piergiovanni A. Angelova MLLM VLM ObjD 35 134 0 30 Sep 2022
ERNIE-ViL 2.0: Multi-view Contrastive Learning for Image-Text Pre-training Bin Shan Weichong Yin Yu Sun Hao Tian Hua-Hong Wu Haifeng Wang VLM 16 19 0 30 Sep 2022
Dynamic MDETR: A Dynamic Multimodal Transformer Decoder for Visual Grounding Fengyuan Shi Ruopeng Gao Weilin Huang Limin Wang 17 23 0 28 Sep 2022
Unified Loss of Pair Similarity Optimization for Vision-Language Retrieval Zheng Li Caili Guo Xin Eric Wang Zerun Feng Jenq-Neng Hwang Zhongtian Du VLM 13 2 0 28 Sep 2022
TokenFlow: Rethinking Fine-grained Cross-modal Alignment in Vision-Language Retrieval Xiaohan Zou Changqiao Wu Lele Cheng Zhongyuan Wang 84 6 0 28 Sep 2022
Revisiting Few-Shot Learning from a Causal Perspective Guoliang Lin Yongheng Xu Hanjiang Lai Jian Yin CML 19 3 0 28 Sep 2022
Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priors Vignav Ramesh Nathan Chi Pranav Rajpurkar MedIm 17 47 0 27 Sep 2022
Unsupervised Hashing with Semantic Concept Mining Rong-Cheng Tu Xian-Ling Mao Kevin Qinghong Lin Chengfei Cai Weize Qin Hongfa Wang Wei Wei Heyan Huang 28 10 0 23 Sep 2022
LGDN: Language-Guided Denoising Network for Video-Language Modeling Haoyu Lu Mingyu Ding Nanyi Fei Yuqi Huo Zhiwu Lu VLM 74 16 0 23 Sep 2022
DetCLIP: Dictionary-Enriched Visual-Concept Paralleled Pre-training for Open-world Detection Lewei Yao Jianhua Han Youpeng Wen Xiaodan Liang Dan Xu Wei Zhang Zhenguo Li Chunjing Xu Hang Xu CLIP VLM 115 151 0 20 Sep 2022
LAVIS: A Library for Language-Vision Intelligence Dongxu Li Junnan Li Hung Le Guangsen Wang Silvio Savarese S. Hoi VLM 113 51 0 15 Sep 2022
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks Junke Wang Dongdong Chen Zuxuan Wu Chong Luo Luowei Zhou Yucheng Zhao Yujia Xie Ce Liu Yu-Gang Jiang Lu Yuan MLLM VLM 30 148 0 15 Sep 2022
Align, Reason and Learn: Enhancing Medical Vision-and-Language Pre-training with Knowledge Zhihong Chen Guanbin Li Xiang Wan 119 65 0 15 Sep 2022
Correlation Information Bottleneck: Towards Adapting Pretrained Multimodal Models for Robust Visual Question Answering Jingjing Jiang Zi-yi Liu Nanning Zheng 21 8 0 14 Sep 2022
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Hongwei Xue Yuchong Sun Bei Liu Jianlong Fu Rui Song Houqiang Li Jiebo Luo CLIP VLM 20 68 0 14 Sep 2022
OmDet: Large-scale vision-language multi-dataset pre-training with multimodal detection network Tiancheng Zhao Peng Liu Kyusong Lee VLM MLLM ObjD 11 7 0 10 Sep 2022
FETA: Towards Specializing Foundation Models for Expert Task Applications Amit Alfassy Assaf Arbelle Oshri Halimi Sivan Harary Roei Herzig ... Christoph Auer Kate Saenko Peter W. J. Staar Rogerio Feris Leonid Karlinsky 21 19 0 08 Sep 2022
Design of the topology for contrastive visual-textual alignment Zhun Sun 25 1 0 05 Sep 2022
RLIP: Relational Language-Image Pre-training for Human-Object Interaction Detection Hangjie Yuan Jianwen Jiang Samuel Albanie Tao Feng Ziyuan Huang Dong Ni Mingqian Tang VLM 26 51 0 05 Sep 2022
Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignment Mustafa Shukor Guillaume Couairon Matthieu Cord VLM CLIP 19 27 0 29 Aug 2022
CMD: Self-supervised 3D Action Representation Learning with Cross-modal Mutual Distillation Yunyao Mao Wen-gang Zhou Zhenbo Lu Jiajun Deng Houqiang Li 25 38 0 26 Aug 2022
MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining Xiaoyi Dong Jianmin Bao Yinglin Zheng Ting Zhang Dongdong Chen ... Weiming Zhang Lu Yuan Dong Chen Fang Wen Nenghai Yu CLIP VLM 32 156 0 25 Aug 2022
MuMUR : Multilingual Multimodal Universal Retrieval Avinash Madasu Estelle Aflalo Gabriela Ben-Melech Stan Shachar Rosenman Shao-Yen Tseng Gedas Bertasius Vasudev Lal 34 3 0 24 Aug 2022