Align and Prompt: Video-and-Language Pre-training with Entity Prompts

17 December 2021

Papers citing "Align and Prompt: Video-and-Language Pre-training with Entity Prompts"

38 / 138 papers shown

Title
Compositional Prompt Tuning with Motion Cues for Open-vocabulary Video Relation Detection Kaifeng Gao Long Chen Hanwang Zhang Jun Xiao Qianru Sun VLM VPVLM 14 26 0 01 Feb 2023
Learning Open-vocabulary Semantic Segmentation Models From Natural Language Supervision Jilan Xu Junlin Hou Yuejie Zhang Rui Feng Yi Wang Yu Qiao Weidi Xie VLM 11 81 0 22 Jan 2023
Temporal Perceiving Video-Language Pre-training Fan Ma Xiaojie Jin Heng Wang Jingjia Huang Linchao Zhu Jiashi Feng Yi Yang VLM 13 15 0 18 Jan 2023
Learning Trajectory-Word Alignments for Video-Language Tasks Xu Yang Zhang Li Haiyang Xu Hanwang Zhang Qinghao Ye Chenliang Li Ming Yan Yu Zhang Fei Huang Songfang Huang 23 7 0 05 Jan 2023
HiTeA: Hierarchical Temporal-Aware Video-Language Pre-training Qinghao Ye Guohai Xu Ming Yan Haiyang Xu Qi Qian Ji Zhang Fei Huang VLM AI4TS 163 69 0 30 Dec 2022
CREPE: Can Vision-Language Foundation Models Reason Compositionally? Zixian Ma Jerry Hong Mustafa Omer Gul Mona Gandhi Irena Gao Ranjay Krishna CoGe 18 125 0 13 Dec 2022
VindLU: A Recipe for Effective Video-and-Language Pretraining Feng Cheng Xizi Wang Jie Lei David J. Crandall Mohit Bansal Gedas Bertasius VLM 27 78 0 09 Dec 2022
Decorate the Newcomers: Visual Domain Prompt for Continual Test Time Adaptation Yulu Gan Yan Bai Yihang Lou Xianzheng Ma Renrui Zhang Nian Shi Lin Luo OOD VLM 25 91 0 08 Dec 2022
SimVTP: Simple Video Text Pre-training with Masked Autoencoders Yue Ma Tianyu Yang Yin Shan Xiu Li 25 27 0 07 Dec 2022
Masked Contrastive Pre-Training for Efficient Video-Text Retrieval Fangxun Shu Biaolong Chen Yue Liao Shuwen Xiao Wenyu Sun Xiaobo Li Yousong Zhu Jinqiao Wang Si Liu CLIP 25 11 0 02 Dec 2022
Seeing What You Miss: Vision-Language Pre-training with Semantic Completion Learning Yatai Ji Rong-Cheng Tu Jie Jiang Weijie Kong Chengfei Cai Wenzhe Zhao Hongfa Wang Yujiu Yang Wei Liu VLM 32 13 0 24 Nov 2022
X $^2$ -VLM: All-In-One Pre-trained Model For Vision-Language Tasks Yan Zeng Xinsong Zhang Hang Li Jiawei Wang Jipeng Zhang Hkust Wangchunshu Zhou VLM MLLM 21 14 0 22 Nov 2022
SMAUG: Sparse Masked Autoencoder for Efficient Video-Language Pre-training Yuanze Lin Chen Wei Huiyu Wang Alan Yuille Cihang Xie 3DGS 26 15 0 21 Nov 2022
CLOP: Video-and-Language Pre-Training with Knowledge Regularizations Guohao Li Hu Yang Feng He Zhifan Feng Yajuan Lyu Hua-Hong Wu Haifeng Wang VLM 17 1 0 07 Nov 2022
LiteVL: Efficient Video-Language Learning with Enhanced Spatial-Temporal Modeling Dongsheng Chen Chaofan Tao Lu Hou Lifeng Shang Xin Jiang Qun Liu VLM 25 18 0 21 Oct 2022
RaP: Redundancy-aware Video-language Pre-training for Text-Video Retrieval Xing Wu Chaochen Gao Zijia Lin Zhongyuan Wang Jizhong Han Songlin Hu 24 7 0 13 Oct 2022
LAVIS: A Library for Language-Vision Intelligence Dongxu Li Junnan Li Hung Le Guangsen Wang Silvio Savarese S. Hoi VLM 113 51 0 15 Sep 2022
OmniVL:One Foundation Model for Image-Language and Video-Language Tasks Junke Wang Dongdong Chen Zuxuan Wu Chong Luo Luowei Zhou Yucheng Zhao Yujia Xie Ce Liu Yu-Gang Jiang Lu Yuan MLLM VLM 30 148 0 15 Sep 2022
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling Tsu-jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin William Yang Wang Lijuan Wang Zicheng Liu VLM 19 63 0 04 Sep 2022
MuMUR : Multilingual Multimodal Universal Retrieval Avinash Madasu Estelle Aflalo Gabriela Ben-Melech Stan Shachar Rosenman Shao-Yen Tseng Gedas Bertasius Vasudev Lal 37 3 0 24 Aug 2022
Dual Modality Prompt Tuning for Vision-Language Pre-Trained Model Yinghui Xing Qirui Wu De-Chun Cheng Shizhou Zhang Guoqiang Liang Peng Wang Yanning Zhang VLM VPVLM 52 51 0 17 Aug 2022
Zero-Shot Video Captioning with Evolving Pseudo-Tokens Yoad Tewel Yoav Shalev Roy Nadler Idan Schwartz Lior Wolf 29 28 0 22 Jul 2022
Clover: Towards A Unified Video-Language Alignment and Fusion Model Jingjia Huang Yinan Li Jiashi Feng Xinglong Wu Xiaoshuai Sun Rongrong Ji VLM 19 48 0 16 Jul 2022
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models Antoine Yang Antoine Miech Josef Sivic Ivan Laptev Cordelia Schmid 34 226 0 16 Jun 2022
Multimodal Learning with Transformers: A Survey P. Xu Xiatian Zhu David A. Clifton ViT 41 522 0 13 Jun 2022
Bringing Image Scene Structure to Video via Frame-Clip Consistency of Object Tokens Elad Ben-Avraham Roei Herzig K. Mangalam Amir Bar Anna Rohrbach Leonid Karlinsky Trevor Darrell Amir Globerson 19 0 0 13 Jun 2022
Revealing Single Frame Bias for Video-and-Language Learning Jie Lei Tamara L. Berg Mohit Bansal 18 109 0 07 Jun 2022
mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connections Chenliang Li Haiyang Xu Junfeng Tian Wei Wang Ming Yan ... Ji Zhang Songfang Huang Feiran Huang Jingren Zhou Luo Si VLM MLLM 26 212 0 24 May 2022
Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners Zhenhailong Wang Manling Li Ruochen Xu Luowei Zhou Jie Lei ... Chenguang Zhu Derek Hoiem Shih-Fu Chang Mohit Bansal Heng Ji MLLM VLM 167 136 0 22 May 2022
A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunities Yisheng Song Ting-Yuan Wang S. Mondal J. P. Sahoo SLR 36 342 0 13 May 2022
OMG: Observe Multiple Granularities for Natural Language-Based Vehicle Retrieval Yunhao Du Binyu Zhang Xiang Ruan Fei Su Zhicheng Zhao Hong Chen 17 5 0 18 Apr 2022
Tencent Text-Video Retrieval: Hierarchical Cross-Modal Interactions with Multi-Level Representations Jie Jiang Shaobo Min Weijie Kong Dihong Gong Hongfa Wang Zhifeng Li Wei Liu VLM 18 18 0 07 Apr 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 388 4,110 0 28 Jan 2022
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 245 557 0 28 Sep 2021
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Huaishao Luo Lei Ji Ming Zhong Yang Chen Wen Lei Nan Duan Tianrui Li CLIP VLM 309 778 0 18 Apr 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 3,683 0 11 Feb 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 278 1,978 0 09 Feb 2021
Multi-modal Transformer for Video Retrieval Valentin Gabeur Chen Sun Alahari Karteek Cordelia Schmid ViT 410 594 0 21 Jul 2020