Align and Prompt: Video-and-Language Pre-training with Entity Prompts

17 December 2021

Papers citing "Align and Prompt: Video-and-Language Pre-training with Entity Prompts"

50 / 138 papers shown

Title
METOR: A Unified Framework for Mutual Enhancement of Objects and Relationships in Open-vocabulary Video Visual Relationship Detection Yongqi Wang Xinxiao Wu Shuo Yang ObjD 19 0 0 10 May 2025
T2VPhysBench: A First-Principles Benchmark for Physical Consistency in Text-to-Video Generation Xuyang Guo Jiayan Huo Zhenmei Shi Zhao-quan Song Jiahao Zhang Jiale Zhao EGVM VGen PINN 77 1 0 01 May 2025
Multi-Resolution Pathology-Language Pre-training Model with Text-Guided Visual Representation Shahad Albastaki Anabia Sohail I. I. Ganapathi B. Alawode Asim Khan Sajid Javed N. Werghi Mohammed Bennamoun Arif Mahmood 66 0 0 26 Apr 2025
SEM-CLIP: Precise Few-Shot Learning for Nanoscale Defect Detection in Scanning Electron Microscope Image Qian Jin Yuqi Jiang Xudong Lu Yumeng Liu Yining Chen Dawei Gao Qi Sun Cheng Zhuo 65 0 0 24 Feb 2025
Pretrained Image-Text Models are Secretly Video Captioners Chunhui Zhang Yiren Jian Z. Ouyang Soroush Vosoughi VLM 69 3 0 20 Feb 2025
Hierarchical Banzhaf Interaction for General Video-Language Representation Learning Peng Jin H. Li Li Yuan Shuicheng Yan Jie Chen 45 1 0 31 Dec 2024
Beyond Accuracy: On the Effects of Fine-tuning Towards Vision-Language Model's Prediction Rationality Qitong Wang Tang Li Kien X. Nguyen Xi Peng 70 0 0 17 Dec 2024
ACE: Action Concept Enhancement of Video-Language Models in Procedural Videos Reza Ghoddoosian Nakul Agarwal Isht Dwivedi Behzad Darisuh 57 0 0 23 Nov 2024
Scene Graph Generation with Role-Playing Large Language Models Guikun Chen Jin Li Wenguan Wang VLM 40 5 0 20 Oct 2024
LocoMotion: Learning Motion-Focused Video-Language Representations Hazel Doughty Fida Mohammad Thoker Cees G. M. Snoek 33 2 0 15 Oct 2024
Prompting Video-Language Foundation Models with Domain-specific Fine-grained Heuristics for Video Question Answering Ting Yu Kunhao Fu Shuhui Wang Qingming Huang Jun Yu 41 0 0 12 Oct 2024
Multi-granularity Contrastive Cross-modal Collaborative Generation for End-to-End Long-term Video Question Answering Ting Yu Kunhao Fu Jian Zhang Qingming Huang Jun Yu 25 2 0 12 Oct 2024
End-to-end Open-vocabulary Video Visual Relationship Detection using Multi-modal Prompting Yongqi Wang Xinxiao Wu Shuo Yang Jiebo Luo 74 1 0 19 Sep 2024
Towards Completeness: A Generalizable Action Proposal Generator for Zero-Shot Temporal Action Localization Jia-Run Du Kun-Yu Lin Jingke Meng Wei-Shi Zheng 26 0 0 25 Aug 2024
NAVERO: Unlocking Fine-Grained Semantics for Video-Language Compositionality Chaofan Tao Gukyeong Kwon Varad Gunjal Hao Yang Zhaowei Cai Yonatan Dukler Ashwin Swaminathan R. Manmatha Colin Jon Taylor Stefano Soatto CoGe 27 0 0 18 Aug 2024
Compositional Physical Reasoning of Objects and Events from Videos Zhenfang Chen Shilong Dong Kexin Yi Yunzhu Li Mingyu Ding Antonio Torralba Joshua B. Tenenbaum Chuang Gan OCL 20 1 0 02 Aug 2024
Enhancing Video-Language Representations with Structural Spatio-Temporal Alignment Hao Fei Shengqiong Wu Meishan Zhang M. Zhang Tat-Seng Chua Shuicheng Yan AI4TS 34 38 0 27 Jun 2024
Revealing Vision-Language Integration in the Brain with Multimodal Networks Vighnesh Subramaniam C. Conwell Christopher Wang Gabriel Kreiman Boris Katz Ignacio Cases Andrei Barbu 19 8 0 20 Jun 2024
Towards Holistic Language-video Representation: the language model-enhanced MSR-Video to Text Dataset Yuchen Yang Yingxuan Duan VGen 23 0 0 19 Jun 2024
2M-NER: Contrastive Learning for Multilingual and Multimodal NER with Language and Modal Fusion Dongsheng Wang Xiaoqin Feng Zeming Liu Chuan Wang 41 2 0 26 Apr 2024
Boosting Architectural Generation via Prompts: Report Xin Zhang Wenwen Liu AI4CE 27 1 0 24 Apr 2024
VideoDistill: Language-aware Vision Distillation for Video Question Answering Bo Zou Chao Yang Yu Qiao Chengbin Quan Youjian Zhao VGen 39 1 0 01 Apr 2024
PLOT-TAL -- Prompt Learning with Optimal Transport for Few-Shot Temporal Action Localization Edward Fish Jon Weinbren Andrew Gilbert 25 1 0 27 Mar 2024
OmniVid: A Generative Framework for Universal Video Understanding Junke Wang Dongdong Chen Chong Luo Bo He Lu Yuan Zuxuan Wu Yu-Gang Jiang VLM VGen 69 14 0 26 Mar 2024
vid-TLDR: Training Free Token merging for Light-weight Video Transformer Joonmyung Choi Sanghyeok Lee Jaewon Chu Minhyuk Choi Hyunwoo J. Kim MoMe ViT 40 12 0 20 Mar 2024
VideoMamba: State Space Model for Efficient Video Understanding Kunchang Li Xinhao Li Yi Wang Yinan He Yali Wang Limin Wang Yu Qiao Mamba 30 179 0 11 Mar 2024
Rethinking CLIP-based Video Learners in Cross-Domain Open-Vocabulary Action Recognition Kun-Yu Lin Henghui Ding Jiaming Zhou Yu-Ming Tang Yi-Xing Peng Zhilin Zhao Chen Change Loy Wei-Shi Zheng VLM 30 15 0 03 Mar 2024
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers Tsai-Shien Chen Aliaksandr Siarohin Willi Menapace Ekaterina Deyneka Hsiang-wei Chao ... Yuwei Fang Hsin-Ying Lee Jian Ren Ming-Hsuan Yang Sergey Tulyakov VGen 70 177 0 29 Feb 2024
Unifying Latent and Lexicon Representations for Effective Video-Text Retrieval Haowei Liu Yaya Shi Haiyang Xu Chunfen Yuan Qinghao Ye ... Mingshi Yan Ji Zhang Fei Huang Bing Li Weiming Hu 20 0 0 26 Feb 2024
LVCHAT: Facilitating Long Video Comprehension Yu-Xiang Wang Zeyuan Zhang Julian McAuley Zexue He VLM 26 4 0 19 Feb 2024
ContPhy: Continuum Physical Concept Learning and Reasoning from Videos Zhicheng Zheng Xin Yan Zhenfang Chen Jingzhou Wang Qin Zhi Eddie Lim Joshua B. Tenenbaum Chuang Gan LRM 27 6 0 09 Feb 2024
Convincing Rationales for Visual Question Answering Reasoning Kun Li G. Vosselman Michael Ying Yang 34 1 0 06 Feb 2024
SNP-S3: Shared Network Pre-training and Significant Semantic Strengthening for Various Video-Text Tasks Xingning Dong Qingpei Guo Tian Gan Qing Wang Jianlong Wu Xiangyuan Ren Yuan-Chia Cheng Wei Chu 21 5 0 31 Jan 2024
CoAVT: A Cognition-Inspired Unified Audio-Visual-Text Pre-Training Model for Multimodal Processing Xianghu Yue Xiaohai Tian Lu Lu Malu Zhang Zhizheng Wu Haizhou Li 22 0 0 22 Jan 2024
Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness Sibo Wang Jie M. Zhang Zheng Yuan Shiguang Shan VLM 31 18 0 09 Jan 2024
Efficient Multiscale Multimodal Bottleneck Transformer for Audio-Video Classification Wentao Zhu 25 5 0 08 Jan 2024
Efficient Selective Audio Masked Multimodal Bottleneck Transformer for Audio-Video Classification Wentao Zhu 19 4 0 08 Jan 2024
Video Understanding with Large Language Models: A Survey Yunlong Tang Jing Bi Siting Xu Luchuan Song Susan Liang ... Feng Zheng Jianguo Zhang Ping Luo Jiebo Luo Chenliang Xu VLM 50 81 0 29 Dec 2023
Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognition Qianrui Zhou Hua Xu Hao Li Hanlei Zhang Xiaohan Zhang Yifan Wang Kai Gao 28 12 0 22 Dec 2023
CL2CM: Improving Cross-Lingual Cross-Modal Retrieval via Cross-Lingual Knowledge Transfer Yabing Wang Fan Wang Jianfeng Dong Hao Luo VLM 15 8 0 14 Dec 2023
Audio-Visual LLM for Video Understanding Fangxun Shu Lei Zhang Hao Jiang Cihang Xie VLM MLLM 17 37 0 11 Dec 2023
Prompt Tuning for Zero-shot Compositional Learning Lingyu Zhang Ting Hua Yilin Shen Hongxia Jin VLM 14 0 0 02 Dec 2023
VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Models Shicheng Li Lei Li Shuhuai Ren Yuanxin Liu Yi Liu Rundong Gao Xu Sun Lu Hou 27 29 0 29 Nov 2023
C3Net: Compound Conditioned ControlNet for Multimodal Content Generation Juntao Zhang Yuehuai Liu Yu-Wing Tai Chi-Keung Tang DiffM 30 5 0 29 Nov 2023
Mug-STAN: Adapting Image-Language Pretrained Models for General Video Understanding Ruyang Liu Jingjia Huang Wei-Nan Gao Thomas H. Li Ge Li VLM 27 3 0 25 Nov 2023
SPOT! Revisiting Video-Language Models for Event Understanding Gengyuan Zhang Jinhe Bi Jindong Gu Yanyu Chen Volker Tresp 19 1 0 21 Nov 2023
ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models .Ilker Kesen Andrea Pedrotti Mustafa Dogan Michele Cafagna Emre Can Acikgoz ... Iacer Calixto Anette Frank Albert Gatt Aykut Erdem Erkut Erdem 33 15 0 13 Nov 2023
Mirasol3B: A Multimodal Autoregressive model for time-aligned and contextual modalities A. Piergiovanni Isaac Noble Dahun Kim Michael S. Ryoo Victor Gomes A. Angelova 33 19 0 09 Nov 2023
An Empirical Study of Frame Selection for Text-to-Video Retrieval Mengxia Wu Min Cao Yang Bai Ziyin Zeng Chen Chen Liqiang Nie Min Zhang 12 3 0 01 Nov 2023
Harvest Video Foundation Models via Efficient Post-Pretraining Yizhuo Li Kunchang Li Yinan He Yi Wang Yali Wang Limin Wang Yu Qiao Ping Luo CLIP VLM VGen 33 2 0 30 Oct 2023