Cross-Modal and Hierarchical Modeling of Video and Text

16 October 2018

Papers citing "Cross-Modal and Hierarchical Modeling of Video and Text"

45 / 45 papers shown

Title
Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation Kun Yuan V. Srivastav Nassir Navab N. Padoy 44 7 0 30 Sep 2024
TempMe: Video Temporal Token Merging for Efficient Text-Video Retrieval Leqi Shen Tianxiang Hao Tao He Sicheng Zhao Pengzhang Liu Yongjun Bao Guiguang Ding Guiguang Ding 118 7 0 02 Sep 2024
VideoDistill: Language-aware Vision Distillation for Video Question Answering Bo Zou Chao Yang Yu Qiao Chengbin Quan Youjian Zhao VGen 42 1 0 01 Apr 2024
Video ReCap: Recursive Captioning of Hour-Long Videos Md. Mohaiminul Islam Ngan Ho Xitong Yang Tushar Nagarajan Lorenzo Torresani Gedas Bertasius VGen VLM 27 44 0 20 Feb 2024
Collaboratively Self-supervised Video Representation Learning for Action Recognition Jie M. Zhang Zhifan Wan Lanqing Hu Stephen Lin Shuzhe Wu Shiguang Shan TTA 58 1 0 15 Jan 2024
View while Moving: Efficient Video Recognition in Long-untrimmed Videos Ye Tian Meng Yang Lanshan Zhang Zhizhen Zhang Yang Liu Xiao-Zhu Xie Xirong Que Wendong Wang 22 7 0 09 Aug 2023
Video-Text as Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning Peng Jin Jinfa Huang Pengfei Xiong Shangxuan Tian Chang-rui Liu Xiang Ji Li-ming Yuan Jie Chen 30 49 0 25 Mar 2023
Text with Knowledge Graph Augmented Transformer for Video Captioning Xin Gu G. Chen Yufei Wang Libo Zhang Tiejian Luo Longyin Wen 19 47 0 22 Mar 2023
VideoXum: Cross-modal Visual and Textural Summarization of Videos Jingyang Lin Hang Hua Ming Chen Yikang Li Jenhao Hsiao C. Ho Jiebo Luo 28 30 0 21 Mar 2023
Tagging before Alignment: Integrating Multi-Modal Tags for Video-Text Retrieval Yizhen Chen Jie Wang Lijian Lin Zhongang Qi Jin Ma Ying Shan VLM 18 18 0 30 Jan 2023
HierVL: Learning Hierarchical Video-Language Embeddings Kumar Ashutosh Rohit Girdhar Lorenzo Torresani Kristen Grauman VLM AI4TS 20 51 0 05 Jan 2023
Learning Trajectory-Word Alignments for Video-Language Tasks Xu Yang Zhang Li Haiyang Xu Hanwang Zhang Qinghao Ye Chenliang Li Ming Yan Yu Zhang Fei Huang Songfang Huang 28 7 0 05 Jan 2023
Cross-Modal Adapter for Text-Video Retrieval Haojun Jiang Jianke Zhang Rui Huang Chunjiang Ge Zanlin Ni Jiwen Lu Jie Zhou S. Song Gao Huang 40 36 0 17 Nov 2022
Contrastive Video-Language Learning with Fine-grained Frame Sampling Zixu Wang Yujie Zhong Yishu Miao Lin Ma Lucia Specia 44 11 0 10 Oct 2022
CLIP-ViP: Adapting Pre-trained Image-Text Model to Video-Language Representation Alignment Hongwei Xue Yuchong Sun Bei Liu Jianlong Fu Rui Song Houqiang Li Jiebo Luo CLIP VLM 25 68 0 14 Sep 2022
An Empirical Study of End-to-End Video-Language Transformers with Masked Visual Modeling Tsu-jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin William Yang Wang Lijuan Wang Zicheng Liu VLM 19 63 0 04 Sep 2022
MuMUR : Multilingual Multimodal Universal Retrieval Avinash Madasu Estelle Aflalo Gabriela Ben-Melech Stan Shachar Rosenman Shao-Yen Tseng Gedas Bertasius Vasudev Lal 37 3 0 24 Aug 2022
M2HF: Multi-level Multi-modal Hybrid Fusion for Text-Video Retrieval Shuo Liu Weize Quan Mingyuan Zhou Sihong Chen Jian Kang Zhenlan Zhao Chen Chen Dong-Ming Yan 11 0 0 16 Aug 2022
TS2-Net: Token Shift and Selection Transformer for Text-Video Retrieval Yuqi Liu Pengfei Xiong Luhui Xu Shengming Cao Qin Jin 22 113 0 16 Jul 2022
X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrieval Yiwei Ma Guohai Xu Xiaoshuai Sun Ming Yan Ji Zhang Rongrong Ji CLIP VLM 10 267 0 15 Jul 2022
ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound Yan-Bo Lin Jie Lei Mohit Bansal Gedas Bertasius 33 39 0 06 Apr 2022
Searching for fingerspelled content in American Sign Language Bowen Shi D. Brentari G. Shakhnarovich Karen Livescu 22 5 0 24 Mar 2022
All in One: Exploring Unified Video-Language Pre-training Alex Jinpeng Wang Yixiao Ge Rui Yan Yuying Ge Xudong Lin Guanyu Cai Jianping Wu Ying Shan Xiaohu Qie Mike Zheng Shou 14 200 0 14 Mar 2022
End-to-End Semantic Video Transformer for Zero-Shot Action Recognition Keval Doshi Yasin Yılmaz ViT 21 2 0 10 Mar 2022
Cross Modal Retrieval with Querybank Normalisation Simion-Vlad Bogolin Ioana Croitoru Hailin Jin Yang Liu Samuel Albanie 25 84 0 23 Dec 2021
Tell me what you see: A zero-shot action recognition method based on natural language descriptions Valter Estevam Rayson Laroca David Menotti Hélio Pedrini 25 13 0 18 Dec 2021
Align and Prompt: Video-and-Language Pre-training with Entity Prompts Dongxu Li Junnan Li Hongdong Li Juan Carlos Niebles S. Hoi 20 191 0 17 Dec 2021
Video-Text Pre-training with Learned Regions Rui Yan Mike Zheng Shou Yixiao Ge Alex Jinpeng Wang Xudong Lin Guanyu Cai Jinhui Tang 25 23 0 02 Dec 2021
Object-aware Video-language Pre-training for Retrieval Alex Jinpeng Wang Yixiao Ge Guanyu Cai Rui Yan Xudong Lin Ying Shan Xiaohu Qie Mike Zheng Shou ViT VLM 17 79 0 01 Dec 2021
VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling Tsu-jui Fu Linjie Li Zhe Gan Kevin Qinghong Lin W. Wang Lijuan Wang Zicheng Liu VLM 34 216 0 24 Nov 2021
Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions Hongwei Xue Tiankai Hang Yanhong Zeng Yuchong Sun Bei Liu Huan Yang Jianlong Fu B. Guo AI4TS VLM 27 189 0 19 Nov 2021
DVCFlow: Modeling Information Flow Towards Human-like Video Captioning Xu Yan Zhengcong Fei Shuhui Wang Qingming Huang Qi Tian VGen 32 4 0 19 Nov 2021
Masking Modalities for Cross-modal Video Retrieval Valentin Gabeur Arsha Nagrani Chen Sun Alahari Karteek Cordelia Schmid 11 29 0 01 Nov 2021
Motion-aware Contrastive Video Representation Learning via Foreground-background Merging Shuangrui Ding Maomao Li Tianyu Yang Rui Qian Haohang Xu Qingyi Chen Jue Wang Hongkai Xiong SSL 15 49 0 30 Sep 2021
ActionCLIP: A New Paradigm for Video Action Recognition Mengmeng Wang Jiazheng Xing Yong Liu VLM 149 362 0 17 Sep 2021
End-to-End Dense Video Captioning with Parallel Decoding Teng Wang Ruimao Zhang Zhichao Lu Feng Zheng Ran Cheng Ping Luo 3DV 38 179 0 17 Aug 2021
HANet: Hierarchical Alignment Networks for Video-Text Retrieval Peng Wu Xiangteng He Mingqian Tang Yiliang Lv Jing Liu 23 52 0 26 Jul 2021
CLIP2Video: Mastering Video-Text Retrieval via Image CLIP Han Fang Pengfei Xiong Luhui Xu Yu Chen CLIP VLM 13 291 0 21 Jun 2021
COOT: Cooperative Hierarchical Transformer for Video-Text Representation Learning Simon Ging Mohammadreza Zolfaghari Hamed Pirsiavash Thomas Brox ViT CLIP 13 168 0 01 Nov 2020
Dual Encoding for Video Retrieval by Text Jianfeng Dong Xirong Li Chaoxi Xu Xun Yang Gang Yang Xun Wang Meng Wang 16 2 0 10 Sep 2020
Multi-modal Transformer for Video Retrieval Valentin Gabeur Chen Sun Alahari Karteek Cordelia Schmid ViT 412 595 0 21 Jul 2020
MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning Jie Lei Liwei Wang Yelong Shen Dong Yu Tamara L. Berg Mohit Bansal 14 186 0 11 May 2020
Rethinking Zero-shot Video Classification: End-to-end Training for Realistic Applications Biagio Brattoli Joseph Tighe Fedor Zhdanov Pietro Perona Krzysztof Chalupka VLM 129 127 0 03 Mar 2020
Use What You Have: Video Retrieval Using Representations From Collaborative Experts Yang Liu Samuel Albanie Arsha Nagrani Andrew Zisserman 28 387 0 31 Jul 2019
Effective Approaches to Attention-based Neural Machine Translation Thang Luong Hieu H. Pham Christopher D. Manning 216 7,924 0 17 Aug 2015