ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound

6 April 2022

Yan-Bo Lin

Jie Lei

Mohit Bansal

Gedas Bertasius

ArXiv PDF HTML

Papers citing "ECLIPSE: Efficient Long-range Video Retrieval using Sight and Sound"

39 / 39 papers shown

Title
Learning Audio-guided Video Representation with Gated Attention for Video-Text Retrieval Boseung Jeong Jicheol Park Sungyeon Kim Suha Kwak 33 0 0 03 Apr 2025
Leveraging Modality Tags for Enhanced Cross-Modal Video Retrieval A. Fragomeni Dima Damen Michael Wray 33 0 0 02 Apr 2025
NeighborRetr: Balancing Hub Centrality in Cross-Modal Retrieval Zengrong Lin Zheng Wang Tianwen Qian Pan Mu Sixian Chan Cong Bai 40 0 0 13 Mar 2025
Masked Differential Privacy David Schneider Sina Sajadmanesh Vikash Sehwag Saquib Sarfraz Rainer Stiefelhagen Lingjuan Lyu Vivek Sharma 28 1 0 22 Oct 2024
Learning Video Context as Interleaved Multimodal Sequences S. Shao Pengchuan Zhang Y. Li Xide Xia A. Meso Ziteng Gao Jinheng Xie N. Holliman Mike Zheng Shou 41 5 0 31 Jul 2024
Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learning Thong Nguyen Yi Bin Xiaobao Wu Xinshuai Dong Zhiyuan Hu Khoi M. Le Cong-Duy Nguyen See-Kiong Ng Luu Anh Tuan 29 5 0 04 Jul 2024
Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectives Thong Nguyen Yi Bin Junbin Xiao Leigang Qu Yicong Li Jay Zhangjie Wu Cong-Duy Nguyen See-Kiong Ng Luu Anh Tuan VLM 34 9 1 09 Jun 2024
Encoding and Controlling Global Semantics for Long-form Video Question Answering Thong Nguyen Zhiyuan Hu Xiaobao Wu Cong-Duy Nguyen See-Kiong Ng A. Luu 32 2 0 30 May 2024
Siamese Vision Transformers are Scalable Audio-visual Learners Yan-Bo Lin Gedas Bertasius 27 5 0 28 Mar 2024
Text Is MASS: Modeling as Stochastic Embedding for Text-Video Retrieval Jiamian Wang Guohao Sun Pichao Wang Dongfang Liu S. Dianat Majid Rabbani Raghuveer M. Rao Zhiqiang Tao VGen 44 2 0 26 Mar 2024
Retrieval-Augmented Egocentric Video Captioning Jilan Xu Yifei Huang Junlin Hou Guo Chen Yue Zhang Rui Feng Weidi Xie EgoV 26 7 0 01 Jan 2024
MoVQA: A Benchmark of Versatile Question-Answering for Long-Form Movie Understanding Hongjie Zhang Yi Liu Lu Dong Yifei Huang Z. Ling Yali Wang Limin Wang Yu Qiao 15 25 0 08 Dec 2023
CAST: Cross-Attention in Space and Time for Video Action Recognition Dongho Lee Jongseo Lee Jinwoo Choi EgoV 19 10 0 30 Nov 2023
ViLMA: A Zero-Shot Benchmark for Linguistic and Temporal Grounding in Video-Language Models .Ilker Kesen Andrea Pedrotti Mustafa Dogan Michele Cafagna Emre Can Acikgoz ... Iacer Calixto Anette Frank Albert Gatt Aykut Erdem Erkut Erdem 22 15 0 13 Nov 2023
Query-aware Long Video Localization and Relation Discrimination for Deep Video Understanding Yuanxing Xu Yuting Wei Bin Wu 17 0 0 19 Oct 2023
Unified Coarse-to-Fine Alignment for Video-Text Retrieval Ziyang Wang Yi-Lin Sung Feng Cheng Gedas Bertasius Mohit Bansal 80 41 0 18 Sep 2023
Multimodal Fish Feeding Intensity Assessment in Aquaculture Meng Cui Xubo Liu Haohe Liu Zhuangzhuang Du Tao Chen Guoping Lian Daoliang Li Wenwu Wang 13 5 0 10 Sep 2023
Audio-Enhanced Text-to-Video Retrieval using Text-Conditioned Feature Alignment Sarah Ibrahimi Xiaohang Sun Pichao Wang Amanmeet Garg Ashutosh Sanan Mohamed Omar 33 12 0 24 Jul 2023
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Sihan Chen Handong Li Qunbo Wang Zijia Zhao Ming-Ting Sun Xinxin Zhu J. Liu 21 95 0 29 May 2023
SViTT: Temporal Learning of Sparse Video-Text Transformers Yi Li Kyle Min Subarna Tripathi Nuno Vasconcelos 12 12 0 18 Apr 2023
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Sihan Chen Xingjian He Longteng Guo Xinxin Zhu Weining Wang Jinhui Tang Jinhui Tang VLM 21 99 0 17 Apr 2023
Multiscale Audio Spectrogram Transformer for Efficient Audio Classification Wenjie Zhu M. Omar 25 22 0 19 Mar 2023
MIST: Multi-modal Iterative Spatial-Temporal Transformer for Long-form Video Question Answering Difei Gao Luowei Zhou Lei Ji Linchao Zhu Yezhou Yang Mike Zheng Shou 30 60 0 19 Dec 2022
Vision Transformers are Parameter-Efficient Audio-Visual Learners Yan-Bo Lin Yi-Lin Sung Jie Lei Mohit Bansal Gedas Bertasius 13 69 0 15 Dec 2022
VindLU: A Recipe for Effective Video-and-Language Pretraining Feng Cheng Xizi Wang Jie Lei David J. Crandall Mohit Bansal Gedas Bertasius VLM 16 78 0 09 Dec 2022
TVLT: Textless Vision-Language Transformer Zineng Tang Jaemin Cho Yixin Nie Mohit Bansal VLM 38 28 0 28 Sep 2022
CONE: An Efficient COarse-to-fiNE Alignment Framework for Long Video Temporal Grounding Zhijian Hou Wanjun Zhong Lei Ji Difei Gao Kun Yan W. Chan Chong-Wah Ngo Zheng Shou Nan Duan AI4TS 24 23 0 22 Sep 2022
MuMUR : Multilingual Multimodal Universal Retrieval Avinash Madasu Estelle Aflalo Gabriela Ben-Melech Stan Shachar Rosenman Shao-Yen Tseng Gedas Bertasius Vasudev Lal 22 3 0 24 Aug 2022
Revealing Single Frame Bias for Video-and-Language Learning Jie Lei Tamara L. Berg Mohit Bansal 13 109 0 07 Jun 2022
A CLIP-Hitchhiker's Guide to Long Video Retrieval Max Bain Arsha Nagrani Gül Varol Andrew Zisserman CLIP 113 60 0 17 May 2022
VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding Hu Xu Gargi Ghosh Po-Yao (Bernie) Huang Dmytro Okhonko Armen Aghajanyan Florian Metze Luke Zettlemoyer Florian Metze Luke Zettlemoyer Christoph Feichtenhofer CLIP VLM 245 554 0 28 Sep 2021
Exploiting Audio-Visual Consistency with Partial Supervision for Spatial Audio Generation Yan-Bo Lin Y. Wang 43 21 0 03 May 2021
VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text Hassan Akbari Liangzhe Yuan Rui Qian Wei-Hong Chuang Shih-Fu Chang Yin Cui Boqing Gong ViT 231 573 0 22 Apr 2021
T2VLAD: Global-Local Sequence Alignment for Text-Video Retrieval Xiaohan Wang Linchao Zhu Yi Yang 138 166 0 20 Apr 2021
CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrieval Huaishao Luo Lei Ji Ming Zhong Yang Chen Wen Lei Nan Duan Tianrui Li CLIP VLM 303 771 0 18 Apr 2021
A Straightforward Framework For Video Retrieval Using CLIP Jesús Andrés Portillo-Quintero J. C. Ortíz-Bayliss Hugo Terashima-Marín CLIP 302 116 0 24 Feb 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 278 1,939 0 09 Feb 2021
PSLA: Improving Audio Tagging with Pretraining, Sampling, Labeling, and Aggregation Yuan Gong Yu-An Chung James R. Glass VLM 99 144 0 02 Feb 2021
Multi-modal Transformer for Video Retrieval Valentin Gabeur Chen Sun Alahari Karteek Cordelia Schmid ViT 398 532 0 21 Jul 2020