HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training

1 May 2020

Papers citing "HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training"

50 / 96 papers shown

Title
HierSum: A Global and Local Attention Mechanism for Video Summarization Apoorva Beedu Irfan Essa 44 0 0 25 Apr 2025
ALLVB: All-in-One Long Video Understanding Benchmark Xichen Tan Yuanjing Luo Yunfan Ye Fang Liu Zhiping Cai MLLM VLM 70 0 0 10 Mar 2025
OneLLM: One Framework to Align All Modalities with Language Jiaming Han Kaixiong Gong Yiyuan Zhang Jiaqi Wang Kaipeng Zhang D. Lin Yu Qiao Peng Gao Xiangyu Yue MLLM 104 107 0 10 Jan 2025
Do Language Models Understand Time? Xi Ding Lei Wang 167 0 0 18 Dec 2024
End-to-end Open-vocabulary Video Visual Relationship Detection using Multi-modal Prompting Yongqi Wang Xinxiao Wu Shuo Yang Jiebo Luo 74 1 0 19 Sep 2024
Uncertainty-Guided Self-Questioning and Answering for Video-Language Alignment Jin Chen Kaijing Ma Haojian Huang Jiayu Shen Han Fang Xianghao Zang Chao Ban 79 2 0 17 Sep 2024
QTG-VQA: Question-Type-Guided Architectural for VideoQA Systems Zhixian He Pengcheng Zhao Fuwei Zhang Shujin Lin 31 0 0 14 Sep 2024
Lighthouse: A User-Friendly Library for Reproducible Video Moment Retrieval and Highlight Detection Taichi Nishimura Shota Nakada Hokuto Munakata Tatsuya Komatsu VLM 14 1 0 06 Aug 2024
Learning Video Context as Interleaved Multimodal Sequences S. Shao Pengchuan Zhang Y. Li Xide Xia A. Meso Ziteng Gao Jinheng Xie N. Holliman Mike Zheng Shou 41 5 0 31 Jul 2024
Causal Understanding For Video Question Answering Bhanu Prakash Reddy Guda Tanmay Kulkarni Adithya Sampath Swarnashree Mysore Sathyendra CML 39 0 0 23 Jul 2024
LVBench: An Extreme Long Video Understanding Benchmark Weihan Wang Zehai He Wenyi Hong Yean Cheng Xiaohan Zhang ... Shiyu Huang Bin Xu Yuxiao Dong Ming Ding Jie Tang ELM VLM 40 63 0 12 Jun 2024
MuJo: Multimodal Joint Feature Space Learning for Human Activity Recognition Stefan Gerd Fritsch Cennet Oğuz Vitor Fortes Rey L. Ray Maximilian Kiefer-Emmanouilidis P. Lukowicz HAI 43 0 0 06 Jun 2024
Instruct-ReID++: Towards Universal Purpose Instruction-Guided Person Re-identification Weizhen He Yiheng Deng Yunfeng Yan Feng Zhu Yizhou Wang Lei Bai Qingsong Xie Donglian Qi Wanli Ouyang Shixiang Tang 93 2 0 28 May 2024
CinePile: A Long Video Question Answering Dataset and Benchmark Ruchit Rawal Khalid Saifullah Ronen Basri David Jacobs Gowthami Somepalli Tom Goldstein 38 39 0 14 May 2024
Koala: Key frame-conditioned long video-LLM Reuben Tan Ximeng Sun Ping Hu Jui-hsien Wang Hanieh Deilamsalehy Bryan A. Plummer Bryan C. Russell Kate Saenko 38 35 0 05 Apr 2024
VideoDistill: Language-aware Vision Distillation for Video Question Answering Bo Zou Chao Yang Yu Qiao Chengbin Quan Youjian Zhao VGen 39 1 0 01 Apr 2024
DAM: Dynamic Adapter Merging for Continual Video QA Learning Feng Cheng Ziyang Wang Yi-Lin Sung Yan-Bo Lin Mohit Bansal Gedas Bertasius CLL MoMe 31 10 0 13 Mar 2024
Video ReCap: Recursive Captioning of Hour-Long Videos Md. Mohaiminul Islam Ngan Ho Xitong Yang Tushar Nagarajan Lorenzo Torresani Gedas Bertasius VGen VLM 27 44 0 20 Feb 2024
MM-VID: Advancing Video Understanding with GPT-4V(ision) Kevin Qinghong Lin Faisal Ahmed Linjie Li Chung-Ching Lin E. Azarnasab ... Lin Liang Zicheng Liu Yumao Lu Ce Liu Lijuan Wang MLLM 26 63 0 30 Oct 2023
CAD -- Contextual Multi-modal Alignment for Dynamic AVQA Asmar Nadeem Adrian Hilton R. Dawes Graham A. Thomas A. Mustafa 16 9 0 25 Oct 2023
SCANet: Scene Complexity Aware Network for Weakly-Supervised Video Moment Retrieval Sunjae Yoon Gwanhyeong Koo Dahyun Kim Changdong Yoo 21 12 0 08 Oct 2023
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model Avamarie Brueggeman Andrea Madotto Zhaojiang Lin Tushar Nagarajan Matt Smith ... Peyman Heidari Yue Liu Kavya Srinet Babak Damavandi Anuj Kumar MLLM 24 92 0 27 Sep 2023
Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts Bipin Rajendran Bashir M. Al-Hashimi MLLM VLM 26 2 0 27 Sep 2023
Dual-Modal Attention-Enhanced Text-Video Retrieval with Triplet Partial Margin Contrastive Learning Chen Jiang Hong Liu Xuzheng Yu Qing Wang Yuan-Chia Cheng ... Zhongyi Liu Qingpei Guo Wei Chu Ming Yang Yuan Qi 16 10 0 20 Sep 2023
EgoPCA: A New Framework for Egocentric Hand-Object Interaction Understanding Yue Xu Yong-Lu Li Zhemin Huang Michael Xu Liu Cewu Lu Yu-Wing Tai Chi-Keung Tang EgoV 18 9 0 05 Sep 2023
Instruct-ReID: A Multi-purpose Person Re-identification Task with Instructions Weizhen He Yihe Deng Shixiang Tang Qihao Chen Qingsong Xie ... Feng Zhu Rui Zhao Wanli Ouyang Donglian Qi Yunfeng Yan 65 19 0 13 Jun 2023
Self-Chained Image-Language Model for Video Localization and Question Answering Shoubin Yu Jaemin Cho Prateek Yadav Mohit Bansal 31 129 0 11 May 2023
Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselines Yaochen Zhu Xiangqing Shen Rui Xia 19 5 0 05 Apr 2023
Procedure-Aware Pretraining for Instructional Video Understanding Honglu Zhou Roberto Martín-Martín Mubbasir Kapadia Silvio Savarese Juan Carlos Niebles 23 38 0 31 Mar 2023
Hierarchical Video-Moment Retrieval and Step-Captioning Abhaysinh Zala Jaemin Cho Satwik Kottur Xilun Chen Barlas Ouguz Yasher Mehdad Mohit Bansal 3DV 8 51 0 29 Mar 2023
VideoXum: Cross-modal Visual and Textural Summarization of Videos Jingyang Lin Hang Hua Ming Chen Yikang Li Jenhao Hsiao C. Ho Jiebo Luo 23 30 0 21 Mar 2023
Accommodating Audio Modality in CLIP for Multimodal Processing Ludan Ruan Anwen Hu Yuqing Song Liang Zhang S. Zheng Qin Jin VLM 16 10 0 12 Mar 2023
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning Antoine Yang Arsha Nagrani Paul Hongsuck Seo Antoine Miech Jordi Pont-Tuset Ivan Laptev Josef Sivic Cordelia Schmid AI4TS VLM 18 220 0 27 Feb 2023
STOA-VLP: Spatial-Temporal Modeling of Object and Action for Video-Language Pre-training Weihong Zhong Mao Zheng Duyu Tang Xuan Luo Heng Gong Xiaocheng Feng Bing Qin 25 8 0 20 Feb 2023
Interactive Video Corpus Moment Retrieval using Reinforcement Learning Zhixin Ma Chong-Wah Ngo 31 3 0 19 Feb 2023
HierVL: Learning Hierarchical Video-Language Embeddings Kumar Ashutosh Rohit Girdhar Lorenzo Torresani Kristen Grauman VLM AI4TS 20 51 0 05 Jan 2023
NaQ: Leveraging Narrations as Queries to Supervise Episodic Memory Santhosh Kumar Ramakrishnan Ziad Al-Halah Kristen Grauman 80 39 0 02 Jan 2023
Masked Contrastive Pre-Training for Efficient Video-Text Retrieval Fangxun Shu Biaolong Chen Yue Liao Shuwen Xiao Wenyu Sun Xiaobo Li Yousong Zhu Jinqiao Wang Si Liu CLIP 22 11 0 02 Dec 2022
Perceiver-VL: Efficient Vision-and-Language Modeling with Iterative Latent Attention Zineng Tang Jaemin Cho Jie Lei Mohit Bansal VLM 16 9 0 21 Nov 2022
SMAUG: Sparse Masked Autoencoder for Efficient Video-Language Pre-training Yuanze Lin Chen Wei Huiyu Wang Alan Yuille Cihang Xie 3DGS 26 15 0 21 Nov 2022
Grafting Pre-trained Models for Multimodal Headline Generation Lingfeng Qiao Chen Wu Ye Liu Haoyuan Peng Di Yin Bo Ren 30 5 0 14 Nov 2022
Watching the News: Towards VideoQA Models that can Read Soumya Jahagirdar Minesh Mathew Dimosthenis Karatzas C. V. Jawahar 14 18 0 10 Nov 2022
Selective Query-guided Debiasing for Video Corpus Moment Retrieval Sunjae Yoon Jiajing Hong Eunseop Yoon Dahyun Kim Junyeong Kim Hee Suk Yoon Changdong Yoo 33 21 0 17 Oct 2022
Hierarchical3D Adapters for Long Video-to-text Summarization Pinelopi Papalampidi Mirella Lapata VGen 27 12 0 10 Oct 2022
Music-to-Text Synaesthesia: Generating Descriptive Text from Music Recordings Zhihuan Kuang Shi Zong Jianbing Zhang Jiajun Chen Hongfu Liu 22 4 0 02 Oct 2022
TVLT: Textless Vision-Language Transformer Zineng Tang Jaemin Cho Yixin Nie Mohit Bansal VLM 49 28 0 28 Sep 2022
MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretraining Xiaoyi Dong Jianmin Bao Yinglin Zheng Ting Zhang Dongdong Chen ... Weiming Zhang Lu Yuan Dong Chen Fang Wen Nenghai Yu CLIP VLM 32 157 0 25 Aug 2022
MuMUR : Multilingual Multimodal Universal Retrieval Avinash Madasu Estelle Aflalo Gabriela Ben-Melech Stan Shachar Rosenman Shao-Yen Tseng Gedas Bertasius Vasudev Lal 37 3 0 24 Aug 2022
Video Question Answering with Iterative Video-Text Co-Tokenization A. Piergiovanni K. Morton Weicheng Kuo Michael S. Ryoo A. Angelova 16 17 0 01 Aug 2022
Clover: Towards A Unified Video-Language Alignment and Fusion Model Jingjia Huang Yinan Li Jiashi Feng Xinglong Wu Xiaoshuai Sun Rongrong Ji VLM 19 48 0 16 Jul 2022