Zero-Shot Video Question Answering via Frozen Bidirectional Language Models

16 June 2022

Papers citing "Zero-Shot Video Question Answering via Frozen Bidirectional Language Models"

50 / 193 papers shown

Title
Cross-Modal Learning for Anomaly Detection in Fused Magnesium Smelting Process: Methodology and Benchmark Gaochang Wu Yapeng Zhang Lan Deng Jingxin Zhang Tianyou Chai 31 1 0 13 Jun 2024
Too Many Frames, Not All Useful: Efficient Strategies for Long-Form Video QA Jongwoo Park Kanchana Ranasinghe Kumara Kahatapitiya Wonjeong Ryoo Donghyun Kim Michael S. Ryoo 60 20 0 13 Jun 2024
Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams Haoji Zhang Yiqin Wang Yansong Tang Yong-Jin Liu Jiashi Feng Jifeng Dai Xiaojie Jin 32 37 0 12 Jun 2024
Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models Shimin Chen Yitian Yuan Shaoxiang Chen Zequn Jie Lin Ma VLM 27 3 0 12 Jun 2024
Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectives Thong Nguyen Yi Bin Junbin Xiao Leigang Qu Yicong Li Jay Zhangjie Wu Cong-Duy Nguyen See-Kiong Ng Luu Anh Tuan VLM 39 9 1 09 Jun 2024
Artemis: Towards Referential Understanding in Complex Videos Jihao Qiu Yuan Zhang Xi Tang Lingxi Xie Tianren Ma Pengyu Yan David Doermann Qixiang Ye Yunjie Tian VLM VGen 37 8 0 01 Jun 2024
VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos Ziyang Wang Shoubin Yu Elias Stengel-Eskin Jaehong Yoon Feng Cheng Gedas Bertasius Mohit Bansal 40 56 0 29 May 2024
Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs Mustafa Shukor Matthieu Cord 64 5 0 26 May 2024
Streaming Long Video Understanding with Large Language Models Rui Qian Xiao-wen Dong Pan Zhang Yuhang Zang Shuangrui Ding Dahua Lin Jiaqi Wang VLM 29 40 0 25 May 2024
Dense Connector for MLLMs Huanjin Yao Wenhao Wu Taojiannan Yang Yuxin Song Mengxi Zhang Haocheng Feng Yifan Sun Zhiheng Li Wanli Ouyang Jingdong Wang MLLM VLM 32 16 0 22 May 2024
Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts Yunxin Li Shenyuan Jiang Baotian Hu Longyue Wang Wanqi Zhong Wenhan Luo Lin Ma Min-Ling Zhang MoE 34 28 0 18 May 2024
VideoQA-SC: Adaptive Semantic Communication for Video Question Answering Jiangyuan Guo Wei-Neng Chen Yuxuan Sun Jia-lin Xu Bo Ai 54 4 0 17 May 2024
FreeVA: Offline MLLM as Training-Free Video Assistant Wenhao Wu VLM OffRL 32 19 0 13 May 2024
WorldQA: Multimodal World Knowledge in Videos through Long-Chain Reasoning Yuanhan Zhang Kaichen Zhang Bo-wen Li Fanyi Pu Christopher Arif Setiadharma Jingkang Yang Ziwei Liu VGen 47 7 0 06 May 2024
Adapting Dual-encoder Vision-language Models for Paraphrased Retrieval Jiacheng Cheng Hijung Valentina Shin Nuno Vasconcelos Bryan C. Russell Fabian Caba Heilbron VLM 22 1 0 06 May 2024
MovieChat+: Question-aware Sparse Memory for Long Video Question Answering Enxin Song Wenhao Chai Tianbo Ye Jenq-Neng Hwang Xi Li Gaoang Wang VLM MLLM 24 28 0 26 Apr 2024
PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning Lin Xu Yilin Zhao Daquan Zhou Zhijie Lin See Kiong Ng Jiashi Feng MLLM VLM 34 156 0 25 Apr 2024
Listen Then See: Video Alignment with Speaker Attention Aviral Agrawal Carlos Mateo Samudio Lezcano Iqui Balam Heredia-Marin P. Sethi 28 2 0 21 Apr 2024
Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answering Jie Ma Min Hu Pinghui Wang Wangchun Sun Lingyun Song Hongbin Pei Jun Liu Youtian Du 32 4 0 18 Apr 2024
CREST: Cross-modal Resonance through Evidential Deep Learning for Enhanced Zero-Shot Learning Haojian Huang Xiaozhen Qiao Zhuo Chen Haodong Chen Bingyu Li Zhe Sun Mulin. Chen Xuelong Li 24 10 0 15 Apr 2024
Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs Kanchana Ranasinghe Satya Narayan Shukla Omid Poursaeed Michael S. Ryoo Tsung-Yu Lin LRM 38 22 0 11 Apr 2024
MoReVQA: Exploring Modular Reasoning Models for Video Question Answering Juhong Min Shyamal Buch Arsha Nagrani Minsu Cho Cordelia Schmid LRM 34 20 0 09 Apr 2024
MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding Bo He Hengduo Li Young Kyun Jang Menglin Jia Xuefei Cao Ashish Shah Abhinav Shrivastava Ser-Nam Lim MLLM 81 88 0 08 Apr 2024
Koala: Key frame-conditioned long video-LLM Reuben Tan Ximeng Sun Ping Hu Jui-hsien Wang Hanieh Deilamsalehy Bryan A. Plummer Bryan C. Russell Kate Saenko 38 35 0 05 Apr 2024
MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens Kirolos Ataallah Xiaoqian Shen Eslam Abdelrahman Essam Sleiman Deyao Zhu Jian Ding Mohamed Elhoseiny VLM 39 66 0 04 Apr 2024
LongVLM: Efficient Long Video Understanding via Large Language Models Yuetian Weng Mingfei Han Haoyu He Xiaojun Chang Bohan Zhuang VLM 60 56 0 04 Apr 2024
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward Ruohong Zhang Liangke Gui Zhiqing Sun Yihao Feng Keyang Xu ... Di Fu Chunyuan Li Alexander G. Hauptmann Yonatan Bisk Yiming Yang MLLM 43 57 0 01 Apr 2024
ST-LLM: Large Language Models Are Effective Temporal Learners Ruyang Liu Chen Li Haoran Tang Yixiao Ge Ying Shan Ge Li 27 68 0 30 Mar 2024
An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM Wonkyun Kim Changin Choi Wonseok Lee Wonjong Rhee VLM 43 50 0 27 Mar 2024
Elysium: Exploring Object-level Perception in Videos via MLLM Hang Wang Yanjie Wang Yongjie Ye Yuxiang Nie Can Huang MLLM 32 18 0 25 Mar 2024
Ranking Distillation for Open-Ended Video Question Answering with Insufficient Labels Tianming Liang Chaolei Tan Beihao Xia Wei-Shi Zheng Jianfang Hu 28 1 0 21 Mar 2024
vid-TLDR: Training Free Token merging for Light-weight Video Transformer Joonmyung Choi Sanghyeok Lee Jaewon Chu Minhyuk Choi Hyunwoo J. Kim MoMe ViT 40 12 0 20 Mar 2024
VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding Yue Fan Xiaojian Ma Rujie Wu Yuntao Du Jiaqi Li Zhi Gao Qing Li VLM LLMAG 46 55 0 18 Mar 2024
VideoAgent: Long-form Video Understanding with Large Language Model as Agent Xiaohan Wang Yuhui Zhang Orr Zohar Serena Yeung-Levy VLM 108 83 0 15 Mar 2024
HawkEye: Training Video-Text LLMs for Grounding Text in Videos Yueqian Wang Xiaojun Meng Jianxin Liang Yuxuan Wang Qun Liu Dongyan Zhao 32 30 0 15 Mar 2024
Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding Guo Chen Yifei Huang Jilan Xu Baoqi Pei Zhe Chen Zhiqi Li Jiahao Wang Kunchang Li Tong Lu Limin Wang Mamba 64 72 0 14 Mar 2024
DAM: Dynamic Adapter Merging for Continual Video QA Learning Feng Cheng Ziyang Wang Yi-Lin Sung Yan-Bo Lin Mohit Bansal Gedas Bertasius CLL MoMe 26 10 0 13 Mar 2024
Answering Diverse Questions via Text Attached with Key Audio-Visual Clues Qilang Ye Zitong Yu Xin Liu 33 1 0 11 Mar 2024
TV-TREES: Multimodal Entailment Trees for Neuro-Symbolic Video Reasoning Kate Sanders Nathaniel Weir Benjamin Van Durme LRM 31 11 0 29 Feb 2024
Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models Yixin Liu Kai Zhang Yuan Li Zhiling Yan Chujie Gao ... Yue Huang Hanchi Sun Jianfeng Gao Lifang He Lichao Sun VLM VGen EGVM 68 254 0 27 Feb 2024
LSTP: Language-guided Spatial-Temporal Prompt Learning for Long-form Video-Text Understanding Yuxuan Wang Yueqian Wang Pengfei Wu Jianxin Liang Dongyan Zhao Zilong Zheng VLM 21 9 0 25 Feb 2024
Video ReCap: Recursive Captioning of Hour-Long Videos Md. Mohaiminul Islam Ngan Ho Xitong Yang Tushar Nagarajan Lorenzo Torresani Gedas Bertasius VGen VLM 27 44 0 20 Feb 2024
VideoPrism: A Foundational Visual Encoder for Video Understanding Long Zhao N. B. Gundavarapu Liangzhe Yuan Hao Zhou Shen Yan ... Huisheng Wang Hartwig Adam Mikhail Sirotenko Ting Liu Boqing Gong VGen 27 29 0 20 Feb 2024
LVCHAT: Facilitating Long Video Comprehension Yu-Xiang Wang Zeyuan Zhang Julian McAuley Zexue He VLM 26 4 0 19 Feb 2024
PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter Junfei Xiao Zheng Xu Alan L. Yuille Shen Yan Boyu Wang 22 3 0 16 Feb 2024
Question-Instructed Visual Descriptions for Zero-Shot Video Question Answering David Romero Thamar Solorio 98 4 0 16 Feb 2024
BDIQA: A New Dataset for Video Question Answering to Explore Cognitive Reasoning through Theory of Mind Yuanyuan Mao Xin Lin Qin Ni Liang He 16 3 0 12 Feb 2024
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization Yang Jin Zhicheng Sun Kun Xu Kun Xu Liwei Chen ... Yuliang Liu Di Zhang Yang Song Kun Gai Yadong Mu VGen 47 42 0 05 Feb 2024
Position: What Can Large Language Models Tell Us about Time Series Analysis Ming Jin Yifan Zhang Wei Chen Kexin Zhang Yuxuan Liang Bin Yang Jindong Wang Shirui Pan Qingsong Wen AI4TS 29 16 0 05 Feb 2024
YTCommentQA: Video Question Answerability in Instructional Videos Saelyne Yang Sunghyun Park Yunseok Jang Moontae Lee 16 3 0 30 Jan 2024