Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models

27 November 2023

Bin Lin

Lu Yuan

Papers citing "Video-Bench: A Comprehensive Benchmark and Toolkit for Evaluating Video-based Large Language Models"

50 / 50 papers shown

Title
SeriesBench: A Benchmark for Narrative-Driven Drama Series Understanding Chenkai Zhang Yiming Lei Z. Liu Haitao Leng Shaoguo Liu Tingting Gao Qingjie Liu Yunhong Wang AI4TS 46 0 0 30 Apr 2025
Can Large Language Models Help Multimodal Language Analysis? MMLA: A Comprehensive Benchmark Hanlei Zhang Zhuohang Li Yeshuang Zhu Hua Xu Peiwu Wang Haige Zhu Jie Zhou Jinchao Zhang 25 0 0 23 Apr 2025
IV-Bench: A Benchmark for Image-Grounded Video Perception and Reasoning in Multimodal LLMs David Ma Y. Zhang J. Ren Jarvis Guo Yifan Yao ... Shiwen Ni J. H. Liu Wenhao Huang Ge Zhang Xiaojie Jin VLM 35 0 0 21 Apr 2025
VideoAds for Fast-Paced Video Understanding: Where Opensource Foundation Models Beat GPT-4o & Gemini-1.5 Pro Zheyuan Zhang Monica Dou Linkai Peng Hongyi Pan Ulas Bagci Boqing Gong VLM 56 0 0 12 Apr 2025
VideoComp: Advancing Fine-Grained Compositional and Temporal Alignment in Video-Text Models Dahun Kim A. Piergiovanni Ganesh Mallya A. Angelova CoGe 34 0 0 04 Apr 2025
ANNEXE: Unified Analyzing, Answering, and Pixel Grounding for Egocentric Interaction Yuejiao Su Yi Wang Qiongyang Hu Chuang Yang Lap-Pui Chau 43 0 0 02 Apr 2025
H2VU-Benchmark: A Comprehensive Benchmark for Hierarchical Holistic Video Understanding Qi Wu Quanlong Zheng Yanhao Zhang Junlin Xie Jinguo Luo ... Peng Liu Qingsong Xie Ru Zhen Haonan Lu Zhenyu Yang VLM 58 0 0 31 Mar 2025
Aurelia: Test-time Reasoning Distillation in Audio-Visual LLMs Sanjoy Chowdhury Hanan Gani Nishit Anand Sayan Nag Ruohan Gao Mohamed Elhoseiny Salman Khan Dinesh Manocha LRM 36 0 0 29 Mar 2025
OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contexts Y. Wang Y. Wang Bo Chen Tong Wu Dongyan Zhao Zilong Zheng VLM MLLM 55 1 0 29 Mar 2025
MAVERIX: Multimodal Audio-Visual Evaluation Reasoning IndeX Liuyue Xie George Z. Wei Avik Kuthiala Ce Zheng Ananya Bal ... Rohan Choudhury Morteza Ziyadi Xu Zhang Hao Yang László A. Jeni 62 0 0 27 Mar 2025
4D-Bench: Benchmarking Multi-modal Large Language Models for 4D Object Understanding Wenxuan Zhu Bing Li Cheng Zheng Jinjie Mai Jun-Cheng Chen ... Abdullah Hamdi Sara Rojas Martinez Chia-Wen Lin Mohamed Elhoseiny Bernard Ghanem VLM 48 0 0 22 Mar 2025
Impossible Videos Zechen Bai Hai Ci Mike Zheng Shou EGVM VGen 70 0 0 18 Mar 2025
video-SALMONN-o1: Reasoning-enhanced Audio-visual Large Language Model Guangzhi Sun Yudong Yang Jimin Zhuang Changli Tang Y. Li W. Li Z. Ma Chao Zhang LRM MLLM VLM 64 2 0 17 Feb 2025
MMVU: Measuring Expert-Level Multi-Discipline Video Understanding Yilun Zhao Lujing Xie Haowei Zhang Guo Gan Yitao Long ... Xiangru Tang Zhenwen Liang Y. Liu Chen Zhao Arman Cohan 51 5 0 21 Jan 2025
MLVU: Benchmarking Multi-task Long Video Understanding Junjie Zhou Yan Shu Bo Zhao Boya Wu Zhengyang Liang ... Xi Yang Y. Xiong Bo Zhang Tiejun Huang Zheng Liu VLM 56 11 0 03 Jan 2025
SCBench: A Sports Commentary Benchmark for Video LLMs Kuangzhi Ge L. Chen Kevin Zhang Yulin Luo Tianyu Shi Liaoyuan Fan Xiang Li Guanqun Wang S. Zhang 41 0 0 23 Dec 2024
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces Jihan Yang Shusheng Yang Anjali W. Gupta Rilyn Han Li Fei-Fei Saining Xie LRM 119 50 0 18 Dec 2024
InternLM-XComposer2.5-OmniLive: A Comprehensive Multimodal System for Long-term Streaming Video and Audio Interactions Pan Zhang Xiaoyi Dong Yuhang Cao Yuhang Zang Rui Qian ... X. Zhang K. Chen Yu Qiao D. Lin Jiaqi Wang KELM 84 12 0 12 Dec 2024
Neptune: The Long Orbit to Benchmarking Long Video Understanding Arsha Nagrani Mingda Zhang Ramin Mehran Rachel Hornung N. B. Gundavarapu ... Boqing Gong Cordelia Schmid Mikhail Sirotenko Yukun Zhu Tobias Weyand 98 4 0 12 Dec 2024
PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos Meng Cao Haoran Tang Haoze Zhao Hangyu Guo J. H. Liu Ge Zhang Ruyang Liu Qiang Sun Ian Reid Xiaodan Liang 93 2 0 02 Dec 2024
On the Consistency of Video Large Language Models in Temporal Comprehension Minjoon Jung Junbin Xiao Byoung-Tak Zhang Angela Yao 83 2 0 20 Nov 2024
VidComposition: Can MLLMs Analyze Compositions in Compiled Videos? Yunlong Tang Junjia Guo Hang Hua Susan Liang Mingqian Feng ... Chao Huang Jing Bi Zeliang Zhang Pooyan Fazli Chenliang Xu CoGe 74 8 0 17 Nov 2024
VCBench: A Controllable Benchmark for Symbolic and Abstract Challenges in Video Cognition Chenglin Li Qianglong Chen Zhi Li Feng Tao Yin Zhang 29 0 0 14 Nov 2024
HourVideo: 1-Hour Video-Language Understanding Keshigeyan Chandrasegaran Agrim Gupta Lea M. Hadzic Taran Kota Jimming He Cristobal Eyzaguirre Zane Durante Manling Li Jiajun Wu L. Fei-Fei VLM 36 31 0 07 Nov 2024
MMAU: A Massive Multi-Task Audio Understanding and Reasoning Benchmark S. Sakshi Utkarsh Tyagi Sonal Kumar Ashish Seth Ramaneswaran Selvakumar Oriol Nieto R. Duraiswami Sreyan Ghosh Dinesh Manocha AuLLM ELM 65 19 0 24 Oct 2024
OmnixR: Evaluating Omni-modality Language Models on Reasoning across Modalities L. Chen Hexiang Hu Mingda Zhang Y. Chen Zifeng Wang ... Pranav Shyam Tianyi Zhou Heng-Chiao Huang Ming Yang Boqing Gong 21 2 0 16 Oct 2024
VidEgoThink: Assessing Egocentric Video Understanding Capabilities for Embodied AI Sijie Cheng Kechen Fang Yangyang Yu Sicheng Zhou B. Li Ye Tian Tingguang Li Lei Han Yang Janet Liu 37 8 0 15 Oct 2024
HERM: Benchmarking and Enhancing Multimodal LLMs for Human-Centric Understanding Keliang Li Zaifei Yang Jiahe Zhao Hongze Shen Ruibing Hou Hong Chang Shiguang Shan Xilin Chen VLM 26 0 0 09 Oct 2024
Temporal Reasoning Transfer from Text to Video Lei Li Yuanxin Liu Linli Yao Peiyuan Zhang Chenxin An Lean Wang Xu Sun Lingpeng Kong Qi Liu LRM 32 6 0 08 Oct 2024
E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding Ye Liu Zongyang Ma Zhongang Qi Yang Wu Ying Shan Chang Wen Chen 31 15 0 26 Sep 2024
A Survey on Multimodal Benchmarks: In the Era of Large AI Models Lin Li Guikun Chen Hanrong Shi Jun Xiao Long Chen 34 9 0 21 Sep 2024
ProteinGPT: Multimodal LLM for Protein Property Prediction and Structure Understanding Yijia Xiao Edward Sun Yiqiao Jin Qifan Wang Wei Wang 38 9 0 21 Aug 2024
Continuous Perception Benchmark Zeyu Wang Zhenzhen Weng Serena Yeung-Levy VLM 23 0 0 15 Aug 2024
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output Pan Zhang Xiaoyi Dong Yuhang Zang Yuhang Cao Rui Qian ... Kai Chen Jifeng Dai Yu Qiao Dahua Lin Jiaqi Wang 43 98 0 03 Jul 2024
We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning? Runqi Qiao Qiuna Tan Guanting Dong Minhui Wu Chong Sun ... Yida Xu Muxi Diao Zhimin Bao Chen Li Honggang Zhang VLM LRM 39 30 0 01 Jul 2024
VideoVista: A Versatile Benchmark for Video Understanding and Reasoning Yunxin Li Xinyu Chen Baotian Hu Longyue Wang Haoyuan Shi Min-Ling Zhang MLLM LRM 38 25 0 17 Jun 2024
Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs Zijia Zhao Haoyu Lu Yuqi Huo Yifan Du Tongtian Yue Longteng Guo Bingning Wang Weipeng Chen Jing Liu 31 2 0 13 Jun 2024
MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos Xuehai He Weixi Feng Kaizhi Zheng Yujie Lu Wanrong Zhu ... Zhengyuan Yang Kevin Lin William Yang Wang Lijuan Wang Xin Eric Wang VGen LRM 33 12 0 12 Jun 2024
Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models Shimin Chen Yitian Yuan Shaoxiang Chen Zequn Jie Lin Ma VLM 27 3 0 12 Jun 2024
ShareGPT4Video: Improving Video Understanding and Generation with Better Captions Lin Chen Xilin Wei Jinsong Li Xiaoyi Dong Pan Zhang ... Li Yuan Yu Qiao Dahua Lin Feng Zhao Jiaqi Wang 69 138 0 06 Jun 2024
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis Chaoyou Fu Yuhan Dai Yondong Luo Lei Li Shuhuai Ren ... Tong Bill Xu Xiawu Zheng Enhong Chen Rongrong Ji Xing Sun VLM MLLM 41 216 0 31 May 2024
MotionLLM: Understanding Human Behaviors from Human Motions and Videos Ling-Hao Chen Shunlin Lu Ailing Zeng Hao Zhang Benyou Wang Ruimao Zhang Lei Zhang 45 33 0 30 May 2024
CinePile: A Long Video Question Answering Dataset and Benchmark Ruchit Rawal Khalid Saifullah Ronen Basri David Jacobs Gowthami Somepalli Tom Goldstein 38 39 0 14 May 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 120 106 0 08 Feb 2024
Video Understanding with Large Language Models: A Survey Yunlong Tang Jing Bi Siting Xu Luchuan Song Susan Liang ... Feng Zheng Jianguo Zhang Ping Luo Jiebo Luo Chenliang Xu VLM 47 81 0 29 Dec 2023
A Survey on Multimodal Large Language Models Shukang Yin Chaoyou Fu Sirui Zhao Ke Li Xing Sun Tong Bill Xu Enhong Chen MLLM LRM 33 551 0 23 Jun 2023
Valley: Video Assistant with Large Language model Enhanced abilitY Ruipu Luo Ziwang Zhao Min Yang Junwei Dong Da Li Pengcheng Lu Tao Wang Linmei Hu Ming-Hui Qiu MLLM 38 185 0 12 Jun 2023
ChatVideo: A Tracklet-centric Multimodal and Versatile Video Understanding System Junke Wang Dongdong Chen Chong Luo Xiyang Dai Lu Yuan Zuxuan Wu Yu-Gang Jiang 93 54 0 27 Apr 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 203 883 0 27 Apr 2023
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022