AutoEval-Video: An Automatic Benchmark for Assessing Large Vision Language Models in Open-Ended Video Question Answering

25 November 2023

Papers citing "AutoEval-Video: An Automatic Benchmark for Assessing Large Vision Language Models in Open-Ended Video Question Answering"

26 / 26 papers shown

Title
SeriesBench: A Benchmark for Narrative-Driven Drama Series Understanding Chenkai Zhang Yiming Lei Z. Liu Haitao Leng Shaoguo Liu Tingting Gao Qingjie Liu Yunhong Wang AI4TS 46 0 0 30 Apr 2025
VideoAds for Fast-Paced Video Understanding: Where Opensource Foundation Models Beat GPT-4o & Gemini-1.5 Pro Zheyuan Zhang Monica Dou Linkai Peng Hongyi Pan Ulas Bagci Boqing Gong VLM 56 0 0 12 Apr 2025
SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding Yangliu Hu Zikai Song Na Feng Yawei Luo Junqing Yu Yi-Ping Phoebe Chen Wei Yang 33 0 0 10 Apr 2025
H2VU-Benchmark: A Comprehensive Benchmark for Hierarchical Holistic Video Understanding Qi Wu Quanlong Zheng Yanhao Zhang Junlin Xie Jinguo Luo ... Peng Liu Qingsong Xie Ru Zhen Haonan Lu Zhenyu Yang VLM 58 0 0 31 Mar 2025
MAVERIX: Multimodal Audio-Visual Evaluation Reasoning IndeX Liuyue Xie George Z. Wei Avik Kuthiala Ce Zheng Ananya Bal ... Rohan Choudhury Morteza Ziyadi Xu Zhang Hao Yang László A. Jeni 62 0 0 27 Mar 2025
4D-Bench: Benchmarking Multi-modal Large Language Models for 4D Object Understanding Wenxuan Zhu Bing Li Cheng Zheng Jinjie Mai Jun-Cheng Chen ... Abdullah Hamdi Sara Rojas Martinez Chia-Wen Lin Mohamed Elhoseiny Bernard Ghanem VLM 48 0 0 22 Mar 2025
TIME: Temporal-sensitive Multi-dimensional Instruction Tuning and Benchmarking for Video-LLMs Yunxiao Wang Meng Liu Rui Shao Haoyu Zhang Bin Wen Fan Yang Tingting Gao Di Zhang Liqiang Nie 62 1 0 13 Mar 2025
MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models Wenyi Hong Yean Cheng Z. Yang Weihan Wang Lefan Wang Xiaotao Gu Shiyu Huang Yuxiao Dong J. Tang CoGe VLM 71 4 0 06 Jan 2025
SCBench: A Sports Commentary Benchmark for Video LLMs Kuangzhi Ge L. Chen Kevin Zhang Yulin Luo Tianyu Shi Liaoyuan Fan Xiang Li Guanqun Wang S. Zhang 41 0 0 23 Dec 2024
Neptune: The Long Orbit to Benchmarking Long Video Understanding Arsha Nagrani Mingda Zhang Ramin Mehran Rachel Hornung N. B. Gundavarapu ... Boqing Gong Cordelia Schmid Mikhail Sirotenko Yukun Zhu Tobias Weyand 98 4 0 12 Dec 2024
PhysGame: Uncovering Physical Commonsense Violations in Gameplay Videos Meng Cao Haoran Tang Haoze Zhao Hangyu Guo J. H. Liu Ge Zhang Ruyang Liu Qiang Sun Ian Reid Xiaodan Liang 93 2 0 02 Dec 2024
VidComposition: Can MLLMs Analyze Compositions in Compiled Videos? Yunlong Tang Junjia Guo Hang Hua Susan Liang Mingqian Feng ... Chao Huang Jing Bi Zeliang Zhang Pooyan Fazli Chenliang Xu CoGe 74 8 0 17 Nov 2024
VCBench: A Controllable Benchmark for Symbolic and Abstract Challenges in Video Cognition Chenglin Li Qianglong Chen Zhi Li Feng Tao Yin Zhang 29 0 0 14 Nov 2024
HourVideo: 1-Hour Video-Language Understanding Keshigeyan Chandrasegaran Agrim Gupta Lea M. Hadzic Taran Kota Jimming He Cristobal Eyzaguirre Zane Durante Manling Li Jiajun Wu L. Fei-Fei VLM 36 31 0 07 Nov 2024
E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding Ye Liu Zongyang Ma Zhongang Qi Yang Wu Ying Shan Chang Wen Chen 26 15 0 26 Sep 2024
A Survey on Multimodal Benchmarks: In the Era of Large AI Models Lin Li Guikun Chen Hanrong Shi Jun Xiao Long Chen 34 9 0 21 Sep 2024
VideoVista: A Versatile Benchmark for Video Understanding and Reasoning Yunxin Li Xinyu Chen Baotian Hu Longyue Wang Haoyuan Shi Min-Ling Zhang MLLM LRM 38 25 0 17 Jun 2024
Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs Zijia Zhao Haoyu Lu Yuqi Huo Yifan Du Tongtian Yue Longteng Guo Bingning Wang Weipeng Chen Jing Liu 31 2 0 13 Jun 2024
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis Chaoyou Fu Yuhan Dai Yondong Luo Lei Li Shuhuai Ren ... Tong Bill Xu Xiawu Zheng Enhong Chen Rongrong Ji Xing Sun VLM MLLM 41 216 0 31 May 2024
Video Understanding with Large Language Models: A Survey Yunlong Tang Jing Bi Siting Xu Luchuan Song Susan Liang ... Feng Zheng Jianguo Zhang Ping Luo Jiebo Luo Chenliang Xu VLM 47 76 0 29 Dec 2023
Perception Test: A Diagnostic Benchmark for Multimodal Video Models Viorica Puatruaucean Lucas Smaira Ankush Gupta Adrià Recasens Continente L. Markeeva ... Y. Aytar Simon Osindero Dima Damen Andrew Zisserman João Carreira VLM 110 138 0 23 May 2023
ANetQA: A Large-scale Benchmark for Fine-grained Compositional Reasoning over Untrimmed Videos Zhou Yu Lixiang Zheng Zhou Zhao A. Fedoseev Jianping Fan Kui Ren Jun Yu CoGe 24 13 0 04 May 2023
Sparks of Artificial General Intelligence: Early experiments with GPT-4 Sébastien Bubeck Varun Chandrasekaran Ronen Eldan J. Gehrke Eric Horvitz ... Scott M. Lundberg Harsha Nori Hamid Palangi Marco Tulio Ribeiro Yi Zhang ELM AI4MH AI4CE ALM 203 2,232 0 22 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Jason W. Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter F. Xia Ed H. Chi Quoc Le Denny Zhou LM&Ro LRM AI4CE ReLM 315 8,261 0 28 Jan 2022
Ego4D: Around the World in 3,000 Hours of Egocentric Video Kristen Grauman Andrew Westbury Eugene Byrne Zachary Chavis Antonino Furnari ... Mike Zheng Shou Antonio Torralba Lorenzo Torresani Mingfei Yan Jitendra Malik EgoV 218 1,017 0 13 Oct 2021