LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models

28 November 2023

Papers citing "LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models"

50 / 202 papers shown

Title
MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding Xinyu Fang Kangrui Mao Haodong Duan Xiangyu Zhao Yining Li Dahua Lin Kai Chen VLM 49 61 0 20 Jun 2024
VoCo-LLaMA: Towards Vision Compression with Large Language Models Xubing Ye Yukang Gan Xiaoke Huang Yixiao Ge Yansong Tang MLLM VLM 19 22 0 18 Jun 2024
VideoVista: A Versatile Benchmark for Video Understanding and Reasoning Yunxin Li Xinyu Chen Baotian Hu Longyue Wang Haoyuan Shi Min-Ling Zhang MLLM LRM 25 25 0 17 Jun 2024
Investigating Video Reasoning Capability of Large Language Models with Tropes in Movies Hung-Ting Su Chun-Tong Chao Ya-Ching Hsu Xudong Lin Yulei Niu Hung-Yi Lee Winston H. Hsu LRM 23 1 0 16 Jun 2024
Beyond Raw Videos: Understanding Edited Videos with Large Multimodal Model Lu Xu Sijie Zhu Chunyuan Li Chia-Wen Kuo Fan Chen Xinyao Wang Guang Chen Dawei Du Ye Yuan Longyin Wen 25 4 0 15 Jun 2024
VANE-Bench: Video Anomaly Evaluation Benchmark for Conversational LMMs Rohit K Bharadwaj Hanan Gani Muzammal Naseer F. Khan Salman Khan 47 3 0 14 Jun 2024
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding Muhammad Maaz H. Rasheed Salman Khan Fahad A Khan VLM MLLM 19 49 0 13 Jun 2024
Explore the Limits of Omni-modal Pretraining at Scale Yiyuan Zhang Handong Li Jing Liu Xiangyu Yue VLM LRM 30 1 0 13 Jun 2024
Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs Zijia Zhao Haoyu Lu Yuqi Huo Yifan Du Tongtian Yue Longteng Guo Bingning Wang Weipeng Chen Jing Liu 23 2 0 13 Jun 2024
Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams Haoji Zhang Yiqin Wang Yansong Tang Yong-Jin Liu Jiashi Feng Jifeng Dai Xiaojie Jin 21 37 0 12 Jun 2024
LVBench: An Extreme Long Video Understanding Benchmark Weihan Wang Zehai He Wenyi Hong Yean Cheng Xiaohan Zhang ... Shiyu Huang Bin Xu Yuxiao Dong Ming Ding Jie Tang ELM VLM 30 63 0 12 Jun 2024
Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models Shimin Chen Yitian Yuan Shaoxiang Chen Zequn Jie Lin Ma VLM 16 3 0 12 Jun 2024
ShareGPT4Video: Improving Video Understanding and Generation with Better Captions Lin Chen Xilin Wei Jinsong Li Xiaoyi Dong Pan Zhang ... Li Yuan Yu Qiao Dahua Lin Feng Zhao Jiaqi Wang 69 138 0 06 Jun 2024
From Redundancy to Relevance: Enhancing Explainability in Multimodal Large Language Models Xiaofeng Zhang Chen Shen Xiaosong Yuan Shaotian Yan Liang Xie Wenxiao Wang Chaochen Gu Hao Tang Jieping Ye 32 8 0 04 Jun 2024
Streaming Long Video Understanding with Large Language Models Rui Qian Xiao-wen Dong Pan Zhang Yuhang Zang Shuangrui Ding Dahua Lin Jiaqi Wang VLM 26 40 0 25 May 2024
Dense Connector for MLLMs Huanjin Yao Wenhao Wu Taojiannan Yang Yuxin Song Mengxi Zhang Haocheng Feng Yifan Sun Zhiheng Li Wanli Ouyang Jingdong Wang MLLM VLM 24 16 0 22 May 2024
CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models Guangzhi Sun Potsawee Manakul Adian Liusie Kunat Pipatanakul Chao Zhang P. Woodland Mark J. F. Gales HILM MLLM 16 7 0 22 May 2024
Uni-MoE: Scaling Unified Multimodal LLMs with Mixture of Experts Yunxin Li Shenyuan Jiang Baotian Hu Longyue Wang Wanqi Zhong Wenhan Luo Lin Ma Min-Ling Zhang MoE 17 27 0 18 May 2024
FreeVA: Offline MLLM as Training-Free Video Assistant Wenhao Wu VLM OffRL 19 19 0 13 May 2024
DoLLM: How Large Language Models Understanding Network Flow Data to Detect Carpet Bombing DDoS Qingyang Li Yihang Zhang Zhidong Jia Yannan Hu Lei Zhang Jianrong Zhang Yongming Xu Yong Cui Zongming Guo Xinggong Zhang AI4CE 26 6 0 13 May 2024
CuMo: Scaling Multimodal LLM with Co-Upcycled Mixture-of-Experts Jiachen Li Xinyao Wang Sijie Zhu Chia-Wen Kuo Lu Xu Fan Chen Jitesh Jain Humphrey Shi Longyin Wen MLLM MoE 28 26 0 09 May 2024
How Good is my Video LMM? Complex Video Reasoning and Robustness Evaluation Suite for Video-LMMs Muhammad Uzair Khattak Muhammad Ferjad Naeem Jameel Hassan Muzammal Naseer Federico Tombari Fahad Shahbaz Khan Salman Khan LRM ELM 24 3 0 06 May 2024
MileBench: Benchmarking MLLMs in Long Context Dingjie Song Shunian Chen Guiming Hardy Chen Fei Yu Xiang Wan Benyou Wang VLM 56 34 0 29 Apr 2024
MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition Zheng Lian Haiyang Sun Licai Sun Zhuofan Wen Siyuan Zhang ... Bin Liu Erik Cambria Guoying Zhao Björn W. Schuller Jianhua Tao VLM 25 11 0 26 Apr 2024
PLLaVA : Parameter-free LLaVA Extension from Images to Videos for Video Dense Captioning Lin Xu Yilin Zhao Daquan Zhou Zhijie Lin See Kiong Ng Jiashi Feng MLLM VLM 26 108 0 25 Apr 2024
Pegasus-v1 Technical Report Raehyuk Jung Hyojun Go Jaehyuk Yi Jiho Jang Daniel Kim ... Maninder Saini Meredith Sanders Soyoung Lee Sue Kim Travis Couture MLLM VLM 21 5 0 23 Apr 2024
Self-Supervised Visual Preference Alignment Ke Zhu Liang Zhao Zheng Ge Xiangyu Zhang 19 12 0 16 Apr 2024
Unified Language-driven Zero-shot Domain Adaptation Senqiao Yang Zhuotao Tian Li Jiang Jiaya Jia 23 7 0 10 Apr 2024
UniMD: Towards Unifying Moment Retrieval and Temporal Action Detection Yingsen Zeng Yujie Zhong Chengjian Feng Lin Ma 50 7 0 07 Apr 2024
MiniGPT4-Video: Advancing Multimodal LLMs for Video Understanding with Interleaved Visual-Textual Tokens Kirolos Ataallah Xiaoqian Shen Eslam Abdelrahman Essam Sleiman Deyao Zhu Jian Ding Mohamed Elhoseiny VLM 31 64 0 04 Apr 2024
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward Ruohong Zhang Liangke Gui Zhiqing Sun Yihao Feng Keyang Xu ... Di Fu Chunyuan Li Alexander G. Hauptmann Yonatan Bisk Yiming Yang MLLM 33 57 0 01 Apr 2024
ST-LLM: Large Language Models Are Effective Temporal Learners Ruyang Liu Chen Li Haoran Tang Yixiao Ge Ying Shan Ge Li 27 68 0 30 Mar 2024
LITA: Language Instructed Temporal-Localization Assistant De-An Huang Shijia Liao Subhashree Radhakrishnan Hongxu Yin Pavlo Molchanov Zhiding Yu Jan Kautz VLM 42 49 0 27 Mar 2024
Mini-Gemini: Mining the Potential of Multi-modality Vision Language Models Yanwei Li Yuechen Zhang Chengyao Wang Zhisheng Zhong Yixin Chen Ruihang Chu Shaoteng Liu Jiaya Jia VLM MLLM MoE 29 210 0 27 Mar 2024
An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM Wonkyun Kim Changin Choi Wonseok Lee Wonjong Rhee VLM 40 50 0 27 Mar 2024
Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning Hao Shao Shengju Qian Han Xiao Guanglu Song Zhuofan Zong Letian Wang Yu Liu Hongsheng Li VGen LRM MLLM 47 35 0 25 Mar 2024
An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models Liang Chen Haozhe Zhao Tianyu Liu Shuai Bai Junyang Lin Chang Zhou Baobao Chang MLLM VLM 40 22 0 11 Mar 2024
CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarios Qilang Ye Zitong Yu Rui Shao Xinyu Xie Philip H. S. Torr Xiaochun Cao MLLM 25 24 0 07 Mar 2024
NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigation Jiazhao Zhang Kunyu Wang Rongtao Xu Gengze Zhou Yicong Hong Xiaomeng Fang Qi Wu Zhizheng Zhang Wang He LM&Ro 19 11 0 24 Feb 2024
Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions Akash Ghosh Arkadeep Acharya Sriparna Saha Vinija Jain Aman Chadha VLM 30 9 0 20 Feb 2024
VideoPrism: A Foundational Visual Encoder for Video Understanding Long Zhao N. B. Gundavarapu Liangzhe Yuan Hao Zhou Shen Yan ... Huisheng Wang Hartwig Adam Mikhail Sirotenko Ting Liu Boqing Gong VGen 21 29 0 20 Feb 2024
Slot-VLM: SlowFast Slots for Video-Language Modeling Jiaqi Xu Cuiling Lan Wenxuan Xie Xuejin Chen Yan Lu MLLM VLM 27 7 0 20 Feb 2024
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization Yang Jin Zhicheng Sun Kun Xu Kun Xu Liwei Chen ... Yuliang Liu Di Zhang Yang Song Kun Gai Yadong Mu VGen 44 42 0 05 Feb 2024
Video Understanding with Large Language Models: A Survey Yunlong Tang Jing Bi Siting Xu Luchuan Song Susan Liang ... Feng Zheng Jianguo Zhang Ping Luo Jiebo Luo Chenliang Xu VLM 47 76 0 29 Dec 2023
AutoEval-Video: An Automatic Benchmark for Assessing Large Vision Language Models in Open-Ended Video Question Answering Xiuyuan Chen Yuan Lin Yuchen Zhang Weiran Huang ELM MLLM 10 26 0 25 Nov 2023
Explainable Multimodal Emotion Recognition Zheng Lian Haiyang Sun Licai Sun Hao Gu Zhuofan Wen ... Shan Liang Ya Li Jiangyan Yi B. Liu Jianhua Tao MLLM 8 6 0 27 Jun 2023
Valley: Video Assistant with Large Language model Enhanced abilitY Ruipu Luo Ziwang Zhao Min Yang Junwei Dong Da Li Pengcheng Lu Tao Wang Linmei Hu Ming-Hui Qiu MLLM 21 185 0 12 Jun 2023
LMEye: An Interactive Perception Network for Large Language Models Yunxin Li Baotian Hu Xinyu Chen Lin Ma Yong-mei Xu M. Zhang MLLM VLM 17 24 0 05 May 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 198 1,089 0 20 Sep 2022