Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding

5 June 2023

Xin Li

Papers citing "Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding"

50 / 697 papers shown

Title
VideoVista: A Versatile Benchmark for Video Understanding and Reasoning Yunxin Li Xinyu Chen Baotian Hu Longyue Wang Haoyuan Shi Min-Ling Zhang MLLM LRM 38 25 0 17 Jun 2024
MMNeuron: Discovering Neuron-Level Domain-Specific Interpretation in Multimodal Large Language Model Jiahao Huo Yibo Yan Boren Hu Yutao Yue Xuming Hu LRM MLLM 32 7 0 17 Jun 2024
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning Zebang Cheng Zhi-Qi Cheng Jun-Yan He Jingdong Sun Kai Wang Yuxiang Lin Zheng Lian Xiaojiang Peng Alexander G. Hauptmann MLLM 29 28 0 17 Jun 2024
WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences Yujie Lu Dongfu Jiang Wenhu Chen William Yang Wang Yejin Choi Bill Yuchen Lin VLM 43 26 0 16 Jun 2024
Investigating Video Reasoning Capability of Large Language Models with Tropes in Movies Hung-Ting Su Chun-Tong Chao Ya-Ching Hsu Xudong Lin Yulei Niu Hung-Yi Lee Winston H. Hsu LRM 31 1 0 16 Jun 2024
Exploring the Potential of Multimodal LLM with Knowledge-Intensive Multimodal ASR Minghan Wang Yuxia Wang Thuy-Trang Vu Ehsan Shareghi Gholamreza Haffari 29 0 0 16 Jun 2024
GPT-4o: Visual perception performance of multimodal large language models in piglet activity understanding Yiqi Wu Xiaodan Hu Ziming Fu Siling Zhou Jiangong Li MLLM 27 9 0 14 Jun 2024
VANE-Bench: Video Anomaly Evaluation Benchmark for Conversational LMMs Rohit K Bharadwaj Hanan Gani Muzammal Naseer F. Khan Salman Khan 49 3 0 14 Jun 2024
Multimodal Large Language Models with Fusion Low Rank Adaptation for Device Directed Speech Detection Shruti Palaskar Oggi Rudovic Sameer Dharur Florian Pesce G. Krishna Aswin Sivaraman Jack Berkowitz Ahmed Hussen Abdelaziz Saurabh N. Adya Ahmed H. Tewfik VLM 55 0 0 13 Jun 2024
VideoGPT+: Integrating Image and Video Encoders for Enhanced Video Understanding Muhammad Maaz H. Rasheed Salman Khan Fahad A Khan VLM MLLM 19 49 0 13 Jun 2024
Explore the Limits of Omni-modal Pretraining at Scale Yiyuan Zhang Handong Li Jing Liu Xiangyu Yue VLM LRM 45 1 0 13 Jun 2024
Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs Zijia Zhao Haoyu Lu Yuqi Huo Yifan Du Tongtian Yue Longteng Guo Bingning Wang Weipeng Chen Jing Liu 36 2 0 13 Jun 2024
MMWorld: Towards Multi-discipline Multi-faceted World Model Evaluation in Videos Xuehai He Weixi Feng Kaizhi Zheng Yujie Lu Wanrong Zhu ... Zhengyuan Yang Kevin Lin William Yang Wang Lijuan Wang Xin Eric Wang VGen LRM 33 12 0 12 Jun 2024
Understanding Sounds, Missing the Questions: The Challenge of Object Hallucination in Large Audio-Language Models Chun-Yi Kuan Wei-Ping Huang Hung-yi Lee AuLLM 29 5 0 12 Jun 2024
Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams Haoji Zhang Yiqin Wang Yansong Tang Yong-Jin Liu Jiashi Feng Jifeng Dai Xiaojie Jin 32 37 0 12 Jun 2024
Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models Shimin Chen Yitian Yuan Shaoxiang Chen Zequn Jie Lin Ma VLM 27 3 0 12 Jun 2024
TRINS: Towards Multimodal Language Models that Can Read Ruiyi Zhang Yanzhe Zhang Jian Chen Yufan Zhou Jiuxiang Gu Changyou Chen Tong Sun VLM 26 6 0 10 Jun 2024
Multimodal Contextualized Semantic Parsing from Speech Jordan Voas Raymond Mooney David Harwath 38 0 0 10 Jun 2024
iMotion-LLM: Motion Prediction Instruction Tuning Abdulwahab Felemban Eslam Mohamed Bakr Xiaoqian Shen Jian Ding Abduallah A. Mohamed Mohamed Elhoseiny 45 1 0 10 Jun 2024
Vript: A Video Is Worth Thousands of Words Dongjie Yang Suyuan Huang Chengqiang Lu Xiaodong Han Haoxin Zhang Yan Gao Yao Hu Hai Zhao VGen 58 21 0 10 Jun 2024
Chain-of-Scrutiny: Detecting Backdoor Attacks for Large Language Models Xi Li Yusen Zhang Renze Lou Chen Wu Jiaqi Wang LRM AAML 37 11 0 10 Jun 2024
Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectives Thong Nguyen Yi Bin Junbin Xiao Leigang Qu Yicong Li Jay Zhangjie Wu Cong-Duy Nguyen See-Kiong Ng Luu Anh Tuan VLM 39 9 1 09 Jun 2024
VP-LLM: Text-Driven 3D Volume Completion with Large Language Models through Patchification Jianmeng Liu Yichen Liu Yuyao Zhang Zeyuan Meng Yu-Wing Tai Chi-Keung Tang 47 0 0 08 Jun 2024
Seeing the Unseen: Visual Metaphor Captioning for Videos Abisek Rajakumar Kalarani Pushpak Bhattacharyya Sumit Shekhar VLM 32 1 0 07 Jun 2024
ShareGPT4Video: Improving Video Understanding and Generation with Better Captions Lin Chen Xilin Wei Jinsong Li Xiaoyi Dong Pan Zhang ... Li Yuan Yu Qiao Dahua Lin Feng Zhao Jiaqi Wang 72 142 0 06 Jun 2024
VideoTetris: Towards Compositional Text-to-Video Generation Ye Tian Ling Yang Haotian Yang Yuan Gao Yufan Deng ... Zhaochen Yu Xin Tao Pengfei Wan Di Zhang Bin Cui DiffM VGen 79 15 0 06 Jun 2024
VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling Zeyue Tian Zhaoyang Liu Ruibin Yuan Jiahao Pan Xiaoqiang Huang Xu Tan Xu Tan Qifeng Chen Y. Guo VGen 100 16 0 06 Jun 2024
AD-H: Autonomous Driving with Hierarchical Agents Zaibin Zhang Shiyu Tang Yuanhang Zhang Talas Fu Yifan Wang Yang Liu Dong Wang Jing Shao Lijun Wang H. Lu 42 3 0 05 Jun 2024
Evaluation of data inconsistency for multi-modal sentiment analysis Yufei Wang Mengyue Wu 27 1 0 05 Jun 2024
Multi-layer Learnable Attention Mask for Multimodal Tasks Wayner Barrios SouYoung Jin 34 0 0 04 Jun 2024
From Redundancy to Relevance: Enhancing Explainability in Multimodal Large Language Models Xiaofeng Zhang Chen Shen Xiaosong Yuan Shaotian Yan Liang Xie Wenxiao Wang Chaochen Gu Hao Tang Jieping Ye 49 0 0 04 Jun 2024
Multimodal Reasoning with Multimodal Knowledge Graph Junlin Lee Yequan Wang Jing Li Min Zhang 29 15 0 04 Jun 2024
Towards Practical Single-shot Motion Synthesis Konstantinos Roditakis Spyridon Thermos N. Zioulis VGen 43 0 0 03 Jun 2024
Artemis: Towards Referential Understanding in Complex Videos Jihao Qiu Yuan Zhang Xi Tang Lingxi Xie Tianren Ma Pengyu Yan David Doermann Qixiang Ye Yunjie Tian VLM VGen 37 8 0 01 Jun 2024
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis Chaoyou Fu Yuhan Dai Yondong Luo Lei Li Shuhuai Ren ... Tong Bill Xu Xiawu Zheng Enhong Chen Rongrong Ji Xing Sun VLM MLLM 46 297 0 31 May 2024
MotionLLM: Understanding Human Behaviors from Human Motions and Videos Ling-Hao Chen Shunlin Lu Ailing Zeng Hao Zhang Benyou Wang Ruimao Zhang Lei Zhang 45 34 0 30 May 2024
Can't make an Omelette without Breaking some Eggs: Plausible Action Anticipation using Large Video-Language Models Himangi Mittal Nakul Agarwal Shao-Yuan Lo Kwonjoon Lee 30 13 0 30 May 2024
Temporal Grounding of Activities using Multimodal Large Language Models Young Chol Song 30 0 0 30 May 2024
X-VILA: Cross-Modality Alignment for Large Language Model Hanrong Ye De-An Huang Yao Lu Zhiding Yu Wei Ping ... Jan Kautz Song Han Dan Xu Pavlo Molchanov Hongxu Yin MLLM VLM 40 29 0 29 May 2024
Matryoshka Query Transformer for Large Vision-Language Models Wenbo Hu Zi-Yi Dou Liunian Harold Li Amita Kamath Nanyun Peng Kai-Wei Chang MLLM 29 8 0 29 May 2024
MindSemantix: Deciphering Brain Visual Experiences with a Brain-Language Model Ziqi Ren Jie Li Xuetong Xue Xin Li Fan Yang Zhicheng Jiao Xinbo Gao 35 3 0 29 May 2024
VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos Ziyang Wang Shoubin Yu Elias Stengel-Eskin Jaehong Yoon Feng Cheng Gedas Bertasius Mohit Bansal 40 56 0 29 May 2024
The Evolution of Multimodal Model Architectures S. Wadekar Abhishek Chaurasia Aman Chadha Eugenio Culurciello 41 14 0 28 May 2024
Video Enriched Retrieval Augmented Generation Using Aligned Video Captions Kevin Dela Rosa 21 5 0 27 May 2024
Cross-Modal Safety Alignment: Is textual unlearning all you need? Trishna Chakraborty Erfan Shayegani Zikui Cai Nael B. Abu-Ghazaleh M. Salman Asif Yue Dong A. Roy-Chowdhury Chengyu Song 39 15 0 27 May 2024
Matryoshka Multimodal Models Mu Cai Jianwei Yang Jianfeng Gao Yong Jae Lee VLM 39 25 0 27 May 2024
Hawk: Learning to Understand Open-World Video Anomalies Jiaqi Tang Hao Lu Ruizheng Wu Xiaogang Xu Ke Ma Cheng Fang Bin Guo Jiangbo Lu Qifeng Chen Ying-Cong Chen VLM 40 9 0 27 May 2024
A Survey of Multimodal Large Language Model from A Data-centric Perspective Tianyi Bai Hao Liang Binwang Wan Yanran Xu Xi Li ... Ping-Chia Huang Jiulong Shan Conghui He Binhang Yuan Wentao Zhang 47 36 0 26 May 2024
C3LLM: Conditional Multimodal Content Generation Using Large Language Models Zixuan Wang Qinkai Duan Yu-Wing Tai Chi-Keung Tang 38 3 0 25 May 2024
Streaming Long Video Understanding with Large Language Models Rui Qian Xiao-wen Dong Pan Zhang Yuhang Zang Shuangrui Ding Dahua Lin Jiaqi Wang VLM 29 40 0 25 May 2024