Valley: Video Assistant with Large Language model Enhanced abilitY

12 June 2023

Papers citing "Valley: Video Assistant with Large Language model Enhanced abilitY"

50 / 147 papers shown

Title
OMCAT: Omni Context Aware Transformer Arushi Goel Karan Sapra Matthieu Le Rafael Valle Andrew Tao Bryan Catanzaro MLLM VLM 16 0 0 15 Oct 2024
Enhancing Temporal Modeling of Video LLMs via Time Gating Zi-Yuan Hu Yiwu Zhong Shijia Huang M. Lyu Liwei Wang VLM 16 0 0 08 Oct 2024
TRACE: Temporal Grounding Video LLM via Causal Event Modeling Yongxin Guo Jingyu Liu Mingda Li Xiaoying Tang Qingbin Liu Xiaoying Tang 20 14 0 08 Oct 2024
Realizing Video Summarization from the Path of Language-based Semantic Understanding Kuan-Chen Mu Zhi-Yi Chin Wei-Chen Chiu 13 0 0 06 Oct 2024
Grounded-VideoLLM: Sharpening Fine-grained Temporal Grounding in Video Large Language Models Haibo Wang Zhiyang Xu Yu Cheng Shizhe Diao Yufan Zhou Yixin Cao Qifan Wang Weifeng Ge Lifu Huang 17 20 0 04 Oct 2024
UAL-Bench: The First Comprehensive Unusual Activity Localization Benchmark Hasnat Md Abdullah Tian Liu Kangda Wei Shu Kong Ruihong Huang 18 2 0 02 Oct 2024
Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understanding Xiao Wang Jianlong Wu Zijia Lin Fuzheng Zhang Di Zhang Liqiang Nie VGen 22 1 0 29 Sep 2024
From Seconds to Hours: Reviewing MultiModal Large Language Models on Comprehensive Long Video Understanding Heqing Zou Tianze Luo Guiyang Xie Victor Zhang ... Guangcong Wang Juanyang Chen Zhuochen Wang Hansheng Zhang Huaijian Zhang VLM 28 6 0 27 Sep 2024
PiTe: Pixel-Temporal Alignment for Large Video-Language Model Yang Liu Pengxiang Ding Siteng Huang Min Zhang H. Zhao Donglin Wang 17 5 0 11 Sep 2024
StimuVAR: Spatiotemporal Stimuli-aware Video Affective Reasoning with Multimodal Large Language Models Y. Guo Faizan Siddiqui Yang Zhao Rama Chellappa Shao-Yuan Lo LRM 22 2 0 31 Aug 2024
A Survey on Evaluation of Multimodal Large Language Models Jiaxing Huang Jingyi Zhang LM&MA ELM LRM 43 20 0 28 Aug 2024
SZTU-CMU at MER2024: Improving Emotion-LLaMA with Conv-Attention for Multimodal Emotion Recognition Zebang Cheng Shuyuan Tu Dawei Huang Minghan Li Xiaojiang Peng Zhi-Qi Cheng Alexander G. Hauptmann 37 2 0 20 Aug 2024
A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks Jiaqi Wang Hanqi Jiang Yi-Hsueh Liu Chong Ma Xu-Yao Zhang ... Xin Zhang Wei Zhang Dinggang Shen Tianming Liu Shu Zhang VLM AI4TS 31 18 0 02 Aug 2024
MMInstruct: A High-Quality Multi-Modal Instruction Tuning Dataset with Extensive Diversity Yangzhou Liu Yue Cao Zhangwei Gao Weiyun Wang Zhe Chen ... Lewei Lu Xizhou Zhu Tong Lu Yu Qiao Jifeng Dai VLM MLLM 42 15 0 22 Jul 2024
ViLLa: Video Reasoning Segmentation with Large Language Model Rongkun Zheng Lu Qi Xi Chen Yi Wang Kun Wang Yu Qiao Hengshuang Zhao VOS LRM 45 2 0 18 Jul 2024
InternLM-XComposer-2.5: A Versatile Large Vision Language Model Supporting Long-Contextual Input and Output Pan Zhang Xiaoyi Dong Yuhang Zang Yuhang Cao Rui Qian ... Kai Chen Jifeng Dai Yu Qiao Dahua Lin Jiaqi Wang 43 98 0 03 Jul 2024
Meerkat: Audio-Visual Large Language Model for Grounding in Space and Time Sanjoy Chowdhury Sayan Nag Subhrajyoti Dasgupta Jun Chen Mohamed Elhoseiny Ruohan Gao Dinesh Manocha VLM MLLM 29 9 0 01 Jul 2024
Tarsier: Recipes for Training and Evaluating Large Video Description Models Jiawei Wang Liping Yuan Yuchen Zhang 29 52 0 30 Jun 2024
VideoHallucer: Evaluating Intrinsic and Extrinsic Hallucinations in Large Video-Language Models Yuxuan Wang Yueqian Wang Dongyan Zhao Cihang Xie Zilong Zheng MLLM VLM 28 25 0 24 Jun 2024
video-SALMONN: Speech-Enhanced Audio-Visual Large Language Models Guangzhi Sun Wenyi Yu Changli Tang Xianzhao Chen Tian Tan Wei Li Lu Lu Zejun Ma Yuxuan Wang Chao Zhang 42 2 0 22 Jun 2024
VoCo-LLaMA: Towards Vision Compression with Large Language Models Xubing Ye Yukang Gan Xiaoke Huang Yixiao Ge Yansong Tang MLLM VLM 19 22 0 18 Jun 2024
Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning Zebang Cheng Zhi-Qi Cheng Jun-Yan He Jingdong Sun Kai Wang Yuxiang Lin Zheng Lian Xiaojiang Peng Alexander G. Hauptmann MLLM 18 14 0 17 Jun 2024
Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs Zijia Zhao Haoyu Lu Yuqi Huo Yifan Du Tongtian Yue Longteng Guo Bingning Wang Weipeng Chen Jing Liu 25 2 0 13 Jun 2024
Flash-VStream: Memory-Based Real-Time Understanding for Long Video Streams Haoji Zhang Yiqin Wang Yansong Tang Yong-Jin Liu Jiashi Feng Jifeng Dai Xiaojie Jin 23 37 0 12 Jun 2024
Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models Shimin Chen Yitian Yuan Shaoxiang Chen Zequn Jie Lin Ma VLM 18 3 0 12 Jun 2024
ShareGPT4Video: Improving Video Understanding and Generation with Better Captions Lin Chen Xilin Wei Jinsong Li Xiaoyi Dong Pan Zhang ... Li Yuan Yu Qiao Dahua Lin Feng Zhao Jiaqi Wang 69 138 0 06 Jun 2024
Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis Chaoyou Fu Yuhan Dai Yondong Luo Lei Li Shuhuai Ren ... Tong Bill Xu Xiawu Zheng Enhong Chen Rongrong Ji Xing Sun VLM MLLM 34 216 0 31 May 2024
MotionLLM: Understanding Human Behaviors from Human Motions and Videos Ling-Hao Chen Shunlin Lu Ailing Zeng Hao Zhang Benyou Wang Ruimao Zhang Lei Zhang 45 33 0 30 May 2024
Hawk: Learning to Understand Open-World Video Anomalies Jiaqi Tang Hao Lu Ruizheng Wu Xiaogang Xu Ke Ma Cheng Fang Bin Guo Jiangbo Lu Qifeng Chen Ying-Cong Chen VLM 30 7 0 27 May 2024
VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Models Zejun Li Ruipu Luo Jiwen Zhang Minghui Qiu Zhongyu Wei Zhongyu Wei LRM MLLM 48 6 0 27 May 2024
A Survey of Multimodal Large Language Model from A Data-centric Perspective Tianyi Bai Hao Liang Binwang Wan Yanran Xu Xi Li ... Ping-Chia Huang Jiulong Shan Conghui He Binhang Yuan Wentao Zhang 47 31 0 26 May 2024
Streaming Long Video Understanding with Large Language Models Rui Qian Xiao-wen Dong Pan Zhang Yuhang Zang Shuangrui Ding Dahua Lin Jiaqi Wang VLM 29 40 0 25 May 2024
CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models Guangzhi Sun Potsawee Manakul Adian Liusie Kunat Pipatanakul Chao Zhang P. Woodland Mark J. F. Gales HILM MLLM 16 7 0 22 May 2024
SOK-Bench: A Situated Video Reasoning Benchmark with Aligned Open-World Knowledge Andong Wang Bo Wu Sunli Chen Zhenfang Chen Haotian Guan Wei-Ning Lee Li Erran Li Chuang Gan LRM RALM 19 16 0 15 May 2024
FreeVA: Offline MLLM as Training-Free Video Assistant Wenhao Wu VLM OffRL 26 19 0 13 May 2024
MileBench: Benchmarking MLLMs in Long Context Dingjie Song Shunian Chen Guiming Hardy Chen Fei Yu Xiang Wan Benyou Wang VLM 56 34 0 29 Apr 2024
MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition Zheng Lian Haiyang Sun Licai Sun Zhuofan Wen Siyuan Zhang ... Bin Liu Erik Cambria Guoying Zhao Björn W. Schuller Jianhua Tao VLM 28 11 0 26 Apr 2024
SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehension Bohao Li Yuying Ge Yi Chen Yixiao Ge Ruimao Zhang Ying Shan VLM 30 27 0 25 Apr 2024
Movie101v2: Improved Movie Narration Benchmark Zihao Yue Yepeng Zhang Ziheng Wang Qin Jin VGen 16 1 0 20 Apr 2024
Audio Dialogues: Dialogues dataset for audio and music understanding Arushi Goel Zhifeng Kong Rafael Valle Bryan Catanzaro AuLLM 24 4 0 11 Apr 2024
JRDB-Social: A Multifaceted Robotic Dataset for Understanding of Context and Dynamics of Human Interactions Within Social Groups Simindokht Jahangard Zhixi Cai Shiki Wen Hamid Rezatofighi 26 6 0 06 Apr 2024
Koala: Key frame-conditioned long video-LLM Reuben Tan Ximeng Sun Ping Hu Jui-hsien Wang Hanieh Deilamsalehy Bryan A. Plummer Bryan C. Russell Kate Saenko 30 35 0 05 Apr 2024
LongVLM: Efficient Long Video Understanding via Large Language Models Yuetian Weng Mingfei Han Haoyu He Xiaojun Chang Bohan Zhuang VLM 45 56 0 04 Apr 2024
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward Ruohong Zhang Liangke Gui Zhiqing Sun Yihao Feng Keyang Xu ... Di Fu Chunyuan Li Alexander G. Hauptmann Yonatan Bisk Yiming Yang MLLM 41 57 0 01 Apr 2024
ST-LLM: Large Language Models Are Effective Temporal Learners Ruyang Liu Chen Li Haoran Tang Yixiao Ge Ying Shan Ge Li 27 68 0 30 Mar 2024
LITA: Language Instructed Temporal-Localization Assistant De-An Huang Shijia Liao Subhashree Radhakrishnan Hongxu Yin Pavlo Molchanov Zhiding Yu Jan Kautz VLM 42 49 0 27 Mar 2024
An Image Grid Can Be Worth a Video: Zero-shot Video Question Answering Using a VLM Wonkyun Kim Changin Choi Wonseok Lee Wonjong Rhee VLM 40 50 0 27 Mar 2024
Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning Hao Shao Shengju Qian Han Xiao Guanglu Song Zhuofan Zong Letian Wang Yu Liu Hongsheng Li VGen LRM MLLM 47 35 0 25 Mar 2024
Elysium: Exploring Object-level Perception in Videos via MLLM Hang Wang Yanjie Wang Yongjie Ye Yuxiang Nie Can Huang MLLM 32 18 0 25 Mar 2024
AVicuna: Audio-Visual LLM with Interleaver and Context-Boundary Alignment for Temporal Referential Dialogue Yunlong Tang Daiki Shimada Jing Bi Chenliang Xu VGen 14 17 0 24 Mar 2024