Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding

14 November 2023

Papers citing "Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understanding"

50 / 50 papers shown

Title
RTV-Bench: Benchmarking MLLM Continuous Perception, Understanding and Reasoning through Real-Time Video Shuhang Xun Sicheng Tao J. Li Yibo Shi Zhixin Lin ... Shikang Wang Y. Liu H. Zhang Ying Ma Xuming Hu VLM LRM 30 0 0 04 May 2025
TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action Jen-Hao Cheng Vivian Wang Huayu Wang Huapeng Zhou Yi-Hao Peng ... Wenhao Chai Yi-Ling Chen Vibhav Vineet Qin Cai Jenq-Neng Hwang AI4TS 30 0 0 02 May 2025
AdCare-VLM: Leveraging Large Vision Language Model (LVLM) to Monitor Long-Term Medication Adherence and Care Md Asaduzzaman Jabin Hanqi Jiang Y. Li Patrick Kaggwa Eugene Douglass Juliet N. Sekandi Tianming Liu LM&MA 60 0 0 01 May 2025
FSBench: A Figure Skating Benchmark for Advancing Artistic Sports Understanding Rong Gao Xin Liu Zhuozhao Hu Bohao Xing Baiqiang Xia Zitong Yu H. Kalviainen 38 0 0 28 Apr 2025
DEEMO: De-identity Multimodal Emotion Recognition and Reasoning Deng Li Bohao Xing Xin Liu Baiqiang Xia Bihan Wen H. Kalviainen VLM 65 0 0 28 Apr 2025
ActionArt: Advancing Multimodal Large Models for Fine-Grained Human-Centric Video Understanding Yi-Xing Peng Q. Yang Yu-Ming Tang Shenghao Fu Kun-Yu Lin Xihan Wei Wei-Shi Zheng 38 0 0 25 Apr 2025
TimeSoccer: An End-to-End Multimodal Large Language Model for Soccer Commentary Generation Ling You Wenxuan Huang Xinni Xie Xiangyi Wei Bangyan Li Shaohui Lin Yang Li Changbo Wang VGen 46 0 0 24 Apr 2025
Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark Enxin Song Wenhao Chai Weili Xu Jianwen Xie Yuxuan Liu Gaoang Wang 54 0 0 20 Apr 2025
Towards Understanding How Knowledge Evolves in Large Vision-Language Models Sudong Wang Y. Zhang Yao Zhu Jianing Li Zizhe Wang Y. Liu Xiangyang Ji 29 0 0 31 Mar 2025
Video-XL-Pro: Reconstructive Token Compression for Extremely Long Video Understanding Xiangrui Liu Yan Shu Zheng Liu Ao Li Yang Tian Bo Zhao VGen VLM 86 0 0 24 Mar 2025
VideoMind: A Chain-of-LoRA Agent for Long Video Reasoning Y. Liu Kevin Qinghong Lin C. Chen Mike Zheng Shou LM&Ro LRM 57 0 0 17 Mar 2025
VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search Yiming Jia J. Li Xiang Yue Bo Li Ping Nie Kai Zou Wenhu Chen LRM 70 2 0 13 Mar 2025
VideoScan: Enabling Efficient Streaming Video Understanding via Frame-level Semantic Carriers Ruanjun Li Yuedong Tan Yuanming Shi Jiawei Shao VLM 65 0 0 12 Mar 2025
BIMBA: Selective-Scan Compression for Long-Range Video Question Answering Md. Mohaiminul Islam Tushar Nagarajan Huiyu Wang Gedas Bertasius Lorenzo Torresani 48 0 0 12 Mar 2025
HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understanding Shehreen Azad Vibhav Vineet Y. S. Rawat VLM 44 1 0 11 Mar 2025
Omni-SILA: Towards Omni-scene Driven Visual Sentiment Identifying, Locating and Attributing in Videos Jiamin Luo Jingjing Wang Junxiao Ma Yujie Jin Shoushan Li Guodong Zhou 26 0 0 26 Feb 2025
TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Data Jeremy Irvin Emily Ruoyu Liu Joyce Chuyi Chen Ines Dormoy Jinyoung Kim Samar Khanna Zhuo Zheng Stefano Ermon MLLM VLM 43 4 0 28 Jan 2025
EDoRA: Efficient Weight-Decomposed Low-Rank Adaptation via Singular Value Decomposition Hamid Nasiri Peter Garraghan 26 1 0 21 Jan 2025
TimeLogic: A Temporal Logic Benchmark for Video QA S. Swetha Hilde Kuehne Mubarak Shah 37 1 0 13 Jan 2025
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos Haobo Yuan X. Li Tao Zhang Zilong Huang Shilin Xu S. Ji Yunhai Tong Lu Qi Jiashi Feng Ming Yang VLM 65 11 0 07 Jan 2025
FOLDER: Accelerating Multi-modal Large Language Models with Enhanced Performance Haicheng Wang Zhemeng Yu Gabriele Spadaro Chen Ju Victor Quétu Enzo Tartaglione Enzo Tartaglione VLM 49 3 0 05 Jan 2025
Image-based Multimodal Models as Intruders: Transferable Multimodal Attacks on Video-based MLLMs Linhao Huang Xue Jiang Zhiqiang Wang Wentao Mo Xi Xiao Bo Han Yongjie Yin Feng Zheng AAML 25 2 0 02 Jan 2025
ReTaKe: Reducing Temporal and Knowledge Redundancy for Long Video Understanding Xiao Wang Qingyi Si Jianlong Wu Shiyu Zhu Li Cao Liqiang Nie VLM 54 6 0 29 Dec 2024
Next Patch Prediction for Autoregressive Visual Generation Yatian Pang Peng Jin Shuo Yang Bin Lin Bin Zhu ... Liuhan Chen Francis E. H. Tay Ser-Nam Lim Harry Yang Li Yuan 109 8 0 19 Dec 2024
Video LLMs for Temporal Reasoning in Long Videos Fawad Javed Fateh Umer Ahmed Hamza Khan M. Zia Quoc-Huy Tran VLM 70 0 0 04 Dec 2024
HyperGLM: HyperGraph for Video Scene Graph Generation and Anticipation Trong-Thuan Nguyen Pha Nguyen J. Cothren Alper Yilmaz Khoa Luu 74 1 0 27 Nov 2024
VideoOrion: Tokenizing Object Dynamics in Videos Yicheng Feng Yijiang Li Wanpeng Zhang Sipeng Zheng Zongqing Lu Sipeng Zheng Zongqing Lu 82 1 0 25 Nov 2024
ReWind: Understanding Long Videos with Instructed Learnable Memory Anxhelo Diko Tinghuai Wang Wassim Swaileh Shiyan Sun Ioannis Patras KELM VLM 73 0 0 23 Nov 2024
TimeSuite: Improving MLLMs for Long Video Understanding via Grounded Tuning Xiangyu Zeng Kunchang Li Chenting Wang Xinhao Li Tianxiang Jiang ... Zhengrong Yue Yi Wang Yali Wang Yu Qiao Limin Wang MLLM VLM AI4TS 50 14 0 25 Oct 2024
Ferret-UI 2: Mastering Universal User Interface Understanding Across Platforms Zhangheng Li Keen You H. Zhang Di Feng Harsh Agrawal Xiujun Li Mohana Prasad Sathya Moorthy Jeff Nichols Y. Yang Zhe Gan MLLM 29 18 0 24 Oct 2024
SPORTU: A Comprehensive Sports Understanding Benchmark for Multimodal Large Language Models H. Xia Zhengbang Yang Junbo Zou Rhys Tracy Yuqing Wang ... Xun Shao Zhuoqing Xie Yuan-fang Wang Weining Shen Hanjie Chen ReLM LRM ELM 23 2 0 11 Oct 2024
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 53 25 0 04 Oct 2024
Frame-Voyager: Learning to Query Frames for Video Large Language Models Sicheng Yu Chengkai Jin Huanyu Wang Zhenghao Chen Sheng Jin ... Zhenbang Sun Bingni Zhang Jiawei Wu Hao Zhang Qianru Sun 39 5 0 04 Oct 2024
Mobile Edge Intelligence for Large Language Models: A Contemporary Survey Guanqiao Qu Qiyuan Chen Wei Wei Zheng Lin Xianhao Chen Kaibin Huang 27 37 0 09 Jul 2024
Sports Intelligence: Assessing the Sports Understanding Capabilities of Language Models through Question Answering from Text to Video Zhengbang Yang Haotian Xia Jingxi Li Zezhi Chen Zhuangdi Zhu Weining Shen ELM LRM 15 1 0 21 Jun 2024
Hallucination Mitigation Prompts Long-term Video Understanding Yiwei Sun Zhihang Liu Chuanbin Liu Bowei Pu Zhihan Zhang Hongtao Xie VLM MLLM 25 2 0 17 Jun 2024
Towards Semantic Equivalence of Tokenization in Multimodal LLM Shengqiong Wu Hao Fei Xiangtai Li Jiayi Ji Hanwang Zhang Tat-Seng Chua Shuicheng Yan MLLM 52 25 0 07 Jun 2024
Direct Preference Optimization of Video Large Multimodal Models from Language Model Reward Ruohong Zhang Liangke Gui Zhiqing Sun Yihao Feng Keyang Xu ... Di Fu Chunyuan Li Alexander G. Hauptmann Yonatan Bisk Yiming Yang MLLM 28 57 0 01 Apr 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 116 106 0 08 Feb 2024
Mementos: A Comprehensive Benchmark for Multimodal Large Language Model Reasoning over Image Sequences Xiyao Wang Yuhang Zhou Xiaoyu Liu Hongjin Lu Yuancheng Xu ... Taixi Lu Gedas Bertasius Mohit Bansal Huaxiu Yao Furong Huang LRM VLM 55 65 0 19 Jan 2024
Video-LLaVA: Learning United Visual Representation by Alignment Before Projection Bin Lin Yang Ye Bin Zhu Jiaxi Cui Munan Ning Peng Jin Li-ming Yuan VLM MLLM 182 576 0 16 Nov 2023
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 152 280 0 14 Oct 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 198 883 0 27 Apr 2023
Image as Set of Points Xu Ma Yuqian Zhou Huan Wang Can Qin Bin Sun Chang Liu Yun Fu VLM 30 48 0 02 Mar 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023
Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering Pan Lu Swaroop Mishra Tony Xia Liang Qiu Kai-Wei Chang Song-Chun Zhu Oyvind Tafjord Peter Clark A. Kalyan ELM ReLM LRM 198 1,089 0 20 Sep 2022
Training language models to follow instructions with human feedback Long Ouyang Jeff Wu Xu Jiang Diogo Almeida Carroll L. Wainwright ... Amanda Askell Peter Welinder Paul Christiano Jan Leike Ryan J. Lowe OSLM ALM 301 11,730 0 04 Mar 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz X. Wang ViT VLM 173 494 0 22 Feb 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 380 4,010 0 28 Jan 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 234 690 0 27 Aug 2021