LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models

10 July 2024

Feng Li

Renrui Zhang

Hao Zhang

Yuanhan Zhang

Bo Li

Wei Li

Zejun Ma

Papers citing "LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Models"

50 / 150 papers shown

Title
PlaceIt3D: Language-Guided Object Placement in Real 3D Scenes Ahmed Abdelreheem Filippo Aleotti Jamie Watson Z. Qureshi Abdelrahman Eldesokey Peter Wonka Gabriel J. Brostow Sara Vicente Guillermo Garcia-Hernando DiffM 42 0 0 08 May 2025
TEMPURA: Temporal Event Masked Prediction and Understanding for Reasoning in Action Jen-Hao Cheng Vivian Wang Huayu Wang Huapeng Zhou Yi-Hao Peng ... Wenhao Chai Yi-Ling Chen Vibhav Vineet Qin Cai Jenq-Neng Hwang AI4TS 30 0 0 02 May 2025
Transferable Adversarial Attacks on Black-Box Vision-Language Models Kai Hu Weichen Yu L. Zhang Alexander Robey Andy Zou Chengming Xu Haoqi Hu Matt Fredrikson AAML VLM 47 0 0 02 May 2025
Zoomer: Adaptive Image Focus Optimization for Black-box MLLM Jiaxu Qian Chendong Wang Y. Yang Chaoyun Zhang Huiqiang Jiang ... Saravan Rajmohan Dongmei Zhang Y. Yang Qi Zhang Lili Qiu VLM 65 0 0 30 Apr 2025
FRAG: Frame Selection Augmented Generation for Long Video and Long Document Understanding De-An Huang Subhashree Radhakrishnan Zhiding Yu Jan Kautz VGen VLM 67 0 0 24 Apr 2025
Benchmarking Multimodal Mathematical Reasoning with Explicit Visual Dependency Zhikai Wang Jiashuo Sun W. Zhang Zhiqiang Hu Xin Li F. Wang Deli Zhao VLM LRM 70 0 0 24 Apr 2025
MR. Video: "MapReduce" is the Principle for Long Video Understanding Ziqi Pang Yu-xiong Wang VLM 32 0 0 22 Apr 2025
T2VShield: Model-Agnostic Jailbreak Defense for Text-to-Video Models Siyuan Liang Jiayang Liu Jiecheng Zhai Tianmeng Fang Rongcheng Tu A. Liu Xiaochun Cao Dacheng Tao VGen 49 0 0 22 Apr 2025
Vidi: Large Multimodal Models for Video Understanding and Editing Vidi Team Celong Liu Chia-Wen Kuo Dawei Du Fan Chen ... Wen Zhong Xiaohui Shen Xin Gu Xing Mei Xueqiong Qu 54 0 0 22 Apr 2025
Ask2Loc: Learning to Locate Instructional Visual Answers by Asking Questions Chang Zong Bin Li Shoujun Zhou Jian Wan Lei Zhang 31 0 0 22 Apr 2025
DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understanding Geng Li Jinglin Xu Yunzhen Zhao Yuxin Peng ObjD 17 0 0 21 Apr 2025
OmniV-Med: Scaling Medical Vision-Language Model for Universal Visual Understanding Songtao Jiang Yuan Wang Sibo Song Y. Zhang Zijie Meng Bohan Lei Jian Wu Jimeng Sun Zuozhu Liu MedIm VLM 27 0 0 20 Apr 2025
Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark Enxin Song Wenhao Chai Weili Xu Jianwen Xie Yuxuan Liu Gaoang Wang 54 0 0 20 Apr 2025
VideoPASTA: 7K Preference Pairs That Matter for Video-LLM Alignment Yogesh Kulkarni Pooyan Fazli 27 0 0 18 Apr 2025
VistaDPO: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models Haojian Huang Haodong Chen Shengqiong Wu Meng Luo Jinlan Fu Xinya Du H. Zhang Hao Fei AI4TS 58 0 0 17 Apr 2025
AnomalyR1: A GRPO-based End-to-end MLLM for Industrial Anomaly Detection Yuhao Chao Jie Liu J. Tang Gangshan Wu 21 1 0 16 Apr 2025
TAMP: Token-Adaptive Layerwise Pruning in Multimodal Large Language Models Jaewoo Lee Keyang Xuan Chanakya Ekbote Sandeep Polisetty Yi Ren Fung Paul Pu Liang VLM 27 0 0 14 Apr 2025
Mavors: Multi-granularity Video Representation for Multimodal Large Language Model Yang Shi Jiaheng Liu Yushuo Guan Z. Wu Y. Zhang ... Bohan Zeng W. Zhang Fuzheng Zhang Wenjing Yang Di Zhang VGen VLM 63 0 0 14 Apr 2025
The Structural Safety Generalization Problem Julius Broomfield Tom Gibbs Ethan Kosak-Hine George Ingebretsen Tia Nasir Jason Zhang Reihaneh Iranmanesh Sara Pieri Reihaneh Rabbany Kellin Pelrine AAML 23 0 0 13 Apr 2025
Mimic In-Context Learning for Multimodal Tasks Yuchu Jiang Jiale Fu Chenduo Hao Xinting Hu Yingzhe Peng Xin Geng Xu Yang 22 0 0 11 Apr 2025
SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding Yangliu Hu Zikai Song Na Feng Yawei Luo Junqing Yu Yi-Ping Phoebe Chen Wei Yang 28 0 0 10 Apr 2025
TokenFocus-VQA: Enhancing Text-to-Image Alignment with Position-Aware Focus and Multi-Perspective Aggregations on LVLMs Zijian Zhang Xuhui Zheng X. Wu Chong Peng Xuezhi Cao 25 0 0 10 Apr 2025
REVEAL: Relation-based Video Representation Learning for Video-Question-Answering Sofian Chaybouti Walid Bousselham Moritz Wolter Hilde Kuehne 27 0 0 07 Apr 2025
Overcoming Vocabulary Constraints with Pixel-level Fallback Jonas F. Lotz Hendra Setiawan Stephan Peitz Yova Kementchedjhieva 30 0 0 02 Apr 2025
Aligned Better, Listen Better for Audio-Visual Large Language Models Yuxin Guo Shuailei Ma Shijie Ma Xiaoyi Bao Chen-Wei Xie Kecheng Zheng Tingyu Weng Siyang Sun Yun Zheng Wei Zou MLLM AuLLM 58 2 0 02 Apr 2025
Q-Adapt: Adapting LMM for Visual Quality Assessment with Progressive Instruction Tuning Yiting Lu X. Li H. Wu Bingchen Li Weisi Lin Zhibo Chen 37 1 0 02 Apr 2025
Improved Visual-Spatial Reasoning via R1-Zero-Like Training Zhenyi Liao Qingsong Xie Yanhao Zhang Zijian Kong Haonan Lu Zhenyu Yang Zhijie Deng ReLM VLM LRM 93 0 1 01 Apr 2025
Short-video Propagation Influence Rating: A New Real-world Dataset and A New Large Graph Model Dizhan Xue Jing Cui Shengsheng Qian Chuanrui Hu Changsheng Xu 27 0 0 31 Mar 2025
OmniMMI: A Comprehensive Multi-modal Interaction Benchmark in Streaming Video Contexts Y. Wang Y. Wang Bo Chen Tong Wu Dongyan Zhao Zilong Zheng VLM MLLM 47 1 0 29 Mar 2025
Unicorn: Text-Only Data Synthesis for Vision Language Model Training Xiaomin Yu Pengxiang Ding Wenjie Zhang Siteng Huang Songyang Gao Chengwei Qin Kejian Wu Zhaoxin Fan Ziyue Qiao Donglin Wang MLLM SyDa 67 0 0 28 Mar 2025
StyleMotif: Multi-Modal Motion Stylization using Style-Content Cross Fusion Ziyu Guo Young Yoon Lee Joseph Liu Yizhak Ben-Shabat Victor Zordan Mubbasir Kapadia DiffM VGen 63 0 0 27 Mar 2025
Fwd2Bot: LVLM Visual Token Compression with Double Forward Bottleneck Adrian Bulat Yassine Ouali Georgios Tzimiropoulos 43 0 0 27 Mar 2025
VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness Dian Zheng Ziqi Huang Hongbo Liu Kai Zou Yinan He ... Y. Zhang Jingwen He Wei-Shi Zheng Yu Qiao Ziwei Liu EGVM VGen 46 3 0 27 Mar 2025
Instruction-Oriented Preference Alignment for Enhancing Multi-Modal Comprehension Capability of MLLMs Zitian Wang Yue Liao Kang Rong Fengyun Rao Yibo Yang Si Liu 70 0 0 26 Mar 2025
PAVE: Patching and Adapting Video Large Language Models Zhuoming Liu Yiquan Li Khoi Duc Nguyen Yiwu Zhong Yin Li KELM LRM 79 0 0 25 Mar 2025
RGB-Th-Bench: A Dense benchmark for Visual-Thermal Understanding of Vision Language Models Mehdi Moshtaghi Siavash H. Khajavi Joni Pajarinen VLM 36 0 0 25 Mar 2025
Improved Alignment of Modalities in Large Vision Language Models Kartik Jangra Aman Kumar Singh Yashwani Mann Geetanjali Rathee VLM 42 0 0 25 Mar 2025
FALCONEye: Finding Answers and Localizing Content in ONE-hour-long videos with multi-modal LLMs Carlos Plou Cesar Borja Ruben Martinez-Cantin Ana C. Murillo 54 0 0 25 Mar 2025
A Simple yet Effective Layout Token in Large Language Models for Document Understanding Zhaoqing Zhu Chuwei Luo Zirui Shao Feiyu Gao Hangdi Xing Qi Zheng Ji Zhang 44 0 0 24 Mar 2025
What can Off-the-Shelves Large Multi-Modal Models do for Dynamic Scene Graph Generation? Xuanming Cui Jaiminkumar Ashokbhai Bhoi Chionh Wei Peng Adriel Kuek Ser-Nam Lim 43 0 0 20 Mar 2025
When Less is Enough: Adaptive Token Reduction for Efficient Image Representation Eduard Allakhverdov Elizaveta Goncharova Andrey Kuznetsov 37 0 0 20 Mar 2025
MathFlow: Enhancing the Perceptual Flow of MLLMs for Visual Mathematical Problems Felix Chen Hangjie Yuan Yunqiu Xu Tao Feng Jun Cen Pengwei Liu Zeying Huang Yi Yang LRM 37 1 0 19 Mar 2025
CoSpace: Benchmarking Continuous Space Perception Ability for Vision-Language Models Yiqi Zhu Z. Wang C. Zhang Peng Li Yang Liu CoGe VLM 63 0 0 18 Mar 2025
Identifying and Mitigating Position Bias of Multi-image Vision-Language Models Xinyu Tian Shu Zou Zhaoyuan Yang Jing Zhang 53 0 0 18 Mar 2025
Can Large Vision Language Models Read Maps Like a Human? Shuo Xing Zezhou Sun Shuangyu Xie Kaiyuan Chen Yanjia Huang Yuping Wang Jiachen Li Dezhen Song Zhengzhong Tu 53 2 0 18 Mar 2025
Aligning Multimodal LLM with Human Preference: A Survey Tao Yu Y. Zhang Chaoyou Fu Junkang Wu Jinda Lu ... Qingsong Wen Z. Zhang Yan Huang Liang Wang T. Tan 64 2 0 18 Mar 2025
MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs Erik Daxberger Nina Wenzel David Griffiths Haiming Gang Justin Lazarow ... Kai Kang Marcin Eichner Y. Yang Afshin Dehghan Peter Grasch 72 2 0 17 Mar 2025
ClusComp: A Simple Paradigm for Model Compression and Efficient Finetuning Baohao Liao Christian Herold Seyyed Hadi Hashemi Stefan Vasilev Shahram Khadivi Christof Monz MQ 39 0 0 17 Mar 2025
Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey Y. Wang Shengqiong Wu Y. Zhang William Yang Wang Ziwei Liu Jiebo Luo Hao Fei LRM 69 7 0 16 Mar 2025
SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversion A. Nassar Andres Marafioti Matteo Omenetti Maksym Lysak Nikolaos Livathinos ... Yusik Kim A. Said Gurbuz Michele Dolfi Miquel Farré Peter W. J. Staar 42 3 0 14 Mar 2025