GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest

7 July 2023

Ping Luo

Papers citing "GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest"

50 / 199 papers shown

Title
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning Haotian Zhang Mingfei Gao Zhe Gan Philipp Dufter Nina Wenzel ... Haoxuan You Zirui Wang Afshin Dehghan Peter Grasch Yinfei Yang VLM MLLM 36 32 1 30 Sep 2024
World to Code: Multi-modal Data Generation via Self-Instructed Compositional Captioning and Filtering Jiacong Wang Bohong Wu Haiyong Jiang Xun Zhou Xin Xiao Haoyuan Guo Jun Xiao VLM VGen 22 4 0 30 Sep 2024
One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videos Zechen Bai Tong He Haiyang Mei Pichao Wang Ziteng Gao Joya Chen Lei Liu Zheng Zhang Mike Zheng Shou VLM VOS MLLM 24 17 0 29 Sep 2024
ZALM3: Zero-Shot Enhancement of Vision-Language Alignment via In-Context Information in Multi-Turn Multimodal Medical Dialogue Zhangpu Li Changhong Zou Suxue Ma Zhicheng Yang Chen Du ... Xingzhi Sun Jing Xiao Kai Zhang Mei Han Mei Han LM&MA 33 0 0 26 Sep 2024
EAGLE: Egocentric AGgregated Language-video Engine Jing Bi Yunlong Tang Luchuan Song A. Vosoughi Nguyen Nguyen Chenliang Xu 20 8 0 26 Sep 2024
EAGLE: Towards Efficient Arbitrary Referring Visual Prompts Comprehension for Multimodal Large Language Models Jiacheng Zhang Yang Jiao Shaoxiang Chen Jingjing Chen Yu-Gang Jiang 13 1 0 25 Sep 2024
Enhancing Perception of Key Changes in Remote Sensing Image Change Captioning Cong Yang Zuchao Li Hongzan Jiao Zhi Gao Lefei Zhang 27 1 0 19 Sep 2024
From Linguistic Giants to Sensory Maestros: A Survey on Cross-Modal Reasoning with Large Language Models Shengsheng Qian Zuyi Zhou Dizhan Xue Bing Wang Changsheng Xu LRM 24 1 0 19 Sep 2024
SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentation Yi-Chia Chen Wei-Hua Li Cheng Sun Yu-Chiang Frank Wang Chu-Song Chen VLM 17 6 0 01 Sep 2024
Visual Agents as Fast and Slow Thinkers Guangyan Sun Mingyu Jin Zhenting Wang Cheng-Long Wang Siqi Ma Qifan Wang Ying Nian Wu Ying Nian Wu Dongfang Liu Dongfang Liu LLMAG LRM 63 11 0 16 Aug 2024
Sportify: Question Answering with Embedded Visualizations and Personified Narratives for Sports Video Chunggi Lee Tica Lin Hanspeter Pfister Chen Zhu-Tian 18 1 0 09 Aug 2024
Targeted Visual Prompting for Medical Visual Question Answering Sergio Tascon-Morales Pablo Márquez-Neila Raphael Sznitman 21 2 0 06 Aug 2024
Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models Fushuo Huo Wenchao Xu Zhong Zhang Haozhao Wang Zhicheng Chen Peilin Zhao VLM MLLM 36 18 0 04 Aug 2024
Learning Video Context as Interleaved Multimodal Sequences S. Shao Pengchuan Zhang Y. Li Xide Xia A. Meso Ziteng Gao Jinheng Xie N. Holliman Mike Zheng Shou 33 4 0 31 Jul 2024
ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models Ming-Kuan Wu Xinyue Cai Jiayi Ji Jiale Li Oucheng Huang Gen Luo Hao Fei Xiaoshuai Sun Rongrong Ji MLLM 37 5 0 31 Jul 2024
Advancing Multimodal Large Language Models in Chart Question Answering with Visualization-Referenced Instruction Tuning Xingchen Zeng Haichuan Lin Yilin Ye Wei Zeng 37 13 0 29 Jul 2024
DOPRA: Decoding Over-accumulation Penalization and Re-allocation in Specific Weighting Layer Jinfeng Wei Xiaofeng Zhang 19 10 0 21 Jul 2024
SegPoint: Segment Any Point Cloud via Large Language Model Shuting He Henghui Ding Xudong Jiang Bihan Wen 3DV MLLM 3DPC 22 17 0 18 Jul 2024
EarthMarker: Visual Prompt Learning for Region-level and Point-level Remote Sensing Imagery Comprehension Wei Zhang Miaoxin Cai Tong Zhang Jun Li Zhuang Yin Xuerui Mao 59 5 0 18 Jul 2024
VISA: Reasoning Video Object Segmentation via Large Language Models Cilin Yan Haochen Wang Shilin Yan Xiaolong Jiang Yao Hu Guoliang Kang Weidi Xie E. Gavves LRM VLM VOS 24 22 0 16 Jul 2024
IDA-VLM: Towards Movie Understanding via ID-Aware Large Vision-Language Model Yatai Ji Shilong Zhang Jie Wu Peize Sun Weifeng Chen Xuefeng Xiao Sidi Yang Y. Yang Ping Luo VLM 23 1 0 10 Jul 2024
Rethinking Visual Prompting for Multimodal Large Language Models with External Knowledge Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip H. S. Torr Lu Yuan LRM VLM 23 8 0 05 Jul 2024
Meerkat: Audio-Visual Large Language Model for Grounding in Space and Time Sanjoy Chowdhury Sayan Nag Subhrajyoti Dasgupta Jun Chen Mohamed Elhoseiny Ruohan Gao Dinesh Manocha VLM MLLM 27 9 0 01 Jul 2024
LLaRA: Supercharging Robot Learning Data for Vision-Language Policy Xiang Li Cristina Mata J. Park Kumara Kahatapitiya Yoo Sung Jang ... Kanchana Ranasinghe R. Burgert Mu Cai Yong Jae Lee Michael S. Ryoo LM&Ro 47 23 0 28 Jun 2024
OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding Tao Zhang Xiangtai Li Hao Fei Haobo Yuan Shengqiong Wu Shunping Ji Chen Change Loy Shuicheng Yan LRM MLLM VLM 44 44 0 27 Jun 2024
A Refer-and-Ground Multimodal Large Language Model for Biomedicine Xiaoshuang Huang Haifeng Huang Lingdong Shen Yehui Yang Fangxin Shang Junwei Liu Jia Liu LM&MA 39 6 0 26 Jun 2024
Evaluating and Analyzing Relationship Hallucinations in Large Vision-Language Models Mingrui Wu Jiayi Ji Oucheng Huang Jiale Li Yuhang Wu Xiaoshuai Sun Rongrong Ji 30 8 0 24 Jun 2024
Does Object Grounding Really Reduce Hallucination of Large Vision-Language Models? Gregor Geigle Radu Timofte Goran Glavas 27 0 0 20 Jun 2024
VideoLLM-online: Online Video Large Language Model for Streaming Video Joya Chen Zhaoyang Lv Shiwei Wu Kevin Qinghong Lin Chenan Song Difei Gao Jia-Wei Liu Ziteng Gao Dongxing Mao Mike Zheng Shou MLLM MoMe 27 47 0 17 Jun 2024
DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Models Renqiu Xia Song Mao Xiangchao Yan Hongbin Zhou Bo Zhang ... Yongwei Wang Bin Wang Junchi Yan Fei Wu Yu Qiao 31 10 0 17 Jun 2024
ClawMachine: Learning to Fetch Visual Tokens for Referential Comprehension Tianren Ma Lingxi Xie Yunjie Tian Boyu Yang Yuan Zhang 29 0 0 17 Jun 2024
VANE-Bench: Video Anomaly Evaluation Benchmark for Conversational LMMs Rohit K Bharadwaj Hanan Gani Muzammal Naseer F. Khan Salman Khan 45 3 0 14 Jun 2024
Needle In A Multimodal Haystack Weiyun Wang Shuibo Zhang Yiming Ren Yuchen Duan Tiantong Li ... Ping Luo Yu Qiao Jifeng Dai Wenqi Shao Wenhai Wang VLM 57 16 0 11 Jun 2024
Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation Peize Sun Yi Jiang Shoufa Chen Shilong Zhang Bingyue Peng Ping Luo Zehuan Yuan VLM 25 214 0 10 Jun 2024
MGIMM: Multi-Granularity Instruction Multimodal Model for Attribute-Guided Remote Sensing Image Detailed Description Cong Yang Zuchao Li Lefei Zhang 24 1 0 07 Jun 2024
SpatialRGPT: Grounded Spatial Reasoning in Vision Language Model An-Chieh Cheng Hongxu Yin Yang Fu Qiushan Guo Ruihan Yang Jan Kautz Xiaolong Wang Sifei Liu LRM 36 43 0 03 Jun 2024
CYCLO: Cyclic Graph Transformer Approach to Multi-Object Relationship Modeling in Aerial Videos Trong-Thuan Nguyen Pha Nguyen Xin Li Jackson Cothren Alper Yilmaz Khoa Luu 33 3 0 03 Jun 2024
OLIVE: Object Level In-Context Visual Embeddings Timothy Ossowski Junjie Hu OCL VLM 52 0 0 02 Jun 2024
Artemis: Towards Referential Understanding in Complex Videos Jihao Qiu Yuan Zhang Xi Tang Lingxi Xie Tianren Ma Pengyu Yan David Doermann Qixiang Ye Yunjie Tian VLM VGen 29 9 0 01 Jun 2024
Reasoning3D -- Grounding and Reasoning in 3D: Fine-Grained Zero-Shot Open-Vocabulary 3D Reasoning Part Segmentation via Large Vision-Language Models Tianrun Chen Chunan Yu Jing Li Jianqi Zhang Lanyun Zhu Deyi Ji Yong Zhang Ying-Dong Zang Zejian Li Lingyun Sun LRM 28 9 0 29 May 2024
The Evolution of Multimodal Model Architectures S. Wadekar Abhishek Chaurasia Aman Chadha Eugenio Culurciello 41 13 0 28 May 2024
Matryoshka Multimodal Models Mu Cai Jianwei Yang Jianfeng Gao Yong Jae Lee VLM 31 25 0 27 May 2024
Reason3D: Searching and Reasoning 3D Segmentation via Large Language Model Kuan-Chih Huang Xiangtai Li Lu Qi Shuicheng Yan Ming-Hsuan Yang LRM 38 9 0 27 May 2024
V-Zen: Efficient GUI Understanding and Precise Grounding With A Novel Multimodal LLM Abdur Rahman Rajat Chawla Muskaan Kumar Arkajit Datta Adarsh Jha NS Mukunda Ishaan Bhola 22 2 0 24 May 2024
DEEM: Diffusion Models Serve as the Eyes of Large Language Models for Image Perception Run Luo Yunshui Li Longze Chen Wanwei He Ting-En Lin ... Zikai Song Xiaobo Xia Tongliang Liu Min Yang Binyuan Hui VLM DiffM 51 14 0 24 May 2024
ReasonPix2Pix: Instruction Reasoning Dataset for Advanced Image Editing Ying Jin Pengyang Ling Xiao-wen Dong Pan Zhang Jiaqi Wang Dahua Lin 22 2 0 18 May 2024
Language-Image Models with 3D Understanding Jang Hyun Cho B. Ivanovic Yulong Cao Edward Schmerling Yue Wang ... Boyi Li Yurong You Philipp Krahenbuhl Yan Wang Marco Pavone LRM 32 3 0 06 May 2024
Hallucination of Multimodal Large Language Models: A Survey Zechen Bai Pichao Wang Tianjun Xiao Tong He Zongbo Han Zheng Zhang Mike Zheng Shou VLM LRM 68 136 0 29 Apr 2024
How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites Zhe Chen Weiyun Wang Hao Tian Shenglong Ye Zhangwei Gao ... Tong Lu Dahua Lin Yu Qiao Jifeng Dai Wenhai Wang MLLM VLM 35 513 0 25 Apr 2024
Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Models Chuofan Ma Yi-Xin Jiang Jiannan Wu Zehuan Yuan Xiaojuan Qi VLM ObjD 26 49 0 19 Apr 2024