GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest

7 July 2023

Ping Luo

Papers citing "GPT4RoI: Instruction Tuning Large Language Model on Region-of-Interest"

50 / 199 papers shown

Title
Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Models Haotian Zhang Haoxuan You Philipp Dufter Bowen Zhang Chen Chen ... Tsu-jui Fu William Yang Wang Shih-Fu Chang Zhe Gan Yinfei Yang ObjD MLLM 93 42 0 11 Apr 2024
Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs Kanchana Ranasinghe Satya Narayan Shukla Omid Poursaeed Michael S. Ryoo Tsung-Yu Lin LRM 25 21 0 11 Apr 2024
Can Feedback Enhance Semantic Grounding in Large Vision-Language Models? Yuan-Hong Liao Rafid Mahmood Sanja Fidler David Acuna VLM 33 7 0 09 Apr 2024
Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs Keen You Haotian Zhang E. Schoop Floris Weers Amanda Swearngin Jeffrey Nichols Yinfei Yang Zhe Gan MLLM 31 82 0 08 Apr 2024
CoReS: Orchestrating the Dance of Reasoning and Segmentation Xiaoyi Bao Siyang Sun Shuailei Ma Kecheng Zheng Yuxin Guo Guosheng Zhao Yun Zheng Xingang Wang LRM 20 6 0 08 Apr 2024
PARIS3D: Reasoning-based 3D Part Segmentation Using Large Multimodal Model Amrin Kareem Jean Lahoud Hisham Cholakkal LRM 28 3 0 04 Apr 2024
Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want Weifeng Lin Xinyu Wei Ruichuan An Peng Gao Bocheng Zou Yulin Luo Siyuan Huang Shanghang Zhang Hongsheng Li VLM 40 31 0 29 Mar 2024
LocCa: Visual Pretraining with Location-aware Captioners Bo Wan Michael Tschannen Yongqin Xian Filip Pavetić Ibrahim M. Alabdulmohsin Xiao Wang André Susano Pinto Andreas Steiner Lucas Beyer Xiao-Qi Zhai VLM 27 5 0 28 Mar 2024
Toward Interactive Regional Understanding in Vision-Large Language Models Jungbeom Lee Sanghyuk Chun Sangdoo Yun VLM 14 1 0 27 Mar 2024
Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning Hao Shao Shengju Qian Han Xiao Guanglu Song Zhuofan Zong Letian Wang Yu Liu Hongsheng Li VGen LRM MLLM 41 35 0 25 Mar 2024
Elysium: Exploring Object-level Perception in Videos via MLLM Hang Wang Yanjie Wang Yongjie Ye Yuxiang Nie Can Huang MLLM 24 18 0 25 Mar 2024
LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models Yuzhang Shang Mu Cai Bingxin Xu Yong Jae Lee Yan Yan VLM 18 88 0 22 Mar 2024
Pensieve: Retrospect-then-Compare Mitigates Visual Hallucination Dingchen Yang Bowen Cao Guang Chen Changjun Jiang 35 7 0 21 Mar 2024
SC-Tune: Unleashing Self-Consistent Referential Comprehension in Large Vision Language Models Tongtian Yue Jie Cheng Longteng Guo Xingyuan Dai Zijia Zhao Xingjian He Gang Xiong Yisheng Lv Jing Liu 28 8 0 20 Mar 2024
Griffon v2: Advancing Multimodal Perception with High-Resolution Scaling and Visual-Language Co-Referring Yufei Zhan Yousong Zhu Hongyin Zhao Fan Yang Ming Tang Jinqiao Wang ObjD 23 12 0 14 Mar 2024
VLM-PL: Advanced Pseudo Labeling Approach for Class Incremental Object Detection via Vision-Language Model Junsu Kim Yunhoe Ku Jihyeon Kim Junuk Cha Seungryul Baek ObjD VLM 22 9 0 08 Mar 2024
Multi-modal Instruction Tuned LLMs with Fine-grained Visual Perception Jun-Yan He Yifan Wang Lijun Wang Huchuan Lu Jun-Yan He Jinpeng Lan Bin Luo Xuansong Xie MLLM VLM 22 16 0 05 Mar 2024
RegionGPT: Towards Region Understanding Vision Language Model Qiushan Guo Shalini De Mello Hongxu Yin Wonmin Byeon Ka Chun Cheung Yizhou Yu Ping Luo Sifei Liu VLM 22 35 0 04 Mar 2024
Contrastive Region Guidance: Improving Grounding in Vision-Language Models without Training David Wan Jaemin Cho Elias Stengel-Eskin Mohit Bansal VLM ObjD 36 29 0 04 Mar 2024
The All-Seeing Project V2: Towards General Relation Comprehension of the Open World Weiyun Wang Yiming Ren Hao Luo Tiantong Li Chenxiang Yan ... Qingyun Li Lewei Lu Xizhou Zhu Yu Qiao Jifeng Dai MLLM 26 46 0 29 Feb 2024
Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiency Akila Wickramasekara F. Breitinger Mark Scanlon 34 7 0 29 Feb 2024
ShapeLLM: Universal 3D Object Understanding for Embodied Interaction Zekun Qi Runpei Dong Shaochen Zhang Haoran Geng Chunrui Han Zheng Ge Li Yi Kaisheng Ma 33 49 0 27 Feb 2024
VCD: Knowledge Base Guided Visual Commonsense Discovery in Images Xiangqing Shen Yurun Song Siwei Wu Rui Xia 31 6 0 27 Feb 2024
GROUNDHOG: Grounding Large Language Models to Holistic Segmentation Yichi Zhang Ziqiao Ma Xiaofeng Gao Suhaila Shakiah Qiaozi Gao Joyce Chai MLLM VLM 22 38 0 26 Feb 2024
RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis Yao Mu Junting Chen Qinglong Zhang Shoufa Chen Qiaojun Yu ... Wenhai Wang Jifeng Dai Yu Qiao Mingyu Ding Ping Luo 37 20 0 25 Feb 2024
A Survey on Knowledge Distillation of Large Language Models Xiaohan Xu Ming Li Chongyang Tao Tao Shen Reynold Cheng Jinyang Li Can Xu Dacheng Tao Tianyi Zhou KELM VLM 23 94 0 20 Feb 2024
The Revolution of Multimodal Large Language Models: A Survey Davide Caffagni Federico Cocchi Luca Barsellotti Nicholas Moratelli Sara Sarto Lorenzo Baraldi Lorenzo Baraldi Marcella Cornia Rita Cucchiara LRM VLM 27 41 0 19 Feb 2024
Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models Xuanyu Lei Zonghan Yang Xinrui Chen Peng Li Yang Liu MLLM LRM 24 30 0 19 Feb 2024
Image Fusion via Vision-Language Model Zixiang Zhao Lilun Deng Haowen Bai Yukun Cui Zhipeng Zhang ... Haotong Qin Dongdong Chen Jiangshe Zhang Peng Wang Luc Van Gool VLM 13 18 0 03 Feb 2024
ControlCap: Controllable Region-level Captioning Yuzhong Zhao Yue Liu Zonghao Guo Weijia Wu Chen Gong Fang Wan QiXiang Ye 16 4 0 31 Jan 2024
LLaVA-MoLE: Sparse Mixture of LoRA Experts for Mitigating Data Conflicts in Instruction Finetuning MLLMs Shaoxiang Chen Zequn Jie Lin Ma MoE 30 46 0 29 Jan 2024
MM-LLMs: Recent Advances in MultiModal Large Language Models Duzhen Zhang Yahan Yu Jiahua Dong Chenxing Li Dan Su Chenhui Chu Dong Yu OffRL LRM 34 173 0 24 Jan 2024
ChatterBox: Multi-round Multimodal Referring and Grounding Yunjie Tian Tianren Ma Lingxi Xie Jihao Qiu Xi Tang Yuan Zhang Jianbin Jiao Qi Tian Qixiang Ye 10 15 0 24 Jan 2024
Advancing Large Multi-modal Models with Explicit Chain-of-Reasoning and Visual Question Generation Kohei Uehara Nabarun Goswami Hanqin Wang Toshiaki Baba Kohtaro Tanaka ... Takagi Naoya Ryo Umagami Yingyi Wen Tanachai Anakewat Tatsuya Harada LRM 13 2 0 18 Jan 2024
Self-Imagine: Effective Unimodal Reasoning with Multimodal Models using Self-Imagination Syeda Nahida Akter Aman Madaan Sangwu Lee Yiming Yang Eric Nyberg ReLM VLM LRM 20 2 0 16 Jan 2024
Seeing the Unseen: Visual Common Sense for Semantic Placement Ram Ramrakhya Aniruddha Kembhavi Dhruv Batra Z. Kira Kuo-Hao Zeng Luca Weihs VLM 20 4 0 15 Jan 2024
UMG-CLIP: A Unified Multi-Granularity Vision Generalist for Open-World Understanding Bowen Shi Peisen Zhao Zichen Wang Yuhang Zhang Yaoming Wang ... Wenrui Dai Junni Zou Hongkai Xiong Qi Tian Xiaopeng Zhang VLM 25 6 0 12 Jan 2024
ChartAssisstant: A Universal Chart Multimodal Language Model via Chart-to-Table Pre-training and Multitask Instruction Tuning Fanqing Meng Wenqi Shao Quanfeng Lu Peng Gao Kaipeng Zhang Yu Qiao Ping Luo 17 45 0 04 Jan 2024
Tracking with Human-Intent Reasoning Jiawen Zhu Zhi-Qi Cheng Jun-Yan He Chenyang Li Bin Luo Huchuan Lu Yifeng Geng Xuansong Xie LRM VOS 19 6 0 29 Dec 2023
LISA++: An Improved Baseline for Reasoning Segmentation with Large Language Model Senqiao Yang Tianyuan Qu Xin Lai Zhuotao Tian Bohao Peng Shu-Lin Liu Jiaya Jia VLM 8 28 0 28 Dec 2023
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action Jiasen Lu Christopher Clark Sangho Lee Zichen Zhang Savya Khosla Ryan Marten Derek Hoiem Aniruddha Kembhavi VLM MLLM 13 143 0 28 Dec 2023
Visual Instruction Tuning towards General-Purpose Multimodal Model: A Survey Jiaxing Huang Jingyi Zhang Kai Jiang Han Qiu Shijian Lu 17 22 0 27 Dec 2023
Voila-A: Aligning Vision-Language Models with User's Gaze Attention Kun Yan Lei Ji Zeyu Wang Yuntao Wang Nan Duan Shuai Ma 37 6 0 22 Dec 2023
InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks Zhe Chen Jiannan Wu Wenhai Wang Weijie Su Guo Chen ... Bin Li Ping Luo Tong Lu Yu Qiao Jifeng Dai VLM MLLM 126 895 0 21 Dec 2023
Generative Multimodal Models are In-Context Learners Quan-Sen Sun Yufeng Cui Xiaosong Zhang Fan Zhang Qiying Yu ... Yueze Wang Yongming Rao Jingjing Liu Tiejun Huang Xinlong Wang MLLM LRM 28 212 0 20 Dec 2023
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model Shraman Pramanick Guangxing Han Rui Hou Sayan Nag Ser-Nam Lim Nicolas Ballas Qifan Wang Rama Chellappa Amjad Almahairi VLM MLLM 19 29 0 19 Dec 2023
G-LLaVA: Solving Geometric Problem with Multi-Modal Large Language Model Jiahui Gao Renjie Pi Jipeng Zhang Jiacheng Ye Wanjun Zhong ... Lanqing Hong Jianhua Han Hang Xu Zhenguo Li Lingpeng Kong SyDa ReLM LRM 36 84 0 18 Dec 2023
One-Shot Learning as Instruction Data Prospector for Large Language Models Yunshui Li Binyuan Hui Xiaobo Xia Jiaxi Yang Min Yang ... Ling-Hao Chen Junhao Liu Tongliang Liu Fei Huang Yongbin Li 25 31 0 16 Dec 2023
Osprey: Pixel Understanding with Visual Instruction Tuning Yuqian Yuan Wentong Li Jian Liu Dongqi Tang Xinjie Luo Chi Qin Lei Zhang Jianke Zhu MLLM VLM 34 76 0 15 Dec 2023
GSVA: Generalized Segmentation via Multimodal Large Language Models Zhuofan Xia Dongchen Han Yizeng Han Xuran Pan Shiji Song Gao Huang VLM 15 40 0 15 Dec 2023