Osprey: Pixel Understanding with Visual Instruction Tuning

15 December 2023

Yuqian Yuan

Lei Zhang

Papers citing "Osprey: Pixel Understanding with Visual Instruction Tuning"

50 / 67 papers shown

Title
RESAnything: Attribute Prompting for Arbitrary Referring Segmentation Ruiqi Wang Hao Zhang VLM 44 0 0 03 May 2025
UniBiomed: A Universal Foundation Model for Grounded Biomedical Image Interpretation Linshan Wu Yuxiang Nie Sunan He Jiaxin Zhuang Hao Chen LM&MA MedIm 66 0 0 30 Apr 2025
DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understanding Geng Li Jinglin Xu Yunzhen Zhao Yuxin Peng ObjD 17 0 0 21 Apr 2025
EarthGPT-X: Enabling MLLMs to Flexibly and Comprehensively Understand Multi-Source Remote Sensing Imagery Wei Zhang Miaoxin Cai Yaqian Ning T. Zhang Yin Zhuang He Chen Jun Li Xuerui Mao 36 0 0 17 Apr 2025
Pixel-SAIL: Single Transformer For Pixel-Grounded Understanding Tao Zhang X. Li Zilong Huang Y. Li Weixian Lei XueQing Deng Shihao Chen S. Ji Jiashi Feng MLLM LRM 40 1 0 14 Apr 2025
OrderChain: A General Prompting Paradigm to Improve Ordinal Understanding Ability of MLLM Jinhong Wang Shuo Tong Jian Liu Dongqi Tang Weiqiang Wang Wentong Li Hongxia Xu D. Z. Chen J. Chen Jian Wu LRM 16 0 0 07 Apr 2025
URECA: Unique Region Caption Anything Sangbeom Lim J. Kim Heeji Yoon Jaewoo Jung Seungryong Kim 24 0 0 07 Apr 2025
InstructRestore: Region-Customized Image Restoration with Human Instructions S. Liu Jianqi Ma Lingchen Sun Xiangtao Kong Lei Zhang DiffM 32 0 0 31 Mar 2025
Towards Understanding How Knowledge Evolves in Large Vision-Language Models Sudong Wang Y. Zhang Yao Zhu Jianing Li Zizhe Wang Y. Liu Xiangyang Ji 29 0 0 31 Mar 2025
A Survey on Remote Sensing Foundation Models: From Vision to Multimodality Ziyue Huang Hongxi Yan Qiqi Zhan Shuai Yang Mingming Zhang Chenkai Zhang Yiming Lei Zeming Liu Qingjie Liu Y. Wang 37 0 0 28 Mar 2025
SceneSplat: Gaussian Splatting-based Scene Understanding with Vision-Language Pretraining Yue Li Qi Ma Runyi Yang Huapeng Li Mengjiao Ma ... E. Konukoglu Theo Gevers Luc Van Gool Martin R. Oswald Danda Pani Paudel 3DGS VLM 62 0 0 23 Mar 2025
GraspCorrect: Robotic Grasp Correction via Vision-Language Model-Guided Feedback Sungjae Lee Yeonjoo Hong Kwang In KIm 39 0 0 19 Mar 2025
MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMs Erik Daxberger Nina Wenzel David Griffiths Haiming Gang Justin Lazarow ... Kai Kang Marcin Eichner Y. Yang Afshin Dehghan Peter Grasch 72 2 0 17 Mar 2025
SegAgent: Exploring Pixel Understanding Capabilities in MLLMs by Imitating Human Annotator Trajectories Muzhi Zhu Yuzhuo Tian Hao Chen Chunluan Zhou Qingpei Guo Y. Liu M. Yang Chunhua Shen MLLM VLM 62 0 0 11 Mar 2025
REF-VLM: Triplet-Based Referring Paradigm for Unified Visual Decoding Yan Tai Luhao Zhu Zhiqiang Chen Ynan Ding Yiying Dong Xiaohong Liu Guodong Guo MLLM ObjD 45 0 0 10 Mar 2025
Multi-Layer Visual Feature Fusion in Multimodal LLMs: Methods, Analysis, and Best Practices Junyan Lin Haoran Chen Yue Fan Yingqi Fan Xin Jin Hui Su Jinlan Fu Xiaoyu Shen 55 0 0 08 Mar 2025
Pixel-Level Reasoning Segmentation via Multi-turn Conversations Dexian Cai Xiaocui Yang Yongkang Liu Daling Wang Shi Feng Yifei Zhang Soujanya Poria LRM 74 0 0 13 Feb 2025
Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks Miran Heo Min-Hung Chen De-An Huang Sifei Liu Subhashree Radhakrishnan Seon Joo Kim Yu-Chun Wang Ryo Hachiuma ObjD VLM 100 2 0 14 Jan 2025
Sa2VA: Marrying SAM2 with LLaVA for Dense Grounded Understanding of Images and Videos Haobo Yuan X. Li Tao Zhang Zilong Huang Shilin Xu S. Ji Yunhai Tong Lu Qi Jiashi Feng Ming Yang VLM 65 11 0 07 Jan 2025
Visual Large Language Models for Generalized and Specialized Applications Yifan Li Zhixin Lai Wentao Bao Zhen Tan Anh Dao Kewei Sui Jiayi Shen Dong Liu Huan Liu Yu Kong VLM 83 10 0 06 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 86 45 0 03 Jan 2025
Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing Hao Fei Shengqiong Wu H. Zhang Tat-Seng Chua Shuicheng Yan 51 35 0 31 Dec 2024
VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM Yuqian Yuan Hang Zhang Wentong Li Zesen Cheng Boqiang Zhang ... Deli Zhao Wenqiao Zhang Yueting Zhuang Jianke Zhu Lidong Bing 41 5 0 31 Dec 2024
CoF: Coarse to Fine-Grained Image Understanding for Multi-modal Large Language Models Yeyuan Wang D. Gao Bin Li Rujiao Long Lei Yi Xiaoyan Cai Libin Yang Jinxia Zhang Shanqing Yu Qi Xuan 64 0 0 22 Dec 2024
ROSE: Revolutionizing Open-Set Dense Segmentation with Patch-Wise Perceptual Large Multimodal Model Kunyang Han Yibo Hu Mengxue Qu Hailin Shi Yao Zhao Y. X. Wei MLLM VLM 3DV 72 1 0 29 Nov 2024
Detailed Object Description with Controllable Dimensions Xinran Wang H. Zhang Baoteng Li Kongming Liang Hao Sun Zhongjiang He Z. Ma Jun Guo 76 0 0 28 Nov 2024
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding Qing Jiang Gen Luo Yuqin Yang Yuda Xiong Yihao Chen Zhaoyang Zeng Tianhe Ren Lei Zhang VLM LRM 84 6 0 27 Nov 2024
FINECAPTION: Compositional Image Captioning Focusing on Wherever You Want at Any Granularity Hang Hua Qing Liu Lingzhi Zhang Jing Shi Zhifei Zhang Yilin Wang Jianming Zhang Jiebo Luo CoGe VLM 79 6 0 23 Nov 2024
SEAGULL: No-reference Image Quality Assessment for Regions of Interest via Vision-Language Instruction Tuning Z. Chen Juan Wang Wen Wang Sunhan Xu Hang Xiong ... Jian Guo Shuxun Wang C. Yuan Bing Li Weiming Hu VLM 28 1 0 15 Nov 2024
Griffon-G: Bridging Vision-Language and Vision-Centric Tasks via Large Multimodal Models Yufei Zhan Hongyin Zhao Yousong Zhu Fan Yang Ming Tang Jinqiao Wang MLLM 43 1 0 21 Oct 2024
Mitigating Object Hallucination via Concentric Causal Attention Yun Xing Yiheng Li Ivan Laptev Shijian Lu 36 17 0 21 Oct 2024
ForgeryGPT: Multimodal Large Language Model For Explainable Image Forgery Detection and Localization Jiawei Li Fanrui Zhang Jiaying Zhu Esther Sun Qiang Zhang Zheng-jun Zha MLLM 27 7 0 14 Oct 2024
DTVLT: A Multi-modal Diverse Text Benchmark for Visual Language Tracking Based on LLM Xuchen Li Shiyu Hu Xiaokun Feng Dailing Zhang Meiqi Wu Jing Zhang Kaiqi Huang 16 5 0 03 Oct 2024
MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuning Haotian Zhang Mingfei Gao Zhe Gan Philipp Dufter Nina Wenzel ... Haoxuan You Zirui Wang Afshin Dehghan Peter Grasch Yinfei Yang VLM MLLM 36 32 1 30 Sep 2024
EAGLE: Towards Efficient Arbitrary Referring Visual Prompts Comprehension for Multimodal Large Language Models Jiacheng Zhang Yang Jiao Shaoxiang Chen Jingjing Chen Yu-Gang Jiang 13 1 0 25 Sep 2024
Instruction-guided Multi-Granularity Segmentation and Captioning with Large Multimodal Model Li Zhou Xu Yuan Zenghui Sun Zikun Zhou Jingsong Lan VLM MLLM 31 2 0 20 Sep 2024
Image Segmentation in Foundation Model Era: A Survey Tianfei Zhou Fei Zhang Boyu Chang Wenguan Wang Ye Yuan E. Konukoglu Daniel Cremers VLM 36 4 0 23 Aug 2024
Sketch2Scene: Automatic Generation of Interactive 3D Game Scenes from User's Casual Sketches Yongzhi Xu Yonhon Ng Yifu Wang Inkyu Sa Yunfei Duan Yang Li Pan Ji Hongdong Li VGen 3DV 23 4 0 08 Aug 2024
ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models Ming-Kuan Wu Xinyue Cai Jiayi Ji Jiale Li Oucheng Huang Gen Luo Hao Fei Xiaoshuai Sun Rongrong Ji MLLM 37 5 0 31 Jul 2024
WTS: A Pedestrian-Centric Traffic Video Dataset for Fine-grained Spatial-Temporal Understanding Quan Kong Yuki Kawana Rajat Saini Ashutosh Kumar Jingjing Pan ... Yohei Ozao Balázs Opra D. Anastasiu Yoichi Sato N. Kobori VGen 19 7 0 22 Jul 2024
EarthMarker: Visual Prompt Learning for Region-level and Point-level Remote Sensing Imagery Comprehension Wei Zhang Miaoxin Cai Tong Zhang Jun Li Zhuang Yin Xuerui Mao 59 5 0 18 Jul 2024
SPIN: Hierarchical Segmentation with Subpart Granularity in Natural Images Josh Myers-Dean Jarek Reynolds Brian Price Yifei Fan Danna Gurari 24 2 0 12 Jul 2024
Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentation Seonghoon Yu Paul Hongsuck Seo Jeany Son DiffM 46 4 0 10 Jul 2024
TokenPacker: Efficient Visual Projector for Multimodal LLM Wentong Li Yuqian Yuan Jian Liu Dongqi Tang Song Wang Jie Qin Jianke Zhu Lei Zhang MLLM 29 6 0 02 Jul 2024
OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding Tao Zhang Xiangtai Li Hao Fei Haobo Yuan Shengqiong Wu Shunping Ji Chen Change Loy Shuicheng Yan LRM MLLM VLM 47 44 0 27 Jun 2024
SpatialBot: Precise Spatial Understanding with Vision Language Models Wenxiao Cai Yaroslav Ponomarenko Jianhao Yuan Xiaoqi Li Wankou Yang Hao Dong Bo-Lu Zhao VLM 30 24 0 19 Jun 2024
ClawMachine: Learning to Fetch Visual Tokens for Referential Comprehension Tianren Ma Lingxi Xie Yunjie Tian Boyu Yang Yuan Zhang 29 0 0 17 Jun 2024
Towards Semantic Equivalence of Tokenization in Multimodal LLM Shengqiong Wu Hao Fei Xiangtai Li Jiayi Ji Hanwang Zhang Tat-Seng Chua Shuicheng Yan MLLM 52 25 0 07 Jun 2024
SpatialRGPT: Grounded Spatial Reasoning in Vision Language Model An-Chieh Cheng Hongxu Yin Yang Fu Qiushan Guo Ruihan Yang Jan Kautz Xiaolong Wang Sifei Liu LRM 36 43 0 03 Jun 2024
The Evolution of Multimodal Model Architectures S. Wadekar Abhishek Chaurasia Aman Chadha Eugenio Culurciello 41 13 0 28 May 2024