Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want

29 March 2024

Shanghang Zhang

Papers citing "Draw-and-Understand: Leveraging Visual Prompts to Enable MLLMs to Comprehend What You Want"

29 / 29 papers shown

Title
G-FOCUS: Towards a Robust Method for Assessing UI Design Persuasiveness Jaehyun Jeon Janghan Yoon Minsoo Kim Sumin Shim Yejin Choi Hanbin Kim Youngjae Yu AAML 26 0 0 08 May 2025
EarthGPT-X: Enabling MLLMs to Flexibly and Comprehensively Understand Multi-Source Remote Sensing Imagery Wei Zhang Miaoxin Cai Yaqian Ning T. Zhang Yin Zhuang He Chen Jun Li Xuerui Mao 31 0 0 17 Apr 2025
Pixel-SAIL: Single Transformer For Pixel-Grounded Understanding Tao Zhang X. Li Zilong Huang Y. Li Weixian Lei XueQing Deng Shihao Chen S. Ji Jiashi Feng MLLM LRM 32 1 0 14 Apr 2025
V2P-Bench: Evaluating Video-Language Understanding with Visual Prompts for Better Human-Model Interaction Yiming Zhao Y. Zeng Yukun Qi Y. Liu Lin Yen-Chen Zehui Chen Xikun Bao Jie Zhao Feng Zhao VLM 51 2 0 22 Mar 2025
Concept-as-Tree: Synthetic Data is All You Need for VLM Personalization Ruichuan An Kai Zeng Ming Lu Sihan Yang Renrui Zhang Huitong Ji Qizhe Zhang Y. Luo Hao Liang Wentao Zhang 45 0 0 17 Mar 2025
VERIFY: A Benchmark of Visual Explanation and Reasoning for Investigating Multimodal Reasoning Fidelity Jing Bi Junjia Guo Susan Liang Guangyu Sun Luchuan Song ... Jinxi He Jiarui Wu A. Vosoughi C. L. P. Chen Chenliang Xu LRM 54 1 0 14 Mar 2025
Enhancing Visual Reasoning with Autonomous Imagination in Multimodal Large Language Models J. Liu Yumeng Li Boyuan Xiao Yichang Jian Ziang Qin Tianjia Shao Yao-Xiang Ding Kun Zhou MLLM LRM 81 2 0 27 Nov 2024
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding Qing Jiang Gen Luo Yuqin Yang Yuda Xiong Yihao Chen Zhaoyang Zeng Tianhe Ren Lei Zhang VLM LRM 78 6 0 27 Nov 2024
DOGE: Towards Versatile Visual Document Grounding and Referring Yinan Zhou Yuxin Chen Haokun Lin Shuyu Yang Li Zhu Zhongang Qi Chen Ma Ying Shan ObjD 71 0 0 26 Nov 2024
Panther: Illuminate the Sight of Multimodal LLMs with Instruction-Guided Visual Prompts Honglin Li Yuting Gao Chenglu Zhu Jingdong Chen M. Yang Lin Yang MLLM 72 0 0 21 Nov 2024
MC-LLaVA: Multi-Concept Personalized Vision-Language Model Ruichuan An Sihan Yang Ming Lu Kai Zeng Yulin Luo ... Hao Liang Qi She Shanghang Zhang W. Zhang Wentao Zhang 60 5 0 18 Nov 2024
EVQAScore: A Fine-grained Metric for Video Question Answering Data Quality Evaluation Hao Liang Zirong Chen W. Zhang Wentao Zhang 21 0 0 11 Nov 2024
InfantCryNet: A Data-driven Framework for Intelligent Analysis of Infant Cries Mengze Hong Chen Jason Zhang Lingxiao Yang Yuanfeng Song Di Jiang 28 2 0 29 Sep 2024
A Survey on Multimodal Benchmarks: In the Era of Large AI Models Lin Li Guikun Chen Hanrong Shi Jun Xiao Long Chen 28 8 0 21 Sep 2024
Visual Prompting in Multimodal Large Language Models: A Survey Junda Wu Zhehao Zhang Yu Xia Xintong Li Zhaoyang Xia ... Subrata Mitra Dimitris N. Metaxas Lina Yao Jingbo Shang Julian McAuley VLM LRM 38 12 0 05 Sep 2024
A Survey on Evaluation of Multimodal Large Language Models Jiaxing Huang Jingyi Zhang LM&MA ELM LRM 43 20 0 28 Aug 2024
ControlMLLM: Training-Free Visual Prompt Learning for Multimodal Large Language Models Ming-Kuan Wu Xinyue Cai Jiayi Ji Jiale Li Oucheng Huang Gen Luo Hao Fei Xiaoshuai Sun Rongrong Ji MLLM 32 5 0 31 Jul 2024
EarthMarker: Visual Prompt Learning for Region-level and Point-level Remote Sensing Imagery Comprehension Wei Zhang Miaoxin Cai Tong Zhang Jun Li Zhuang Yin Xuerui Mao 59 5 0 18 Jul 2024
The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective Zhen Qin Daoyuan Chen Wenhao Zhang Liuyi Yao Yilun Huang Bolin Ding Yaliang Li Shuiguang Deng 35 5 0 11 Jul 2024
OMG-LLaVA: Bridging Image-level, Object-level, Pixel-level Reasoning and Understanding Tao Zhang Xiangtai Li Hao Fei Haobo Yuan Shengqiong Wu Shunping Ji Chen Change Loy Shuicheng Yan LRM MLLM VLM 34 44 0 27 Jun 2024
AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulation Chuyan Xiong Chengyu Shen Xiaoqi Li Kaichen Zhou Jiaming Liu Ruiping Wang Hao Dong LRM 28 10 0 17 Jun 2024
Image-of-Thought Prompting for Visual Reasoning Refinement in Multimodal Large Language Models Qiji Zhou Ruochen Zhou Zike Hu Panzhong Lu Siyang Gao Yue Zhang LRM 30 12 0 22 May 2024
SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Models Chris Liu Renrui Zhang Longtian Qiu Siyuan Huang Weifeng Lin ... Hao Shao Pan Lu Hongsheng Li Yu Qiao Peng Gao MLLM 113 106 0 08 Feb 2024
SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents Kanzhi Cheng Qiushi Sun Yougang Chu Fangzhi Xu Yantao Li Jianbing Zhang Zhiyong Wu LLMAG 162 137 0 17 Jan 2024
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 150 280 0 14 Oct 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023
CPT: Colorful Prompt Tuning for Pre-trained Vision-Language Models Yuan Yao Ao Zhang Zhengyan Zhang Zhiyuan Liu Tat-Seng Chua Maosong Sun MLLM VPVLM VLM 186 218 0 24 Sep 2021
Semantic Understanding of Scenes through the ADE20K Dataset Bolei Zhou Hang Zhao Xavier Puig Tete Xiao Sanja Fidler Adela Barriuso Antonio Torralba SSeg 243 1,817 0 18 Aug 2016
COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Images Andreas Veit Tomas Matera Lukás Neumann Jirí Matas Serge J. Belongie 169 458 0 26 Jan 2016