Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic

27 June 2023

Papers citing "Shikra: Unleashing Multimodal LLM's Referential Dialogue Magic"

50 / 114 papers shown

Title
LISAT: Language-Instructed Segmentation Assistant for Satellite Imagery Jerome Quenum Wen-Han Hsieh Tsung-Han Wu Ritwik Gupta Trevor Darrell David M. Chan MLLM VLM 54 0 0 05 May 2025
RESAnything: Attribute Prompting for Arbitrary Referring Segmentation Ruiqi Wang Hao Zhang VLM 54 0 0 03 May 2025
RoboGround: Robotic Manipulation with Grounded Vision-Language Priors Haifeng Huang Xinyi Chen Y. Chen H. Li Xiaoshen Han Z. Wang Tai Wang Jiangmiao Pang Zhou Zhao LM&Ro 75 0 0 30 Apr 2025
Vision-Language Models Are Not Pragmatically Competent in Referring Expression Generation Ziqiao Ma Jing Ding Xuejun Zhang Dezhi Luo Jiahe Ding Sihan Xu Yuchen Huang Run Peng Joyce Chai 49 0 0 22 Apr 2025
InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models Jinguo Zhu Weiyun Wang Zhe Chen Z. Liu Shenglong Ye ... D. Lin Yu Qiao Jifeng Dai Wenhai Wang W. Wang MLLM VLM 66 7 1 14 Apr 2025
ZS-VCOS: Zero-Shot Outperforms Supervised Video Camouflaged Object Segmentation Wenqi Guo Shan Du VLM 52 0 0 10 Apr 2025
How Can Objects Help Video-Language Understanding? Zitian Tang Shijie Wang Junho Cho Jaewook Yoo Chen Sun 40 0 0 10 Apr 2025
On-device Sora: Enabling Training-Free Diffusion-based Text-to-Video Generation for Mobile Devices Bosung Kim Kyuhwan Lee Isu Jeong Jungmin Cheon Yeojin Lee Seulki Lee VGen 45 1 0 31 Mar 2025
From Head to Tail: Towards Balanced Representation in Large Vision-Language Models through Adaptive Data Calibration Mingyang Song Xiaoye Qu Jiawei Zhou Yu-Xi Cheng VLM 50 1 0 17 Mar 2025
Large-scale Pre-training for Grounded Video Caption Generation Evangelos Kazakos Cordelia Schmid Josef Sivic 54 0 0 13 Mar 2025
Learning to Inference Adaptively for Multimodal Large Language Models Zhuoyan Xu Khoi Duc Nguyen Preeti Mukherjee Saurabh Bagchi Somali Chaterji Yingyu Liang Yin Li LRM 42 1 0 13 Mar 2025
Referring to Any Person Qing Jiang Lin Wu Zhaoyang Zeng Tianhe Ren Yuda Xiong Yihao Chen Qin Liu Lei Zhang 113 0 0 11 Mar 2025
VisRL: Intention-Driven Visual Perception via Reinforced Reasoning Zhangquan Chen Xufang Luo Dongsheng Li OffRL LRM 64 3 0 10 Mar 2025
Think Before You Segment: High-Quality Reasoning Segmentation with GPT Chain of Thoughts Shiu-hong Kao Yu-Wing Tai Chi-Keung Tang LRM MLLM 52 0 0 10 Mar 2025
Filter Images First, Generate Instructions Later: Pre-Instruction Data Selection for Visual Instruction Tuning Bardia Safaei Faizan Siddiqui Jiacong Xu Vishal M. Patel Shao-Yuan Lo VLM 113 0 0 10 Mar 2025
Adversarial Training for Multimodal Large Language Models against Jailbreak Attacks Liming Lu Shuchao Pang Siyuan Liang Haotian Zhu Xiyu Zeng Aishan Liu Yunhuai Liu Yongbin Zhou AAML 49 1 0 05 Mar 2025
M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performance Qingpei Guo Kaiyou Song Zipeng Feng Ziping Ma Qinglong Zhang ... Yunxiao Sun Tai-WeiChang Jingdong Chen Ming Yang Jun Zhou MLLM VLM 82 3 0 26 Feb 2025
MQADet: A Plug-and-Play Paradigm for Enhancing Open-Vocabulary Object Detection via Multimodal Question Answering Caixiong Li Xiongwei Zhao Jinhang Zhang Xing Zhang Qihao Sun Zhou Wu ObjD MLLM VLM 51 0 0 23 Feb 2025
LOVA3: Learning to Visual Question Answering, Asking and Assessment Henry Hengyuan Zhao Pan Zhou Difei Gao Zechen Bai Mike Zheng Shou 77 8 0 21 Feb 2025
Contrastive Localized Language-Image Pre-Training Hong-You Chen Zhengfeng Lai H. Zhang X. Wang Marcin Eichner Keen You Meng Cao Bowen Zhang Y. Yang Zhe Gan CLIP VLM 65 7 0 20 Feb 2025
Magma: A Foundation Model for Multimodal AI Agents Jianwei Yang Reuben Tan Qianhui Wu Ruijie Zheng Baolin Peng ... Seonghyeon Ye Joel Jang Yuquan Deng Lars Liden Jianfeng Gao VLM AI4TS 107 9 0 18 Feb 2025
Vision-Language In-Context Learning Driven Few-Shot Visual Inspection Model Shiryu Ueno Yoshikazu Hayashi Shunsuke Nakatsuka Yusei Yamada Hiroaki Aizawa K. Kato MLLM VLM 89 0 0 13 Feb 2025
InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling Yi Wang Xinhao Li Ziang Yan Yinan He Jiashuo Yu ... Kai Chen Wenhai Wang Yu Qiao Yali Wang Limin Wang 76 19 0 21 Jan 2025
Myriad: Large Multimodal Model by Applying Vision Experts for Industrial Anomaly Detection Yuanze Li Haolin Wang Shihao Yuan Ming-Yu Liu Debin Zhao Yiwen Guo Chen Xu Guangming Shi Wangmeng Zuo 79 28 0 20 Jan 2025
Omni-RGPT: Unifying Image and Video Region-level Understanding via Token Marks Miran Heo Min-Hung Chen De-An Huang Sifei Liu Subhashree Radhakrishnan Seon Joo Kim Yu-Chun Wang Ryo Hachiuma ObjD VLM 130 2 0 14 Jan 2025
MaPPER: Multimodal Prior-guided Parameter Efficient Tuning for Referring Expression Comprehension Ting Liu Zunnan Xu Yue Hu Liangtao Shi Zhiqiang Wang Quanjun Yin 57 2 0 03 Jan 2025
VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Tasks Jiannan Wu Muyan Zhong Sen Xing Zeqiang Lai Zhaoyang Liu ... Lewei Lu Tong Lu Ping Luo Yu Qiao Jifeng Dai MLLM VLM LRM 91 46 0 03 Jan 2025
VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM Yuqian Yuan Hang Zhang Wentong Li Zesen Cheng Boqiang Zhang ... Deli Zhao Wenqiao Zhang Yueting Zhuang Jianke Zhu Lidong Bing 70 5 0 31 Dec 2024
Nullu: Mitigating Object Hallucinations in Large Vision-Language Models via HalluSpace Projection Le Yang Ziwei Zheng Boxu Chen Zhengyu Zhao Chenhao Lin Chao Shen VLM 138 3 0 18 Dec 2024
Olympus: A Universal Task Router for Computer Vision Tasks Yuanze Lin Yunsheng Li Dongdong Chen Weijian Xu Ronald Clark Philip H. S. Torr VLM ObjD 147 0 0 12 Dec 2024
Efficient and Comprehensive Feature Extraction in Large Vision-Language Model for Clinical Pathology Analysis Shengxuming Zhang Weihan Li Tianhong Gao Jiacong Hu Haoming Luo Mingli Song Xiuming Zhang Mingli Song Zunlei Feng LM&MA 101 0 0 12 Dec 2024
Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Grounding Zilin Du Haoxin Li Jianfei Yu Boyang Li 111 0 0 01 Dec 2024
ROSE: Revolutionizing Open-Set Dense Segmentation with Patch-Wise Perceptual Large Multimodal Model Kunyang Han Yibo Hu Mengxue Qu Hailin Shi Yao Zhao Y. X. Wei MLLM VLM 3DV 83 1 0 29 Nov 2024
ChatRex: Taming Multimodal LLM for Joint Perception and Understanding Qing Jiang Gen Luo Yuqin Yang Yuda Xiong Yihao Chen Zhaoyang Zeng Tianhe Ren Lei Zhang VLM LRM 105 6 0 27 Nov 2024
Beyond Walking: A Large-Scale Image-Text Benchmark for Text-based Person Anomaly Search Shuyu Yang Yaxiong Wang Li Zhu Zhedong Zheng 91 2 0 26 Nov 2024
Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens Zhangqi Jiang Junkai Chen Beier Zhu Tingjin Luo Yankun Shen Xu Yang 98 4 0 23 Nov 2024
GeoGround: A Unified Large Vision-Language Model for Remote Sensing Visual Grounding Y. Zhou Mengcheng Lan Xiang Li Yiping Ke Xue Jiang Litong Feng Qingyun Li Xue Yang Wayne Zhang ObjD VLM 112 4 0 16 Nov 2024
VideoGLaMM: A Large Multimodal Model for Pixel-Level Visual Grounding in Videos Shehan Munasinghe Hanan Gani Wenqi Zhu Jiale Cao Eric P. Xing F. Khan Salman Khan MLLM VGen VLM 44 6 0 07 Nov 2024
GiVE: Guiding Visual Encoder to Perceive Overlooked Information Junjie Li Jianghong Ma Xiaofeng Zhang Yuhang Li Jianyang Shi 23 0 0 26 Oct 2024
Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities Zheyuan Zhang Fengyuan Hu Jayjun Lee Freda Shi Parisa Kordjamshidi Joyce Chai Ziqiao Ma 51 11 0 22 Oct 2024
PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction Long Xing Qidong Huang Xiaoyi Dong Jiajie Lu Pan Zhang ... Yuhang Cao Conghui He Jiaqi Wang Feng Wu Dahua Lin VLM 45 26 0 22 Oct 2024
TransAgent: Transfer Vision-Language Foundation Models with Heterogeneous Agent Collaboration Yiwei Guo Shaobin Zhuang Kunchang Li Yu Qiao Yali Wang VLM CLIP 21 0 0 16 Oct 2024
UnSeg: One Universal Unlearnable Example Generator is Enough against All Image Segmentation Ye Sun Hao Zhang Tiehua Zhang Xingjun Ma Yu-Gang Jiang VLM 32 3 0 13 Oct 2024
MRAG-Bench: Vision-Centric Evaluation for Retrieval-Augmented Multimodal Models Wenbo Hu Jia-Chen Gu Zi-Yi Dou Mohsen Fayyaz Pan Lu Kai-Wei Chang Nanyun Peng VLM 61 4 0 10 Oct 2024
Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents Boyu Gou Ruohan Wang Boyuan Zheng Yanan Xie Cheng Chang Yiheng Shu Huan Sun Yu Su LM&Ro LLMAG 76 48 0 07 Oct 2024
When SAM2 Meets Video Camouflaged Object Segmentation: A Comprehensive Evaluation and Adaptation Yuli Zhou Guolei Sun Yawei Li Guo-Sen Xie Luca Benini Ender Konukoglu 22 4 0 27 Sep 2024
LLaVA-3D: A Simple yet Effective Pathway to Empowering LMMs with 3D-awareness Chenming Zhu Tai Wang Wenwei Zhang Jiangmiao Pang Xihui Liu 108 29 0 26 Sep 2024
FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension Junzhuo Liu X. Yang Weiwei Li Peng Wang ObjD 44 3 0 23 Sep 2024
SAM4MLLM: Enhance Multi-Modal Large Language Model for Referring Expression Segmentation Yi-Chia Chen Wei-Hua Li Cheng Sun Yu-Chiang Frank Wang Chu-Song Chen VLM 35 10 0 01 Sep 2024
IAA: Inner-Adaptor Architecture Empowers Frozen Large Language Model with Multimodal Capabilities Bin Wang Chunyu Xie Dawei Leng Yuhui Yin MLLM 45 1 0 23 Aug 2024