PIN: Positional Insert Unlocks Object Localisation Abilities in VLMs

13 February 2024

Papers citing "PIN: Positional Insert Unlocks Object Localisation Abilities in VLMs"

21 / 21 papers shown

Title
QID: Efficient Query-Informed ViTs in Data-Scarce Regimes for OCR-free Visual Document Understanding Binh M. Le Shaoyuan Xu Jinmiao Fu Zhishen Huang Moyan Li Yanhui Guo Hongdong Li Sameera Ramasinghe Bryan Wang 28 0 0 03 Apr 2025
SMILE: Infusing Spatial and Motion Semantics in Masked Video Learning Fida Mohammad Thoker Letian Jiang Chen Zhao Bernard Ghanem 50 0 0 01 Apr 2025
Parameter-Efficient Fine-Tuning for Foundation Models Dan Zhang Tao Feng Lilong Xue Yuandong Wang Yuxiao Dong J. Tang 37 6 0 23 Jan 2025
Visual Prompting with Iterative Refinement for Design Critique Generation Peitong Duan Chin-yi Chen Bjoern Hartmann Yang Li 71 0 0 22 Dec 2024
LocoMotion: Learning Motion-Focused Video-Language Representations Hazel Doughty Fida Mohammad Thoker Cees G. M. Snoek 27 2 0 15 Oct 2024
Why context matters in VQA and Reasoning: Semantic interventions for VLM input modalities Kenza Amara Lukas Klein Carsten T. Lüth Paul Jäger Hendrik Strobelt Mennatallah El-Assady 25 1 0 02 Oct 2024
Have Large Vision-Language Models Mastered Art History? Ombretta Strafforello Derya Soydaner Michiel Willems Anne-Sofie Maerten Stefanie De Winter CoGe VLM MLLM 21 0 0 05 Sep 2024
In-Context Learning Improves Compositional Understanding of Vision-Language Models Matteo Nulli Anesa Ibrahimi Avik Pal Hoshe Lee Ivona Najdenkoska VLM CoGe 30 0 0 22 Jul 2024
UICrit: Enhancing Automated Design Evaluation with a UICritique Dataset Peitong Duan Chin-yi Chen Gang Li Bjoern Hartmann Yang Li 42 7 0 11 Jul 2024
MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning Jun Chen Deyao Zhu Xiaoqian Shen Xiang Li Zechun Liu Pengchuan Zhang Raghuraman Krishnamoorthi Vikas Chandra Yunyang Xiong Mohamed Elhoseiny MLLM 154 280 0 14 Oct 2023
mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality Qinghao Ye Haiyang Xu Guohai Xu Jiabo Ye Ming Yan ... Junfeng Tian Qiang Qi Ji Zhang Feiyan Huang Jingren Zhou VLM MLLM 203 883 0 27 Apr 2023
BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models Junnan Li Dongxu Li Silvio Savarese Steven C. H. Hoi VLM MLLM 244 4,186 0 30 Jan 2023
MaPLe: Multi-modal Prompt Learning Muhammad Uzair Khattak H. Rasheed Muhammad Maaz Salman Khan F. Khan VPVLM VLM 186 521 0 06 Oct 2022
Prompt-aligned Gradient for Prompt Tuning Beier Zhu Yulei Niu Yucheng Han Yuehua Wu Hanwang Zhang VLM 175 263 0 30 May 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 382 4,010 0 28 Jan 2022
P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks Xiao Liu Kaixuan Ji Yicheng Fu Weng Lam Tam Zhengxiao Du Zhilin Yang Jie Tang VLM 236 780 0 14 Oct 2021
Pix2seq: A Language Modeling Framework for Object Detection Ting-Li Chen Saurabh Saxena Lala Li David J. Fleet Geoffrey E. Hinton MLLM ViT VLM 233 341 0 22 Sep 2021
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 322 2,249 0 02 Sep 2021
The Power of Scale for Parameter-Efficient Prompt Tuning Brian Lester Rami Al-Rfou Noah Constant VPVLM 278 3,784 0 18 Apr 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 3,683 0 11 Feb 2021
Simple Copy-Paste is a Strong Data Augmentation Method for Instance Segmentation Golnaz Ghiasi Yin Cui A. Srinivas Rui Qian Tsung-Yi Lin E. D. Cubuk Quoc V. Le Barret Zoph ISeg 223 962 0 13 Dec 2020