Title
Kosmos-2: Grounding Multimodal Large Language Models to the World Zhiliang Peng Wenhui Wang Li Dong Y. Hao Shaohan Huang Shuming Ma Furu Wei MLLM ObjD VLM 14 688 0 26 Jun 2023
DesCo: Learning Object Recognition with Rich Language Descriptions Liunian Harold Li Zi-Yi Dou Nanyun Peng Kai-Wei Chang ObjD VLM 11 20 0 24 Jun 2023
World-to-Words: Grounded Open Vocabulary Acquisition through Fast Mapping in Vision-Language Models Ziqiao Ma Jiayi Pan J. Chai ObjD VLM 21 8 0 14 Jun 2023
Recent Advances of Local Mechanisms in Computer Vision: A Survey and Outlook of Recent Work Qiangchang Wang Yilong Yin 21 0 0 02 Jun 2023
ConES: Concept Embedding Search for Parameter Efficient Tuning Large Vision Language Models Huahui Yi Ziyuan Qin Wei Xu Miaotian Guo Kun Wang Shaoting Zhang Kang Li Qicheng Lao VLM 19 0 0 30 May 2023
Gender Biases in Automatic Evaluation Metrics for Image Captioning Haoyi Qiu Zi-Yi Dou Tianlu Wang Asli Celikyilmaz Nanyun Peng EGVM 19 14 0 24 May 2023
Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining Emanuele Bugliarello Aida Nematzadeh Lisa Anne Hendricks SSL 22 5 0 23 May 2023
Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization Zimeng Qiu Quanqi Hu Zhuoning Yuan Denny Zhou Lijun Zhang Tianbao Yang 27 17 0 19 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 16 113 0 18 May 2023
Improved baselines for vision-language pre-training Enrico Fini Pietro Astolfi Adriana Romero Soriano Jakob Verbeek M. Drozdzal SSL CLIP VLM 45 22 0 15 May 2023
A Memory Model for Question Answering from Streaming Data Supported by Rehearsal and Anticipation of Coreference Information Vladimir Araujo Alvaro Soto Marie-Francine Moens KELM 20 2 0 12 May 2023
Measuring Progress in Fine-grained Vision-and-Language Understanding Emanuele Bugliarello Laurent Sartran Aishwarya Agrawal Lisa Anne Hendricks Aida Nematzadeh VLM 20 22 0 12 May 2023
COLA: A Benchmark for Compositional Text-to-image Retrieval Arijit Ray Filip Radenovic Abhimanyu Dubey Bryan A. Plummer Ranjay Krishna Kate Saenko CoGe VLM 30 34 0 05 May 2023
OmniLabel: A Challenging Benchmark for Language-Based Object Detection S. Schulter G. VijayKumarB. Yumin Suh Konstantinos M. Dafnis Zhixing Zhang Shiyu Zhao Dimitris N. Metaxas ObjD 22 11 0 22 Apr 2023
What does CLIP know about a red circle? Visual prompt engineering for VLMs Aleksandar Shtedritski Christian Rupprecht Andrea Vedaldi VLM MLLM 21 137 0 13 Apr 2023
DetCLIPv2: Scalable Open-Vocabulary Object Detection Pre-training via Word-Region Alignment Lewei Yao Jianhua Han Xiaodan Liang Danqian Xu Wei Zhang Zhenguo Li Hang Xu VLM ObjD CLIP 28 72 0 10 Apr 2023
Vision-Language Models for Vision Tasks: A Survey Jingyi Zhang Jiaxing Huang Sheng Jin Shijian Lu VLM 34 451 0 03 Apr 2023
Equivariant Similarity for Vision-Language Foundation Models Tan Wang Kevin Qinghong Lin Linjie Li Chung-Ching Lin Zhengyuan Yang Hanwang Zhang Zicheng Liu Lijuan Wang CoGe 33 44 0 25 Mar 2023
Three ways to improve feature alignment for open vocabulary detection Relja Arandjelović A. Andonian A. Mensch Olivier J. Hénaff Jean-Baptiste Alayrac Andrew Zisserman VLM ObjD 28 19 0 23 Mar 2023
Retrieving Multimodal Information for Augmented Generation: A Survey Ruochen Zhao Hailin Chen Weishi Wang Fangkai Jiao Do Xuan Long ... Bosheng Ding Xiaobao Guo Minzhi Li Xingxuan Li Shafiq R. Joty 13 80 0 20 Mar 2023
FAME-ViL: Multi-Tasking Vision-Language Model for Heterogeneous Fashion Tasks Xiaoping Han Xiatian Zhu Licheng Yu Li Zhang Yi-Zhe Song Tao Xiang VLM 11 38 0 04 Mar 2023
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning Antoine Yang Arsha Nagrani Paul Hongsuck Seo Antoine Miech Jordi Pont-Tuset Ivan Laptev Josef Sivic Cordelia Schmid AI4TS VLM 18 219 0 27 Feb 2023
Advancing Radiograph Representation Learning with Masked Record Modeling Hong-Yu Zhou Chenyu Lian Lian-cheng Wang Yizhou Yu MedIm 17 54 0 30 Jan 2023
See, Think, Confirm: Interactive Prompting Between Vision and Language Models for Knowledge-based Visual Reasoning Zhenfang Chen Qinhong Zhou Yikang Shen Yining Hong Hao Zhang Chuang Gan LRM VLM 29 35 0 12 Jan 2023
Generalized Decoding for Pixel, Image, and Language Xueyan Zou Zi-Yi Dou Jianwei Yang Zhe Gan Linjie Li ... Lu Yuan Nanyun Peng Lijuan Wang Yong Jae Lee Jianfeng Gao VLM MLLM ObjD 13 238 0 21 Dec 2022
Exploring Discrete Diffusion Models for Image Captioning Zixin Zhu Yixuan Wei Jianfeng Wang Zhe Gan Zheng-Wei Zhang Le Wang G. Hua Lijuan Wang Zicheng Liu Han Hu DiffM VLM 21 16 0 21 Nov 2022
Leveraging per Image-Token Consistency for Vision-Language Pre-training Yunhao Gou Tom Ko Hansi Yang James T. Kwok Yu Zhang Mingxuan Wang VLM 14 9 0 20 Nov 2022
ConStruct-VL: Data-Free Continual Structured VL Concepts Learning James Smith Paola Cascante-Bonilla Assaf Arbelle Donghyun Kim Rameswar Panda David D. Cox Diyi Yang Z. Kira Rogerio Feris Leonid Karlinsky VLM 33 20 0 17 Nov 2022
CAMANet: Class Activation Map Guided Attention Network for Radiology Report Generation Jun Wang A. Bhalerao Terry Yin Simon See Yulan He MedIm 17 14 0 02 Nov 2022
Perceptual Grouping in Contrastive Vision-Language Models Kanchana Ranasinghe Brandon McKinzie S. S. Ravi Yinfei Yang Alexander Toshev Jonathon Shlens VLM 19 50 0 18 Oct 2022
VoLTA: Vision-Language Transformer with Weakly-Supervised Local-Feature Alignment Shraman Pramanick Li Jing Sayan Nag Jiachen Zhu Hardik Shah Yann LeCun Ramalingam Chellappa 24 21 0 09 Oct 2022
A New Path: Scaling Vision-and-Language Navigation with Synthetic Instructions and Imitation Learning Aishwarya Kamath Peter Anderson Su Wang Jing Yu Koh Alexander Ku Austin Waters Yinfei Yang Jason Baldridge Zarana Parekh LM&Ro 15 45 0 06 Oct 2022
Efficient Vision-Language Pretraining with Visual Concepts and Hierarchical Alignment Mustafa Shukor Guillaume Couairon Matthieu Cord VLM CLIP 19 26 0 29 Aug 2022
Self-supervised Multi-modal Training from Uncurated Image and Reports Enables Zero-shot Oversight Artificial Intelligence in Radiology Sangjoon Park Eunha Lee Kyung Sook Shin Jeonghyeon Lee Jong Chul Ye 15 2 0 10 Aug 2022
Dynamic Contrastive Distillation for Image-Text Retrieval Jun Rao Liang Ding Shuhan Qi Meng Fang Yang Liu Liqiong Shen Dacheng Tao VLM 44 30 0 04 Jul 2022
Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Tasks Jiasen Lu Christopher Clark Rowan Zellers Roozbeh Mottaghi Aniruddha Kembhavi ObjD VLM MLLM 36 391 0 17 Jun 2022
BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation Junnan Li Dongxu Li Caiming Xiong S. Hoi MLLM BDL VLM CLIP 388 4,010 0 28 Jan 2022
Pix2seq: A Language Modeling Framework for Object Detection Ting-Li Chen Saurabh Saxena Lala Li David J. Fleet Geoffrey E. Hinton MLLM ViT VLM 233 341 0 22 Sep 2021
How Much Can CLIP Benefit Vision-and-Language Tasks? Sheng Shen Liunian Harold Li Hao Tan Mohit Bansal Anna Rohrbach Kai-Wei Chang Z. Yao Kurt Keutzer CLIP VLM MLLM 185 403 0 13 Jul 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 3,683 0 11 Feb 2021
Unifying Vision-and-Language Tasks via Text Generation Jaemin Cho Jie Lei Hao Tan Mohit Bansal MLLM 249 518 0 04 Feb 2021
Unified Vision-Language Pre-Training for Image Captioning and VQA Luowei Zhou Hamid Palangi Lei Zhang Houdong Hu Jason J. Corso Jianfeng Gao MLLM VLM 250 922 0 24 Sep 2019
GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding Alex Jinpeng Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel R. Bowman ELM 294 6,927 0 20 Apr 2018