SimVLM: Simple Visual Language Model Pretraining with Weak Supervision

24 August 2021

Papers citing "SimVLM: Simple Visual Language Model Pretraining with Weak Supervision"

50 / 565 papers shown

Title
Semantics-enhanced Cross-modal Masked Image Modeling for Vision-Language Pre-training Haowei Liu Yaya Shi Haiyang Xu Chunfen Yuan Qinghao Ye ... Mingshi Yan Ji Zhang Fei Huang Bing Li Weiming Hu VLM 22 0 0 01 Mar 2024
Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlook Xingchen Zou Yibo Yan Xixuan Hao Yuehong Hu Haomin Wen ... Junbo Zhang Yong Li Tianrui Li Yu Zheng Yuxuan Liang HAI AI4TS 43 35 0 29 Feb 2024
VIXEN: Visual Text Comparison Network for Image Difference Captioning Alexander Black Jing Shi Yifei Fai Tu Bui John Collomosse 39 5 0 29 Feb 2024
BLO-SAM: Bi-level Optimization Based Overfitting-Preventing Finetuning of SAM Li Zhang Youwei Liang Ruiyi Zhang Amirhosein Javadi Pengtao Xie VLM 14 8 0 26 Feb 2024
Bridging the Gap between 2D and 3D Visual Question Answering: A Fusion Approach for 3D VQA Wentao Mo Yang Liu 16 5 0 24 Feb 2024
PQA: Zero-shot Protein Question Answering for Free-form Scientific Enquiry with Large Language Models Eli M. Carrami Sahand Sharifzadeh 19 2 0 21 Feb 2024
Exploring the Frontier of Vision-Language Models: A Survey of Current Methodologies and Future Directions Akash Ghosh Arkadeep Acharya Sriparna Saha Vinija Jain Aman Chadha VLM 38 9 0 20 Feb 2024
How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Prompts Yusu Qian Haotian Zhang Yinfei Yang Zhe Gan 64 26 0 20 Feb 2024
PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter Junfei Xiao Zheng Xu Alan L. Yuille Shen Yan Boyu Wang 19 2 0 16 Feb 2024
ProtChatGPT: Towards Understanding Proteins with Large Language Models Chao Wang Hehe Fan Ruijie Quan Yi Yang 26 12 0 15 Feb 2024
PIN: Positional Insert Unlocks Object Localisation Abilities in VLMs Michael Dorkenwald Nimrod Barazani Cees G. M. Snoek Yuki M. Asano VLM MLLM 17 12 0 13 Feb 2024
Cacophony: An Improved Contrastive Audio-Text Model Ge Zhu Jordan Darefsky Zhiyao Duan AuLLM 33 11 0 10 Feb 2024
Large Language Models for Captioning and Retrieving Remote Sensing Images João Daniel Silva João Magalhães D. Tuia Bruno Martins 27 29 0 09 Feb 2024
Convincing Rationales for Visual Question Answering Reasoning Kun Li G. Vosselman Michael Ying Yang 31 1 0 06 Feb 2024
Image Fusion via Vision-Language Model Zixiang Zhao Lilun Deng Haowen Bai Yukun Cui Zhipeng Zhang ... Haotong Qin Dongdong Chen Jiangshe Zhang Peng Wang Luc Van Gool VLM 24 18 0 03 Feb 2024
Proximity QA: Unleashing the Power of Multi-Modal Large Language Models for Spatial Proximity Analysis Jianing Li Xi Nan Ming Lu Li Du Shanghang Zhang 37 1 0 31 Jan 2024
Multimodal Pathway: Improve Transformers with Irrelevant Data from Other Modalities Yiyuan Zhang Xiaohan Ding Kaixiong Gong Yixiao Ge Ying Shan Xiangyu Yue ViT 16 7 0 25 Jan 2024
WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models Hongliang He Wenlin Yao Kaixin Ma Wenhao Yu Yong Dai Hongming Zhang Zhenzhong Lan Dong Yu LLMAG 22 48 0 25 Jan 2024
Exploring scalable medical image encoders beyond text supervision Fernando Pérez-García Harshita Sharma Sam Bond-Taylor Kenza Bouzid Valentina Salvatelli ... Maria T. A. Wetscherek Noel Codella Stephanie L. Hyland Javier Alvarez-Valle Ozan Oktay LM&MA MedIm 35 9 0 19 Jan 2024
MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer Changyao Tian Xizhou Zhu Yuwen Xiong Weiyun Wang Zhe Chen ... Tong Lu Jie Zhou Hongsheng Li Yu Qiao Jifeng Dai AuLLM 80 40 0 18 Jan 2024
Question-Answer Cross Language Image Matching for Weakly Supervised Semantic Segmentation Songhe Deng Wei Zhuo Jinheng Xie Linlin Shen VLM 13 6 0 18 Jan 2024
Efficient Vision-and-Language Pre-training with Text-Relevant Image Patch Selection Wei Ye Chaoya Jiang Haiyang Xu Chenhao Ye Chenliang Li Mingshi Yan Shikun Zhang Songhang Huang Fei Huang VLM 21 0 0 11 Jan 2024
Mining Fine-Grained Image-Text Alignment for Zero-Shot Captioning via Text-Only Training Longtian Qiu Shan Ning Xuming He VLM 33 3 0 04 Jan 2024
SyCoCa: Symmetrizing Contrastive Captioners with Attentive Masking for Multimodal Alignment Ziping Ma Furong Xu Jian Liu Ming Yang Qingpei Guo VLM 28 3 0 04 Jan 2024
Data-Centric Foundation Models in Computational Healthcare: A Survey Yunkun Zhang Jin Gao Zheling Tan Lingfeng Zhou Kexin Ding Mu Zhou Shaoting Zhang Dequan Wang AI4CE 21 20 0 04 Jan 2024
GPT-4V(ision) is a Generalist Web Agent, if Grounded Boyuan Zheng Boyu Gou Jihyung Kil Huan Sun Yu-Chuan Su MLLM VLM LLMAG 32 79 0 03 Jan 2024
Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Action Jiasen Lu Christopher Clark Sangho Lee Zichen Zhang Savya Khosla Ryan Marten Derek Hoiem Aniruddha Kembhavi VLM MLLM 27 143 0 28 Dec 2023
Cycle-Consistency Learning for Captioning and Grounding Ning Wang Jiajun Deng Mingbo Jia ObjD 15 7 0 23 Dec 2023
LingoQA: Video Question Answering for Autonomous Driving Ana-Maria Marcu Long Chen Jan Hünermann Alice Karnsund Benoît Hanotte ... Vijay Badrinarayanan Alex Kendall Jamie Shotton Elahe Arani Oleg Sinavski 21 31 0 21 Dec 2023
InfoVisDial: An Informative Visual Dialogue Dataset by Bridging Large Multimodal and Language Models Bingbing Wen Zhengyuan Yang Jianfeng Wang Zhe Gan Bill Howe Lijuan Wang MLLM 28 1 0 21 Dec 2023
ECAMP: Entity-centered Context-aware Medical Vision Language Pre-training Rongsheng Wang Qingsong Yao Haoran Lai Zhiyang He Xiaodong Tao Zihang Jiang S.Kevin Zhou VLM MedIm 23 4 0 20 Dec 2023
Data-Efficient Multimodal Fusion on a Single GPU Noël Vouitsis Zhaoyan Liu S. Gorti Valentin Villecroze Jesse C. Cresswell Guangwei Yu G. Loaiza-Ganem M. Volkovs 29 1 0 15 Dec 2023
ToViLaG: Your Visual-Language Generative Model is Also An Evildoer Xinpeng Wang Xiaoyuan Yi Han Jiang Shanlin Zhou Zhihua Wei Xing Xie 17 12 0 13 Dec 2023
RCA-NOC: Relative Contrastive Alignment for Novel Object Captioning Jiashuo Fan Yaoyuan Liang Leyao Liu Shao-Lun Huang Lei Zhang 25 2 0 11 Dec 2023
MAFA: Managing False Negatives for Vision-Language Pre-training Jaeseok Byun Dohoon Kim Taesup Moon VLM 13 3 0 11 Dec 2023
SYNC-CLIP: Synthetic Data Make CLIP Generalize Better in Data-Limited Scenarios Mushui Liu Weijie He Ziqian Lu Yunlong Yu VLM 11 1 0 06 Dec 2023
Training on Synthetic Data Beats Real Data in Multimodal Relation Extraction Zilin Du Haoxin Li Xu Guo Boyang Li 25 1 0 05 Dec 2023
Lenna: Language Enhanced Reasoning Detection Assistant Fei Wei Xinyu Zhang Ailing Zhang Bo-Wen Zhang Xiangxiang Chu MLLM LRM 17 21 0 05 Dec 2023
Object Recognition as Next Token Prediction Kaiyu Yue Borchun Chen Jonas Geiping Hengduo Li Tom Goldstein Ser-Nam Lim 14 8 0 04 Dec 2023
Bootstrapping Interactive Image-Text Alignment for Remote Sensing Image Captioning Cong Yang Zuchao Li Lefei Zhang 16 22 0 02 Dec 2023
Brainformer: Mimic Human Visual Brain Functions to Machine Vision Models via fMRI Xuan-Bac Nguyen Xin Li Pawan Sinha Samee U. Khan Khoa Luu ViT MedIm 22 0 0 30 Nov 2023
PALM: Predicting Actions through Language Models Sanghwan Kim Daoji Huang Yongqin Xian Otmar Hilliges Luc Van Gool Xi Wang VLM 16 4 0 29 Nov 2023
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI Xiang Yue Yuansheng Ni Kai Zhang Tianyu Zheng Ruoqi Liu ... Yibo Liu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen OSLM ELM VLM 24 723 0 27 Nov 2023
Tessel: Boosting Distributed Execution of Large DNN Models via Flexible Schedule Search Zhiqi Lin Youshan Miao Guanbin Xu Cheng Li Olli Saarikivi Saeed Maleki Fan Yang 4 1 0 26 Nov 2023
Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding Hoang-Quan Nguyen Thanh-Dat Truong Xuan-Bac Nguyen Ashley Dowling Xin Li Khoa Luu VLM 11 19 0 26 Nov 2023
ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation Yangyi Chen Xingyao Wang Manling Li Derek Hoiem Heng Ji 17 5 0 22 Nov 2023
Multimodal Large Language Models: A Survey Jiayang Wu Wensheng Gan Zefeng Chen Shicheng Wan Philip S. Yu 14 160 0 22 Nov 2023
From Wrong To Right: A Recursive Approach Towards Vision-Language Explanation Jiaxin Ge Sanjay Subramanian Trevor Darrell Boyi Li LRM 10 4 0 21 Nov 2023
A Survey on Multimodal Large Language Models for Autonomous Driving Can Cui Yunsheng Ma Xu Cao Wenqian Ye Yang Zhou ... Xinrui Yan Shuqi Mei Jianguo Cao Ziran Wang Chao Zheng 24 248 0 21 Nov 2023
DRESS: Instructing Large Vision-Language Models to Align and Interact with Humans via Natural Language Feedback Yangyi Chen Karan Sikka Michael Cogswell Heng Ji Ajay Divakaran 24 56 0 16 Nov 2023