SimVLM: Simple Visual Language Model Pretraining with Weak Supervision

24 August 2021

Papers citing "SimVLM: Simple Visual Language Model Pretraining with Weak Supervision"

50 / 565 papers shown

Title
Joint Adaptive Representations for Image-Language Learning A. Piergiovanni A. Angelova VLM 14 0 0 31 May 2023
LMCap: Few-shot Multilingual Image Captioning by Retrieval Augmented Language Model Prompting R. Ramos Bruno Martins Desmond Elliott VLM 13 16 0 31 May 2023
PaLI-X: On Scaling up a Multilingual Vision and Language Model Xi Chen Josip Djolonga Piotr Padlewski Basil Mustafa Soravit Changpinyo ... Mojtaba Seyedhosseini A. Angelova Xiaohua Zhai N. Houlsby Radu Soricut VLM 33 186 0 29 May 2023
VAST: A Vision-Audio-Subtitle-Text Omni-Modality Foundation Model and Dataset Sihan Chen Handong Li Qunbo Wang Zijia Zhao Ming-Ting Sun Xinxin Zhu J. Liu 24 95 0 29 May 2023
Deeply Coupled Cross-Modal Prompt Learning Xuejing Liu Wei Tang Jinghui Lu Rui Zhao Zhaojun Guo Fei Tan VLM 6 17 0 29 May 2023
FuseCap: Leveraging Large Language Models for Enriched Fused Image Captions Noam Rotstein David Bensaid Shaked Brody Roy Ganz Ron Kimmel VLM 11 26 0 28 May 2023
PuMer: Pruning and Merging Tokens for Efficient Vision Language Models Qingqing Cao Bhargavi Paranjape Hannaneh Hajishirzi MLLM VLM 8 20 0 27 May 2023
BiomedGPT: A Unified and Generalist Biomedical Generative Pre-trained Transformer for Vision, Language, and Multimodal Tasks Kai Zhang Jun Yu Eashan Adhikarla Rong-Er Zhou Zhilin Yan ... Xun Chen Yong Chen Quanzheng Li Hongfang Liu Lichao Sun LM&MA MedIm 15 149 0 26 May 2023
LANISTR: Multimodal Learning from Structured and Unstructured Data Sayna Ebrahimi Sercan Ö. Arik Yihe Dong Tomas Pfister 14 4 0 26 May 2023
HAAV: Hierarchical Aggregation of Augmented Views for Image Captioning Chia-Wen Kuo Z. Kira 25 21 0 25 May 2023
PaCE: Unified Multi-modal Dialogue Pre-training with Progressive and Compositional Experts Yunshui Li Binyuan Hui Zhichao Yin Min Yang Fei Huang Yongbin Li MoE 6 18 0 24 May 2023
UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning Ahmed Masry P. Kavehzadeh Do Xuan Long Enamul Hoque Shafiq R. Joty LRM 19 100 0 24 May 2023
Training Transitive and Commutative Multimodal Transformers with LoReTTa Manuel Tran Yashin Dicente Cid Amal Lahiani Fabian J. Theis Tingying Peng Eldad Klaiman 11 2 0 23 May 2023
S-CLIP: Semi-supervised Vision-Language Learning using Few Specialist Captions Sangwoo Mo Minkyu Kim Kyungmin Lee Jinwoo Shin VLM CLIP 31 21 0 23 May 2023
Can Language Models Understand Physical Concepts? Lei Li Jingjing Xu Qingxiu Dong Ce Zheng Qi Liu Lingpeng Kong Xu Sun ALM 17 17 0 23 May 2023
i-Code Studio: A Configurable and Composable Framework for Integrative AI Yuwei Fang Mahmoud Khademi Chenguang Zhu Ziyi Yang Reid Pryzant ... Yao Qian Takuya Yoshioka Lu Yuan Michael Zeng Xuedong Huang 25 2 0 23 May 2023
UNIMO-3: Multi-granularity Interaction for Vision-Language Representation Learning Hao-Yu Yang Can Gao Hao Liu Xinyan Xiao Yanyan Zhao Bing Qin 18 2 0 23 May 2023
EDIS: Entity-Driven Image Search over Multimodal Web Content Siqi Liu Weixi Feng Tsu-jui Fu Wenhu Chen W. Wang VLM 32 9 0 23 May 2023
ReSee: Responding through Seeing Fine-grained Visual Knowledge in Open-domain Dialogue Haoqin Tu Yitong Li Fei Mi Zhongliang Yang 28 4 0 23 May 2023
Enhance Reasoning Ability of Visual-Language Models via Large Language Models Yueting Yang Xintong Zhang Wenjuan Han VLM ReLM LRM 17 1 0 22 May 2023
VLAB: Enhancing Video Language Pre-training by Feature Adapting and Blending Xingjian He Sihan Chen Fan Ma Zhicheng Huang Xiaojie Jin Zikang Liu Dongmei Fu Yi Yang J. Liu Jiashi Feng VLM CLIP 15 17 0 22 May 2023
Album Storytelling with Iterative Story-aware Captioning and Large Language Models Munan Ning Yujia Xie Dongdong Chen Zeyin Song Lu Yuan Yonghong Tian QiXiang Ye Liuliang Yuan 11 8 0 22 May 2023
Gloss-Free End-to-End Sign Language Translation Kezhou Lin Xiaohan Wang Linchao Zhu Ke Sun Bang Zhang Yezhou Yang SLR 17 17 0 22 May 2023
Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approach Haoning Wu Erli Zhang Liang Liao Chaofeng Chen Jingwen Hou Annan Wang Wenxiu Sun Qiong Yan Weisi Lin 15 38 0 22 May 2023
i-Code V2: An Autoregressive Generation Framework over Vision, Language, and Speech Data Ziyi Yang Mahmoud Khademi Yichong Xu Reid Pryzant Yuwei Fang ... Yu Shi Lu Yuan Takuya Yoshioka Michael Zeng Xuedong Huang 14 2 0 21 May 2023
Pengi: An Audio Language Model for Audio Tasks Soham Deshmukh Benjamin Elizalde Rita Singh Huaming Wang MLLM AuLLM 25 155 0 19 May 2023
Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense Captioner Zikang Liu Sihan Chen Longteng Guo Handong Li Xingjian He J. Liu 6 1 0 19 May 2023
Surgical-VQLA: Transformer with Gated Vision-Language Embedding for Visual Question Localized-Answering in Robotic Surgery Long Bai Mobarakol Islam Lalithkumar Seenivasan Hongliang Ren 15 27 0 19 May 2023
Going Denser with Open-Vocabulary Part Segmentation Pei Sun Shoufa Chen Chenchen Zhu Fanyi Xiao Ping Luo Saining Xie Zhicheng Yan ObjD VLM 12 45 0 18 May 2023
ONE-PEACE: Exploring One General Representation Model Toward Unlimited Modalities Peng Wang Shijie Wang Junyang Lin Shuai Bai Xiaohuan Zhou Jingren Zhou Xinggang Wang Chang Zhou VLM MLLM ObjD 13 113 0 18 May 2023
Paxion: Patching Action Knowledge in Video-Language Foundation Models Zhenhailong Wang Ansel Blume Sha Li Genglin Liu Jaemin Cho Zineng Tang Mohit Bansal Heng Ji KELM VGen 9 26 0 18 May 2023
Rethinking Multimodal Content Moderation from an Asymmetric Angle with Mixed-modality Jialing Yuan Ye Yu Gaurav Mittal Matthew Hall Sandra Sajeev Mei Chen 12 8 0 17 May 2023
IMAD: IMage-Augmented multi-modal Dialogue Viktor Moskvoretskii Anton Frolov Denis Kuznetsov 12 4 0 17 May 2023
An Empirical Study on the Language Modal in Visual Question Answering Daowan Peng Wei Wei Xian-Ling Mao Yuanyuan Fu Dangyang Chen 24 4 0 17 May 2023
ULIP-2: Towards Scalable Multimodal Pre-training for 3D Understanding Le Xue Ning Yu Shu Zhen Zhang Artemis Panagopoulou Junnan Li ... Jiajun Wu Caiming Xiong Ran Xu Juan Carlos Niebles Silvio Savarese 8 113 0 14 May 2023
RC3: Regularized Contrastive Cross-lingual Cross-modal Pre-training Chulun Zhou Yunlong Liang Fandong Meng Jinan Xu Jinsong Su Jie Zhou VLM 16 4 0 13 May 2023
Simple Token-Level Confidence Improves Caption Correctness Suzanne Petryk Spencer Whitehead Joseph E. Gonzalez Trevor Darrell Anna Rohrbach Marcus Rohrbach 13 7 0 11 May 2023
Musketeer: Joint Training for Multi-task Vision Language Model with Task Explanation Prompts Zhaoyang Zhang Yantao Shen Kunyu Shi Zhaowei Cai Jun Fang Siqi Deng Hao-Yu Yang Davide Modolo Z. Tu Stefano Soatto VLM 19 2 0 11 May 2023
Region-Aware Pretraining for Open-Vocabulary Object Detection with Vision Transformers Dahun Kim A. Angelova Weicheng Kuo ObjD ViT VLM 17 71 0 11 May 2023
A Multi-Modal Context Reasoning Approach for Conditional Inference on Joint Textual and Visual Clues Yunxin Li Baotian Hu Xinyu Chen Yuxin Ding Lin Ma Min Zhang LRM 35 14 0 08 May 2023
Vision Language Pre-training by Contrastive Learning with Cross-Modal Similarity Regulation Chaoya Jiang Wei Ye Haiyang Xu Miang yan Shikun Zhang Jie Zhang Fei Huang VLM 19 14 0 08 May 2023
OpenViVQA: Task, Dataset, and Multimodal Fusion Models for Visual Question Answering in Vietnamese Nghia Hieu Nguyen Duong T.D. Vo Kiet Van Nguyen N. Nguyen 24 18 0 07 May 2023
Structure-CLIP: Towards Scene Graph Knowledge to Enhance Multi-modal Structured Representations Yufen Huang Jiji Tang Zhuo Chen Rongsheng Zhang Xinfeng Zhang ... Zeng Zhao Zhou Zhao Tangjie Lv Zhipeng Hu Wen Zhang VLM 12 20 0 06 May 2023
Making the Most of What You Have: Adapting Pre-trained Visual Language Models in the Low-data Regime Chuhan Zhang Antoine Miech Jiajun Shen Jean-Baptiste Alayrac Pauline Luc VLM VPVLM 36 2 0 03 May 2023
An Empirical Study of Multimodal Model Merging Yi-Lin Sung Linjie Li Kevin Qinghong Lin Zhe Gan Mohit Bansal Lijuan Wang MoMe 6 40 0 28 Apr 2023
Towards Robust Text-Prompted Semantic Criterion for In-the-Wild Video Quality Assessment Haoning Wu Liang Liao Annan Wang Chaofeng Chen Jingwen Hou Wenxiu Sun Qiong Yan Weisi Lin 54 15 0 28 Apr 2023
Retrieval-based Knowledge Augmented Vision Language Pre-training Jiahua Rao Zifei Shan Long Liu Yao Zhou Yuedong Yang VLM 78 13 0 27 Apr 2023
A Cookbook of Self-Supervised Learning Randall Balestriero Mark Ibrahim Vlad Sobal Ari S. Morcos Shashank Shekhar ... Pierre Fernandez Amir Bar Hamed Pirsiavash Yann LeCun Micah Goldblum SyDa FedML SSL 31 270 0 24 Apr 2023
VALOR: Vision-Audio-Language Omni-Perception Pretraining Model and Dataset Sihan Chen Xingjian He Longteng Guo Xinxin Zhu Weining Wang Jinhui Tang Jinhui Tang VLM 21 99 0 17 Apr 2023
MoMo: A shared encoder Model for text, image and multi-Modal representations Rakesh Chada Zhao-Heng Zheng P. Natarajan ViT 8 4 0 11 Apr 2023