CoCa: Contrastive Captioners are Image-Text Foundation Models

4 May 2022

Mojtaba Seyedhosseini

Papers citing "CoCa: Contrastive Captioners are Image-Text Foundation Models"

50 / 910 papers shown

Title
Spacewalk-18: A Benchmark for Multimodal and Long-form Procedural Video Understanding in Novel Domains Rohan Myer Krishnan Zitian Tang Zhiqiu Yu Chen Sun 40 1 0 30 Nov 2023
GELDA: A generative language annotation framework to reveal visual biases in datasets Krish Kabra Kathleen M. Lewis Guha Balakrishnan VLM 11 1 0 29 Nov 2023
CLIPC8: Face liveness detection algorithm based on image-text pairs and contrastive learning Xu Liu Shu Zhou Yurong Song Wenzhe Luo Xin Zhang 16 1 0 29 Nov 2023
Contrastive Vision-Language Alignment Makes Efficient Instruction Learner Lizhao Liu Xinyu Sun Tianhang Xiang Zhuangwei Zhuang Liuren Yin Mingkui Tan VLM 16 2 0 29 Nov 2023
MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training Pavan Kumar Anasosalu Vasu Hadi Pouransari Fartash Faghri Raviteja Vemulapalli Oncel Tuzel CLIP VLM 16 43 0 28 Nov 2023
The curse of language biases in remote sensing VQA: the role of spatial attributes, language diversity, and the need for clear evaluation Christel Chappuis Eliot Walt Vincent Mendez Sylvain Lobry B. L. Saux D. Tuia 23 3 0 28 Nov 2023
Large Model Based Referring Camouflaged Object Detection Shupeng Cheng Ge-Peng Ji Pengda Qin Deng-Ping Fan Bowen Zhou Peng-Tao Xu ObjD 16 8 0 28 Nov 2023
Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models Jiayun Luo Siddhesh Khandelwal Leonid Sigal Boyang Albert Li MLLM VLM 27 7 0 28 Nov 2023
IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers Chenglin Yang Siyuan Qiao Yuan Cao Yu Zhang Tao Zhu Alan L. Yuille Jiahui Yu VLM 11 3 0 27 Nov 2023
ViT-Lens: Towards Omni-modal Representations Weixian Lei Yixiao Ge Kun Yi Jianfeng Zhang Difei Gao Dylan Sun Yuying Ge Ying Shan Mike Zheng Shou 21 18 0 27 Nov 2023
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI Xiang Yue Yuansheng Ni Kai Zhang Tianyu Zheng Ruoqi Liu ... Yibo Liu Wenhao Huang Huan Sun Yu-Chuan Su Wenhu Chen OSLM ELM VLM 66 729 0 27 Nov 2023
Efficient Pre-training for Localized Instruction Generation of Videos Anil Batra Davide Moltisanti Laura Sevilla-Lara Marcus Rohrbach Frank Keller 22 0 0 27 Nov 2023
Align before Adapt: Leveraging Entity-to-Region Alignments for Generalizable Video Action Recognition Yifei Chen Dapeng Chen Ruijin Liu Sai Zhou Wenyuan Xue Wei Peng 25 6 0 27 Nov 2023
Insect-Foundation: A Foundation Model and Large-scale 1M Dataset for Visual Insect Understanding Hoang-Quan Nguyen Thanh-Dat Truong Xuan-Bac Nguyen Ashley Dowling Xin Li Khoa Luu VLM 24 19 0 26 Nov 2023
Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets A. Blattmann Tim Dockhorn Sumith Kulal Daniel Mendelevitch Maciej Kilian ... Zion English Vikram S. Voleti Adam Letts Varun Jampani Robin Rombach VGen 150 1,009 0 25 Nov 2023
Mug-STAN: Adapting Image-Language Pretrained Models for General Video Understanding Ruyang Liu Jingjia Huang Wei-Nan Gao Thomas H. Li Ge Li VLM 27 3 0 25 Nov 2023
Effective Backdoor Mitigation in Vision-Language Models Depends on the Pre-training Objective Sahil Verma Gantavya Bhatt Avi Schwarzschild Soumye Singhal Arnav M. Das Chirag Shah John P Dickerson Jeff Bilmes J. Bilmes AAML 59 1 0 25 Nov 2023
SEGIC: Unleashing the Emergent Correspondence for In-Context Segmentation Lingchen Meng Shiyi Lan Hengduo Li Jose M. Alvarez Zuxuan Wu Yu-Gang Jiang VLM ISeg MLLM 28 6 0 24 Nov 2023
T-Rex: Counting by Visual Prompting Qing Jiang Feng Li Tianhe Ren Shilong Liu Zhaoyang Zeng Kent Yu Lei Zhang 16 10 0 22 Nov 2023
Vamos: Versatile Action Models for Video Understanding Shijie Wang Qi Zhao Minh Quan Do Nakul Agarwal Kwonjoon Lee Chen Sun 27 19 0 22 Nov 2023
Breathing Life Into Sketches Using Text-to-Video Priors Rinon Gal Yael Vinker Yuval Alaluf Amit H. Bermano Daniel Cohen-Or Ariel Shamir Gal Chechik VGen DiffM 27 29 0 21 Nov 2023
Controlling the Output of a Generative Model by Latent Feature Vector Shifting Róbert Belanec Peter Lacko Kristína Malinovská 11 1 0 15 Nov 2023
Violet: A Vision-Language Model for Arabic Image Captioning with Gemini Decoder Abdelrahman Mohamed Fakhraddin Alwajih El Moatez Billah Nagoudi Alcides Alcoba Inciarte Muhammad Abdul-Mageed VLM MLLM 25 7 0 15 Nov 2023
Correlation-Guided Query-Dependency Calibration for Video Temporal Grounding WonJun Moon Sangeek Hyun Subeen Lee Jae-Pil Heo 19 4 0 15 Nov 2023
Towards Open-Ended Visual Recognition with Large Language Model Qihang Yu Xiaohui Shen Liang-Chieh Chen VLM 22 8 0 14 Nov 2023
DRUformer: Enhancing the driving scene Important object detection with driving relationship self-understanding Yingjie Niu Ming Ding Keisuke Fujii Kento Ohtani Alexander Carballo K. Takeda ViT 29 0 0 11 Nov 2023
Florence-2: Advancing a Unified Representation for a Variety of Vision Tasks Bin Xiao Haiping Wu Weijian Xu Xiyang Dai Houdong Hu Yumao Lu Michael Zeng Ce Liu Lu Yuan VLM 31 142 0 10 Nov 2023
LRM: Large Reconstruction Model for Single Image to 3D Yicong Hong Kai Zhang Jiuxiang Gu Sai Bi Yang Zhou Difan Liu Feng Liu Kalyan Sunkavalli Trung Bui Hao Tan 3DV 3DH 40 411 0 08 Nov 2023
OmniVec: Learning robust representations with cross modal sharing Siddharth Srivastava Gaurav Sharma SSL 21 64 0 07 Nov 2023
Meta-Adapter: An Online Few-shot Learner for Vision-Language Model Cheng Cheng Lin Song Ruoyi Xue Hang Wang Hongbin Sun Yixiao Ge Ying Shan VLM ObjD 27 18 0 07 Nov 2023
GLaMM: Pixel Grounding Large Multimodal Model H. Rasheed Muhammad Maaz Sahal Shaji Mullappilly Abdelrahman M. Shaker Salman Khan Hisham Cholakkal Rao M. Anwer Erix Xing Ming-Hsuan Yang Fahad S. Khan MLLM VLM 36 199 0 06 Nov 2023
CogVLM: Visual Expert for Pretrained Language Models Weihan Wang Qingsong Lv Wenmeng Yu Wenyi Hong Ji Qi ... Bin Xu Juanzi Li Yuxiao Dong Ming Ding Jie Tang VLM MLLM 17 445 0 06 Nov 2023
Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models Andy Zhou Jindong Wang Yu-xiong Wang Haohan Wang VLM 36 6 0 02 Nov 2023
Learning to Adapt CLIP for Few-Shot Monocular Depth Estimation Xue-mei Hu Ce Zhang Yi Zhang Bowen Hai Ke Yu Zhihai He MDE VLM 22 17 0 02 Nov 2023
RoboVQA: Multimodal Long-Horizon Reasoning for Robotics P. Sermanet Tianli Ding Jeffrey Zhao Fei Xia Debidatta Dwibedi ... Pannag R. Sanketi Karol Hausman Izhak Shafran Brian Ichter Yuan Cao LM&Ro 17 50 0 01 Nov 2023
De-Diffusion Makes Text a Strong Cross-Modal Interface Chen Wei Chenxi Liu Siyuan Qiao Zhishuai Zhang Alan Yuille Jiahui Yu VLM DiffM 29 10 0 01 Nov 2023
CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders A. Fuller K. Millard James R. Green 15 60 0 01 Nov 2023
fMRI-PTE: A Large-scale fMRI Pretrained Transformer Encoder for Multi-Subject Brain Activity Decoding Xuelin Qian Yun Wang Jingyang Huo Jianfeng Feng Yanwei Fu MedIm 11 8 0 01 Nov 2023
From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities Md Farhan Ishmam Md Sakib Hossain Shovon M. F. Mridha Nilanjan Dey 35 36 0 01 Nov 2023
Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks Micah Goldblum Hossein Souri Renkun Ni Manli Shu Viraj Prabhu ... Adrien Bardes Judy Hoffman Ramalingam Chellappa Andrew Gordon Wilson Tom Goldstein VLM 68 62 0 30 Oct 2023
What's "up" with vision-language models? Investigating their struggle with spatial reasoning Amita Kamath Jack Hessel Kai-Wei Chang LRM CoGe 13 94 0 30 Oct 2023
Intra-Modal Proxy Learning for Zero-Shot Visual Categorization with CLIP Qi Qian Yuanhong Xu Juhua Hu VLM CLIP 21 16 0 30 Oct 2023
Harvest Video Foundation Models via Efficient Post-Pretraining Yizhuo Li Kunchang Li Yinan He Yi Wang Yali Wang Limin Wang Yu Qiao Ping Luo CLIP VLM VGen 33 2 0 30 Oct 2023
Foundation Models for Generalist Geospatial Artificial Intelligence Johannes Jakubik Sujit Roy C. Phillips P. Fraccaro Denys Godwin ... Hamed Alemohammad M. Maskey R. Ganti Kommy Weldemariam Rahul Ramachandran AI4CE VLM 8 91 0 28 Oct 2023
CoDet: Co-Occurrence Guided Region-Word Alignment for Open-Vocabulary Object Detection Chuofan Ma Yi-Xin Jiang Xin Wen Zehuan Yuan Xiaojuan Qi ObjD VLM 18 48 0 25 Oct 2023
MotionAGFormer: Enhancing 3D Human Pose Estimation with a Transformer-GCNFormer Network Soroush Mehraban Vida Adeli Babak Taati ViT 25 41 0 25 Oct 2023
Leveraging Image-Text Similarity and Caption Modification for the DataComp Challenge: Filtering Track and BYOD Track Shuhei Yokoo Peifei Zhu Yuchi Ishikawa Mikihiro Tanaka Masayoshi Kondo Hirokatsu Kataoka 16 0 0 23 Oct 2023
CLIP meets Model Zoo Experts: Pseudo-Supervision for Visual Enhancement Mohammadreza Salehi Mehrdad Farajtabar Maxwell Horton Fartash Faghri Hadi Pouransari Raviteja Vemulapalli Oncel Tuzel Ali Farhadi Mohammad Rastegari Sachin Mehta CLIP VLM 22 1 0 21 Oct 2023
SILC: Improving Vision Language Pretraining with Self-Distillation Muhammad Ferjad Naeem Yongqin Xian Xiaohua Zhai Lukas Hoyer Luc Van Gool F. Tombari VLM 17 32 0 20 Oct 2023
CLARA: Multilingual Contrastive Learning for Audio Representation Acquisition K. A. Noriy Xiaosong Yang Marcin Budka Jian Jun Zhang VLM 15 3 0 18 Oct 2023