CoCa: Contrastive Captioners are Image-Text Foundation Models

4 May 2022

Mojtaba Seyedhosseini

Papers citing "CoCa: Contrastive Captioners are Image-Text Foundation Models"

50 / 910 papers shown

Title
QUASAR: QUality and Aesthetics Scoring with Advanced Representations Sergey Kastryulin Denis Prokopenko Artem Babenko Dmitry V. Dylov 22 0 0 11 Mar 2024
RESTORE: Towards Feature Shift for Vision-Language Prompt Learning Yuncheng Yang Chuyan Zhang Zuopeng Yang Yuting Gao Yulei Qin Ke Li Xing Sun Jie-jin Yang Yun Gu VLM VPVLM 44 0 0 10 Mar 2024
CLIP the Bias: How Useful is Balancing Data in Multimodal Learning? Ibrahim M. Alabdulmohsin Xiao Wang Andreas Steiner Priya Goyal Alexander DÁmour Xiao-Qi Zhai 26 16 0 07 Mar 2024
Modeling Collaborator: Enabling Subjective Vision Classification With Minimal Human Effort via LLM Tool-Use Imad Eddine Toubal Aditya Avinash N. Alldrin Jan Dlabal Wenlei Zhou ... Chun-Ta Lu Howard Zhou Ranjay Krishna Ariel Fuxman Tom Duerig VLM 73 7 0 05 Mar 2024
HeAR -- Health Acoustic Representations Sebastien Baur Zaid Nabulsi Wei-Hung Weng Jake Garrison Louis Blankemeier ... Shwetak N. Patel S. Shetty Shruthi Prabhakara Monde Muyoyeta Diego Ardila LM&MA 21 10 0 04 Mar 2024
Differentially Private Representation Learning via Image Captioning Tom Sander Yaodong Yu Maziar Sanjabi Alain Durmus Yi-An Ma Kamalika Chaudhuri Chuan Guo 48 3 0 04 Mar 2024
Rethinking CLIP-based Video Learners in Cross-Domain Open-Vocabulary Action Recognition Kun-Yu Lin Henghui Ding Jiaming Zhou Yu-Ming Tang Yi-Xing Peng Zhilin Zhao Chen Change Loy Wei-Shi Zheng VLM 30 15 0 03 Mar 2024
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers Tsai-Shien Chen Aliaksandr Siarohin Willi Menapace Ekaterina Deyneka Hsiang-wei Chao ... Yuwei Fang Hsin-Ying Lee Jian Ren Ming-Hsuan Yang Sergey Tulyakov VGen 70 177 0 29 Feb 2024
The All-Seeing Project V2: Towards General Relation Comprehension of the Open World Weiyun Wang Yiming Ren Hao Luo Tiantong Li Chenxiang Yan ... Qingyun Li Lewei Lu Xizhou Zhu Yu Qiao Jifeng Dai MLLM 36 47 0 29 Feb 2024
SeD: Semantic-Aware Discriminator for Image Super-Resolution Bingchen Li Xin Li Hanxin Zhu Yeying Jin Ruoyu Feng Zhizheng Zhang Zhibo Chen SupR 35 21 0 29 Feb 2024
Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlook Xingchen Zou Yibo Yan Xixuan Hao Yuehong Hu Haomin Wen ... Junbo Zhang Yong Li Tianrui Li Yu Zheng Yuxuan Liang HAI AI4TS 43 35 0 29 Feb 2024
Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interaction Hao Li Ying Chen Yifei Chen Wenxian Yang Bowen Ding Yuchen Han Liansheng Wang Rongshan Yu 31 15 0 29 Feb 2024
Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models Yixin Liu Kai Zhang Yuan Li Zhiling Yan Chujie Gao ... Yue Huang Hanchi Sun Jianfeng Gao Lifang He Lichao Sun VLM VGen EGVM 68 254 0 27 Feb 2024
CLIPose: Category-Level Object Pose Estimation with Pre-trained Vision-Language Knowledge Xiao Lin Minghao Zhu Ronghao Dang Guangliang Zhou Shaolong Shu Feng Lin Chengju Liu Qi Chen CLIP 35 7 0 24 Feb 2024
User-LLM: Efficient LLM Contextualization with User Embeddings Lin Ning Luyang Liu Jiaxing Wu Neo Wu D. Berlowitz Sushant Prakash Bradley Green S. O’Banion Jun Xie 37 32 0 21 Feb 2024
VideoPrism: A Foundational Visual Encoder for Video Understanding Long Zhao N. B. Gundavarapu Liangzhe Yuan Hao Zhou Shen Yan ... Huisheng Wang Hartwig Adam Mikhail Sirotenko Ting Liu Boqing Gong VGen 27 29 0 20 Feb 2024
PaLM2-VAdapter: Progressively Aligned Language Model Makes a Strong Vision-language Adapter Junfei Xiao Zheng Xu Alan L. Yuille Shen Yan Boyu Wang 22 3 0 16 Feb 2024
Revisiting Feature Prediction for Learning Visual Representations from Video Adrien Bardes Q. Garrido Jean Ponce Xinlei Chen Michael G. Rabbat Yann LeCun Mahmoud Assran Nicolas Ballas MDE VLM 82 73 0 15 Feb 2024
ProtChatGPT: Towards Understanding Proteins with Large Language Models Chao Wang Hehe Fan Ruijie Quan Yi Yang 26 12 0 15 Feb 2024
Can Text-to-image Model Assist Multi-modal Learning for Visual Recognition with Visual Modality Missing? Tiantian Feng Daniel Yang Digbalay Bose Shrikanth Narayanan 27 4 0 14 Feb 2024
Open-Vocabulary Segmentation with Unpaired Mask-Text Supervision Zhaoqing Wang Xiaobo Xia Ziye Chen Xiao He Yandong Guo Mingming Gong Tongliang Liu VLM 14 11 0 14 Feb 2024
PRDP: Proximal Reward Difference Prediction for Large-Scale Reward Finetuning of Diffusion Models Fei Deng Qifei Wang Wei Wei Matthias Grundmann Tingbo Hou EGVM 17 15 0 13 Feb 2024
PIN: Positional Insert Unlocks Object Localisation Abilities in VLMs Michael Dorkenwald Nimrod Barazani Cees G. M. Snoek Yuki M. Asano VLM MLLM 25 12 0 13 Feb 2024
Towards a Foundation Model for Brain Age Prediction using coVariance Neural Networks Saurabh Sihag Gonzalo Mateos Alejandro Ribeiro 15 5 0 12 Feb 2024
An Empirical Study Into What Matters for Calibrating Vision-Language Models Weijie Tu Weijian Deng Dylan Campbell Stephen Gould Tom Gedeon VLM 33 7 0 12 Feb 2024
Distilling Symbolic Priors for Concept Learning into Neural Networks Ioana Marinescu R. Thomas McCoy Thomas L. Griffiths 34 2 0 10 Feb 2024
Cacophony: An Improved Contrastive Audio-Text Model Ge Zhu Jordan Darefsky Zhiyao Duan AuLLM 38 11 0 10 Feb 2024
CIC: A Framework for Culturally-Aware Image Captioning Youngsik Yun Jihie Kim VLM 11 5 0 08 Feb 2024
LLMs Meet VLMs: Boost Open Vocabulary Object Detection with Fine-grained Descriptors Sheng Jin Xue-Qiu Jiang Jiaxing Huang Lewei Lu Shijian Lu VLM ObjD 18 21 0 07 Feb 2024
Multimodal Unsupervised Domain Generalization by Retrieving Across the Modality Gap Christopher Liao Christian So Theodoros Tsiligkaridis Brian Kulis 21 0 0 06 Feb 2024
Progress and Opportunities of Foundation Models in Bioinformatics Qing Li Zhihang Hu Yixuan Wang Lei Li Yimin Fan Irwin King Le Song Yu-Hu Li AI4CE 30 9 0 06 Feb 2024
Image-Caption Encoding for Improving Zero-Shot Generalization Eric Yang Yu Christopher Liao Sathvik Ravi Theodoros Tsiligkaridis Brian Kulis OODD VLM 13 0 0 05 Feb 2024
M2-RAAP: A Multi-Modal Recipe for Advancing Adaptation-based Pre-training towards Effective and Efficient Zero-shot Video-text Retrieval Xingning Dong Zipeng Feng Chunluan Zhou Xuzheng Yu Ming Yang Qingpei Guo VLM 25 2 0 31 Jan 2024
A Survey on Data Augmentation in Large Model Era Yue Zhou Chenlu Guo Xu Wang Yi-Ju Chang Yuan Wu LM&MA VLM 40 23 0 27 Jan 2024
Segment Any Cell: A SAM-based Auto-prompting Fine-tuning Framework for Nuclei Segmentation Saiyang Na Yuzhi Guo Feng Jiang Hehuan Ma Junzhou Huang VLM MedIm 16 14 0 24 Jan 2024
On the Efficacy of Text-Based Input Modalities for Action Anticipation Apoorva Beedu Karan Samel Irfan Essa 45 2 0 23 Jan 2024
Facing the Elephant in the Room: Visual Prompt Tuning or Full Finetuning? Cheng Han Qifan Wang Yiming Cui Wenguan Wang Lifu Huang Siyuan Qi Dongfang Liu VLM 44 19 0 23 Jan 2024
Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers Katherine Crowson Stefan Andreas Baumann Alex Birch Tanishq Mathew Abraham Daniel Z. Kaplan Enrico Shippole 18 48 0 21 Jan 2024
Exploring scalable medical image encoders beyond text supervision Fernando Pérez-García Harshita Sharma Sam Bond-Taylor Kenza Bouzid Valentina Salvatelli ... Maria T. A. Wetscherek Noel Codella Stephanie L. Hyland Javier Alvarez-Valle Ozan Oktay LM&MA MedIm 48 9 0 19 Jan 2024
Supervised Fine-tuning in turn Improves Visual Foundation Models Xiaohu Jiang Yixiao Ge Yuying Ge Dachuan Shi Chun Yuan Ying Shan VLM CLIP 38 8 0 18 Jan 2024
MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizer Changyao Tian Xizhou Zhu Yuwen Xiong Weiyun Wang Zhe Chen ... Tong Lu Jie Zhou Hongsheng Li Yu Qiao Jifeng Dai AuLLM 83 41 0 18 Jan 2024
GPT4Ego: Unleashing the Potential of Pre-trained Models for Zero-Shot Egocentric Action Recognition Guangzhao Dai Xiangbo Shu Wenhao Wu Rui Yan Jiachao Zhang VLM 14 5 0 18 Jan 2024
Improving fine-grained understanding in image-text pre-training Ioana Bica Anastasija Ilić Matthias Bauer Goker Erdogan Matko Bovsnjak ... A. Gritsenko Matthias Minderer Charles Blundell Razvan Pascanu Jovana Mitrović VLM 23 21 0 18 Jan 2024
Instance Brownian Bridge as Texts for Open-vocabulary Video Instance Segmentation Ze-Long Cheng Kehan Li Hao Li Peng Jin Chang Liu Xiawu Zheng Rongrong Ji Jie Chen VOS 28 2 0 18 Jan 2024
Scalable Pre-training of Large Autoregressive Image Models Alaaeldin El-Nouby Michal Klein Shuangfei Zhai Miguel Angel Bautista Alexander Toshev Vaishaal Shankar J. Susskind Armand Joulin VLM 12 71 0 16 Jan 2024
Concept-Guided Prompt Learning for Generalization in Vision-Language Models Yi Zhang Ce Zhang Ke Yu Yushun Tang Zhihai He VLM MLLM 32 20 0 15 Jan 2024
Generalizing Visual Question Answering from Synthetic to Human-Written Questions via a Chain of QA with a Large Language Model Taehee Kim Yeongjae Cho Heejun Shin Yohan Jo Dongmyung Shin 32 3 0 12 Jan 2024
Distilling Vision-Language Models on Millions of Videos Yue Zhao Long Zhao Xingyi Zhou Jialin Wu Chun-Te Chu ... Hartwig Adam Ting Liu Boqing Gong Philipp Krahenbuhl Liangzhe Yuan VLM 21 13 0 11 Jan 2024
Evaluating Data Augmentation Techniques for Coffee Leaf Disease Classification Adrian Gheorghiu Iulian-Marius Taiatu Dumitru-Clementin Cercel Iuliana Marin Florin-Catalin Pop 49 2 0 11 Jan 2024
Learning to Prompt with Text Only Supervision for Vision-Language Models Muhammad Uzair Khattak Muhammad Ferjad Naeem Muzammal Naseer Luc Van Gool F. Tombari VLM VPVLM 28 18 0 04 Jan 2024