CoCa: Contrastive Captioners are Image-Text Foundation Models

4 May 2022

Mojtaba Seyedhosseini

Papers citing "CoCa: Contrastive Captioners are Image-Text Foundation Models"

50 / 910 papers shown

Title
RoboLLM: Robotic Vision Tasks Grounded on Multimodal Large Language Models Zijun Long George Killick R. McCreadie Gerardo Aragon Camarasa VLM 22 11 0 16 Oct 2023
Few-shot Action Recognition with Captioning Foundation Models Xiang Wang Shiwei Zhang Hangjie Yuan Yingya Zhang Changxin Gao Deli Zhao Nong Sang VLM 19 7 0 16 Oct 2023
CAPro: Webly Supervised Learning with Cross-Modality Aligned Prototypes Yulei Qin Xingyu Chen Yunhang Shen Chaoyou Fu Yun Gu Ke Li Xing Sun Rongrong Ji 29 3 0 15 Oct 2023
Vision-by-Language for Training-Free Compositional Image Retrieval Shyamgopal Karthik Karsten Roth Massimiliano Mancini Zeynep Akata CoGe 26 52 0 13 Oct 2023
PaLI-3 Vision Language Models: Smaller, Faster, Stronger Xi Chen Xiao Wang Lucas Beyer Alexander Kolesnikov Jialin Wu ... Keran Rong Tianli Yu Daniel Keysers Xiao-Qi Zhai Radu Soricut MLLM VLM 30 92 0 13 Oct 2023
Visual Data-Type Understanding does not emerge from Scaling Vision-Language Models Vishaal Udandarao Max F. Burg Samuel Albanie Matthias Bethge VLM 24 8 0 12 Oct 2023
Generalized Logit Adjustment: Calibrating Fine-tuned Models by Removing Label Bias in Foundation Models Beier Zhu Kaihua Tang Qianru Sun Hanwang Zhang 18 21 0 12 Oct 2023
Incorporating Domain Knowledge Graph into Multimodal Movie Genre Classification with Self-Supervised Attention and Contrastive Learning Jiaqi Li Guilin Qi Chuanyi Zhang Yongrui Chen Yiming Tan Chenlong Xia Ye Tian 28 2 0 12 Oct 2023
PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm Haoyi Zhu Honghui Yang Xiaoyang Wu Di Huang Sha Zhang ... Hengshuang Zhao Chunhua Shen Yu Qiao Tong He Wanli Ouyang SSL 69 42 0 12 Oct 2023
VeCLIP: Improving CLIP Training via Visual-enriched Captions Zhengfeng Lai Haotian Zhang Bowen Zhang Wentao Wu Haoping Bai ... Zhe Gan Jiulong Shan Chen-Nee Chuah Yinfei Yang Meng Cao CLIP VLM 29 28 0 11 Oct 2023
Lightweight In-Context Tuning for Multimodal Unified Models Yixin Chen Shuai Zhang Boran Han Jiaya Jia 13 2 0 08 Oct 2023
Video-Teller: Enhancing Cross-Modal Generation with Fusion and Decoupling Haogeng Liu Qihang Fan Tingkai Liu Linjie Yang Yunzhe Tao Huaibo Huang Ran He Hongxia Yang VGen 21 12 0 08 Oct 2023
Module-wise Adaptive Distillation for Multimodality Foundation Models Chen Liang Jiahui Yu Ming-Hsuan Yang Matthew A. Brown Yin Cui Tuo Zhao Boqing Gong Tianyi Zhou 11 10 0 06 Oct 2023
Leveraging Unpaired Data for Vision-Language Generative Models via Cycle Consistency Tianhong Li Sangnie Bhardwaj Yonglong Tian Han Zhang Jarred Barber Dina Katabi Guillaume Lajoie Huiwen Chang Dilip Krishnan VLM 36 4 0 05 Oct 2023
On the Cognition of Visual Question Answering Models and Human Intelligence: A Comparative Study Liben Chen Long Chen Tian Ellison-Chen Zhuoyuan Xu LRM 17 0 0 04 Oct 2023
Towards reporting bias in visual-language datasets: bimodal augmentation by decoupling object-attribute association Qiyu Wu Mengjie Zhao Yutong He Lang Huang Junya Ono Hiromi Wakaki Yuki Mitsufuji 12 4 0 02 Oct 2023
Large Scale Masked Autoencoding for Reducing Label Requirements on SAR Data Matt Allen Francisco Dorr Joseph A. Gallego-Mejia Laura Martínez-Ferrer Anna Jungbluth F. Kalaitzis Raúl Ramos-Pollán 17 9 0 02 Oct 2023
Reformulating Vision-Language Foundation Models and Datasets Towards Universal Multimodal Assistants Tianyu Yu Jinyi Hu Yuan Yao Haoye Zhang Yue Zhao ... Jiao Xue Dahai Li Zhiyuan Liu Hai-Tao Zheng Maosong Sun VLM MLLM 17 18 0 01 Oct 2023
Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning Mustafa Shukor Alexandre Ramé Corentin Dancette Matthieu Cord LRM MLLM 33 20 0 01 Oct 2023
Knowledge Engineering using Large Language Models Bradley Paul Allen Lise Stork Paul T. Groth 15 24 0 01 Oct 2023
InstructCV: Instruction-Tuned Text-to-Image Diffusion Models as Vision Generalists Yulu Gan Sungwoo Park Alexander Schubert Anthony Philippakis Ahmed Alaa VLM 17 22 0 30 Sep 2023
Training a Large Video Model on a Single Machine in a Day Yue Zhao Philipp Krahenbuhl VLM 25 15 0 28 Sep 2023
AutoCLIP: Auto-tuning Zero-Shot Classifiers for Vision-Language Models Sanghwan Kim Hao Tang Fisher Yu VLM CLIP 13 4 0 28 Sep 2023
Parameter-Saving Adversarial Training: Reinforcing Multi-Perturbation Robustness via Hypernetworks Huihui Gong Minjing Dong Siqi Ma S. Çamtepe Surya Nepal Chang Xu AAML OOD 18 1 0 28 Sep 2023
BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning Ruyang Liu Chen Li Yixiao Ge Ying Shan Thomas H. Li Ge Li 25 29 0 27 Sep 2023
Object-Centric Open-Vocabulary Image-Retrieval with Aggregated Features Hila Levi Guy Heller Dan Levi Ethan Fetaya OCL VLM 14 3 0 26 Sep 2023
CWCL: Cross-Modal Transfer with Continuously Weighted Contrastive Loss R. S. Srinivasa Jaejin Cho Chouchang Yang Yashas Malur Saidutta Ching Hua Lee Yilin Shen Hongxia Jin VLM 21 8 0 26 Sep 2023
DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention Z. Yao Xiaoxia Wu Conglong Li Minjia Zhang Heyang Qi Olatunji Ruwase A. A. Awan Samyam Rajbhandari Yuxiong He 26 11 0 25 Sep 2023
Beyond Grids: Exploring Elastic Input Sampling for Vision Transformers Adam Pardyl Grzegorz Kurzejamski Jan Olszewski Tomasz Trzciñski Bartosz Zieliñski 23 1 0 23 Sep 2023
TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance Kan Wu Houwen Peng Zhenghong Zhou Bin Xiao Mengchen Liu ... Xi Xi Chen Xinggang Wang Hongyang Chao Han Hu VLM OODD 26 53 0 21 Sep 2023
ContextRef: Evaluating Referenceless Metrics For Image Description Generation Elisa Kreiss E. Zelikman Christopher Potts Nick Haber 13 5 0 21 Sep 2023
DreamLLM: Synergistic Multimodal Comprehension and Creation Runpei Dong Chunrui Han Yuang Peng Zekun Qi Zheng Ge ... Hao-Ran Wei Xiangwen Kong Xiangyu Zhang Kaisheng Ma Li Yi MLLM 34 168 0 20 Sep 2023
In-Style: Bridging Text and Uncurated Videos with Style Transfer for Text-Video Retrieval Nina Shvetsova Anna Kukleva Bernt Schiele Hilde Kuehne DiffM 23 3 0 16 Sep 2023
Disentangling Spatial and Temporal Learning for Efficient Image-to-Video Transfer Learning Zhiwu Qing Shiwei Zhang Ziyuan Huang Yingya Zhang Changxin Gao Deli Zhao Nong Sang 19 18 0 14 Sep 2023
Improving Multimodal Classification of Social Media Posts by Leveraging Image-Text Auxiliary Tasks Danae Sánchez Villegas Daniel Preoctiuc-Pietro Nikolaos Aletras 31 2 0 14 Sep 2023
PRE: Vision-Language Prompt Learning with Reparameterization Encoder Anh Pham Thi Minh An Duc Nguyen Georgios Tzimiropoulos VPVLM VLM 19 3 0 14 Sep 2023
Sight Beyond Text: Multi-Modal Training Enhances LLMs in Truthfulness and Ethics Haoqin Tu Bingchen Zhao Chen Wei Cihang Xie MLLM 23 13 0 13 Sep 2023
Language Models as Black-Box Optimizers for Vision-Language Models Shihong Liu Zhiqiu Lin Samuel Yu Ryan Lee Tiffany Ling Deepak Pathak Deva Ramanan VLM 22 28 0 12 Sep 2023
Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization Yang Jin Kun Xu Kun Xu Liwei Chen Chao Liao ... Xiaoqiang Lei Di Zhang Wenwu Ou Kun Gai Yadong Mu MLLM VLM 14 41 0 09 Sep 2023
InstructDiffusion: A Generalist Modeling Interface for Vision Tasks Zigang Geng Binxin Yang Tiankai Hang Chen Li Shuyang Gu ... Jianmin Bao Zheng-Wei Zhang Han Hu Dongdong Chen Baining Guo DiffM VLM 38 92 0 07 Sep 2023
Dual Relation Alignment for Composed Image Retrieval Xintong Jiang Yaxiong Wang Yujiao Wu M. Wang Xueming Qian 15 5 0 05 Sep 2023
ExMobileViT: Lightweight Classifier Extension for Mobile Vision Transformer Gyeongdong Yang Yungwook Kwon Hyunjin Kim ViT 8 1 0 04 Sep 2023
BDC-Adapter: Brownian Distance Covariance for Better Vision-Language Reasoning Yi Zhang Ce Zhang Zihan Liao Yushun Tang Zhihai He BDL VLM 16 10 0 03 Sep 2023
Contrastive Feature Masking Open-Vocabulary Vision Transformer Dahun Kim A. Angelova Weicheng Kuo ObjD VLM 21 27 0 02 Sep 2023
ViLTA: Enhancing Vision-Language Pre-training through Textual Augmentation Weihan Wang Z. Yang Bin Xu Juanzi Li Yankui Sun VLM 20 8 0 31 Aug 2023
Do the Frankenstein, or how to achieve better out-of-distribution performance with manifold mixing model soup Hannes Fassold MoMe UQCV 14 2 0 28 Aug 2023
Fine-tuning can cripple your foundation model; preserving features may be the solution Jishnu Mukhoti Y. Gal Philip H. S. Torr P. Dokania CLL 28 29 0 25 Aug 2023
Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond Jinze Bai Shuai Bai Shusheng Yang Shijie Wang Sinan Tan Peng Wang Junyang Lin Chang Zhou Jingren Zhou MLLM VLM ObjD 43 790 0 24 Aug 2023
DLIP: Distilling Language-Image Pre-training Huafeng Kuang Jie Wu Xiawu Zheng Ming Li Xuefeng Xiao Rui Wang Min Zheng Rongrong Ji VLM 36 4 0 24 Aug 2023
Large Multilingual Models Pivot Zero-Shot Multimodal Learning across Languages Jinyi Hu Yuan Yao Chong Wang Shanonan Wang Yinxu Pan ... Yankai Lin Jiao Xue Dahai Li Zhiyuan Liu Maosong Sun MLLM VLM 24 48 0 23 Aug 2023