DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only
Training

DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Training

6 March 2023

Yi Yang

Papers citing "DeCap: Decoding CLIP Latents for Zero-Shot Captioning via Text-Only Training"

17 / 17 papers shown

Title
Instruct-CLIP: Improving Instruction-Guided Image Editing with Automated Data Refinement Using Contrastive Learning Sherry X Chen Misha Sra Pradeep Sen 48 0 0 24 Mar 2025
Cross the Gap: Exposing the Intra-modal Misalignment in CLIP via Modality Inversion Marco Mistretta Alberto Baldrati Lorenzo Agnolucci Marco Bertini Andrew D. Bagdanov CLIP VLM 93 2 0 06 Feb 2025
Unleashing Text-to-Image Diffusion Prior for Zero-Shot Image Captioning Jianjie Luo Jingwen Chen Yehao Li Yingwei Pan Jianlin Feng Hongyang Chao Ting Yao DiffM VLM 36 0 0 03 Jan 2025
AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark Wenhao Chai Enxin Song Y. Du Chenlin Meng Vashisht Madhavan Omer Bar-Tal Jeng-Neng Hwang Saining Xie Christopher D. Manning 3DV 53 25 0 04 Oct 2024
Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation Kun Yuan V. Srivastav Nassir Navab N. Padoy 26 7 0 30 Sep 2024
MeaCap: Memory-Augmented Zero-shot Image Captioning Zequn Zeng Yan Xie Hao Zhang Chiyu Chen Zhengjue Wang Boli Chen VLM 18 13 0 06 Mar 2024
Zero-shot Translation of Attention Patterns in VQA Models to Natural Language Leonard Salewski A. Sophia Koepke Hendrik P. A. Lensch Zeynep Akata 12 2 0 08 Nov 2023
Weakly-supervised Automated Audio Captioning via text only training Theodoros Kouzelis V. Katsouros CLIP 17 6 0 21 Sep 2023
MultiCapCLIP: Auto-Encoding Prompts for Zero-Shot Multilingual Visual Captioning Bang-ju Yang Fenglin Liu X. Wu Yaowei Wang Xu Sun Yuexian Zou VLM CLIP 19 13 0 25 Aug 2023
Linear Alignment of Vision-language Models for Image Captioning Fabian Paischer M. Hofmarcher Sepp Hochreiter Thomas Adler CLIP VLM 26 0 0 10 Jul 2023
Multimodal Knowledge Alignment with Reinforcement Learning Youngjae Yu Jiwan Chung Heeseung Yun Jack Hessel J. Park ... Prithviraj Ammanabrolu Rowan Zellers Ronan Le Bras Gunhee Kim Yejin Choi VLM 109 35 0 25 May 2022
Auto-Encoding Knowledge Graph for Unsupervised Medical Report Generation Fenglin Liu Chenyu You Xian Wu Shen Ge Sheng Wang Xu Sun MedIm 69 89 0 08 Nov 2021
How Much Can CLIP Benefit Vision-and-Language Tasks? Sheng Shen Liunian Harold Li Hao Tan Mohit Bansal Anna Rohrbach Kai-Wei Chang Z. Yao Kurt Keutzer CLIP VLM MLLM 174 342 0 13 Jul 2021
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation Xiuye Gu Tsung-Yi Lin Weicheng Kuo Yin Cui VLM ObjD 203 698 0 28 Apr 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts Soravit Changpinyo P. Sharma Nan Ding Radu Soricut VLM 273 845 0 17 Feb 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 2,875 0 11 Feb 2021
Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning Jiasen Lu Caiming Xiong Devi Parikh R. Socher 81 443 0 06 Dec 2016