Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD
Generalization

Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalization

29 January 2024

Papers citing "Overcoming the Pitfalls of Vision-Language Model Finetuning for OOD Generalization"

16 / 16 papers shown

Title
Is CLIP ideal? No. Can we fix it? Yes! Raphi Kang Yue Song Georgia Gkioxari Pietro Perona VLM 45 0 0 10 Mar 2025
FilterRAG: Zero-Shot Informed Retrieval-Augmented Generation to Mitigate Hallucinations in VQA S M Sarwar 54 1 0 25 Feb 2025
BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games Davide Paglieri Bartłomiej Cupiał Samuel Coward Ulyana Piterbarg Maciej Wolczyk ... Lerrel Pinto Rob Fergus Jakob Foerster Jack Parker-Holder Tim Rocktaschel LLMAG LRM 92 10 0 20 Nov 2024
Aggregate-and-Adapt Natural Language Prompts for Downstream Generalization of CLIP Chen Huang Skyler Seto Samira Abnar David Grangier Navdeep Jaitly J. Susskind VLM 31 0 0 31 Oct 2024
Craft: Cross-modal Aligned Features Improve Robustness of Prompt Tuning Jingchen Sun Rohan Sharma Vishnu Suresh Lokhande Changyou Chen 20 0 0 22 Jul 2024
Unveiling the Unknown: Conditional Evidence Decoupling for Unknown Rejection Zhaowei Wu Binyi Su Hua Zhang Zhong Zhou EDL 29 0 0 26 Jun 2024
TIMA: Text-Image Mutual Awareness for Balancing Zero-Shot Adversarial Robustness and Generalization Ability Fengji Ma Li Liu Hei Victor Cheng VLM 17 0 0 27 May 2024
RESTORE: Towards Feature Shift for Vision-Language Prompt Learning Yuncheng Yang Chuyan Zhang Zuopeng Yang Yuting Gao Yulei Qin Ke Li Xing Sun Jie-jin Yang Yun Gu VLM VPVLM 31 0 0 10 Mar 2024
Visual-Language Prompt Tuning with Knowledge-guided Context Optimization Hantao Yao Rui Zhang Changsheng Xu VLM VPVLM 122 193 0 23 Mar 2023
Open-domain Visual Entity Recognition: Towards Recognizing Millions of Wikipedia Entities Hexiang Hu Yi Luan Yang Chen Urvashi Khandelwal Mandar Joshi Kenton Lee Kristina Toutanova Ming-Wei Chang VLM 35 54 0 22 Feb 2023
MaPLe: Multi-modal Prompt Learning Muhammad Uzair Khattak H. Rasheed Muhammad Maaz Salman Khan F. Khan VPVLM VLM 178 521 0 06 Oct 2022
Open-Vocabulary One-Stage Detection with Hierarchical Visual-Language Knowledge Distillation Zongyang Ma Guan Luo Jin Gao Liang Li Yuxin Chen Shaoru Wang Congxuan Zhang Weiming Hu VLM ObjD 72 81 0 20 Mar 2022
VOS: Learning What You Don't Know by Virtual Outlier Synthesis Xuefeng Du Zhaoning Wang Mu Cai Yixuan Li OODD 171 220 0 02 Feb 2022
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 319 2,108 0 02 Sep 2021
Open-vocabulary Object Detection via Vision and Language Knowledge Distillation Xiuye Gu Tsung-Yi Lin Weicheng Kuo Yin Cui VLM ObjD 206 698 0 28 Apr 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 2,875 0 11 Feb 2021