Finetune like you pretrain: Improved finetuning of zero-shot vision models

1 December 2022

Papers citing "Finetune like you pretrain: Improved finetuning of zero-shot vision models"

50 / 101 papers shown

Title
Efficient Mixture of Geographical Species for On Device Wildlife Monitoring Emmanuel Azuh Mensah Joban Mand Yueheng Ou Min Jang Kurtis Heimerl 29 0 0 11 Apr 2025
Filter Like You Test: Data-Driven Data Filtering for CLIP Pretraining Mikey Shechter Yair Carmon CLIP 42 0 0 11 Mar 2025
Data-Efficient Generalization for Zero-shot Composed Image Retrieval Zining Chen Zhicheng Zhao Fei Su Xiaoqin Zhang Shijian Lu VLM 40 0 0 07 Mar 2025
Generalizable Prompt Learning of CLIP: A Brief Overview Fangming Cui Yonggang Zhang Xuan Wang Xule Wang Liang Xiao VPVLM VLM 84 0 0 03 Mar 2025
Solving Instance Detection from an Open-World Perspective Qianqian Shen Yunhan Zhao Nahyun Kwon Jeeeun Kim Yanan Li Shu Kong 32 0 0 01 Mar 2025
PRISM: High-Resolution & Precise Counterfactual Medical Image Generation using Language-guided Stable Diffusion Amar Kumar Anita Kriz Mohammad Havaei Tal Arbel MedIm 38 2 0 28 Feb 2025
Directional Gradient Projection for Robust Fine-Tuning of Foundation Models Chengyue Huang Junjiao Tian Brisa Maneechotesuwan Shivang Chopra Z. Kira 49 0 0 21 Feb 2025
Demographic User Modeling for Social Robotics with Multimodal Pre-trained Models Hamed Rahimi Mouad Abrini Mahdi Khoramshahi Mohamed Chetouani 36 0 0 15 Feb 2025
Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation Reza Akbarian Bafghi Carden Bagwell Avinash Ravichandran Ashish Shrivastava M. Raissi 40 0 0 28 Jan 2025
The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better Scott Geng Cheng-Yu Hsieh Vivek Ramanujan Matthew Wallingford Chun-Liang Li Pang Wei Koh Ranjay Krishna DiffM 60 6 0 03 Jan 2025
Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models Yulei Qin Yuncheng Yang Pengcheng Guo Gang Li Hang Shao Yuchen Shi Zihan Xu Yun Gu Ke Li Xing Sun ALM 88 11 0 31 Dec 2024
Beyond Accuracy: On the Effects of Fine-tuning Towards Vision-Language Model's Prediction Rationality Qitong Wang Tang Li Kien X. Nguyen Xi Peng 70 0 0 17 Dec 2024
Prompt as Free Lunch: Enhancing Diversity in Source-Free Cross-domain Few-shot Learning through Semantic-Guided Prompting Linhai Zhuo Zheng Wang Yuqian Fu Tianwen Qian VLM 69 1 0 01 Dec 2024
Dual Risk Minimization: Towards Next-Level Robustness in Fine-tuning Zero-Shot Models Kaican Li Weiyan Xie Yongxiang Huang Didan Deng Lanqing Hong Z. Li Ricardo Silva N. Zhang 62 0 0 29 Nov 2024
Words Matter: Leveraging Individual Text Embeddings for Code Generation in CLIP Test-Time Adaptation Shambhavi Mishra Julio Silva-Rodrıguez Ismail ben Ayed M. Pedersoli Jose Dolz VLM 75 1 0 26 Nov 2024
LAGUNA: LAnguage Guided UNsupervised Adaptation with structured spaces Anxhelo Diko Antonino Furnari Luigi Cinque G. Farinella 85 0 0 23 Nov 2024
Robust Fine-tuning of Zero-shot Models via Variance Reduction B. Zhu Jiequan Cui H. Zhang VLM OODD 25 0 0 11 Nov 2024
Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models Junjiao Tian Chengyue Huang Z. Kira 28 1 0 03 Nov 2024
Diffusion Curriculum: Synthetic-to-Real Generative Curriculum Learning via Image-Guided Diffusion Yijun Liang Shweta Bhardwaj Tianyi Zhou 26 0 0 17 Oct 2024
LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Texts Anh-Quan Cao M. Jaritz Matthieu Guillaumin Raoul de Charette Loris Bazzani VLM CLIP 34 2 0 10 Oct 2024
TuneVLSeg: Prompt Tuning Benchmark for Vision-Language Segmentation Models Rabin Adhikari Safal Thapaliya Manish Dhakal Bishesh Khanal MLLM VLM 25 0 0 07 Oct 2024
SAFLEX: Self-Adaptive Augmentation via Feature Label Extrapolation Mucong Ding Bang An Yuancheng Xu Anirudh Satheesh Furong Huang 19 1 0 03 Oct 2024
DaWin: Training-free Dynamic Weight Interpolation for Robust Adaptation Changdae Oh Yixuan Li Kyungwoo Song Sangdoo Yun Dongyoon Han OOD MoMe 36 4 0 03 Oct 2024
Toward a Holistic Evaluation of Robustness in CLIP Models Weijie Tu Weijian Deng Tom Gedeon VLM 31 5 0 02 Oct 2024
CRoP: Context-wise Robust Static Human-Sensing Personalization Sawinder Kaur Avery Gump Jingyu Xin Yi Xiao Harshit Sharma Nina R. Benway J. Preston Asif Salekin 24 0 0 26 Sep 2024
Finetuning CLIP to Reason about Pairwise Differences Dylan Sam Devin Willmott João Dias Semedo J. Zico Kolter VLM 56 3 0 15 Sep 2024
Minimizing Embedding Distortion for Robust Out-of-Distribution Performance Tom Shaked Yuval Goldman Oran Shayer OODD 18 0 0 11 Sep 2024
Improving the Classification Effect of Clinical Images of Diseases for Multi-Source Privacy Protection Tian Bowen Xu Zhengyang Yin Zhihao Wang Jingying Yue Yutao FedML 24 0 0 23 Aug 2024
Efficient and Versatile Robust Fine-Tuning of Zero-shot Models Sungyeon Kim Boseung Jeong Donghyun Kim Suha Kwak VLM 26 2 0 11 Aug 2024
DECIDER: Leveraging Foundation Model Priors for Improved Model Failure Detection and Explanation Rakshith Subramanyam Kowshik Thopalli V. Narayanaswamy Jayaraman J.Thiagarajan 18 2 0 01 Aug 2024
I can listen but cannot read: An evaluation of two-tower multimodal systems for instrument recognition Yannis Vasilakis Rachel M. Bittner Johan Pauwels 35 0 0 25 Jul 2024
Fully Fine-tuned CLIP Models are Efficient Few-Shot Learners Mushui Liu Bozheng Li Yunlong Yu VLM CLIP 21 2 0 04 Jul 2024
SAFT: Towards Out-of-Distribution Generalization in Fine-Tuning Bac Nguyen Stefan Uhlich Fabien Cardinaux Lukas Mauch Marzieh Edraki Aaron Courville OODD CLL VLM 52 3 0 03 Jul 2024
GalLoP: Learning Global and Local Prompts for Vision-Language Models Marc Lafon Elias Ramzi Clément Rambour Nicolas Audebert Nicolas Thome VLM 29 7 0 01 Jul 2024
Controlling Forgetting with Test-Time Data in Continual Learning Vaibhav Singh Rahaf Aljundi Eugene Belilovsky CLL VLM KELM 33 3 0 19 Jun 2024
Few-Shot Recognition via Stage-Wise Retrieval-Augmented Finetuning Tian Liu Huixin Zhang Shubham Parashar Shu Kong 24 2 0 17 Jun 2024
Enhancing Cross-Modal Fine-Tuning with Gradually Intermediate Modality Generation Lincan Cai Shuang Li Wenxuan Ma Jingxuan Kang Binhui Xie Zixun Sun Chengwei Zhu MoE MoMe 35 0 0 13 Jun 2024
On the Use of Anchoring for Training Vision Models V. Narayanaswamy Kowshik Thopalli Rushil Anirudh Yamen Mubarka W. Sakla Jayaraman J. Thiagarajan 32 0 0 01 Jun 2024
Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving Shaoyuan Xie Lingdong Kong Wenwei Zhang Jiawei Ren Liang Pan Kai-xiang Chen Ziwei Liu AAML 50 9 0 27 May 2024
CRoFT: Robust Fine-Tuning with Concurrent Optimization for OOD Generalization and Open-Set OOD Detection Lin Zhu Yifeng Yang Qinying Gu Xinbing Wang Cheng Zhou Nanyang Ye VLM 22 2 0 26 May 2024
Feature Protection For Out-of-distribution Generalization Lu Tan Huei Zhou Yinxiang Huang Zeming Zheng Yujiu Yang OODD 22 0 0 25 May 2024
Selective Classification Under Distribution Shifts Hengyue Liang Le Peng Ju Sun UQCV 31 1 0 08 May 2024
Adapting to Distribution Shift by Visual Domain Prompt Generation Zhixiang Chi Li Gu Tao Zhong Huan Liu Yuanhao Yu Konstantinos N Plataniotis Yang Wang VLM OOD 29 7 0 05 May 2024
Zero-Shot Distillation for Image Encoders: How to Make Effective Use of Synthetic Data Niclas Popp J. H. Metzen Matthias Hein VLM 29 1 0 25 Apr 2024
FLoRA: Enhancing Vision-Language Models with Parameter-Efficient Federated Learning Duy Phuong Nguyen J. P. Muñoz Ali Jannesari VLM 22 6 0 12 Apr 2024
Anchor-based Robust Finetuning of Vision-Language Models Jinwei Han Zhiwen Lin Zhongyi Sun Yingguo Gao Ke Yan Shouhong Ding Yuan Gao Gui-Song Xia VLM 46 6 0 09 Apr 2024
Test-Time Zero-Shot Temporal Action Localization Benedetta Liberatori Alessandro Conti Paolo Rota Yiming Wang Elisa Ricci 19 3 0 08 Apr 2024
T-VSL: Text-Guided Visual Sound Source Localization in Mixtures Tanvir Mahmud Yapeng Tian Diana Marculescu 42 7 0 02 Apr 2024
Model Stock: All we need is just a few fine-tuned models Dong-Hwan Jang Sangdoo Yun Dongyoon Han OODD MoMe 19 38 0 28 Mar 2024
TaskCLIP: Extend Large Vision-Language Model for Task Oriented Object Detection Hanning Chen Wenjun Huang Yang Ni Sanggeon Yun Fei Wen Hugo Latapie Mohsen Imani ObjD MLLM VLM 35 16 0 12 Mar 2024