Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)

3 May 2022

Papers citing "Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP)"

50 / 99 papers shown

Title
X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP Hanxun Huang Sarah Monazam Erfani Yige Li Xingjun Ma James Bailey AAML 34 0 0 08 May 2025
Scaling Language-Free Visual Representation Learning David Fan Shengbang Tong Jiachen Zhu Koustuv Sinha Zhuang Liu ... Michael G. Rabbat Nicolas Ballas Yann LeCun Amir Bar Saining Xie CLIP VLM 56 2 0 01 Apr 2025
Data Distributional Properties As Inductive Bias for Systematic Generalization Felipe del-Rio Alain Raymond-Sáez Daniel Florea Rodrigo Toro Icarte Julio Hurtado Cristian B. Calderon Á. Soto AI4CE 33 0 0 27 Feb 2025
Vision-Language Model Selection and Reuse for Downstream Adaptation Hao-Zhe Tan Zhi-Hua Zhou Lan-Zhe Guo Yu-Feng Li VLM 91 0 0 30 Jan 2025
Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation Reza Akbarian Bafghi Carden Bagwell Avinash Ravichandran Ashish Shrivastava M. Raissi 43 0 0 28 Jan 2025
ProKeR: A Kernel Perspective on Few-Shot Adaptation of Large Vision-Language Models Yassir Bendou Amine Ouasfi Vincent Gripon A. Boukhayma VLM 51 0 0 19 Jan 2025
Point-PRC: A Prompt Learning Based Regulation Framework for Generalizable Point Cloud Analysis Hongyu Sun Qiuhong Ke Y. Wang Wang Chen Kang Yang Deying Li Jianfei Cai 3DPC 70 3 0 17 Jan 2025
The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better Scott Geng Cheng-Yu Hsieh Vivek Ramanujan Matthew Wallingford Chun-Liang Li Pang Wei Koh Ranjay Krishna DiffM 60 6 0 03 Jan 2025
Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuning Zhifang Zhang Shuo He Bingquan Shen Lei Feng Lei Feng AAML 38 0 0 29 Dec 2024
Dual Risk Minimization: Towards Next-Level Robustness in Fine-tuning Zero-Shot Models Kaican Li Weiyan Xie Yongxiang Huang Didan Deng Lanqing Hong Z. Li Ricardo Silva N. Zhang 67 0 0 29 Nov 2024
On the Surprising Effectiveness of Attention Transfer for Vision Transformers Alexander C. Li Yuandong Tian B. Chen Deepak Pathak Xinlei Chen 29 0 0 14 Nov 2024
RaVL: Discovering and Mitigating Spurious Correlations in Fine-Tuned Vision-Language Models Maya Varma Jean-Benoit Delbrouck Zhihong Chen Akshay S. Chaudhari C. Langlotz VLM 42 6 0 06 Nov 2024
Toward a Holistic Evaluation of Robustness in CLIP Models Weijie Tu Weijian Deng Tom Gedeon VLM 34 5 0 02 Oct 2024
Unsupervised Domain Adaptation Via Data Pruning Andrea Napoli Paul White 31 1 0 18 Sep 2024
The Benefits of Balance: From Information Projections to Variance Reduction Lang Liu Ronak R. Mehta Soumik Pal Zaïd Harchaoui 21 0 0 27 Aug 2024
Adversarial Robustification via Text-to-Image Diffusion Models Daewon Choi Jongheon Jeong Huiwon Jang Jinwoo Shin DiffM 35 1 0 26 Jul 2024
LCA-on-the-Line: Benchmarking Out-of-Distribution Generalization with Class Taxonomies Jia Shi Gautam Gare Jinjin Tian Siqi Chai Zhiqiu Lin Arun Vasudevan Di Feng Francesco Ferroni Shu Kong VLM OODD OOD 45 3 0 22 Jul 2024
Towards Adversarially Robust Vision-Language Models: Insights from Design Choices and Prompt Formatting Techniques Rishika Bhagwatkar Shravan Nayak Reza Bayat Alexis Roger Daniel Z Kaplan P. Bashivan Irina Rish AAML VLM 34 1 0 15 Jul 2024
Deciphering the Role of Representation Disentanglement: Investigating Compositional Generalization in CLIP Models Reza Abbasi M. Rohban M. Baghshah CoGe 38 5 0 08 Jul 2024
Automatic benchmarking of large multimodal models via iterative experiment programming Alessandro Conti Enrico Fini Paolo Rota Yiming Wang Massimiliano Mancini Elisa Ricci 35 0 0 18 Jun 2024
Evaluating Durability: Benchmark Insights into Multimodal Watermarking Jielin Qiu William Jongwon Han Xuandong Zhao Shangbang Long Christos Faloutsos Lei Li 51 1 0 06 Jun 2024
Generalization Beyond Data Imbalance: A Controlled Study on CLIP for Transferable Insights Xin Wen Bingchen Zhao Yilun Chen Jiangmiao Pang Xiaojuan Qi 30 3 0 31 May 2024
Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving Shaoyuan Xie Lingdong Kong Wenwei Zhang Jiawei Ren Liang Pan Kai-xiang Chen Ziwei Liu AAML 50 9 0 27 May 2024
Synergy and Diversity in CLIP: Enhancing Performance Through Adaptive Backbone Ensembling Cristian Rodriguez-Opazo Ehsan Abbasnejad Damien Teney Edison Marrese-Taylor Hamed Damirchi A. Hengel VLM 33 1 0 27 May 2024
CLIP with Quality Captions: A Strong Pretraining for Vision Tasks Pavan Kumar Anasosalu Vasu Hadi Pouransari Fartash Faghri Oncel Tuzel VLM CLIP 25 6 0 14 May 2024
Omniview-Tuning: Boosting Viewpoint Invariance of Vision-Language Pre-training Models Shouwei Ruan Yinpeng Dong Hanqing Liu Yao Huang Hang Su Xingxing Wei VLM 45 1 0 18 Apr 2024
Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic Sachin Goyal Pratyush Maini Zachary Chase Lipton Aditi Raghunathan J. Zico Kolter 43 40 0 10 Apr 2024
No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performance Vishaal Udandarao Ameya Prabhu Adhiraj Ghosh Yash Sharma Philip H. S. Torr Adel Bibi Samuel Albanie Matthias Bethge VLM 118 44 0 04 Apr 2024
Predicting the Performance of Foundation Models via Agreement-on-the-Line Aman Mehra Rahul Saxena Taeyoun Kim Christina Baek Zico Kolter Aditi Raghunathan UQCV 31 1 0 02 Apr 2024
Language Plays a Pivotal Role in the Object-Attribute Compositional Generalization of CLIP Reza Abbasi Mohammad Samiei M. Rohban M. Baghshah VLM CoGe 25 0 0 27 Mar 2024
Application-Driven Innovation in Machine Learning David Rolnick Alán Aspuru-Guzik Sara Beery B. Dilkina P. Donti ... Hannah Kerner C. Monteleoni Esther Rolf Milind Tambe Adam White 34 8 0 26 Mar 2024
Analyzing the Roles of Language and Vision in Learning from Limited Data Allison Chen Ilia Sucholutsky Olga Russakovsky Thomas L. Griffiths VLM 21 2 0 15 Feb 2024
An Empirical Study Into What Matters for Calibrating Vision-Language Models Weijie Tu Weijian Deng Dylan Campbell Stephen Gould Tom Gedeon VLM 33 7 0 12 Feb 2024
A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP) Weijie Tu Weijian Deng Tom Gedeon UQCV VLM 20 32 0 12 Feb 2024
A Survey on Safe Multi-Modal Learning System Tianyi Zhao Liangliang Zhang Yao Ma Lu Cheng 52 9 0 08 Feb 2024
On Catastrophic Inheritance of Large Foundation Models Hao Chen Bhiksha Raj Xing Xie Jindong Wang AI4CE 48 12 0 02 Feb 2024
Cross-modality debiasing: using language to mitigate sub-population shifts in imaging Yijiang Pang Hoang Bao Jiayu Zhou 17 0 0 02 Feb 2024
On mitigating stability-plasticity dilemma in CLIP-guided image morphing via geodesic distillation loss Yeongtak Oh Saehyung Lee Uiwon Hwang Sungroh Yoon 30 0 0 19 Jan 2024
CLIP-DINOiser: Teaching CLIP a few DINO tricks for open-vocabulary semantic segmentation Monika Wysoczañska Oriane Siméoni Michael Ramamonjisoa Andrei Bursuc Tomasz Trzciñski Patrick Pérez VLM CLIP 24 29 0 19 Dec 2023
Robustness of Deep Learning for Accelerated MRI: Benefits of Diverse Training Data Kang Lin Reinhard Heckel OOD 27 5 0 16 Dec 2023
BioCLIP: A Vision Foundation Model for the Tree of Life Samuel Stevens Jiaman Wu Matthew J Thompson Elizabeth G Campolongo Chan Hee Song ... Wasila M Dahdul Charles V. Stewart Tanya Berger-Wolf Wei-Lun Chao Yu-Chuan Su 26 62 0 30 Nov 2023
MLLMs-Augmented Visual-Language Representation Learning Yanqing Liu Kai Wang Wenqi Shao Ping Luo Yu Qiao Mike Zheng Shou Kaipeng Zhang Yang You VLM 21 11 0 30 Nov 2023
A Simple Recipe for Language-guided Domain Generalized Segmentation Mohammad Fahes Tuan-Hung Vu Andrei Bursuc Patrick Pérez Raoul de Charette VLM 16 14 0 29 Nov 2023
Zero-shot Retrieval: Augmenting Pre-trained Models with Search Engines Hamed Damirchi Cristian Rodriguez-Opazo Ehsan Abbasnejad Damien Teney Javen Qinfeng Shi Stephen Gould A. Hengel VLM 27 0 0 29 Nov 2023
MobileCLIP: Fast Image-Text Models through Multi-Modal Reinforced Training Pavan Kumar Anasosalu Vasu Hadi Pouransari Fartash Faghri Raviteja Vemulapalli Oncel Tuzel CLIP VLM 24 43 0 28 Nov 2023
Towards Transferable Multi-modal Perception Representation Learning for Autonomy: NeRF-Supervised Masked AutoEncoder Xiaohao Xu 33 0 0 23 Nov 2023
Domain Aligned CLIP for Few-shot Classification Muhammad Waleed Gondal Jochen Gast Inigo Alonso Ruiz Richard Droste Tommaso Macri Suren Kumar Luitpold Staudigl VLM 11 11 0 15 Nov 2023
Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models Andy Zhou Jindong Wang Yu-xiong Wang Haohan Wang VLM 36 6 0 02 Nov 2023
Deep Neural Networks Can Learn Generalizable Same-Different Visual Relations Alexa R. Tartaglini Sheridan Feucht Michael A. Lepori Wai Keen Vong Charles Lovering Brenden Lake Ellie Pavlick ViT OOD 17 3 0 14 Oct 2023
Does CLIP's Generalization Performance Mainly Stem from High Train-Test Similarity? Prasanna Mayilvahanan Thaddäus Wiedemer E. Rusak Matthias Bethge Wieland Brendel OODD 35 22 0 14 Oct 2023