A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Models

13 February 2023

J. Allingham

Jie Jessie Ren

Michael W. Dusenberry

Balaji Lakshminarayanan

LLMAG

VLM

ArXiv PDF HTML

Papers citing "A Simple Zero-shot Prompt Weighting Technique to Improve Prompt Ensembling in Text-Image Models"

29 / 29 papers shown

Title
EcoWikiRS: Learning Ecological Representation of Satellite Images from Weak Supervision with Species Observations and Wikipedia Valerie Zermatten J. Castillo-Navarro Pallavi Jain D. Tuia Diego Marcos 57 0 0 28 Apr 2025
SecPE: Secure Prompt Ensembling for Private and Robust Large Language Models Jiawen Zhang Kejia Chen Zunlei Feng Jian Lou Mingli Song J. Liu X. J. Yang AAML SILM FedML 43 1 0 02 Feb 2025
SenCLIP: Enhancing zero-shot land-use mapping for Sentinel-2 with ground-level prompting Pallavi Jain Dino Ienco R. Interdonato Tristan Berchoux Diego Marcos VLM 76 3 0 11 Dec 2024
Retaining and Enhancing Pre-trained Knowledge in Vision-Language Models with Prompt Ensembling Donggeun Kim Yujin Jo Myungjoo Lee Taesup Kim VLM 73 0 0 10 Dec 2024
Robotic State Recognition with Image-to-Text Retrieval Task of Pre-Trained Vision-Language Model and Black-Box Optimization Kento Kawaharazuka Yoshiki Obinata Naoaki Kanazawa Kei Okada Masayuki Inaba 26 0 0 30 Oct 2024
Enhancing Zero-Shot Vision Models by Label-Free Prompt Distribution Learning and Bias Correcting Xingyu Zhu B. Zhu Yi Tan Shuo Wang Y. Hao Hanwang Zhang VLM VPVLM 29 1 0 25 Oct 2024
Review of Zero-Shot and Few-Shot AI Algorithms in The Medical Domain Maged Badawi Mohammedyahia Abushanab Sheethal Bhat Andreas K. Maier VLM 37 2 0 23 Jun 2024
An Information Compensation Framework for Zero-Shot Skeleton-based Action Recognition Haojun Xu Yanlei Gao Jie Li Xinbo Gao 35 2 0 02 Jun 2024
What Do You See? Enhancing Zero-Shot Image Classification with Multimodal Large Language Models Abdelrahman Abdelhamed Mahmoud Afifi Alec Go MLLM VLM 29 3 0 24 May 2024
Improving Recall of Large Language Models: A Model Collaboration Approach for Relational Triple Extraction Zepeng Ding Wenhao Huang Jiaqing Liang Deqing Yang Yanghua Xiao KELM 29 4 0 15 Apr 2024
Prompt Learning via Meta-Regularization Jinyoung Park Juyeon Ko Hyunwoo J. Kim VLM VPVLM 35 14 0 01 Apr 2024
OTSeg: Multi-prompt Sinkhorn Attention for Zero-Shot Semantic Segmentation Kwanyoung Kim Y. Oh Jong Chul Ye VLM 41 7 0 21 Mar 2024
C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersion Hee Suk Yoon Eunseop Yoon Joshua Tian Jin Tee M. Hasegawa-Johnson Yingzhen Li C. Yoo VLM 53 23 0 21 Mar 2024
On Sensitivity of Learning with Limited Labelled Data to the Effects of Randomness: Impact of Interactions and Systematic Choices Branislav Pecher Ivan Srba M. Bieliková 51 3 0 20 Feb 2024
Tailoring Education with GenAI: A New Horizon in Lesson Planning K. Karpouzis Dimitris Pantazatos Joanna Taouki Kalliopi Meli 16 8 0 12 Feb 2024
A Simple Recipe for Language-guided Domain Generalized Segmentation Mohammad Fahes Tuan-Hung Vu Andrei Bursuc Patrick Pérez Raoul de Charette VLM 16 14 0 29 Nov 2023
Descriptor and Word Soups: Overcoming the Parameter Efficiency Accuracy Tradeoff for Out-of-Distribution Few-shot Learning Christopher Liao Theodoros Tsiligkaridis Brian Kulis OODD 31 5 0 21 Nov 2023
Generalized Logit Adjustment: Calibrating Fine-tuned Models by Removing Label Bias in Foundation Models Beier Zhu Kaihua Tang Qianru Sun Hanwang Zhang 14 21 0 12 Oct 2023
Investigating the Limitation of CLIP Models: The Worst-Performing Categories Jiejing Shao Jiang-Xin Shi Xiao-Wen Yang Lan-Zhe Guo Yu-Feng Li VLM 21 10 0 05 Oct 2023
AutoCLIP: Auto-tuning Zero-Shot Classifiers for Vision-Language Models Sanghwan Kim Hao Tang Fisher Yu VLM CLIP 8 4 0 28 Sep 2023
Improving CLIP Robustness with Knowledge Distillation and Self-Training Clement Laroudie Andrei Bursuc Mai Lan Ha Gianni Franchi VLM 22 5 0 19 Sep 2023
Large Language Models in Cryptocurrency Securities Cases: Can a GPT Model Meaningfully Assist Lawyers? Arianna Trozze Toby P Davies Bennett Kleinberg AILaw ELM 25 11 0 11 Aug 2023
SLPT: Selective Labeling Meets Prompt Tuning on Label-Limited Lesion Segmentation Fan Bai K. Yan Xiaoyu Bai Xinyu Mao Xiaoli Yin Jingren Zhou Yu Shi Le Lu Max Q.-H. Meng VLM 13 2 0 09 Aug 2023
Out-of-Distribution Detection and Selective Generation for Conditional Language Models Jie Jessie Ren Jiaming Luo Yao-Min Zhao Kundan Krishna Mohammad Saleh Balaji Lakshminarayanan Peter J. Liu OODD 64 94 0 30 Sep 2022
Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models Manli Shu Weili Nie De-An Huang Zhiding Yu Tom Goldstein Anima Anandkumar Chaowei Xiao VLM VPVLM 175 279 0 15 Sep 2022
Learning to Prompt for Vision-Language Models Kaiyang Zhou Jingkang Yang Chen Change Loy Ziwei Liu VPVLM CLIP VLM 322 2,249 0 02 Sep 2021
Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision Chao Jia Yinfei Yang Ye Xia Yi-Ting Chen Zarana Parekh Hieu H. Pham Quoc V. Le Yun-hsuan Sung Zhen Li Tom Duerig VLM CLIP 293 3,683 0 11 Feb 2021
Making Pre-trained Language Models Better Few-shot Learners Tianyu Gao Adam Fisch Danqi Chen 241 1,913 0 31 Dec 2020
ImageNet Large Scale Visual Recognition Challenge Olga Russakovsky Jia Deng Hao Su J. Krause S. Satheesh ... A. Karpathy A. Khosla Michael S. Bernstein Alexander C. Berg Li Fei-Fei VLM ObjD 282 39,170 0 01 Sep 2014