Improving CLIP Training with Language Rewrites

31 May 2023

Papers citing "Improving CLIP Training with Language Rewrites"

50 / 113 papers shown

Title
MAKE: Multi-Aspect Knowledge-Enhanced Vision-Language Pretraining for Zero-shot Dermatological Assessment Siyuan Yan X. Li Ming Hu Yiwen Jiang Zhen Yu Zongyuan Ge MedIm VLM 18 0 0 14 May 2025
FedMVP: Federated Multi-modal Visual Prompt Tuning for Vision-Language Models Mainak Singha Subhankar Roy Sarthak Mehrotra Ankit Jha Moloud Abdar Biplab Banerjee Elisa Ricci VLM VPVLM 108 0 0 29 Apr 2025
Decoupled Global-Local Alignment for Improving Compositional Understanding Xiaoxing Hu Kaicheng Yang J. Z. Wang Haoran Xu Ziyong Feng Y. Wang VLM 104 0 0 23 Apr 2025
A Multimodal Recaptioning Framework to Account for Perceptual Diversity in Multilingual Vision-Language Modeling Kyle Buettner Jacob Emmerson Adriana Kovashka 15 0 0 19 Apr 2025
Perception Encoder: The best visual embeddings are not at the output of the network Daniel Bolya Po-Yao (Bernie) Huang Peize Sun Jang Hyun Cho Andrea Madotto ... Shiyu Dong Nikhila Ravi Daniel Li Piotr Dollár Christoph Feichtenhofer ObjD VOS 103 0 0 17 Apr 2025
EditCLIP: Representation Learning for Image Editing Qian Wang Aleksandar Cvejic Abdelrahman Eldesokey Peter Wonka 61 0 0 26 Mar 2025
OCRT: Boosting Foundation Models in the Open World with Object-Concept-Relation Triad Luyao Tang Yuxuan Yuan C. L. P. Chen Zeyu Zhang Yue Huang Kun Zhang 48 0 0 24 Mar 2025
Bridging Writing Manner Gap in Visual Instruction Tuning by Creating LLM-aligned Instructions Dong Jing Nanyi Fei Zhiwu Lu 42 0 0 24 Mar 2025
Seeing What Matters: Empowering CLIP with Patch Generation-to-Selection Gensheng Pei Tao Chen Yujia Wang Xinhao Cai Xiangbo Shu Tianfei Zhou Yazhou Yao VLM 51 1 0 21 Mar 2025
OSLoPrompt: Bridging Low-Supervision Challenges and Open-Set Domain Generalization in CLIP M. Cui Divyam Gupta Mainak Singha Sai Bhargav Rongali Ankit Jha Muhammad Haris Khan Biplab Banerjee VLM 53 1 0 20 Mar 2025
Squeeze Out Tokens from Sample for Finer-Grained Data Governance Weixiong Lin Chen Ju Haicheng Wang Shengchao Hu Shuai Xiao ... Yuheng Jiao Mingshuai Yao Jinsong Lan Qingwen Liu Ying Chen 50 0 0 18 Mar 2025
Dynamic Relation Inference via Verb Embeddings Omri Suissa Muhiim Ali Ariana Azarbal Hui Shen Shekhar Pradhan 41 0 0 17 Mar 2025
Concept-as-Tree: Synthetic Data is All You Need for VLM Personalization Ruichuan An Kai Zeng Ming Lu Sihan Yang Renrui Zhang Huitong Ji Qizhe Zhang Y. Luo Hao Liang Wentao Zhang 63 0 0 17 Mar 2025
Enhanced Continual Learning of Vision-Language Models with Model Fusion Haoyuan Gao Zicong Zhang Yuqi Wei Linglan Zhao Guilin Li Y. Li Linghe Kong Weiran Huang CLL VLM 124 0 0 12 Mar 2025
Narrating the Video: Boosting Text-Video Retrieval via Comprehensive Utilization of Frame-Level Captions Chan hur Jeong-hun Hong Dong-hun Lee Dabin Kang Semin Myeong Sang-hyo Park Hyeyoung Park 56 0 0 07 Mar 2025
FAA-CLIP: Federated Adversarial Adaptation of CLIP Yihang Wu A. Chaddad Christian Desrosiers Tareef Daqqaq R. Kateb VLM 49 0 0 26 Feb 2025
Contrastive Localized Language-Image Pre-Training Hong-You Chen Zhengfeng Lai H. Zhang X. Wang Marcin Eichner Keen You Meng Cao Bowen Zhang Y. Yang Zhe Gan CLIP VLM 68 7 0 20 Feb 2025
GAIA: A Global, Multi-modal, Multi-scale Vision-Language Dataset for Remote Sensing Image Analysis Angelos Zavras Dimitrios Michail Xiao Xiang Zhu Begum Demir Ioannis Papoutsis VLM 81 0 0 13 Feb 2025
Captured by Captions: On Memorization and its Mitigation in CLIP Models Wenhao Wang Adam Dziedzic Grace C. Kim Michael Backes Franziska Boenisch 79 0 0 11 Feb 2025
Spend Wisely: Maximizing Post-Training Gains in Iterative Synthetic Data Boostrapping Pu Yang Yunzhen Feng Ziyuan Chen Yuhang Wu Zhuoyuan Li DiffM 101 0 0 31 Jan 2025
MM-GEN: Enhancing Task Performance Through Targeted Multimodal Data Curation S. Joshi Besmira Nushi Vidhisha Balachandran Varun Chandrasekaran Vibhav Vineet Neel Joshi Baharan Mirzasoleiman MLLM VLM 44 0 0 07 Jan 2025
GFG -- Gender-Fair Generation: A CALAMITA Challenge Simona Frenda Andrea Piergentili Beatrice Savoldi Marco Madeddu Martina Rosola Silvia Casola Chiara Ferrando V. Patti Matteo Negri L. Bentivogli 35 1 0 31 Dec 2024
A Decade of Deep Learning: A Survey on The Magnificent Seven Dilshod Azizov Muhammad Arslan Manzoor Velibor Bojkovic Yingxu Wang Z. Wang ... Liang Li Siwei Liu Yu Zhong Wei Liu Shangsong Liang OOD AI4TS MedIm 116 0 0 13 Dec 2024
FLAIR: VLM with Fine-grained Language-informed Image Representations Rui Xiao Sanghwan Kim Mariana-Iuliana Georgescu Zeynep Akata Stephan Alaniz VLM CLIP 71 2 0 04 Dec 2024
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training Sanghwan Kim Rui Xiao Mariana-Iuliana Georgescu Stephan Alaniz Zeynep Akata VLM 70 1 0 02 Dec 2024
Advancing Myopia To Holism: Fully Contrastive Language-Image Pre-training Haicheng Wang Chen Ju Weixiong Lin Shuai Xiao Mengting Chen ... Mingshuai Yao Jinsong Lan Ying Chen Qingwen Liu Yanfeng Wang VLM CLIP 70 4 0 30 Nov 2024
OphCLIP: Hierarchical Retrieval-Augmented Learning for Ophthalmic Surgical Video-Language Pretraining Ming Hu Kun Yuan Yaling Shen Feilong Tang Xiaohao Xu ... Jin Ye N. Padoy Nassir Navab Junjun He Zongyuan Ge VLM CLIP 85 11 0 23 Nov 2024
ITACLIP: Boosting Training-Free Semantic Segmentation with Image, Text, and Architectural Enhancements M. Arda Aydın Efe Mert Çırpar Elvin Abdinli Gözde B. Ünal Y. Sahin VLM 66 0 0 18 Nov 2024
Past, Present, and Future of Sensor-Based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Task H. Haresamudram Chi Ian Tang Sungho Suh P. Lukowicz Thomas Ploetz 74 2 0 11 Nov 2024
TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negatives Maitreya Patel Abhiram Kusumba Sheng Cheng Changhoon Kim Tejas Gokhale Chitta Baral Yezhou Yang CoGe CLIP 50 7 0 04 Nov 2024
Multilingual Vision-Language Pre-training for the Remote Sensing Domain João Daniel Silva João Magalhães D. Tuia Bruno Martins CLIP VLM 30 1 0 30 Oct 2024
Beyond Filtering: Adaptive Image-Text Quality Enhancement for MLLM Pretraining Han Huang Yuqi Huo Zijia Zhao Haoyu Lu Shu Wu B. Wang Qiang Liu Weipeng Chen Liang Wang VLM 25 1 0 21 Oct 2024
TIPS: Text-Image Pretraining with Spatial awareness Kevis-Kokitsi Maninis Kaifeng Chen Soham Ghosh Arjun Karpur Koert Chen ... Jan Dlabal Dan Gnanapragasam Mojtaba Seyedhosseini Howard Zhou Andre Araujo VLM 35 3 0 21 Oct 2024
CtrlSynth: Controllable Image Text Synthesis for Data-Efficient Multimodal Learning Qingqing Cao Mahyar Najibi Sachin Mehta CLIP DiffM 30 1 0 15 Oct 2024
LG-CAV: Train Any Concept Activation Vector with Language Guidance Qihan Huang Jie Song Mengqi Xue H. Zhang Bingde Hu Huiqiong Wang Hao Jiang Xingen Wang Mingli Song VLM 29 0 0 14 Oct 2024
LatteCLIP: Unsupervised CLIP Fine-Tuning via LMM-Synthetic Texts Anh-Quan Cao M. Jaritz Matthieu Guillaumin Raoul de Charette Loris Bazzani VLM CLIP 39 2 0 10 Oct 2024
HERM: Benchmarking and Enhancing Multimodal LLMs for Human-Centric Understanding Keliang Li Zaifei Yang Jiahe Zhao Hongze Shen Ruibing Hou Hong Chang Shiguang Shan Xilin Chen VLM 26 0 0 09 Oct 2024
LoTLIP: Improving Language-Image Pre-training for Long Text Understanding Wei Wu Kecheng Zheng Shuailei Ma Fan Lu Yuxin Guo Yifei Zhang Wei Chen Qingpei Guo Yujun Shen Zheng-Jun Zha VLM 25 9 0 07 Oct 2024
Revisit Large-Scale Image-Caption Data in Pre-training Multimodal Foundation Models Zhengfeng Lai Vasileios Saveris C. L. P. Chen Hong-You Chen Haotian Zhang ... Wenze Hu Zhe Gan Peter Grasch Meng Cao Yinfei Yang VLM 33 3 0 03 Oct 2024
Quantifying the Gaps Between Translation and Native Perception in Training for Multimodal, Multilingual Retrieval Kyle Buettner Adriana Kovashka VLM 17 4 0 02 Oct 2024
SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Images Kaiyu Li Ruixun Liu Xiangyong Cao Deyu Meng Zhi Wang Deyu Meng Zhi Wang 30 3 0 02 Oct 2024
Procedure-Aware Surgical Video-language Pretraining with Hierarchical Knowledge Augmentation Kun Yuan V. Srivastav Nassir Navab N. Padoy 44 7 0 30 Sep 2024
Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understanding Xiao Wang Jianlong Wu Zijia Lin Fuzheng Zhang Di Zhang Liqiang Nie VGen 27 1 0 29 Sep 2024
The Hard Positive Truth about Vision-Language Compositionality Amita Kamath Cheng-Yu Hsieh Kai-Wei Chang Ranjay Krishna CLIP CoGe VLM 25 5 0 26 Sep 2024
Finetuning CLIP to Reason about Pairwise Differences Dylan Sam Devin Willmott João Dias Semedo J. Zico Kolter VLM 61 3 0 15 Sep 2024
TempMe: Video Temporal Token Merging for Efficient Text-Video Retrieval Leqi Shen Tianxiang Hao Tao He Sicheng Zhao Pengzhang Liu Yongjun Bao Guiguang Ding Guiguang Ding 100 7 0 02 Sep 2024
LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillation Fangxun Shu Yue Liao Le Zhuo Chenning Xu Guanghao Zhang ... Bolin Li Zhelun Yu Si Liu Hongsheng Li Hao Jiang VLM MoE 32 8 0 28 Aug 2024
Limitations in Employing Natural Language Supervision for Sensor-Based Human Activity Recognition -- And Ways to Overcome Them H. Haresamudram Apoorva Beedu Mashfiqui Rabbi Sankalita Saha Irfan Essa Thomas Ploetz 31 4 0 21 Aug 2024
LocalValueBench: A Collaboratively Built and Extensible Benchmark for Evaluating Localized Value Alignment and Ethical Safety in Large Language Models Achintya Gopal Nicholas Wai Long Lau Eva Adelina Susanto Chi Lok Yu Aditya Paul ELM 23 7 0 27 Jul 2024
AgentPeerTalk: Empowering Students through Agentic-AI-Driven Discernment of Bullying and Joking in Peer Interactions in Schools Aditya Paul Chi Lok Yu Eva Adelina Susanto Nicholas Wai Long Lau Gwenyth Isobel Meadows LLMAG 35 3 0 27 Jul 2024