Scaling Language-Image Pre-training via Masking

1 December 2022

Yanghao Li

Haoqi Fan

Ronghang Hu

Christoph Feichtenhofer

Papers citing "Scaling Language-Image Pre-training via Masking"

50 / 58 papers shown

Title
$Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ual-$\mathbf{\texttt{H}}$ead $\mathbf{\texttt{O}}$ptimization$ Simple Semi-supervised Knowledge Distillation from Vision-Language Models via $\mathbf{\texttt{D}}$ ual- $\mathbf{\texttt{H}}$ ead $\mathbf{\texttt{O}}$ ptimization Seongjae Kang Dong Bok Lee Hyungjoon Jang Sung Ju Hwang VLM 38 0 0 12 May 2025
X-Transfer Attacks: Towards Super Transferable Adversarial Attacks on CLIP Hanxun Huang Sarah Monazam Erfani Yige Li Xingjun Ma James Bailey AAML 34 0 0 08 May 2025
DeCLIP: Decoupled Learning for Open-Vocabulary Dense Perception Junjie Wang Bin Chen Yulin Li Bin Kang Y. Chen Zhuotao Tian VLM 38 0 0 07 May 2025
Decoupled Global-Local Alignment for Improving Compositional Understanding Xiaoxing Hu Kaicheng Yang J. Z. Wang Haoran Xu Ziyong Feng Y. Wang VLM 95 0 0 23 Apr 2025
Towards Cardiac MRI Foundation Models: Comprehensive Visual-Tabular Representations for Whole-Heart Assessment and Beyond Yundi Zhang Paul Hager Che Liu Suprosanna Shit C. L. P. Chen Daniel Rueckert Jiazhen Pan 40 0 0 17 Apr 2025
Perception Encoder: The best visual embeddings are not at the output of the network Daniel Bolya Po-Yao (Bernie) Huang Peize Sun Jang Hyun Cho Andrea Madotto ... Shiyu Dong Nikhila Ravi Daniel Li Piotr Dollár Christoph Feichtenhofer ObjD VOS 103 0 0 17 Apr 2025
ELIP: Enhanced Visual-Language Foundation Models for Image Retrieval Guanqi Zhan Yuanpei Liu Kai Han Weidi Xie Andrew Zisserman VLM 111 0 0 21 Feb 2025
Myna: Masking-Based Contrastive Learning of Musical Representations Ori Yonay Tracy Hammond Tianbao Yang AAML 51 0 0 20 Feb 2025
ViLa-MIL: Dual-scale Vision-Language Multiple Instance Learning for Whole Slide Image Classification Jiangbo Shi Chen Li Tieliang Gong Yefeng Zheng Huazhu Fu VLM 60 5 0 12 Feb 2025
A Comprehensive Survey of Foundation Models in Medicine Wasif Khan Seowung Leem Kyle B. See Joshua K. Wong Shaoting Zhang R. Fang AI4CE LM&MA VLM 97 17 0 17 Jan 2025
VideoAuteur: Towards Long Narrative Video Generation Junfei Xiao Feng Cheng Lu Qi Liangke Gui Jiepeng Cen Zhibei Ma Alan L. Yuille Lu Jiang VGen 56 2 0 10 Jan 2025
BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs Sheng Zhang Yanbo Xu Naoto Usuyama Hanwen Xu J. Bagga ... Carlo Bifulco M. Lungren Tristan Naumann Sheng Wang Hoifung Poon LM&MA MedIm 151 198 0 10 Jan 2025
A Simple Recipe for Contrastively Pre-training Video-First Encoders Beyond 16 Frames Pinelopi Papalampidi Skanda Koppula Shreya Pathak Justin T Chiu Joseph Heyward Viorica Patraucean Jiajun Shen Antoine Miech Andrew Zisserman Aida Nematzdeh VLM 58 24 0 31 Dec 2024
CLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance Chu Myaet Thwal Ye Lin Tun Minh N. H. Nguyen Eui-nam Huh Choong Seon Hong VLM 74 0 0 05 Dec 2024
COSMOS: Cross-Modality Self-Distillation for Vision Language Pre-training Sanghwan Kim Rui Xiao Mariana-Iuliana Georgescu Stephan Alaniz Zeynep Akata VLM 70 0 0 02 Dec 2024
Sparse Attention Vectors: Generative Multimodal Model Features Are Discriminative Vision-Language Classifiers Chancharik Mitra Brandon Huang Tianning Chai Zhiqiu Lin Assaf Arbelle Rogerio Feris Leonid Karlinsky Trevor Darrell Deva Ramanan Roei Herzig VLM 121 4 0 28 Nov 2024
RankByGene: Gene-Guided Histopathology Representation Learning Through Cross-Modal Ranking Consistency Wentao Huang Meilong Xu Xiaoling Hu Shahira Abousamra Aniruddha Ganguly ... Prateek Prasanna Tahsin M. Kurc Joel H. Saltz Michael L. Miller C. L. P. Chen 78 0 0 22 Nov 2024
CROPS: A Deployable Crop Management System Over All Possible State Availabilities Jing Wu Zhixin Lai Shengjie Liu Suiyao Chen Ran Tao Pan Zhao Chuyuan Tao Yikun Cheng N. Hovakimyan OffRL 35 0 0 09 Nov 2024
Revisiting MAE pre-training for 3D medical image segmentation Tassilo Wald Constantin Ulrich Stanislav Lukyanenko Andrei Goncharov Alberto Paderno Leander Maerkisch Paul F. Jäger Paul F. Jäger Klaus Maier-Hein 37 2 0 30 Oct 2024
Probabilistic Language-Image Pre-Training Sanghyuk Chun Wonjae Kim Song Park Sangdoo Yun MLLM VLM CLIP 81 4 2 24 Oct 2024
TIPS: Text-Image Pretraining with Spatial awareness Kevis-Kokitsi Maninis Kaifeng Chen Soham Ghosh Arjun Karpur Koert Chen ... Jan Dlabal Dan Gnanapragasam Mojtaba Seyedhosseini Howard Zhou Andre Araujo VLM 30 3 0 21 Oct 2024
Locality Alignment Improves Vision-Language Models Ian Covert Tony Sun James Y. Zou Tatsunori Hashimoto VLM 64 3 0 14 Oct 2024
DetailCLIP: Detail-Oriented CLIP for Fine-Grained Tasks Amin Karimi Monsefi Kishore Prakash Sailaja Ali Alilooee Ser-Nam Lim R. Ramnath VLM 33 6 0 10 Sep 2024
ComKD-CLIP: Comprehensive Knowledge Distillation for Contrastive Language-Image Pre-traning Model Yifan Chen Xiaozhen Qiao Zhe Sun Xuelong Li VLM 37 3 0 08 Aug 2024
Graph-Based Captioning: Enhancing Visual Descriptions by Interconnecting Region Captions Yu-Guan Hsieh Cheng-Yu Hsieh Shih-Ying Yeh Louis Béthune Hadi Pour Ansari Pavan Kumar Anasosalu Vasu Chun-Liang Li Ranjay Krishna Oncel Tuzel Marco Cuturi 58 4 0 09 Jul 2024
An Image is Worth 32 Tokens for Reconstruction and Generation Qihang Yu Mark Weber XueQing Deng Xiaohui Shen Daniel Cremers Liang-Chieh Chen VLM ViT 44 79 0 11 Jun 2024
Benchmarking Vision-Language Contrastive Methods for Medical Representation Learning Shuvendu Roy Yasaman Parhizkar Franklin Ogidi Vahid Reza Khazaie Michael Colacci Ali Etemad Elham Dolatabadi Arash Afkanpour VLM 40 1 0 11 Jun 2024
Scaling White-Box Transformers for Vision Jinrui Yang Xianhang Li Druv Pai Yuyin Zhou Yi-An Ma Yaodong Yu Cihang Xie ViT 34 9 0 30 May 2024
Modeling Caption Diversity in Contrastive Vision-Language Pretraining Samuel Lavoie Polina Kirichenko Mark Ibrahim Mahmoud Assran Andrew Gordon Wilson Aaron Courville Nicolas Ballas CLIP VLM 59 19 0 30 Apr 2024
Semi-supervised Text-based Person Search Daming Gao Yang Bai Min Cao Hao Dou Mang Ye Min Zhang 39 1 0 28 Apr 2024
Salience-Based Adaptive Masking: Revisiting Token Dynamics for Enhanced Pre-training Hyesong Choi Hyejin Park Kwang Moo Yi Sungmin Cha Dongbo Min 34 9 0 12 Apr 2024
Siamese Vision Transformers are Scalable Audio-visual Learners Yan-Bo Lin Gedas Bertasius 37 5 0 28 Mar 2024
Rethinking Multi-view Representation Learning via Distilled Disentangling Guanzhou Ke Bo Wang Xiaoli Wang Shengfeng He 32 3 0 16 Mar 2024
MedFLIP: Medical Vision-and-Language Self-supervised Fast Pre-Training with Masked Autoencoder Lei Li Tianfang Zhang Xinglin Zhang Jiaqi Liu Bingqi Ma Yan-chun Luo Tao Chen MedIm 24 0 0 07 Mar 2024
Generalizable Whole Slide Image Classification with Fine-Grained Visual-Semantic Interaction Hao Li Ying Chen Yifei Chen Wenxian Yang Bowen Ding Yuchen Han Liansheng Wang Rongshan Yu 31 15 0 29 Feb 2024
VideoPrism: A Foundational Visual Encoder for Video Understanding Long Zhao N. B. Gundavarapu Liangzhe Yuan Hao Zhou Shen Yan ... Huisheng Wang Hartwig Adam Mikhail Sirotenko Ting Liu Boqing Gong VGen 27 29 0 20 Feb 2024
Rethinking Patch Dependence for Masked Autoencoders Letian Fu Long Lian Renhao Wang Baifeng Shi Xudong Wang Adam Yala Trevor Darrell Alexei A. Efros Ken Goldberg 26 14 0 25 Jan 2024
Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model Shraman Pramanick Guangxing Han Rui Hou Sayan Nag Ser-Nam Lim Nicolas Ballas Qifan Wang Rama Chellappa Amjad Almahairi VLM MLLM 38 29 0 19 Dec 2023
Alpha-CLIP: A CLIP Model Focusing on Wherever You Want Zeyi Sun Ye Fang Tong Wu Pan Zhang Yuhang Zang Shu Kong Yuanjun Xiong Dahua Lin Jiaqi Wang VLM CLIP 25 82 0 06 Dec 2023
Unified Medical Image Pre-training in Language-Guided Common Semantic Space Xiaoxuan He Yifan Yang Xinyang Jiang Xufang Luo Haoji Hu Siyun Zhao Dongsheng Li Yuqing Yang Lili Qiu 32 1 0 24 Nov 2023
Pretrain like Your Inference: Masked Tuning Improves Zero-Shot Composed Image Retrieval Junyang Chen Hanjiang Lai VLM 34 15 0 13 Nov 2023
PonderV2: Pave the Way for 3D Foundation Model with A Universal Pre-training Paradigm Haoyi Zhu Honghui Yang Xiaoyang Wu Di Huang Sha Zhang ... Hengshuang Zhao Chunhua Shen Yu Qiao Tong He Wanli Ouyang SSL 69 42 0 12 Oct 2023
Cross-Modal Retrieval Meets Inference:Improving Zero-Shot Classification with Cross-Modal Retrieval Seong-Hoon Eom Namgyu Ho Jaehoon Oh Se-Young Yun CLIP VLM 23 0 0 29 Aug 2023
Foundational Models Defining a New Era in Vision: A Survey and Outlook Muhammad Awais Muzammal Naseer Salman Khan Rao Muhammad Anwer Hisham Cholakkal M. Shah Ming Yang F. Khan VLM 18 117 0 25 Jul 2023
Towards a Visual-Language Foundation Model for Computational Pathology Ming Y. Lu Bowen Chen Drew F. K. Williamson Richard J. Chen Ivy Liang ... Andrew Zhang L. Le Georg Gerber Anil V. Parwani Faisal Mahmood VLM MedIm 33 46 0 24 Jul 2023
Knowledge Graph Self-Supervised Rationalization for Recommendation Yuhao Yang Chao Huang Lianghao Xia Chunzhen Huang 22 88 0 06 Jul 2023
Enhancing Vision-Language Pre-Training with Jointly Learned Questioner and Dense Captioner Zikang Liu Sihan Chen Longteng Guo Handong Li Xingjian He J. Liu 13 1 0 19 May 2023
Self-Chained Image-Language Model for Video Localization and Question Answering Shoubin Yu Jaemin Cho Prateek Yadav Mohit Bansal 36 129 0 11 May 2023
RECLIP: Resource-efficient CLIP by Training with Small Images Runze Li Dahun Kim B. Bhanu Weicheng Kuo VLM CLIP 22 12 0 12 Apr 2023
EVA-CLIP: Improved Training Techniques for CLIP at Scale Quan-Sen Sun Yuxin Fang Ledell Yu Wu Xinlong Wang Yue Cao CLIP VLM 52 463 0 27 Mar 2023