Contrastive Learning of Medical Visual Representations from Paired Images and Text

2 October 2020

Yuhao Zhang

Hang Jiang

Yasuhide Miura

Christopher D. Manning

C. Langlotz

MedIm

ArXiv PDF HTML

Papers citing "Contrastive Learning of Medical Visual Representations from Paired Images and Text"

50 / 117 papers shown

Title
Perception Encoder: The best visual embeddings are not at the output of the network Daniel Bolya Po-Yao (Bernie) Huang Peize Sun Jang Hyun Cho Andrea Madotto ... Shiyu Dong Nikhila Ravi Daniel Li Piotr Dollár Christoph Feichtenhofer ObjD VOS 103 0 0 17 Apr 2025
Negate or Embrace: On How Misalignment Shapes Multimodal Representation Learning Yichao Cai Yuhang Liu Erdun Gao T. Jiang Zhen Zhang Anton van den Hengel J. Shi 55 0 0 14 Apr 2025
A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text? Julio Silva-Rodríguez Jose Dolz Ismail ben Ayed VLM MedIm 31 0 0 07 Apr 2025
MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representations Ziyang Zhang Yang Yu Yucheng Chen Xulei Yang S. Yeo MedIm 51 1 0 02 Mar 2025
Revisiting CLIP: Efficient Alignment of 3D MRI and Tabular Data using Domain-Specific Foundation Models Jakob Krogh Petersen Valdemar Licht Mads Nielsen Asbjørn Munk VLM 30 0 0 23 Jan 2025
MedFILIP: Medical Fine-grained Language-Image Pre-training Xinjie Liang Xiangyu Li Fanding Li Jie Jiang Qing Dong Wei Wang K. Wang Suyu Dong Gongning Luo Shuo Li LM&MA VLM MedIm 64 3 0 18 Jan 2025
A Comprehensive Survey of Foundation Models in Medicine Wasif Khan Seowung Leem Kyle B. See Joshua K. Wong Shaoting Zhang R. Fang AI4CE LM&MA VLM 97 18 0 17 Jan 2025
MedGrad E-CLIP: Enhancing Trust and Transparency in AI-Driven Skin Lesion Diagnosis Sadia Kamal Tim Oates MedIm 39 0 0 12 Jan 2025
BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs Sheng Zhang Yanbo Xu Naoto Usuyama Hanwen Xu J. Bagga ... Carlo Bifulco M. Lungren Tristan Naumann Sheng Wang Hoifung Poon LM&MA MedIm 151 205 0 10 Jan 2025
Language-guided Medical Image Segmentation with Target-informed Multi-level Contrastive Alignments Mingjian Li Mingyuan Meng Shuchang Ye David Dagan Feng Lei Bi Jinman Kim 82 0 0 18 Dec 2024
jina-clip-v2: Multilingual Multimodal Embeddings for Text and Images Andreas Koukounas Georgios Mastrapas Bo Wang Mohammad Kalim Akram Sedigheh Eslami Michael Gunther Isabelle Mohr Saba Sturua Scott Martens Nan Wang VLM 103 7 0 11 Dec 2024
CLIP-PING: Boosting Lightweight Vision-Language Models with Proximus Intrinsic Neighbors Guidance Chu Myaet Thwal Ye Lin Tun Minh N. H. Nguyen Eui-nam Huh Choong Seon Hong VLM 74 0 0 05 Dec 2024
Sparse Attention Vectors: Generative Multimodal Model Features Are Discriminative Vision-Language Classifiers Chancharik Mitra Brandon Huang Tianning Chai Zhiqiu Lin Assaf Arbelle Rogerio Feris Leonid Karlinsky Trevor Darrell Deva Ramanan Roei Herzig VLM 121 4 0 28 Nov 2024
Cross- and Intra-image Prototypical Learning for Multi-label Disease Diagnosis and Interpretation Chong Wang Fengbei Liu Yuanhong Chen Helen Frazer Gustavo Carneiro 32 2 0 07 Nov 2024
Can Medical Vision-Language Pre-training Succeed with Purely Synthetic Data? Che Liu Zhongwei Wan Haozhe Wang Yinda Chen T. Qaiser Chen Jin Fariba Yousefi Nikolay Burlutskiy Rossella Arcucci VLM SyDa LM&MA MedIm 53 2 0 17 Oct 2024
Advancing Medical Radiograph Representation Learning: A Hybrid Pre-training Paradigm with Multilevel Semantic Granularity Hanqi Jiang Xixuan Hao Yuzhou Huang Chong Ma Jiaxun Zhang Yi Pan Ruimao Zhang MedIm 35 0 0 01 Oct 2024
Multi-View and Multi-Scale Alignment for Contrastive Language-Image Pre-training in Mammography Yuexi Du John Onofrey Nicha Dvornek VLM 45 1 0 26 Sep 2024
Robust image representations with counterfactual contrastive learning Mélanie Roschewitz Fabio De Sousa Ribeiro Tian Xia G. Khara Ben Glocker OOD MedIm 43 2 0 16 Sep 2024
EEG-Language Modeling for Pathology Detection Sam Gijsen Kerstin Ritter 39 0 0 02 Sep 2024
How Does Diverse Interpretability of Textual Prompts Impact Medical Vision-Language Zero-Shot Tasks? Sicheng Wang Che Liu Rossella Arcucci VLM MedIm 34 0 0 31 Aug 2024
From Latent to Engine Manifolds: Analyzing ImageBind's Multimodal Embedding Space Andrew Hamara Pablo Rivas 16 1 0 30 Aug 2024
A new approach for encoding code and assisting code understanding Mengdan Fan Changde Du Haiyan Zhao Zhi Jin 41 0 0 01 Aug 2024
Prompting Medical Large Vision-Language Models to Diagnose Pathologies by Visual Question Answering Danfeng Guo Sumitaka Honji LRM 62 0 0 31 Jul 2024
Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation Liwen Sun James Zhao Megan Han Chenyan Xiong MedIm 45 8 0 21 Jul 2024
Duoduo CLIP: Efficient 3D Understanding with Multi-View Images Han-Hung Lee Yiming Zhang Angel X. Chang 3DPC 36 3 0 17 Jun 2024
Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training Jinxia Yang Bing-Huang Su Wayne Xin Zhao Ji-Rong Wen 32 2 0 30 May 2024
Topological Perspectives on Optimal Multimodal Embedding Spaces Abdul Aziz Abdul Rahim BDL 29 0 0 29 May 2024
Mammo-CLIP: A Vision Language Foundation Model to Enhance Data Efficiency and Robustness in Mammography Shantanu Ghosh Clare B. Poynton Shyam Visweswaran Kayhan Batmanghelich VLM 32 8 0 20 May 2024
Pseudo-Prompt Generating in Pre-trained Vision-Language Models for Multi-Label Medical Image Classification Yaoqin Ye Junjie Zhang Hongwei Shi MedIm VLM 35 0 0 10 May 2024
X-MIC: Cross-Modal Instance Conditioning for Egocentric Action Generalization Anna Kukleva Fadime Sener Edoardo Remelli Bugra Tekin Eric Sauser Bernt Schiele Shugao Ma VLM EgoV 29 1 0 28 Mar 2024
N-Modal Contrastive Losses with Applications to Social Media Data in Trimodal Space William Theisen Walter J. Scheirer 26 1 0 18 Mar 2024
MedFLIP: Medical Vision-and-Language Self-supervised Fast Pre-Training with Masked Autoencoder Lei Li Tianfang Zhang Xinglin Zhang Jiaqi Liu Bingqi Ma Yan-chun Luo Tao Chen MedIm 32 0 0 07 Mar 2024
Beyond DAGs: A Latent Partial Causal Model for Multimodal Learning Yuhang Liu Zhen Zhang Dong Gong Biwei Huang Mingming Gong A. Hengel Kun Zhang Javen Qinfeng Shi J. Shi 41 7 0 09 Feb 2024
Enhancing the vision-language foundation model with key semantic knowledge-emphasized report refinement Cheng Li Weijian Huang Hao Yang Jiarun Liu Shanshan Wang MedIm 30 4 0 21 Jan 2024
Enhancing medical vision-language contrastive learning via inter-matching relation modelling Mingjian Li Mingyuan Meng M. Fulham David Dagan Feng Lei Bi Jinman Kim VLM 35 1 0 19 Jan 2024
UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic Cross-modal Learnable Prompts Chenlu Zhan Yufei Zhang Yu Lin Gaoang Wang Hongwei Wang VLM MedIm 26 5 0 18 Dec 2023
UniChest: Conquer-and-Divide Pre-training for Multi-Source Chest X-Ray Classification Tianjie Dai Ruipeng Zhang Feng Hong Jiangchao Yao Ya-Qin Zhang Yanfeng Wang 22 8 0 18 Dec 2023
Multimodal Pretraining of Medical Time Series and Notes Ryan N. King Tianbao Yang Bobak J. Mortazavi 23 12 0 11 Dec 2023
Unified Medical Image Pre-training in Language-Guided Common Semantic Space Xiaoxuan He Yifan Yang Xinyang Jiang Xufang Luo Haoji Hu Siyun Zhao Dongsheng Li Yuqing Yang Lili Qiu 32 1 0 24 Nov 2023
Are Natural Domain Foundation Models Useful for Medical Image Classification? Joana Palés Huix Adithya Raju Ganeshan Johan Fredin Haslum Magnus P Soderberg Christos Matsoukas Kevin Smith OOD MedIm VLM 19 30 0 30 Oct 2023
Multi-domain improves out-of-distribution and data-limited scenarios for medical image analysis Ece Ozkan Xavier Boix OOD 20 0 0 10 Oct 2023
Generalized Neural Collapse for a Large Number of Classes Jiachen Jiang Jinxin Zhou Peng Wang Qing Qu Dustin Mixon Chong You Zhihui Zhu AI4CE 32 22 0 09 Oct 2023
CWCL: Cross-Modal Transfer with Continuously Weighted Contrastive Loss R. S. Srinivasa Jaejin Cho Chouchang Yang Yashas Malur Saidutta Ching Hua Lee Yilin Shen Hongxia Jin VLM 21 8 0 26 Sep 2023
A Foundation Language-Image Model of the Retina (FLAIR): Encoding Expert Knowledge in Text Supervision Julio Silva-Rodríguez H. Chakor Riadh Kobbi Jose Dolz Ismail Ben Ayed VLM MedIm 64 33 0 15 Aug 2023
Exploring Transfer Learning in Medical Image Segmentation using Vision-Language Models K. Poudel Manish Dhakal Prasiddha Bhandari Rabin Adhikari Safal Thapaliya Bishesh Khanal VLM 28 17 0 15 Aug 2023
Towards a Visual-Language Foundation Model for Computational Pathology Ming Y. Lu Bowen Chen Drew F. K. Williamson Richard J. Chen Ivy Liang ... Andrew Zhang L. Le Georg Gerber Anil V. Parwani Faisal Mahmood VLM MedIm 33 46 0 24 Jul 2023
PRIOR: Prototype Representation Joint Learning from Medical Images and Reports Pujin Cheng Li Lin Junyan Lyu Yijin Huang Wenhan Luo Xiaoying Tang MedIm 19 44 0 24 Jul 2023
Identifying Interpretable Subspaces in Image Representations N. Kalibhat S. Bhardwaj Bayan Bruss Hamed Firooz Maziar Sanjabi S. Feizi FAtt 30 26 0 20 Jul 2023
Knowledge Boosting: Rethinking Medical Contrastive Vision-Language Pre-Training Xiaofei Chen Yuting He Cheng Xue Rongjun Ge Shuo Li Guanyu Yang VLM MedIm 21 11 0 14 Jul 2023
FILM: How can Few-Shot Image Classification Benefit from Pre-Trained Language Models? Zihao Jiang Yunkai Dang Dong Pang Huishuai Zhang Weiran Huang VLM 26 2 0 09 Jul 2023