Contrastive Learning of Medical Visual Representations from Paired Images and Text

2 October 2020

Yuhao Zhang

Hang Jiang

Yasuhide Miura

Christopher D. Manning

C. Langlotz

MedIm

ArXiv PDF HTML

Papers citing "Contrastive Learning of Medical Visual Representations from Paired Images and Text"

50 / 117 papers shown

Title
Quilt-1M: One Million Image-Text Pairs for Histopathology Wisdom O. Ikezogwo M. S. Seyfioglu Fatemeh Ghezloo Dylan Stefan Chan Geva Fatwir Sheikh Mohammed Pavan Kumar Anand Ranjay Krishna Linda G. Shapiro CLIP VLM 128 112 0 20 Jun 2023
How to Train Your CheXDragon: Training Chest X-Ray Models for Transfer to Novel Tasks and Healthcare Systems Cara Van Uden Jeremy Irvin Mars Huang N. Dean J. Carr A. Ng C. Langlotz OOD 24 1 0 13 May 2023
CamDiff: Camouflage Image Augmentation via Diffusion Model Xuejiao Luo Shuo Wang Zongwei Wu Christos Sakaridis Yun Cheng Deng-Ping Fan Luc Van Gool DiffM 30 19 0 11 Apr 2023
ELVIS: Empowering Locality of Vision Language Pre-training with Intra-modal Similarity Sumin Seo Jaewoong Shin Jaewoo Kang Tae Soo Kim Thijs Kooi 27 1 0 11 Apr 2023
CAVL: Learning Contrastive and Adaptive Representations of Vision and Language Shentong Mo Jingfei Xia Ihor Markevych CLIP VLM 16 1 0 10 Apr 2023
SPAN: Learning Similarity between Scene Graphs and Images with Transformers Yuren Cong Wentong Liao Bodo Rosenhahn M. Yang 26 6 0 02 Apr 2023
Sigmoid Loss for Language Image Pre-Training Xiaohua Zhai Basil Mustafa Alexander Kolesnikov Lucas Beyer CLIP VLM 19 931 0 27 Mar 2023
Local Contrastive Learning for Medical Image Recognition S. A. Rizvi Ruixiang Tang X. Jiang X. Ma X. Hu 19 6 0 24 Mar 2023
LIMITR: Leveraging Local Information for Medical Image-Text Representation Gefen Dawidowicz Elad Hirsch A. Tal 23 15 0 21 Mar 2023
Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images Xiaoman Zhang Chaoyi Wu Ya-Qin Zhang Yanfeng Wang Weidi Xie MedIm 28 119 0 27 Feb 2023
Towards Unifying Medical Vision-and-Language Pre-training via Soft Prompts Zhihong Chen Shizhe Diao Benyou Wang Guanbin Li Xiang Wan MedIm 17 29 0 17 Feb 2023
A Review of Predictive and Contrastive Self-supervised Learning for Medical Images Wei-Chien Wang E. Ahn Da-wei Feng Jinman Kim MedIm 16 27 0 10 Feb 2023
Advancing Radiograph Representation Learning with Masked Record Modeling Hong-Yu Zhou Chenyu Lian Lian-cheng Wang Yizhou Yu MedIm 25 55 0 30 Jan 2023
RILS: Masked Visual Reconstruction in Language Semantic Space Shusheng Yang Yixiao Ge Kun Yi Dian Li Ying Shan Xiaohu Qie Xinggang Wang CLIP 35 11 0 17 Jan 2023
Scene-centric vs. Object-centric Image-Text Cross-modal Retrieval: A Reproducibility Study Mariya Hendriksen Svitlana Vakulenko E. Kuiper Maarten de Rijke 21 5 0 12 Jan 2023
EXIF as Language: Learning Cross-Modal Associations Between Images and Camera Metadata Chenhao Zheng Ayush Shrivastava Andrew Owens VLM 28 11 0 11 Jan 2023
CiT: Curation in Training for Effective Vision-Language Data Hu Xu Saining Xie Po-Yao (Bernie) Huang Licheng Yu Russ Howes Gargi Ghosh Luke Zettlemoyer Christoph Feichtenhofer VLM DiffM 25 24 0 05 Jan 2023
TIER: Text-Image Entropy Regularization for CLIP-style models Anil Palepu Andrew L. Beam MedIm 16 6 0 13 Dec 2022
Using Multiple Instance Learning to Build Multimodal Representations Peiqi Wang W. Wells Seth Berkowitz Steven Horng Polina Golland SSL 24 6 0 11 Dec 2022
Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works Chao Tao Ji Qi Mingning Guo Qing Zhu Haifeng Li SSL 19 56 0 15 Nov 2022
The Role of Local Alignment and Uniformity in Image-Text Contrastive Learning on Medical Images Philip Muller Georgios Kaissis Daniel Rueckert MedIm 16 7 0 14 Nov 2022
MuMIC -- Multimodal Embedding for Multi-label Image Classification with Tempered Sigmoid Feng Wang Sarai Mizrachi Moran Beladev Guy Nadav Gil Amsalem Karen Lastmann Assaraf Hadas Harush Boker VLM 22 13 0 02 Nov 2022
Towards Reliable Zero Shot Classification in Self-Supervised Models with Conformal Prediction Bhawesh Kumar Anil Palepu Rudraksh Tuwani Andrew L. Beam 20 8 0 27 Oct 2022
MedCLIP: Contrastive Learning from Unpaired Medical Images and Text Zifeng Wang Zhenbang Wu Dinesh Agarwal Jimeng Sun CLIP VLM MedIm 29 394 0 18 Oct 2022
Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learning Fuying Wang Yuyin Zhou Shujun Wang V. Vardhanabhuti Lequan Yu 21 136 0 12 Oct 2022
HiCo: Hierarchical Contrastive Learning for Ultrasound Video Model Pretraining Chunhui Zhang Yixiong Chen Li Liu Qiong Liu Xiaoping Zhou VLM 40 8 0 10 Oct 2022
Disentangle and Remerge: Interventional Knowledge Distillation for Few-Shot Object Detection from A Conditional Causal Perspective Jiangmeng Li Yanan Zhang Wenwen Qiang Lingyu Si Chengbo Jiao Xiaohui Hu Changwen Zheng Fuchun Sun CML 23 28 0 26 Aug 2022
Towards Open-vocabulary Scene Graph Generation with Prompt-based Finetuning Tao He Lianli Gao Jingkuan Song Yuan-Fang Li VLM 18 50 0 17 Aug 2022
Contrastive Adapters for Foundation Model Group Robustness Michael Zhang Christopher Ré VLM 8 61 0 14 Jul 2022
LViT: Language meets Vision Transformer in Medical Image Segmentation Zihan Li Yunxiang Li Qingde Li Puyang Wang Dazhou Guo Le Lu D. Jin You Zhang Qingqi Hong VLM MedIm 59 131 0 29 Jun 2022
Language-Based Audio Retrieval with Converging Tied Layers and Contrastive Loss Andrew Koh Chng Eng Siong 18 1 0 29 Jun 2022
Post-hoc Concept Bottleneck Models Mert Yuksekgonul Maggie Wang James Y. Zou 133 185 0 31 May 2022
CyCLIP: Cyclic Contrastive Language-Image Pretraining Shashank Goel Hritik Bansal S. Bhatia Ryan A. Rossi Vishwa Vinay Aditya Grover CLIP VLM 168 132 0 28 May 2022
Breaking with Fixed Set Pathology Recognition through Report-Guided Contrastive Training C. Seibold Simon Reiß M. Sarfraz Rainer Stiefelhagen Jens Kleesiek 16 31 0 14 May 2022
Data Determines Distributional Robustness in Contrastive Language Image Pre-training (CLIP) Alex Fang Gabriel Ilharco Mitchell Wortsman Yu Wan Vaishaal Shankar Achal Dave Ludwig Schmidt VLM OOD 20 138 0 03 May 2022
Hierarchical Text-Conditional Image Generation with CLIP Latents Aditya A. Ramesh Prafulla Dhariwal Alex Nichol Casey Chu Mark Chen VLM DiffM 67 6,622 0 13 Apr 2022
MuCoT: Multilingual Contrastive Training for Question-Answering in Low-resource Languages Gokul Karthik Kumar Abhishek Singh Gehlot Sahal Shaji Mullappilly Karthik Nandakumar 26 13 0 12 Apr 2022
Multi-View Transformer for 3D Visual Grounding Shijia Huang Yilun Chen Jiaya Jia Liwei Wang 20 112 0 05 Apr 2022
Interactive Audio-text Representation for Automated Audio Captioning with Contrastive Learning Chen Chen Nana Hou Yuchen Hu Heqing Zou Xiaofeng Qi Chng Eng Siong VLM 26 21 0 29 Mar 2022
AUC Maximization in the Era of Big Data and AI: A Survey Tianbao Yang Yiming Ying 33 178 0 28 Mar 2022
Contrastive Visual Semantic Pretraining Magnifies the Semantics of Natural Language Representations Robert Wolfe Aylin Caliskan VLM 21 13 0 14 Mar 2022
Provable Stochastic Optimization for Global Contrastive Learning: Small Batch Does Not Harm Performance Zhuoning Yuan Yuexin Wu Zi-qi Qiu Xianzhi Du Lijun Zhang Denny Zhou Tianbao Yang 25 26 0 24 Feb 2022
GroupViT: Semantic Segmentation Emerges from Text Supervision Jiarui Xu Shalini De Mello Sifei Liu Wonmin Byeon Thomas Breuel Jan Kautz X. Wang ViT VLM 180 499 0 22 Feb 2022
A Survey on Visual Transfer Learning using Knowledge Graphs Sebastian Monka Lavdim Halilaj Achim Rettinger 19 23 0 27 Jan 2022
Contrastive and Selective Hidden Embeddings for Medical Image Segmentation Zhuowei Li Zihao Liu Zhiqiang Hu Qing Xia Ruiqin Xiong Shaoting Zhang Dimitris N. Metaxas Tingting Jiang SSL 21 8 0 21 Jan 2022
TriCoLo: Trimodal Contrastive Loss for Text to Shape Retrieval Yue Ruan Han-Hung Lee Yiming Zhang Ke Zhang Angel X. Chang 22 22 0 19 Jan 2022
MERLOT Reserve: Neural Script Knowledge through Vision and Language and Sound Rowan Zellers Jiasen Lu Ximing Lu Youngjae Yu Yanpeng Zhao Mohammadreza Salehi Aditya Kusupati Jack Hessel Ali Farhadi Yejin Choi 26 207 0 07 Jan 2022
C2-CRS: Coarse-to-Fine Contrastive Learning for Conversational Recommender System Yuanhang Zhou Kun Zhou Wayne Xin Zhao Cheng Wang Peng Jiang He Hu 25 52 0 04 Jan 2022
Extending CLIP for Category-to-image Retrieval in E-commerce Mariya Hendriksen Maurits J. R. Bleeker Svitlana Vakulenko N. V. Noord E. Kuiper Maarten de Rijke VLM 11 30 0 21 Dec 2021
Joint Learning of Localized Representations from Medical Images and Reports Philipp Muller Georgios Kaissis Cong Zou Daniel Munich 132 81 0 06 Dec 2021