We present a machine learning method capable of accurately detecting chromosome abnormalities that cause blood cancers directly from microscope images of the metaphase stage of cell division. The pipeline is built on a series of fine-tuned Vision Transformers. Current state of the art (and standard clinical practice) requires expensive, manual expert analysis, whereas our pipeline takes only 15 seconds per metaphase image. Using a novel pretraining-finetuning strategy to mitigate the challenge of data scarcity, we achieve a high precision-recall score of 94% AUC for the clinically significant del(5q) and t(9;22) anomalies. Our method also unlocks zero-shot detection of rare aberrations based on model latent embeddings. The ability to quickly, accurately, and scalably diagnose genetic abnormalities directly from metaphase images could transform karyotyping practice and improve patient outcomes. We will make code publicly available.
View on arXiv@article{shamsi2025_2211.14312,
title={ Karyotype AI for Precision Oncology },
author={ Zahra Shamsi and Isaac Reid and Drew Bryant and Jacob Wilson and Xiaoyu Qu and Avinava Dubey and Konik Kothari and Mostafa Dehghani and Mariya Chavarha and Valerii Likhosherstov and Brian Williams and Michael Frumkin and Fred Appelbaum and Krzysztof Choromanski and Ali Bashir and Min Fang },
journal={arXiv preprint arXiv:2211.14312},
year={ 2025 }
}