Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling

26 July 2023

Hu Wang

Yuanhong Chen

Congbo Ma

Jodie Avery

Louise Hull

G. Carneiro

ArXiv PDF HTML

Papers citing "Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling"

49 / 49 papers shown

Title
Synergy-CLIP: Extending CLIP with Multi-modal Integration for Robust Representation Learning Sangyeon Cho Jangyeong Jeon Mingi Kim Junyeong Kim CLIP VLM 74 0 0 30 Apr 2025
Multimodal 3D Genome Pre-training Minghao Yang Pengteng Li Yan Liang Qianyi Cai Zhihang Zheng Shichen Zhang Pengfei Zhang Zhi-An Huang Hui Xiong 26 0 0 12 Apr 2025
Integrating AI for Human-Centric Breast Cancer Diagnostics: A Multi-Scale and Multi-View Swin Transformer Framework Farnoush Bayatmakou Reza Taleei Milad Amir Toutounchian Arash Mohammadi 41 0 0 17 Mar 2025
Robust Multimodal Survival Prediction with the Latent Differentiation Conditional Variational AutoEncoder Junjie Zhou Jiao Tang Yingli Zuo Peng Wan Daoqiang Zhang Wei Shao 58 0 0 12 Mar 2025
Distilled Prompt Learning for Incomplete Multimodal Survival Prediction Yingxue Xu Fengtao Zhou Chenyu Zhao Yihui Wang Can Yang Hao Chen VLM OffRL 52 0 0 03 Mar 2025
Knowledge Bridger: Towards Training-free Missing Multi-modality Completion Guanzhou Ke Shengfeng He X. Wang Bo Wang Guoqing Chao Y. Zhang Yi Xie HeXing Su 48 0 0 27 Feb 2025
AI-Driven HSI: Multimodality, Fusion, Challenges, and the Deep Learning Revolution David S. Bhatti Yougin Choi Rahman S M Wahidur Maleeka Bakhtawar Sumin Kim Surin Lee Yongtae Lee Heung-No Lee 65 1 0 09 Feb 2025
Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarks Andong Lu Jiacong Zhao Chenglong Li Jin Tang Bin Luo VLM 52 4 0 31 Dec 2024
Personalize to generalize: Towards a universal medical multi-modality generalization through personalization Zhaorui Tan Xi Yang Tan Pan Tianyi Liu Chen Jiang ... Qiufeng Wang Anh Nguyen Yuan Qi K. Huang Yuan-Chia Cheng AI4CE LM&MA 29 0 0 09 Nov 2024
Generalized Multimodal Fusion via Poisson-Nernst-Planck Equation Jiayu Xiong Jing Wang Hengjing Xiang Jun Xue Chen Xu Zhouqiang Jiang 22 0 0 20 Oct 2024
Part-Whole Relational Fusion Towards Multi-Modal Scene Understanding Yi Liu Chengxin Li Shoukun Xu J. Han ViT 30 1 0 19 Oct 2024
Multi-modal Vision Pre-training for Medical Image Analysis Shaohao Rui Lingzhi Chen Zhenyu Tang Lilong Wang M. Liu S. Zhang Xiaosong Wang 22 0 0 14 Oct 2024
Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Experts Sukwon Yun Inyoung Choi Jie Peng Yangfan Wu J. Bao Qiyiwen Zhang Jiayi Xin Qi Long Tianlong Chen MoE 40 4 0 10 Oct 2024
Deep Correlated Prompting for Visual Recognition with Missing Modalities Lianyu Hu Tongkai Shi Wei Feng Fanhua Shang Liang Wan VLM 22 0 0 09 Oct 2024
MMP: Towards Robust Multi-Modal Learning with Masked Modality Projection Niki Nezakati Md Kaykobad Reza Ameya Patil Mashhour Solh M. Salman Asif 27 1 0 03 Oct 2024
Human-AI Collaborative Multi-modal Multi-rater Learning for Endometriosis Diagnosis Hu Wang David Butler Yuan Zhang Jodie C Avery Steven Knox Congbo Ma Louise Hull Gustavo Carneiro 15 2 0 03 Sep 2024
Global-Local Distillation Network-Based Audio-Visual Speaker Tracking with Incomplete Modalities Yidi Li Yihan Li Yixin Guo Bin Ren Zhenhuan Xu Hao Guo Hong Liu N. Sebe 29 0 0 26 Aug 2024
Anatomical Consistency Distillation and Inconsistency Synthesis for Brain Tumor Segmentation with Missing Modalities Zheyu Zhang Xinzhao Liu Zheng Chen Yueyi Zhang Huanjing Yue Yunwei Ou Xiaoyan Sun 29 0 0 25 Aug 2024
Multimodal Contrastive In-Context Learning Yosuke Miyanishi Minh Le Nguyen 32 2 0 23 Aug 2024
Modality Invariant Multimodal Learning to Handle Missing Modalities: A Single-Branch Approach Muhammad Saad Saeed Shah Nawaz Muhammad Zaigham Zaheer Muhammad Haris Khan Karthik Nandakumar Muhammad Haroon Yousaf Hassan Sajjad Tom De Schepper Markus Schedl 18 0 0 14 Aug 2024
HyperMM : Robust Multimodal Learning with Varying-sized Inputs Hava Chaptoukaev Vincenzo Marcianó Francesco Galati Maria A. Zuluaga 19 0 0 30 Jul 2024
Multi-modal Data Binding for Survival Analysis Modeling with Incomplete Data and Annotations Linhao Qu Dan Huang Shaoting Zhang Xiaosong Wang 14 2 0 25 Jul 2024
Learning Trimodal Relation for AVQA with Missing Modality Kyu Ri Park Hong Joo Lee Jung Uk Kim 29 1 0 23 Jul 2024
Robust Facial Reactions Generation: An Emotion-Aware Framework with Modality Compensation Guanyu Hu Jie Wei Siyang Song Dimitrios Kollias Xinyu Yang Zhonglin Sun Odysseus Kaloidas 21 0 0 22 Jul 2024
PASSION: Towards Effective Incomplete Multi-Modal Medical Image Segmentation with Imbalanced Missing Rates Junjie Shi Caozhi Shang Zhaobin Sun Li Yu Xin Yang Zengqiang Yan 22 1 0 20 Jul 2024
Missing Modality Prediction for Unpaired Multimodal Learning via Joint Embedding of Unimodal Models Donggeun Kim Taesup Kim 19 3 0 17 Jul 2024
Robust Multimodal Learning via Representation Decoupling Shicai Wei Yang Luo Yuji Wang Chunbo Luo OOD 29 2 0 05 Jul 2024
ADAPT: Multimodal Learning for Detecting Physiological Changes under Missing Modalities Julie Mordacq Léo Milecki Maria Vakalopoulou Steve Oudot Vicky Kalogeiton OffRL MedIm 28 3 0 04 Jul 2024
Multimodal Data Integration for Precision Oncology: Challenges and Future Directions Huajun Zhou Fengtao Zhou Chenyu Zhao Yingxue Xu Luyang Luo Hao Chen 30 4 0 28 Jun 2024
MDA: An Interpretable Multi-Modal Fusion with Missing Modalities and Intrinsic Noise Lin Fan Yafei Ou Cenyang Zheng Pengyu Dai Tamotsu Kamishima Masayuki Ikebe Kenji Suzuki Xun Gong 28 0 0 15 Jun 2024
$MOTI$\mathcal{VE}$: A Drug-Target Interaction Graph For Inductive Link Prediction$ MOTI $\mathcal{VE}$ : A Drug-Target Interaction Graph For Inductive Link Prediction John Arevalo Ellen Su Anne E Carpenter Shantanu Singh 22 2 0 12 Jun 2024
Dealing with All-stage Missing Modality: Towards A Universal Model with Robust Reconstruction and Personalization Yunpeng Zhao Cheng Chen Qing You Pang Quanzheng Li Carol Tang B. Ang Yueming Jin 19 0 0 04 Jun 2024
Modally Reduced Representation Learning of Multi-Lead ECG Signals through Simultaneous Alignment and Reconstruction Nabil Ibtehaz Masood S. Mortazavi 14 0 0 24 May 2024
All in One Framework for Multimodal Re-identification in the Wild He Li Mang Ye Ming Zhang Bo Du 23 9 0 08 May 2024
A Multimodal Feature Distillation with CNN-Transformer Network for Brain Tumor Segmentation with Incomplete Modalities Ming Kang F. F. Ting Raphaël C.-W. Phan Zongyuan Ge Chee-Ming Ting 27 2 0 22 Apr 2024
Foundation Model for Advancing Healthcare: Challenges, Opportunities, and Future Directions Yuting He Fuxiang Huang Xinrui Jiang Yuxiang Nie Minghao Wang Jiguang Wang Hao Chen LM&MA AI4CE 71 26 0 04 Apr 2024
Reconstruct before Query: Continual Missing Modality Learning with Decomposed Prompt Collaboration Shu Zhao Xiaohan Zou Tan Yu Huijuan Xu 25 1 0 17 Mar 2024
DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency Wenfang Yao Kejing Yin William K. Cheung Jia Liu Jing Qin MedIm 16 14 0 10 Mar 2024
Can Text-to-image Model Assist Multi-modal Learning for Visual Recognition with Visual Modality Missing? Tiantian Feng Daniel Yang Digbalay Bose Shrikanth Narayanan 27 4 0 14 Feb 2024
Fourier Prompt Tuning for Modality-Incomplete Scene Segmentation Ruiping Liu Jiaming Zhang Kunyu Peng Yufan Chen Ke Cao Junwei Zheng M. Sarfraz Kailun Yang Rainer Stiefelhagen VLM 32 7 0 30 Jan 2024
Complementary Information Mutual Learning for Multimodality Medical Image Segmentation Chuyun Shen Wenhao Li Haoqing Chen Xiaoling Wang Fengping Zhu Yuxin Li Xiangfeng Wang Bo Jin 32 3 0 05 Jan 2024
Multimodal Federated Learning with Missing Modality via Prototype Mask and Contrast Guangyin Bao Qi Zhang Duoqian Miao Zixuan Gong Liang Hu Ke Liu Yang Liu Chongyang Shi 29 7 0 21 Dec 2023
Segment Beyond View: Handling Partially Missing Modality for Audio-Visual Semantic Segmentation Renjie Wu Hu Wang Feras Dayoub Hsiang-Ting Chen 17 5 0 14 Dec 2023
Single-Model and Any-Modality for Video Object Tracking Zongwei Wu Jilai Zheng Xiangxuan Ren Florin-Alexandru Vasluianu Chao Ma D. Paudel Luc Van Gool Radu Timofte 46 33 0 27 Nov 2023
What Makes for Robust Multi-Modal Models in the Face of Missing Modalities? Siting Li Chenzhuang Du Yue Zhao Yu Huang Hang Zhao 19 4 0 10 Oct 2023
Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation Md Kaykobad Reza Ashley Prater-Bennette M. Salman Asif 16 4 0 06 Oct 2023
Causal Intersectionality and Dual Form of Gradient Descent for Multimodal Analysis: a Case Study on Hateful Memes Yosuke Miyanishi M. Nguyen 24 2 0 19 Aug 2023
RGBT Tracking via Progressive Fusion Transformer with Dynamically Guided Learning Yabin Zhu Chenglong Li Xiao Wang Jin Tang Zhixiang Huang 24 7 0 26 Mar 2023
Husformer: A Multi-Modal Transformer for Multi-Modal Human State Recognition Ruiqi Wang Wonse Jo Dezhong Zhao Weizheng Wang B. Yang Guohua Chen Byung-Cheol Min HAI 11 28 0 30 Sep 2022