Are Multimodal Transformers Robust to Missing Modality?

12 April 2022

Papers citing "Are Multimodal Transformers Robust to Missing Modality?"

50 / 94 papers shown

Title
Synergy-CLIP: Extending CLIP with Multi-modal Integration for Robust Representation Learning Sangyeon Cho Jangyeong Jeon Mingi Kim Junyeong Kim CLIP VLM 74 0 0 30 Apr 2025
Are you SURE? Enhancing Multimodal Pretraining with Missing Modalities through Uncertainty Estimation Duy Nguyen Quan Huu Do Khoa D. Doan Minh N. Do 27 0 0 18 Apr 2025
Knowledge Distillation for Multimodal Egocentric Action Recognition Robust to Missing Modalities Maria Santos-Villafranca Dustin Carrión-Ojeda Alejandro Pérez-Yus J. Bermudez-Cameo Jose J. Guerrero Simone Schaub-Meyer EgoV VLM 35 0 0 11 Apr 2025
Multifaceted Evaluation of Audio-Visual Capability for MLLMs: Effectiveness, Efficiency, Generalizability and Robustness Yusheng Zhao Junyu Luo Xiao Luo Weizhi Zhang Zhiping Xiao Wei Ju Philip S. Yu Ming Zhang AuLLM 37 0 0 03 Apr 2025
Robust Multimodal Survival Prediction with the Latent Differentiation Conditional Variational AutoEncoder Junjie Zhou Jiao Tang Yingli Zuo Peng Wan Daoqiang Zhang Wei Shao 58 0 0 12 Mar 2025
Attention Bootstrapping for Multi-Modal Test-Time Adaptation Yusheng Zhao Junyu Luo Xiao Luo Jinsheng Huang Jingyang Yuan Zhiping Xiao M. Zhang TTA 85 0 0 04 Mar 2025
Multimodal Learning for Just-In-Time Software Defect Prediction in Autonomous Driving Systems Faisal Mohammad Duksan Ryu 54 0 0 28 Feb 2025
Knowledge Bridger: Towards Training-free Missing Multi-modality Completion Guanzhou Ke Shengfeng He X. Wang Bo Wang Guoqing Chao Y. Zhang Yi Xie HeXing Su 48 0 0 27 Feb 2025
Missing Data as Augmentation in the Earth Observation Domain: A Multi-View Learning Approach Francisco Mena Diego Arenas A. Dengel 26 0 0 03 Jan 2025
Modality-missing RGBT Tracking: Invertible Prompt Learning and High-quality Benchmarks Andong Lu Jiacong Zhao Chenglong Li Jin Tang Bin Luo VLM 50 4 0 31 Dec 2024
Explorations in Self-Supervised Learning: Dataset Composition Testing for Object Classification Raynor Kirkson E. Chavez Kyle Gabriel M. Reynoso 66 0 0 01 Dec 2024
SeafloorAI: A Large-scale Vision-Language Dataset for Seafloor Geological Survey Kien X. Nguyen Fengchun Qiao Arthur Trembanis Xi Peng 21 0 0 31 Oct 2024
Beyond Accuracy: Ensuring Correct Predictions With Correct Rationales Tang Li Mengmeng Ma Xi Peng 26 2 0 31 Oct 2024
Generalized Multimodal Fusion via Poisson-Nernst-Planck Equation Jiayu Xiong Jing Wang Hengjing Xiang Jun Xue Chen Xu Zhouqiang Jiang 22 0 0 20 Oct 2024
X-Fi: A Modality-Invariant Foundation Model for Multimodal Human Sensing Xinyan Chen Jianfei Yang 28 0 0 14 Oct 2024
Deep Correlated Prompting for Visual Recognition with Missing Modalities Lianyu Hu Tongkai Shi Wei Feng Fanhua Shang Liang Wan VLM 19 0 0 09 Oct 2024
MMP: Towards Robust Multi-Modal Learning with Masked Modality Projection Niki Nezakati Md Kaykobad Reza Ameya Patil Mashhour Solh M. Salman Asif 27 1 0 03 Oct 2024
A Multimodal Single-Branch Embedding Network for Recommendation in Cold-Start and Missing Modality Scenarios Christian Ganhor Marta Moscati Anna Hausberger Shah Nawaz Markus Schedl 17 2 0 26 Sep 2024
MuAP: Multi-step Adaptive Prompt Learning for Vision-Language Model with Missing Modality Ruiting Dai Yuqiao Tan Lisi Mo Tao He Ke Qin Shuang Liang VLM 23 2 0 07 Sep 2024
Enhance Modality Robustness in Text-Centric Multimodal Alignment with Adversarial Prompting Yun-Da Tsai Ting-Yu Yen Keng-Te Liao Shou-De Lin 21 1 0 19 Aug 2024
MedMAP: Promoting Incomplete Multi-modal Brain Tumor Segmentation with Alignment Tianyi Liu Zhaorui Tan Muyin Chen Xi Yang Haochuan Jiang Kaizhu Huang 16 0 0 18 Aug 2024
Modality Invariant Multimodal Learning to Handle Missing Modalities: A Single-Branch Approach Muhammad Saad Saeed Shah Nawaz Muhammad Zaigham Zaheer Muhammad Haris Khan Karthik Nandakumar Muhammad Haroon Yousaf Hassan Sajjad Tom De Schepper Markus Schedl 18 0 0 14 Aug 2024
Chameleon: Images Are What You Need For Multimodal Learning Robust To Missing Modalities Muhammad Irzam Liaqat Shah Nawaz Muhammad Zaigham Zaheer M. S. Saeed Hassan Sajjad Tom De Schepper Karthik Nandakumar Muhammad Haris Khan 18 1 0 23 Jul 2024
Learning Trimodal Relation for AVQA with Missing Modality Kyu Ri Park Hong Joo Lee Jung Uk Kim 26 1 0 23 Jul 2024
Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observation Francisco Mena Diego Arenas A. Dengel OOD 16 0 0 22 Jul 2024
Missing Modality Prediction for Unpaired Multimodal Learning via Joint Embedding of Unimodal Models Donggeun Kim Taesup Kim 16 3 0 17 Jul 2024
CoLA: Conditional Dropout and Language-driven Robust Dual-modal Salient Object Detection Shuang Hao Chunlin Zhong He Tang 19 1 0 09 Jul 2024
Enhance the Robustness of Text-Centric Multimodal Alignments Ting-Yu Yen Yun-Da Tsai Keng-Te Liao Shou-De Lin 23 2 0 06 Jul 2024
Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clients Mengmeng Ma Tang Li Xi Peng 76 4 0 06 Jul 2024
ADAPT: Multimodal Learning for Detecting Physiological Changes under Missing Modalities Julie Mordacq Léo Milecki Maria Vakalopoulou Steve Oudot Vicky Kalogeiton OffRL MedIm 28 3 0 04 Jul 2024
Automatic benchmarking of large multimodal models via iterative experiment programming Alessandro Conti Enrico Fini Paolo Rota Yiming Wang Massimiliano Mancini Elisa Ricci 24 0 0 18 Jun 2024
FlexCare: Leveraging Cross-Task Synergy for Flexible Multimodal Healthcare Prediction Muhao Xu Zhenfeng Zhu Youru Li Shuai Zheng Yawei Zhao Kunlun He Yao Zhao 27 0 0 17 Jun 2024
Leveraging Foundation Models for Multi-modal Federated Learning with Incomplete Modality Liwei Che Jiaqi Wang Xinyue Liu Fenglong Ma 23 2 0 16 Jun 2024
Efficient Multi-View Fusion and Flexible Adaptation to View Missing in Cardiovascular System Signals Qihan Hu Daomiao Wang Hong Wu Jian Liu Cuiwei Yang 27 0 0 13 Jun 2024
Stock Movement Prediction with Multimodal Stable Fusion via Gated Cross-Attention Mechanism Chang Zong Jian Shao Weiming Lu Yueting Zhuang 32 2 0 06 Jun 2024
Towards Precision Healthcare: Robust Fusion of Time Series and Image Data Ali Rasekh Reza Heidari Amir Hosein Haji Mohammad Rezaie Parsa Sharifi Sedeh Zahra Ahmadi Prasenjit Mitra Wolfgang Nejdl AI4TS 18 2 0 24 May 2024
TI-ASU: Toward Robust Automatic Speech Understanding through Text-to-speech Imputation Against Missing Speech Modality Tiantian Feng Xuan Shi Rahul Gupta Shrikanth S. Narayanan 41 0 0 27 Apr 2024
iMD4GC: Incomplete Multimodal Data Integration to Advance Precise Treatment Response Prediction and Survival Analysis for Gastric Cancer Fengtao Zhou Ying Xu Yanfen Cui Shenyang Zhang Yun Zhu ... Louis Ho Shing Lau Chu Han Dafu Zhang Zhenhui Li Hao Chen 30 1 0 01 Apr 2024
LITE: Modeling Environmental Ecosystems with Multimodal Large Language Models Haoran Li Junqi Liu Zexian Wang Shiyuan Luo Xiaowei Jia Huaxiu Yao 16 3 0 01 Apr 2024
Towards Multimodal Video Paragraph Captioning Models Robust to Missing Modality Sishuo Chen Lei Li Shuhuai Ren Rundong Gao Yuanxin Liu Xiaohan Bi Xu Sun Lu Hou 24 3 0 28 Mar 2024
In the Search for Optimal Multi-view Learning Models for Crop Classification with Global Remote Sensing Data Francisco Mena Diego Arenas A. Dengel 35 1 0 25 Mar 2024
Reconstruct before Query: Continual Missing Modality Learning with Decomposed Prompt Collaboration Shu Zhao Xiaohan Zou Tan Yu Huijuan Xu 25 1 0 17 Mar 2024
Borrowing Treasures from Neighbors: In-Context Learning for Multimodal Learning with Missing Modalities and Data Scarcity Zhuo Zhi Ziquan Liu M. Elbadawi Adam Daneshmend Mine Orlu Abdul Basit Andreas Demosthenous Miguel R. D. Rodrigues 22 2 0 14 Mar 2024
Specification Overfitting in Artificial Intelligence Benjamin Roth Pedro Henrique Luz de Araujo Yuxi Xia Saskia Kaltenbrunner Christoph Korab 44 0 0 13 Mar 2024
A Study of Dropout-Induced Modality Bias on Robustness to Missing Video Frames for Audio-Visual Speech Recognition Yusheng Dai Hang Chen Jun Du Ruoyu Wang Shihao Chen Jie Ma Haotian Wang Chin-Hui Lee 38 4 0 07 Mar 2024
Gradient-Guided Modality Decoupling for Missing-Modality Robustness Hao Wang Shengda Luo Guosheng Hu Jianguo Zhang 19 3 0 26 Feb 2024
Can Text-to-image Model Assist Multi-modal Learning for Visual Recognition with Visual Modality Missing? Tiantian Feng Daniel Yang Digbalay Bose Shrikanth Narayanan 27 4 0 14 Feb 2024
Quantifying and Enhancing Multi-modal Robustness with Modality Preference Zequn Yang Yake Wei Ce Liang Di Hu AAML 19 9 0 09 Feb 2024
A Survey on Safe Multi-Modal Learning System Tianyi Zhao Liangliang Zhang Yao Ma Lu Cheng 52 7 0 08 Feb 2024
AM^2-EmoJE: Adaptive Missing-Modality Emotion Recognition in Conversation via Joint Embedding Learning Naresh Kumar Devulapally Sidharth Anand S. Bhattacharjee Junsong Yuan 28 1 0 26 Jan 2024