What Makes Multi-modal Learning Better than Single (Provably)

8 June 2021

Hang Zhao

Papers citing "What Makes Multi-modal Learning Better than Single (Provably)"

33 / 33 papers shown

Title
Synergy-CLIP: Extending CLIP with Multi-modal Integration for Robust Representation Learning Sangyeon Cho Jangyeong Jeon Mingi Kim Junyeong Kim CLIP VLM 76 0 0 30 Apr 2025
Optimizing Mouse Dynamics for User Authentication by Machine Learning: Addressing Data Sufficiency, Accuracy-Practicality Trade-off, and Model Performance Challenges Yi Wang Chengyv Wu Yang Liao Maowei You AAML 39 0 0 30 Apr 2025
CLIPure: Purification in Latent Space via CLIP for Adversarially Robust Zero-Shot Classification Mingkun Zhang Keping Bi Wei Chen J. Guo Xueqi Cheng BDL VLM 50 1 0 25 Feb 2025
GNN-Transformer Cooperative Architecture for Trustworthy Graph Contrastive Learning Jianqing Liang Xinkai Wei Min Chen Zhiqiang Wang Jiye Liang 73 0 0 18 Dec 2024
Part-Whole Relational Fusion Towards Multi-Modal Scene Understanding Yi Liu Chengxin Li Shoukun Xu J. Han ViT 37 2 0 19 Oct 2024
CoPRA: Bridging Cross-domain Pretrained Sequence Models with Complex Structures for Protein-RNA Binding Affinity Prediction Rong Han Xiaohong Liu Tong Pan Jing Xu Xiaoyu Wang ... Zhenyu Li Zixuan Wang Jiangning Song Guangyu Wang Ting Chen 21 0 0 21 Aug 2024
Completed Feature Disentanglement Learning for Multimodal MRIs Analysis Tianling Liu Hongying Liu Fanhua Shang Lequan Yu Tong Han Liang Wan 37 1 0 06 Jul 2024
Understanding Retrieval-Augmented Task Adaptation for Vision-Language Models Yifei Ming Yixuan Li VLM 30 7 0 02 May 2024
Borrowing Treasures from Neighbors: In-Context Learning for Multimodal Learning with Missing Modalities and Data Scarcity Zhuo Zhi Ziquan Liu M. Elbadawi Adam Daneshmend Mine Orlu Abdul Basit Andreas Demosthenous Miguel R. D. Rodrigues 34 2 0 14 Mar 2024
Beyond DAGs: A Latent Partial Causal Model for Multimodal Learning Yuhang Liu Zhen Zhang Dong Gong Biwei Huang Mingming Gong A. Hengel Kun Zhang Javen Qinfeng Shi J. Shi 41 7 0 09 Feb 2024
Triple Disentangled Representation Learning for Multimodal Affective Analysis Ying Zhou Xuefeng Liang Han Chen Yin Zhao Xin Chen Lida Yu 45 3 0 29 Jan 2024
Fus-MAE: A cross-attention-based data fusion approach for Masked Autoencoders in remote sensing Hugo Chan-To-Hing B. Veeravalli 22 8 0 05 Jan 2024
Improving Discriminative Multi-Modal Learning with Large-Scale Pre-Trained Models Chenzhuang Du Yue Zhao Chonghua Liao Jiacheng You Jie Fu Hang Zhao 30 2 0 08 Oct 2023
Missing-modality Enabled Multi-modal Fusion Architecture for Medical Data Muyu Wang Shiyu Fan Yichen Li Hui Chen MedIm 17 1 0 27 Sep 2023
kTrans: Knowledge-Aware Transformer for Binary Code Embedding Wenyu Zhu Hao Wang Yuchen Zhou Jiaming Wang Zihan Sha Zeyu Gao Chao Zhang 30 10 0 24 Aug 2023
Interpretation on Multi-modal Visual Fusion Hao Chen Hao Zhou Yongjian Deng 28 0 0 19 Aug 2023
GiGaMAE: Generalizable Graph Masked Autoencoder via Collaborative Latent Space Reconstruction Yucheng Shi Yushun Dong Qiaoyu Tan Jundong Li Ninghao Liu 40 24 0 18 Aug 2023
Divert More Attention to Vision-Language Object Tracking Mingzhe Guo Zhipeng Zhang Li Jing Haibin Ling Heng Fan VLM 26 3 0 19 Jul 2023
Dissecting Multimodality in VideoQA Transformer Models by Impairing Modality Fusion Isha Rawal Alexander Matyasko Shantanu Jaiswal Basura Fernando Cheston Tan 21 1 0 15 Jun 2023
Multimodal Prototype-Enhanced Network for Few-Shot Action Recognition Xin Ni Yong Liu Hao Wen Yatai Ji Jing Xiao Yujiu Yang 26 9 0 09 Dec 2022
Towards Good Practices for Missing Modality Robust Action Recognition Sangmin Woo Sumin Lee Yeonju Park Muhammad Adi Nugroho Changick Kim 22 43 0 25 Nov 2022
HALSIE: Hybrid Approach to Learning Segmentation by Simultaneously Exploiting Image and Event Modalities Shristi Das Biswas Adarsh Kosta C. Liyanagedera M. Apolinario Kaushik Roy 27 18 0 19 Nov 2022
Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works Chao Tao Ji Qi Mingning Guo Qing Zhu Haifeng Li SSL 21 56 0 15 Nov 2022
Greedy Modality Selection via Approximate Submodular Maximization Runxiang Cheng Gargi Balasubramaniam Yifei He Yao-Hung Hubert Tsai Han Zhao 14 1 0 22 Oct 2022
MMRNet: Improving Reliability for Multimodal Object Detection and Segmentation for Bin Picking via Multimodal Redundancy Yuhao Chen Hayden Gunraj E. Z. Zeng Robbie Meyer Maximilian Gilles Alexander Wong 31 1 0 19 Oct 2022
Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture M. Jung He Zhao Joanna Dipnall Belinda Gabbe Lan Du UQCV EDL 50 12 0 06 Oct 2022
Modality Mixer for Multi-modal Action Recognition Sumin Lee Sangmin Woo Yeonju Park Muhammad Adi Nugroho Changick Kim 19 10 0 24 Aug 2022
Divert More Attention to Vision-Language Tracking Mingzhe Guo Zhipeng Zhang Heng Fan Li Jing 21 53 0 03 Jul 2022
More to Less (M2L): Enhanced Health Recognition in the Wild with Reduced Modality of Wearable Sensors Huiyuan Yang Han Yu K. Sridhar T. Vaessen I. Myin‐Germeys Akane Sano 17 7 0 16 Feb 2022
M5Product: Self-harmonized Contrastive Learning for E-commercial Multi-modal Pretraining Xiao Dong Xunlin Zhan Yangxin Wu Yunchao Wei Michael C. Kampffmeyer Xiaoyong Wei Minlong Lu Yaowei Wang Xiaodan Liang 25 36 0 09 Sep 2021
Deep Continuous Fusion for Multi-Sensor 3D Object Detection Ming Liang Binh Yang Shenlong Wang R. Urtasun 3DPC 187 840 0 20 Dec 2020
Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies Itai Gat Idan Schwartz A. Schwing Tamir Hazan 53 89 0 21 Oct 2020
Norm-Based Capacity Control in Neural Networks Behnam Neyshabur Ryota Tomioka Nathan Srebro 116 577 0 27 Feb 2015