Balanced Multimodal Learning via On-the-fly Gradient Modulation

Computer Vision and Pattern Recognition (CVPR), 2022

29 March 2022

ArXiv (abs)PDF HTML Github (274★)

Papers citing "Balanced Multimodal Learning via On-the-fly Gradient Modulation"

50 / 143 papers shown

Title
The More, the Merrier: Contrastive Fusion for Higher-Order Multimodal Alignment Stefanos Koutoupis Michaela Areti Zervou Konstantinos Kontras M. D. Vos Panagiotis Tsakalides Grigorios Tsagatakis 48 0 0 26 Nov 2025
Modality-Balanced Collaborative Distillation for Multi-Modal Domain Generalization X. Wang Zhangtao Cheng Ting Zhong Leiting Chen Fan Zhou 56 0 0 25 Nov 2025
Quantifying Modality Contributions via Disentangling Multimodal Representations Padegal Amit Omkar Mahesh Kashyap Namitha Rayasam Nidhi Shekhar Surabhi Narayan 72 0 0 22 Nov 2025
Boomda: Balanced Multi-objective Optimization for Multimodal Domain Adaptation Jun Sun Xinxin Zhang Simin Hong Jian Zhu Xiang Gao 71 0 0 11 Nov 2025
Mitigating Modality Imbalance in Multi-modal Learning via Multi-objective Optimization Heshan Devaka Fernando Parikshit Ram Yi Zhou Soham Dan Horst Samulowitz Nathalie Baracaldo Tianyi Chen 153 0 0 10 Nov 2025
Balanced Multimodal Learning via Mutual Information Rongrong Xie Guido Sanguinetti 64 0 0 02 Nov 2025
Modality-Aware SAM: Sharpness-Aware-Minimization Driven Gradient Modulation for Harmonized Multimodal Learning Hossein R. Nowdeh Jie Ji Xiaolong Ma Fatemeh Afghah 80 0 0 28 Oct 2025
Quantifying Multimodal Imbalance: A GMM-Guided Adaptive Loss for Audio-Visual Learning Zhaocheng Liu Zhiwen Yu Xiaoqing Liu 148 0 0 20 Oct 2025
MILES: Modality-Informed Learning Rate Scheduler for Balancing Multimodal Learning Alejandro Guerra-Manzanares Farah E. Shamout 88 0 0 20 Oct 2025
Revisit Modality Imbalance at the Decision Layer Xiaoyu Ma Hao Chen 83 0 0 16 Oct 2025
Mixup Helps Understanding Multimodal Video Better Xiaoyu Ma Ding Ding Hao Chen 76 0 0 13 Oct 2025
MCE: Towards a General Framework for Handling Missing Modalities under Imbalanced Missing RatesPattern Recognition (Pattern Recogn.), 2025 Binyu Zhao Wei Zhang Zhaonian Zou 101 0 0 12 Oct 2025
SAMSOD: Rethinking SAM Optimization for RGB-T Salient Object Detection Zhengyi Liu Xinrui Wang Xianyong Fang Zhengzheng Tu Linbo Wang 86 0 0 04 Oct 2025
MIDAS: Misalignment-based Data Augmentation Strategy for Imbalanced Multimodal Learning Seong-Hyeon Hwang Soyoung Choi Steven Euijong Whang 118 0 0 30 Sep 2025
Shaping Initial State Prevents Modality Competition in Multi-modal Fusion: A Two-stage Scheduling Framework via Fast Partial Information Decomposition Jiaqi Tang Yinsong Xu Yang Liu Qingchao Chen 111 0 0 25 Sep 2025
Audio-Visual Separation with Hierarchical Fusion and Representation Alignment Han Hu Dongheng Lin Qiming Huang Yuqi Hou Hyung Jin Chang Jianbo Jiao 76 0 0 24 Sep 2025
PersonaX: Multimodal Datasets with LLM-Inferred Behavior Traits Loka Li Wong Yu Kang Minghao Fu Guangyi Chen Zhenhao Chen Gongxu Luo Yuewen Sun Salman Khan Peter Spirtes Kun Zhang 140 0 0 14 Sep 2025
Multi-modal Uncertainty Robust Tree Cover Segmentation For High-Resolution Remote Sensing Images Yuanyuan Gui Wei Li Y Samuel Wang X. Xia M. Marty C. Ginzler Z. Wang 109 0 0 05 Sep 2025
Decoding Visual Neural Representations by Multimodal with Dynamic BalancingExpert systems with applications (ESWA), 2025 Kaili sun Xingyu Miao Bing Zhai Haoran Duan Yang Long 92 0 0 03 Sep 2025
Robult: Leveraging Redundancy and Modality Specific Features for Robust Multimodal LearningInternational Joint Conference on Artificial Intelligence (IJCAI), 2025 Duy Nguyen Abhi Kamboj Minh N. Do 76 0 0 03 Sep 2025
Balanced Multimodal Learning: An Unidirectional Dynamic Interaction Perspective Shijie Wang Li Zhang Xinyan Liang Y. Qian Shen Hu 181 0 0 02 Sep 2025
AIM: Adaptive Intra-Network Modulation for Balanced Multimodal Learning Shu Shen Chao Chen Tong Zhang 188 0 0 27 Aug 2025
SEAM: Semantically Equivalent Across Modalities Benchmark for Vision-Language Models Zhenwei Tang Difan Jiao Blair Yang Ashton Anderson VLM CoGe 118 1 0 25 Aug 2025
eMotions: A Large-Scale Dataset and Audio-Visual Fusion Network for Emotion Analysis in Short-form Videos Xuecheng Wu Dingkang Yang Danlei Huang Xinyi Yin Yifan Wang ... Liangyu Fu Yang Liu Junxiao Xue Hadi Amirpour Wei Zhou 141 1 0 09 Aug 2025
A Scalable Pretraining Framework for Link Prediction with Efficient Adaptation Yu Song Zhigang Hua Harry Shomer Yan Xie Jingzhe Liu Bo Long Hui Liu AI4CE 72 1 0 06 Aug 2025
From Waveforms to Pixels: A Survey on Audio-Visual Segmentation Jia Li Yapeng Tian VOS 186 1 0 29 Jul 2025
Improving Multimodal Learning via Imbalanced Learning Shicai Wei Chunbo Luo Yang Luo 149 2 0 14 Jul 2025
Confidence-driven Gradient Modulation for Multimodal Human Activity Recognition: A Dynamic Contrastive Dual-Path Learning Approach Panpan Ji Junni Song Yifan Lu Hang Xiao Hanyu Liu Chao Li 157 0 0 03 Jul 2025
$G$^{2}$D: Boosting Multimodal Learning with Gradient-Guided Distillation$ G $^{2}$ D: Boosting Multimodal Learning with Gradient-Guided Distillation Mohammed Rakib A. Bagavathi 214 0 0 26 Jun 2025
DMAF-Net: An Effective Modality Rebalancing Framework for Incomplete Multi-Modal Medical Image Segmentation Libin Lan Hongxing Li Zunhui Xia Yudong Zhang 103 0 0 13 Jun 2025
Improving Multimodal Learning Balance and Sufficiency through Data Remixing Xiaoyu Ma Hao Chen Yongjian Deng 204 4 0 13 Jun 2025
RollingQ: Reviving the Cooperation Dynamics in Multimodal Transformer Haotian Ni Yake Wei Hang Liu Gong Chen Chong Peng Hao Lin Di Hu OffRL 242 1 0 13 Jun 2025
Dynamic Mixture of Curriculum LoRA Experts for Continual Multimodal Instruction Tuning Chendi Ge Xin Eric Wang Zeyang Zhang Hong Chen Jiapei Fan Longtao Huang Hui Xue Wenwu Zhu MoE CLL 178 5 0 13 Jun 2025
MokA: Multimodal Low-Rank Adaptation for MLLMs Yake Wei Yu Miao Dongzhan Zhou Di Hu 193 0 0 05 Jun 2025
Evaluating and Steering Modality Preferences in Multimodal Large Language Model Yu Zhang Jinlong Ma Yongshuai Hou Xuefeng Bai Kehai Chen Yang Xiang Jun Yu Min Zhang 351 6 0 27 May 2025
Learning Optimal Multimodal Information Bottleneck Representations Qilong Wu Yiyang Shao Jun Wang Xiaobo Sun 228 2 0 26 May 2025
MM-Prompt: Cross-Modal Prompt Tuning for Continual Visual Question Answering Xu Li Fan Lyu LRM 152 0 0 26 May 2025
MLLMs are Deeply Affected by Modality Bias Xu Zheng Chenfei Liao Yuqian Fu Kaiyu Lei Yuanhuiyi Lyu ... Yu Jiang Andrii Zadaianchuk Dacheng Tao Luc Van Gool Xuming Hu 260 10 0 24 May 2025
ICPL-ReID: Identity-Conditional Prompt Learning for Multi-Spectral Object Re-IdentificationIEEE transactions on multimedia (TMM), 2025 Shihao Li Chenglong Li Aihua Zheng Jin Tang Bin Luo 197 3 0 23 May 2025
Spiking Neural Networks with Temporal Attention-Guided Adaptive Fusion for imbalanced Multi-modal Learning Jiangrong Shen Yulin Xie Qi Xu Gang Pan Huajin Tang Badong Chen 180 4 0 20 May 2025
Multiscale Adaptive Conflict-Balancing Model For Multimedia Deepfake DetectionInternational Conference on Multimedia Retrieval (ICMR), 2025 Zihan Xiong Xiaohua Wu Lei Chen Fangqi Lou 202 0 0 19 May 2025
RMMSS: Towards Advanced Robust Multi-Modal Semantic Segmentation with Hybrid Prototype Distillation and Feature Selection Jiaqi Tan Xu Zheng Yuhang Liu 272 0 0 19 May 2025
Diffmv: A Unified Diffusion Framework for Healthcare Predictions with Random Missing Views and View Laziness Chuang Zhao Hui Tang Hongke Zhao Xiaomeng Li DiffM MedIm 170 0 0 17 May 2025
Incorporating brain-inspired mechanisms for multimodal learning in artificial intelligence Xiang He Dongcheng Zhao Yang Li Qingqun Kong Xin Yang Yi Zeng 262 0 0 15 May 2025
Towards Explainable Fusion and Balanced Learning in Multimodal Sentiment Analysis Miaosen Luo Yuncheng Jiang Sijie Mai 284 1 0 16 Apr 2025
Audio-visual Event Localization on Portrait Mode Short Videos Wuyang Liu Yi Chai Yongpeng Yan Yanzhen Ren 245 1 0 09 Apr 2025
FLAIRBrainSeg: Fine-grained brain segmentation using FLAIR MRI only Edern Le Bot Rémi Giraud Boris Mansencal T. Tourdias J. V. Manjón Pierrick Coupé 145 1 0 04 Apr 2025
CMD-HAR: Cross-Modal Disentanglement for Wearable Human Activity Recognition Xiaoyang Li Siyao Li Ying Yu Yixuan Jiang Hang Xiao Jingxi Long Haotian Tang Chao Li 294 0 0 27 Mar 2025
Adaptive Unimodal Regulation for Balanced Multimodal Information AcquisitionComputer Vision and Pattern Recognition (CVPR), 2025 Chengxiang Huang Yake Wei Zequn Yang D. Hu 248 6 0 24 Mar 2025
Continual Learning for Multiple Modalities Hyundong Jin Eunwoo Kim CLL 398 0 0 11 Mar 2025