Learning Factorized Multimodal Representations

16 June 2018

Yao-Hung Hubert Tsai

Paul Pu Liang

Amir Zadeh

Louis-Philippe Morency

Ruslan Salakhutdinov

DRL

ArXiv PDF HTML

Papers citing "Learning Factorized Multimodal Representations"

50 / 167 papers shown

Title
Multimodal Transformers are Hierarchical Modal-wise Heterogeneous Graphs Yijie Jin Junjie Peng Xuanchao Lin Haochen Yuan Lan Wang Cangzhi Zheng 30 0 0 02 May 2025
Aggregation of Dependent Expert Distributions in Multimodal Variational Autoencoders R. A. Mancisidor Robert Jenssen Shujian Yu Michael C. Kampffmeyer 10 0 0 02 May 2025
DeepMLF: Multimodal language model with learnable tokens for deep fusion in sentiment analysis Efthymios Georgiou V. Katsouros Yannis Avrithis Alexandros Potamianos 21 0 0 15 Apr 2025
DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learning Chengxuan Qian Shuo Xing Shawn Li Yue Zhao Zhengzhong Tu 43 0 0 14 Mar 2025
CrossOver: 3D Scene Cross-Modal Alignment S. Sarkar O. Mikšík Marc Pollefeys Daniel Barath Iro Armeni 3DPC 67 0 0 20 Feb 2025
Multimodal Emotion Recognition using Audio-Video Transformer Fusion with Cross Attention Joe Dhanith Shravan Venkatraman Modigari Narendra Vigya Sharma Santhosh Malarvannan 65 0 0 20 Feb 2025
RAMer: Reconstruction-based Adversarial Model for Multi-party Multi-modal Multi-label Emotion Recognition Xudong Yang Yizhang Zhu Nan Tang Yuyu Luo 34 0 0 09 Feb 2025
Unbiased Sliced Wasserstein Kernels for High-Quality Audio Captioning Manh Luong Khai Nguyen Dinh Q. Phung Gholamreza Haffari Lizhen Qu 39 0 0 08 Feb 2025
Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attention Yuzhe Weng Haotian Wang Tian Gao Kewei Li Shutong Niu Jun Du 25 0 0 19 Oct 2024
Enhancing Unimodal Latent Representations in Multimodal VAEs through Iterative Amortized Inference Yuta Oshima Masahiro Suzuki Y. Matsuo 23 0 0 15 Oct 2024
Learning in Order! A Sequential Strategy to Learn Invariant Features for Multimodal Sentiment Analysis Xianbing Zhao Lizhen Qu Tao Feng Jianfei Cai Buzhou Tang 29 0 0 05 Sep 2024
Seeking the Sufficiency and Necessity Causal Features in Multimodal Representation Learning Boyu Chen Junjie Liu Zhu Li Mengyue yang 13 1 0 29 Aug 2024
Meta-Learn Unimodal Signals with Weak Supervision for Multimodal Sentiment Analysis Sijie Mai Yu Zhao Ying Zeng Jianhua Yao Haifeng Hu 20 2 0 28 Aug 2024
GSIFN: A Graph-Structured and Interlaced-Masked Multimodal Transformer-based Fusion Network for Multimodal Sentiment Analysis Yijie Jin 17 0 0 27 Aug 2024
End-to-end Semantic-centric Video-based Multimodal Affective Computing Ronghao Lin Ying Zeng Sijie Mai Haifeng Hu VGen 28 0 0 14 Aug 2024
DisCoM-KD: Cross-Modal Knowledge Distillation via Disentanglement Representation and Adversarial Learning Dino Ienco C. Dantas 17 1 0 05 Aug 2024
Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representations Dingkang Yang Mingcheng Li Linhao Qu Kun Yang Peng Zhai Song Wang Lihua Zhang 27 0 0 06 Jul 2024
CLHOP: Combined Audio-Video Learning for Horse 3D Pose and Shape Estimation Ci Li Elin Hernlund Hedvig Kjellström Silvia Zuffi 3DH 21 2 0 01 Jul 2024
Coupled Mamba: Enhanced Multi-modal Fusion with Coupled State Space Model Wenbing Li Hang Zhou Junqing Yu Zikai Song Wei Yang Mamba 28 3 0 28 May 2024
Enhancing Apparent Personality Trait Analysis with Cross-Modal Embeddings Ádám Fodor R. R. Saboundji András Lőrincz 18 0 0 06 May 2024
Trustworthy Multimodal Fusion for Sentiment Analysis in Ordinal Sentiment Space Zhuyang Xie Yan Yang Jie Wang Xiaorong Liu Xiaofan Li 21 6 0 13 Apr 2024
TCAN: Text-oriented Cross Attention Network for Multimodal Sentiment Analysis Ming Zhou Yunfei Feng Ziqi Zhou Kai Wang Tong Wang Dong-ming Yan 36 0 0 06 Apr 2024
Multi-modal perception for soft robotic interactions using generative models Enrico Donato Egidio Falotico T. G. Thuruthel 13 2 0 05 Apr 2024
Borrowing Treasures from Neighbors: In-Context Learning for Multimodal Learning with Missing Modalities and Data Scarcity Zhuo Zhi Ziquan Liu M. Elbadawi Adam Daneshmend Mine Orlu Abdul Basit Andreas Demosthenous Miguel R. D. Rodrigues 14 2 0 14 Mar 2024
Towards Multimodal Sentiment Analysis Debiasing via Bias Purification Dingkang Yang Mingcheng Li Dongling Xiao Yang Liu Kun Yang Zhaoyu Chen Yuzheng Wang Peng Zhai Ke Li Lihua Zhang 28 16 0 08 Mar 2024
Gradient-Guided Modality Decoupling for Missing-Modality Robustness Hao Wang Shengda Luo Guosheng Hu Jianguo Zhang 19 3 0 26 Feb 2024
Quantifying and Enhancing Multi-modal Robustness with Modality Preference Zequn Yang Yake Wei Ce Liang Di Hu AAML 11 9 0 09 Feb 2024
Closed-Loop Unsupervised Representation Disentanglement with $β$ -VAE Distillation and Diffusion Probabilistic Feedback Xin Jin Bo Li Baao Xie Wenyao Zhang Jinming Liu Ziqiang Li Tao Yang Wenjun Zeng DRL DiffM CoGe 19 7 0 04 Feb 2024
Towards Urban General Intelligence: A Review and Outlook of Urban Foundation Models Weijiao Zhang Jindong Han Zhao Xu Hang Ni Hao Liu Hui Xiong Hui Xiong AI4CE 77 14 0 30 Jan 2024
Exploring Missing Modality in Multimodal Egocentric Datasets Merey Ramazanova Alejandro Pardo Humam Alwassel Bernard Ghanem EgoV 12 4 0 21 Jan 2024
Toward Robust Multimodal Learning using Multimodal Foundational Models Xianbing Zhao Soujanya Poria Xuejiao Li Yixin Chen Buzhou Tang VLM 29 0 0 20 Jan 2024
MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition Zheng Lian Licai Sun Yong Ren Hao Gu Haiyang Sun Lan Chen Bin Liu Jianhua Tao 11 12 0 07 Jan 2024
Multimodal Sentiment Analysis with Missing Modality: A Knowledge-Transfer Approach Weide Liu Huijing Zhan Hao Chen Fengmao Lv 16 1 0 28 Dec 2023
Modality-Collaborative Transformer with Hybrid Feature Reconstruction for Robust Emotion Recognition Chengxin Chen Pengyuan Zhang 24 5 0 26 Dec 2023
GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognition Zheng Lian Licai Sun Haiyang Sun Kang Chen Zhuofan Wen Hao Gu Bin Liu Jianhua Tao 12 8 0 07 Dec 2023
Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimodal Emotion Recognition Dongyuan Li Yusong Wang Kotaro Funakoshi Manabu Okumura 20 12 0 18 Nov 2023
ID Embedding as Subtle Features of Content and Structure for Multimodal Recommendation Yuting Liu Enneng Yang Yizhou Dang Guibing Guo Qiang Liu Yuliang Liang Linying Jiang Xingwei Wang 19 5 0 10 Nov 2023
Self-MI: Efficient Multimodal Fusion via Self-Supervised Multi-Task Learning with Auxiliary Mutual Information Maximization Cam-Van Thi Nguyen Ngoc-Hoa Thi Nguyen Duc-Trong Le Quang-Thuy Ha SSL 22 0 0 07 Nov 2023
Overview of ImageArg-2023: The First Shared Task in Multimodal Argument Mining Zhexiong Liu Mohamed Elarby Yang Zhong Diane Litman 11 10 0 15 Oct 2023
Multimodal Variational Auto-encoder based Audio-Visual Segmentation Yuxin Mao Jing Zhang Mochu Xiang Yiran Zhong Yuchao Dai 22 33 0 12 Oct 2023
What Makes for Robust Multi-Modal Models in the Face of Missing Modalities? Siting Li Chenzhuang Du Yue Zhao Yu Huang Hang Zhao 19 4 0 10 Oct 2023
Learning Language-guided Adaptive Hyper-modality Representation for Multimodal Sentiment Analysis Haoyu Zhang Yu Wang Guanghao Yin Kejun Liu Yuanyuan Liu Tianshu Yu 13 30 0 09 Oct 2023
MIS-AVoiDD: Modality Invariant and Specific Representation for Audio-Visual Deepfake Detection Vinaya Sree Katamneni A. Rattani 12 7 0 03 Oct 2023
GRID: A Platform for General Robot Intelligence Development Sai H. Vemprala Shuhang Chen Abhinav Shukla Dinesh Narayanan Ashish Kapoor 17 10 0 02 Oct 2023
Sarcasm in Sight and Sound: Benchmarking and Expansion to Improve Multimodal Sarcasm Detection Swapnil Bhosale Abhra Chaudhuri Alex Lee Robert Williams Divyank Tiwari Anjan Dutta Xiatian Zhu Pushpak Bhattacharyya Diptesh Kanojia 28 1 0 29 Sep 2023
Multimodal Multi-loss Fusion Network for Sentiment Analysis Zehui Wu Ziwei Gong Jaywon Koo Julia Hirschberg 22 2 0 01 Aug 2023
Unlocking the Emotional World of Visual Media: An Overview of the Science, Research, and Impact of Understanding Emotion James Z. Wang Sicheng Zhao Chenyan Wu Reginald B. Adams M. Newman T. Shafir Rachelle Tsachor 13 27 0 25 Jul 2023
Deep Equilibrium Multimodal Fusion Jinhong Ni Yalong Bai Wei Zhang Ting Yao Tao Mei 14 1 0 29 Jun 2023
MultiZoo & MultiBench: A Standardized Toolkit for Multimodal Deep Learning Paul Pu Liang Yiwei Lyu Xiang Fan Arav Agarwal Yun Cheng Louis-Philippe Morency Ruslan Salakhutdinov VLM 14 4 0 28 Jun 2023
Factorized Contrastive Learning: Going Beyond Multi-view Redundancy Paul Pu Liang Zihao Deng Martin Q. Ma James Y. Zou Louis-Philippe Morency Ruslan Salakhutdinov SSL 11 16 0 08 Jun 2023