Learning Factorized Multimodal Representations

16 June 2018

Yao-Hung Hubert Tsai

Paul Pu Liang

Amir Zadeh

Louis-Philippe Morency

Ruslan Salakhutdinov

DRL

ArXiv PDF HTML

Papers citing "Learning Factorized Multimodal Representations"

50 / 167 papers shown

Title
Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications Paul Pu Liang Chun Kai Ling Yun Cheng A. Obolenskiy Yudong Liu Rohan Pandey Alex Wilf Louis-Philippe Morency Ruslan Salakhutdinov OffRL 19 7 0 07 Jun 2023
Multimodal Fusion Interactions: A Study of Human and Automatic Quantification Paul Pu Liang Yun Cheng Ruslan Salakhutdinov Louis-Philippe Morency 15 3 0 07 Jun 2023
Learning Representations without Compositional Assumptions Tennison Liu Jeroen Berrevoets Zhaozhi Qian M. Schaar OCL SSL 20 1 0 31 May 2023
Dynamic Enhancement Network for Partial Multi-modality Person Re-identification A. Zheng Ziling He Zi Wang Chenglong Li Jin Tang 13 2 0 25 May 2023
Denoising Bottleneck with Mutual Information Maximization for Video Multimodal Fusion Shao-Yu Wu Damai Dai Ziwei Qin Tianyu Liu Binghuai Lin Yunbo Cao Zhifang Sui 22 6 0 24 May 2023
Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition Yaoting Wang Yuanchao Li Paul Pu Liang Louis-Philippe Morency P. Bell Catherine Lai CVBM 21 8 0 23 May 2023
Shared and Private Information Learning in Multimodal Sentiment Analysis with Deep Modal Alignment and Self-supervised Multi-Task Learning Songning Lai Jiakang Li Guinan Guo Xifeng Hu Yulong Li ... Yutong Liu Zhaoxia Ren Chun Wan Danmin Miao Zhi Liu SSL 28 9 0 15 May 2023
MMG-Ego4D: Multi-Modal Generalization in Egocentric Action Recognition Xinyu Gong S. Mohan Naina Dhingra Jean-Charles Bazin Yilei Li Zhangyang Wang Rakesh Ranjan EgoV 39 17 0 12 May 2023
Patchwork Learning: A Paradigm Towards Integrative Analysis across Diverse Biomedical Data Sources Suraj Rajendran Weishen Pan M. Sabuncu Yong Chen Jiayu Zhou Fei Wang 36 14 0 10 May 2023
Noise-Resistant Multimodal Transformer for Emotion Recognition Y. Liu Haoyu Zhang Yibing Zhan Zijing Chen Guanghao Yin Lin Wei Zhe Chen ViT 20 3 0 04 May 2023
TextMI: Textualize Multimodal Information for Integrating Non-verbal Cues in Pre-trained Language Models M. Hasan Md. Saiful Islam Sangwu Lee Wasifur Rahman Iftekhar Naim Mohammed Ibrahim Khan E. Hoque VLM 12 2 0 27 Mar 2023
Unimodal Training-Multimodal Prediction: Cross-modal Federated Learning with Hierarchical Aggregation Rongyu Zhang Xiaowei Chi Guiliang Liu Wenyi Zhang Yuan Du Fang Wang 26 12 0 27 Mar 2023
Decoupled Multimodal Distilling for Emotion Recognition Yong Li Yuan-Zheng Wang Zhen Cui 11 70 0 24 Mar 2023
Mirror U-Net: Marrying Multimodal Fission with Multi-task Learning for Semantic Segmentation in Medical Imaging Zdravko Marinov Simon Reiß D. Kersting Jens Kleesiek Rainer Stiefelhagen 16 12 0 13 Mar 2023
Robust Multimodal Fusion for Human Activity Recognition Sanju Xaviar Xin Yang Omid Ardakanian HAI 8 5 0 08 Mar 2023
Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework Paul Pu Liang Yun Cheng Xiang Fan Chun Kai Ling Suzanne Nie ... Nicholas B. Allen Randy P. Auerbach Faisal Mahmood Ruslan Salakhutdinov Louis-Philippe Morency 25 29 0 23 Feb 2023
Multi-modal Machine Learning in Engineering Design: A Review and Future Directions Binyang Song Ruilin Zhou Faez Ahmed AI4CE 32 38 0 14 Feb 2023
Self-supervised Multi-view Disentanglement for Expansion of Visual Collections Nihal Jain Praneetha Vaddamanu Paridhi Maheshwari Vishwa Vinay K. Kulkarni 8 3 0 04 Feb 2023
Learning Multimodal Data Augmentation in Feature Space Zichang Liu Zhiqiang Tang Xingjian Shi Aston Zhang Mu Li Anshumali Shrivastava A. Wilson 27 19 0 29 Dec 2022
InterMulti:Multi-view Multimodal Interactions with Text-dominated Hierarchical High-order Fusion for Emotion Analysis Feng Qiu Wanzeng Kong Yu-qiong Ding 21 2 0 20 Dec 2022
EffMulti: Efficiently Modeling Complex Multimodal Interactions for Emotion Analysis Feng Qiu Chengyang Xie Yu-qiong Ding Wanzeng Kong 8 1 0 16 Dec 2022
Adapted Multimodal BERT with Layer-wise Fusion for Sentiment Analysis Odysseas S. Chlapanis Georgios Paraskevopoulos Alexandros Potamianos 16 9 0 01 Dec 2022
Disentangled Representation Learning Xin Eric Wang Hong Chen Siao Tang Zihao Wu Wenwu Zhu DRL 8 77 0 21 Nov 2022
UniMSE: Towards Unified Multimodal Sentiment Analysis and Emotion Recognition Guimin Hu Ting-En Lin Yi Zhao Guangming Lu Yuchuan Wu Yongbin Li 25 108 0 21 Nov 2022
On the Use of Modality-Specific Large-Scale Pre-Trained Encoders for Multimodal Sentiment Analysis Atsushi Ando Ryo Masumura Akihiko Takashima Satoshi Suzuki Naoki Makishima Keita Suzuki Takafumi Moriya Takanori Ashihara Hiroshi Sato 26 9 0 28 Oct 2022
Multimodal Contrastive Learning via Uni-Modal Coding and Cross-Modal Prediction for Multimodal Sentiment Analysis Ronghao Lin Haifeng Hu SSL 17 15 0 26 Oct 2022
MM-Align: Learning Optimal Transport-based Alignment Dynamics for Fast and Accurate Inference on Missing Modality Sequences Wei Han Hui Chen MingSung Kan Soujanya Poria 24 7 0 23 Oct 2022
ImageArg: A Multi-modal Tweet Dataset for Image Persuasiveness Mining Zhexiong Liu M. Guo Y. Dai Diane Litman 8 13 0 14 Sep 2022
Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions Paul Pu Liang Amir Zadeh Louis-Philippe Morency 8 59 0 07 Sep 2022
Progressive Fusion for Multimodal Integration Shiv Shankar Laure Thompson M. Fiterau 19 3 0 01 Sep 2022
Cross-Modality Gated Attention Fusion for Multimodal Sentiment Analysis Ming-Xin Jiang Shaoxiong Ji 8 3 0 25 Aug 2022
A Graphical Model for Fusing Diverse Microbiome Data Mehmet Aktukmak Haonan Zhu M. Chevrette Julia Nepper S. Magesh J. Handelsman Alfred Hero 11 2 0 21 Aug 2022
CMSBERT-CLR: Context-driven Modality Shifting BERT with Contrastive Learning for linguistic, visual, acoustic Representations Junghun Kim Jihie Kim 17 2 0 21 Aug 2022
Face-to-Face Contrastive Learning for Social Intelligence Question-Answering Alex Wilf Qianli Ma Paul Pu Liang Amir Zadeh Louis-Philippe Morency 31 10 0 29 Jul 2022
Personality-Driven Social Multimedia Content Recommendation Qi Yang Sergey I. Nikolenko Alfred Huang Aleksandr Farseev 25 14 0 25 Jul 2022
GraphCFC: A Directed Graph Based Cross-Modal Feature Complementation Approach for Multimodal Conversational Emotion Recognition Jiang Li Xiaoping Wang Guoqing Lv Zhigang Zeng 23 36 0 06 Jul 2022
A survey of multimodal deep generative models Masahiro Suzuki Y. Matsuo SyDa DRL 43 75 0 05 Jul 2022
Multi-scale Cooperative Multimodal Transformers for Multimodal Sentiment Analysis in Videos Lianyang Ma Yu Yao Tao Liang Tongliang Liu 11 4 0 16 Jun 2022
COLD Fusion: Calibrated and Ordinal Latent Distribution Fusion for Uncertainty-Aware Multimodal Emotion Recognition M. Tellamekala Shahin Amiriparian Björn W. Schuller Elisabeth André T. Giesbrecht M. Valstar 13 25 0 12 Jun 2022
Analyzing Modality Robustness in Multimodal Sentiment Analysis Devamanyu Hazarika Yingting Li Bo Cheng Shuai Zhao Roger Zimmermann Soujanya Poria 26 32 0 30 May 2022
SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Text Pinaki Nath Chowdhury A. Bhunia Aneeshan Sain Subhadeep Koley Tao Xiang Yi-Zhe Song 30 29 0 25 Apr 2022
Are Multimodal Transformers Robust to Missing Modality? Mengmeng Ma Jian Ren Long Zhao Davide Testuggine Xi Peng ViT 17 145 0 12 Apr 2022
Integrating Physiological Time Series and Clinical Notes with Transformer for Early Prediction of Sepsis Yuqing Wang Yun Zhao R. Callcut Linda R. Petzold 6 18 0 28 Mar 2022
M-SENA: An Integrated Platform for Multimodal Sentiment Analysis Huisheng Mao Ziqi Yuan Hua Xu Wenmeng Yu Yihe Liu Kai Gao 14 41 0 23 Mar 2022
DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations Yiwei Lyu Paul Pu Liang Zihao Deng Ruslan Salakhutdinov Louis-Philippe Morency 16 26 0 03 Mar 2022
COMPASS: Contrastive Multimodal Pretraining for Autonomous Systems Shuang Ma Sai H. Vemprala Wenshan Wang Jayesh K. Gupta Yale Song Daniel J. McDuff Ashish Kapoor SSL 6 9 0 20 Feb 2022
Geometric Multimodal Contrastive Representation Learning Petra Poklukar Miguel Vasco Hang Yin Francisco S. Melo Ana Paiva Danica Kragic 11 46 0 07 Feb 2022
Multi-channel Attentive Graph Convolutional Network With Sentiment Fusion For Multimodal Sentiment Analysis Luwei Xiao Xingjiao Wu Wen Wu Jing Yang Liangbo He 21 14 0 25 Jan 2022
Tailor Versatile Multi-modal Learning for Multi-label Emotion Recognition Yi Zhang Mingyuan Chen Jundong Shen Chongjun Wang 10 58 0 15 Jan 2022
Multimodal Representations Learning Based on Mutual Information Maximization and Minimization and Identity Embedding for Multimodal Sentiment Analysis Jiahao Zheng Sen Zhang Xiaoping Wang Zhigang Zeng 12 7 0 10 Jan 2022