Learning Factorized Multimodal Representations

16 June 2018

Yao-Hung Hubert Tsai

Paul Pu Liang

Amir Zadeh

Louis-Philippe Morency

Ruslan Salakhutdinov

DRL

ArXiv PDF HTML

Papers citing "Learning Factorized Multimodal Representations"

50 / 167 papers shown

Title
Multimodal Adversarially Learned Inference with Factorized Discriminators Wenxue Chen Jianke Zhu 35 3 0 20 Dec 2021
LMR-CBT: Learning Modality-fused Representations with CB-Transformer for Multimodal Emotion Recognition from Unaligned Multimodal Sequences Ziwang Fu Feng Liu Hanyang Wang Siyuan Shen Jiahao Zhang Jiayin Qi Xiangling Fu Aimin Zhou 23 9 0 03 Dec 2021
Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis Ying Zeng Sijie Mai Haifeng Hu 14 19 0 10 Nov 2021
Reconstructing Nonlinear Dynamical Systems from Multi-Modal Time Series Daniel Kramer P. Bommer Carlo Tombolini G. Koppe Daniel Durstewitz BDL AI4TS AI4CE 9 18 0 04 Nov 2021
Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach Qitian Wu Chenxiao Yang Junchi Yan 11 32 0 09 Oct 2021
On the Limitations of Multimodal VAEs Imant Daunhawer Thomas M. Sutter Kieran Chin-Cheong Emanuele Palumbo Julia E. Vogt 28 27 0 08 Oct 2021
How to Sense the World: Leveraging Hierarchy in Multimodal Perception for Robust Reinforcement Learning Agents Miguel Vasco Hang Yin Francisco S. Melo Ana Paiva 8 7 0 07 Oct 2021
Unsupervised Multimodal Language Representations using Convolutional Autoencoders Panagiotis Koromilas Theodoros Giannakopoulos SSL 21 10 0 06 Oct 2021
Communication-Efficient Federated Linear and Deep Generalized Canonical Correlation Analysis Sagar Shrestha Xiao Fu FedML 18 0 0 25 Sep 2021
TEASEL: A Transformer-Based Speech-Prefixed Language Model Mehdi Arjmand M. Dousti H. Moradi 25 18 0 12 Sep 2021
Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis Wei Han Hui Chen Soujanya Poria 19 307 0 01 Sep 2021
Maximum Likelihood Estimation for Multimodal Learning with Missing Modality Fei Ma Xiangxiang Xu Shao-Lun Huang Lin Zhang 8 11 0 24 Aug 2021
Graph Capsule Aggregation for Unaligned Multimodal Sequences Jianfeng Wu Sijie Mai Haifeng Hu 14 22 0 17 Aug 2021
Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directions Anil Rahate Rahee Walambe S. Ramanna K. Kotecha 9 96 0 29 Jul 2021
Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis Wei Han Hui Chen Alexander Gelbukh Amir Zadeh Louis-Philippe Morency Soujanya Poria 6 129 0 28 Jul 2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Zetian Wu Yun Cheng ... Peter Wu Michelle A. Lee Yuke Zhu Ruslan Salakhutdinov Louis-Philippe Morency VLM 21 126 0 15 Jul 2021
Evaluating the progress of Deep Reinforcement Learning in the real world: aligning domain-agnostic and domain-specific research J. Luis E. Crawley B. Cameron OffRL 5 5 0 07 Jul 2021
Dizygotic Conditional Variational AutoEncoder for Multi-Modal and Partial Modality Absent Few-Shot Learning Yi Zhang Sheng Huang Xiao-song Peng Dan Yang 15 9 0 28 Jun 2021
Learning Multimodal VAEs through Mutual Supervision Thomas Joy Yuge Shi Philip H. S. Torr Tom Rainforth Sebastian M. Schmon N. Siddharth SSL 19 20 0 23 Jun 2021
Two-Faced Humans on Twitter and Facebook: Harvesting Social Multimedia for Human Personality Profiling Qi Yang Aleksandr Farseev Andrey Filchenkov 6 3 0 20 Jun 2021
Generalized Multimodal ELBO Thomas M. Sutter Imant Daunhawer Julia E. Vogt 11 88 0 06 May 2021
Fast Multi-Step Critiquing for VAE-based Recommender Systems Diego Antognini Boi Faltings LRM 38 16 0 03 May 2021
Weakly-supervised Multi-task Learning for Multimodal Affect Recognition Wenliang Dai Samuel Cahyawijaya Yejin Bang Pascale Fung CVBM 28 10 0 23 Apr 2021
Emotion Dynamics Modeling via BERT Haiqing Yang Jianping Shen 14 8 0 15 Apr 2021
Multimodal End-to-End Sparse Model for Emotion Recognition Wenliang Dai Samuel Cahyawijaya Zihan Liu Pascale Fung CVBM 8 79 0 17 Mar 2021
SMIL: Multimodal Learning with Severely Missing Modality Mengmeng Ma Jian Ren Long Zhao Sergey Tulyakov Cathy H. Wu Xi Peng 16 186 0 09 Mar 2021
Video Sentiment Analysis with Bimodal Information-augmented Multi-Head Attention Ting-Wei Wu Jun-jie Peng Wenqiang Zhang Huiran Zhang Chuan Ma Yansong Huang 8 55 0 03 Mar 2021
Variational Selective Autoencoder: Learning from Partially-Observed Heterogeneous Data Yu Gong Hossein Hajimirsadeghi Jiawei He Thibaut Durand Greg Mori CML 17 9 0 25 Feb 2021
Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis Wenmeng Yu Hua Xu Ziqi Yuan Jiele Wu SSL 45 430 0 09 Feb 2021
A Variational Information Bottleneck Approach to Multi-Omics Data Integration Changhee Lee M. Schaar DRL 8 60 0 05 Feb 2021
MSAF: Multimodal Split Attention Fusion Lang Su Chuqing Hu Guofa Li Dongpu Cao 16 37 0 13 Dec 2020
Detect, Reject, Correct: Crossmodal Compensation of Corrupted Sensors Michelle A. Lee Matthew Tan Yuke Zhu Jeannette Bohg 30 25 0 01 Dec 2020
Analyzing Unaligned Multimodal Sequence via Graph Convolution and Graph Pooling Fusion Sijie Mai Songlong Xing Jiaxuan He Ying Zeng Haifeng Hu GNN 6 21 0 27 Nov 2020
Improving Multimodal Accuracy Through Modality Pre-training and Attention Aya Abdelsalam Ismail Mahmudul Hasan F. Ishtiaq 20 17 0 11 Nov 2020
MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences Jianing Yang Yongxin Wang Ruitao Yi Yuying Zhu Azaan Rehman Amir Zadeh Soujanya Poria Louis-Philippe Morency 21 11 0 22 Oct 2020
Multimodal Research in Vision and Language: A Review of Current and Emerging Trends Shagun Uppal Sarthak Bhagat Devamanyu Hazarika Navonil Majumdar Soujanya Poria Roger Zimmermann Amir Zadeh 11 6 0 19 Oct 2020
DialogueTRM: Exploring the Intra- and Inter-Modal Emotional Behaviors in the Conversation Yuzhao Mao Qi Sun Guang Liu Xiaojie Wang Weiguo Gao Xuan Li Jianping Shen 14 17 0 15 Oct 2020
Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition Wenliang Dai Zihan Liu Tiezheng Yu Pascale Fung 27 29 0 21 Sep 2020
Multi-modal Attention for Speech Emotion Recognition Zexu Pan Zhaojie Luo Jichen Yang Haizhou Li CVBM 9 60 0 09 Sep 2020
Jointly Fine-Tuning "BERT-like" Self Supervised Models to Improve Multimodal Speech Emotion Recognition Shamane Siriwardhana Andrew Reis Rivindu Weerasekera Suranga Nanayakkara 11 103 0 15 Aug 2020
Invertible Zero-Shot Recognition Flows Yuming Shen Jie Qin Lei Huang BDL AI4CE 6 86 0 09 Jul 2020
Relating by Contrasting: A Data-efficient Framework for Multimodal Generative Models Yuge Shi Brooks Paige Philip H. S. Torr N. Siddharth VLM 13 36 0 02 Jul 2020
Multimodal Generative Learning Utilizing Jensen-Shannon-Divergence Thomas M. Sutter Imant Daunhawer Julia E. Vogt 23 57 0 15 Jun 2020
Bayesian Experience Reuse for Learning from Multiple Demonstrators Michael Gimelfarb Scott Sanner Chi-Guhn Lee 6 0 0 10 Jun 2020
Neural Methods for Point-wise Dependency Estimation Yao-Hung Hubert Tsai Han Zhao M. Yamada Louis-Philippe Morency Ruslan Salakhutdinov 17 30 0 09 Jun 2020
A Dataset and Benchmarks for Multimedia Social Analysis Bofan Xue David M. Chan John F. Canny VGen 13 0 0 05 Jun 2020
Instance Explainable Temporal Network For Multivariate Timeseries N. Madiraju H. Karimabadi AI4TS 8 0 0 26 May 2020
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis Devamanyu Hazarika Roger Zimmermann Soujanya Poria 14 484 0 07 May 2020
Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis Yao-Hung Hubert Tsai Martin Q. Ma Muqiao Yang Ruslan Salakhutdinov Louis-Philippe Morency 4 4 0 29 Apr 2020
Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion Cheng Chen Qi Dou Yueming Jin Hao Chen J. Qin Pheng-Ann Heng 4 124 0 22 Feb 2020