Multimodal Language Analysis with Recurrent Multistage Fusion

12 August 2018

Paul Pu Liang

Liu Ziyin

Amir Zadeh

Louis-Philippe Morency

ArXiv PDF HTML

Papers citing "Multimodal Language Analysis with Recurrent Multistage Fusion"

50 / 67 papers shown

Title
Classifier-guided Gradient Modulation for Enhanced Multimodal Learning Zirun Guo Tao Jin Jingyuan Chen Zhou Zhao 21 0 0 03 Nov 2024
Recent Trends of Multimodal Affective Computing: A Survey from NLP Perspective Guimin Hu Yi Xin Weimin Lyu Haojian Huang Chang Sun Z. Zhu Lin Gui Ruichu Cai Erik Cambria Hasti Seifi 21 2 0 11 Sep 2024
Multimodal Prompt Learning with Missing Modalities for Sentiment Analysis and Emotion Recognition Zirun Guo Tao Jin Zhou Zhao 21 0 0 07 Jul 2024
Asynchronous Multimodal Video Sequence Fusion via Learning Modality-Exclusive and -Agnostic Representations Dingkang Yang Mingcheng Li Linhao Qu Kun Yang Peng Zhai Song Wang Lihua Zhang 19 0 0 06 Jul 2024
Neuro-Inspired Information-Theoretic Hierarchical Perception for Multimodal Learning Xiongye Xiao Gengshuo Liu Gaurav Gupta De-An Cao Shixuan Li Yaxing Li Tianqing Fang Mingxi Cheng Paul Bogdan 22 1 0 15 Apr 2024
Multimodal Sentiment Analysis with Missing Modality: A Knowledge-Transfer Approach Weide Liu Huijing Zhan Hao Chen Fengmao Lv 11 1 0 28 Dec 2023
Modality-Collaborative Transformer with Hybrid Feature Reconstruction for Robust Emotion Recognition Chengxin Chen Pengyuan Zhang 11 1 0 26 Dec 2023
A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations Hui Ma Jian Wang Hongfei Lin Bo Zhang Yijia Zhang Bo Xu 15 12 0 31 Oct 2023
Sarcasm in Sight and Sound: Benchmarking and Expansion to Improve Multimodal Sarcasm Detection Swapnil Bhosale Abhra Chaudhuri Alex Lee Robert Williams Divyank Tiwari Anjan Dutta Xiatian Zhu Pushpak Bhattacharyya Diptesh Kanojia 17 1 0 29 Sep 2023
Neuro-Inspired Hierarchical Multimodal Learning Xiongye Xiao Gengshuo Liu Gaurav Gupta Benjamin Cho Shixuan Li Yaxing Li Tianqing Fang M. Erez Michael Orshansky 11 1 0 27 Sep 2023
Deep Equilibrium Multimodal Fusion Jinhong Ni Yalong Bai Wei Zhang Ting Yao Tao Mei 12 1 0 29 Jun 2023
Multimodal Fusion Interactions: A Study of Human and Automatic Quantification Paul Pu Liang Yun Cheng Ruslan Salakhutdinov Louis-Philippe Morency 15 3 0 07 Jun 2023
Cross-Attention is Not Enough: Incongruity-Aware Dynamic Hierarchical Fusion for Multimodal Affect Recognition Yaoting Wang Yuanchao Li Paul Pu Liang Louis-Philippe Morency P. Bell Catherine Lai CVBM 13 8 0 23 May 2023
Multimodal Sentiment Analysis: A Survey Songning Lai Xifeng Hu Hao Xu Zhaoxia Ren Zhi Liu 21 18 0 12 May 2023
TextMI: Textualize Multimodal Information for Integrating Non-verbal Cues in Pre-trained Language Models M. Hasan Md. Saiful Islam Sangwu Lee Wasifur Rahman Iftekhar Naim Mohammed Ibrahim Khan E. Hoque VLM 10 2 0 27 Mar 2023
Unimodal Training-Multimodal Prediction: Cross-modal Federated Learning with Hierarchical Aggregation Rongyu Zhang Xiaowei Chi Guiliang Liu Wenyi Zhang Yuan Du Fang Wang 18 9 0 27 Mar 2023
InterMulti:Multi-view Multimodal Interactions with Text-dominated Hierarchical High-order Fusion for Emotion Analysis Feng Qiu Wanzeng Kong Yu-qiong Ding 13 2 0 20 Dec 2022
Multimodal Learning for Multi-Omics: A Survey Sina Tabakhi M. N. I. Suvon Pegah Ahadian Haiping Lu 10 7 0 29 Nov 2022
A Self-Adjusting Fusion Representation Learning Model for Unaligned Text-Audio Sequences Kaicheng Yang Ruxuan Zhang Hua Xu Kai Gao 12 1 0 12 Nov 2022
Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations Sijie Mai Ying Zeng Haifeng Hu 19 36 0 31 Oct 2022
On the Use of Modality-Specific Large-Scale Pre-Trained Encoders for Multimodal Sentiment Analysis Atsushi Ando Ryo Masumura Akihiko Takashima Satoshi Suzuki Naoki Makishima Keita Suzuki Takafumi Moriya Takanori Ashihara Hiroshi Sato 22 7 0 28 Oct 2022
Hybrid Multimodal Fusion for Humor Detection Hao Xu Weifeng Liu Jingwei Liu Mingzheng Li Yu Feng Yasi Peng Yunwei Shi Xiao Sun Meng Wang 19 14 0 24 Sep 2022
Multi-scale Cooperative Multimodal Transformers for Multimodal Sentiment Analysis in Videos Lianyang Ma Yu Yao Tao Liang Tongliang Liu 6 4 0 16 Jun 2022
Brainish: Formalizing A Multimodal Language for Intelligence and Consciousness Paul Pu Liang 12 6 0 14 Apr 2022
Transformer-based Multimodal Information Fusion for Facial Expression Analysis Wei Zhang Feng Qiu Suzhe Wang Hao Zeng Zhimeng Zhang Rudong An Bowen Ma Yu-qiong Ding CVBM 14 76 0 23 Mar 2022
DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations Yiwei Lyu Paul Pu Liang Zihao Deng Ruslan Salakhutdinov Louis-Philippe Morency 11 26 0 03 Mar 2022
Multi-channel Attentive Graph Convolutional Network With Sentiment Fusion For Multimodal Sentiment Analysis Luwei Xiao Xingjiao Wu Wen Wu Jing Yang Liangbo He 13 14 0 25 Jan 2022
ScaleVLAD: Improving Multimodal Sentiment Analysis via Multi-Scale Fusion of Locally Descriptors Huaishao Luo Lei Ji Yanyong Huang Bin Wang Shenggong Ji Tianrui Li 11 14 0 02 Dec 2021
Many Heads but One Brain: Fusion Brain -- a Competition and a Single Multimodal Multitask Architecture Daria Bakshandaeva Denis Dimitrov V.Ya. Arkhipkin Alex Shonenkov M. Potanin ... Mikhail Martynov Anton Voronov Vera Davydova E. Tutubalina Aleksandr Petiushko 22 0 0 22 Nov 2021
Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributions Huan Ma Zongbo Han Changqing Zhang H. Fu Joey Tianyi Zhou Q. Hu EDL UQCV 23 35 0 11 Nov 2021
Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis Ying Zeng Sijie Mai Haifeng Hu 14 12 0 10 Nov 2021
Multistage linguistic conditioning of convolutional layers for speech emotion recognition Andreas Triantafyllopoulos U. Reichel Shuo Liu Simon Huber F. Eyben Björn W. Schuller 17 8 0 13 Oct 2021
Unsupervised Multimodal Language Representations using Convolutional Autoencoders Panagiotis Koromilas Theodoros Giannakopoulos SSL 19 10 0 06 Oct 2021
Graph Capsule Aggregation for Unaligned Multimodal Sequences Jianfeng Wu Sijie Mai Haifeng Hu 6 22 0 17 Aug 2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning Paul Pu Liang Yiwei Lyu Xiang Fan Zetian Wu Yun Cheng ... Peter Wu Michelle A. Lee Yuke Zhu Ruslan Salakhutdinov Louis-Philippe Morency VLM 13 126 0 15 Jul 2021
Towards Understanding and Mitigating Social Biases in Language Models Paul Pu Liang Chiyu Wu Louis-Philippe Morency Ruslan Salakhutdinov 12 297 0 24 Jun 2021
Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data Paul Pu Liang Terrance Liu Anna Cai Michal Muszynski Ryo Ishii Nicholas B. Allen Randy P. Auerbach David Brent Ruslan Salakhutdinov Louis-Philippe Morency 22 17 0 24 Jun 2021
Quantum Cognitively Motivated Decision Fusion for Video Sentiment Analysis Dimitris Gkoumas Qiuchi Li Shahram Dehdashti Massimo Melucci Yijun Yu D. Song 21 17 0 12 Jan 2021
Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study Terrance Liu Paul Pu Liang Michal Muszynski Ryo Ishii David Brent Randy P. Auerbach Nicholas B. Allen Louis-Philippe Morency 9 8 0 04 Dec 2020
Analyzing Unaligned Multimodal Sequence via Graph Convolution and Graph Pooling Fusion Sijie Mai Songlong Xing Jiaxuan He Ying Zeng Haifeng Hu GNN 6 21 0 27 Nov 2020
Improving Multimodal Accuracy Through Modality Pre-training and Attention Aya Abdelsalam Ismail Mahmudul Hasan F. Ishtiaq 12 17 0 11 Nov 2020
An Investigation of how Label Smoothing Affects Generalization Blair Chen Liu Ziyin Zihao W. Wang Paul Pu Liang UQCV 4 15 0 23 Oct 2020
MTAG: Modal-Temporal Attention Graph for Unaligned Human Multimodal Language Sequences Jianing Yang Yongxin Wang Ruitao Yi Yuying Zhu Azaan Rehman Amir Zadeh Soujanya Poria Louis-Philippe Morency 13 11 0 22 Oct 2020
DialogueTRM: Exploring the Intra- and Inter-Modal Emotional Behaviors in the Conversation Yuzhao Mao Qi Sun Guang Liu Xiaojie Wang Weiguo Gao Xuan Li Jianping Shen 9 17 0 15 Oct 2020
Modality-Transferable Emotion Embeddings for Low-Resource Multimodal Emotion Recognition Wenliang Dai Zihan Liu Tiezheng Yu Pascale Fung 13 29 0 21 Sep 2020
Collaborative Attention Mechanism for Multi-View Action Recognition Yue Bai Zhiqiang Tao Lichen Wang Sheng R. Li Yu Yin Y. Fu HAI 6 12 0 14 Sep 2020
Towards Debiasing Sentence Representations Paul Pu Liang Irene Z Li Emily Zheng Y. Lim Ruslan Salakhutdinov Louis-Philippe Morency 6 198 0 16 Jul 2020
A Dataset and Benchmarks for Multimedia Social Analysis Bofan Xue David M. Chan John F. Canny VGen 11 0 0 05 Jun 2020
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis Devamanyu Hazarika Roger Zimmermann Soujanya Poria 6 484 0 07 May 2020
On Emergent Communication in Competitive Multi-Agent Teams Paul Pu Liang Jeffrey Chen Ruslan Salakhutdinov Louis-Philippe Morency Satwik Kottur LLMAG 12 13 0 04 Mar 2020