Learn to Combine Modalities in Multimodal Deep Learning

29 May 2018

Papers citing "Learn to Combine Modalities in Multimodal Deep Learning"

47 / 47 papers shown

Title
TACFN: Transformer-based Adaptive Cross-modal Fusion Network for Multimodal Emotion Recognition Feng Liu Ziwang Fu Yansen Wang Qijian Zheng 42 4 0 10 May 2025
Summarization of Multimodal Presentations with Vision-Language Models: Study of the Effect of Modalities and Structure Théo Gigant Camille Guinaudeau Frédéric Dufaux 44 0 0 14 Apr 2025
Multi-Task Adversarial Variational Autoencoder for Estimating Biological Brain Age with Multimodal Neuroimaging Muhammad Usman Azka Rehman Abdullah Shahid A. Rehman Sung-Min Gho Aleum Lee Tariq Mahmood Khan Imran Razzak 35 0 0 15 Nov 2024
Camera Model Identification Using Audio and Visual Content from Videos Ioannis Tsingalis Christos Korgialas C. Kotropoulos 11 3 0 25 Jun 2024
Sequence-to-Sequence Multi-Modal Speech In-Painting Mahsa Kadkhodaei Elyaderani S. Shirani 30 1 0 03 Jun 2024
Robust Multi-Modal Speech In-Painting: A Sequence-to-Sequence Approach Mahsa Kadkhodaei Elyaderani Shahram Shirani 43 0 0 02 Jun 2024
3D object quality prediction for Metal Jet Printer with Multimodal thermal encoder R. Chen Chen Wenjia Zheng Sandeep Jalui Pavan Suri Jun Zeng AI4CE 23 0 0 17 Apr 2024
UniCat: Crafting a Stronger Fusion Baseline for Multimodal Re-Identification Jennifer Crawford Haoli Yin Luke McDermott Daniel Cummings 33 13 0 28 Oct 2023
Deep Metric Loss for Multimodal Learning Sehwan Moon Hyun-Yong Lee 24 0 0 21 Aug 2023
Employing Multimodal Machine Learning for Stress Detection Rahee Walambe Pranav Nayak Ashmit Bhardwaj K. Kotecha 14 48 0 15 Jun 2023
Shape-Net: Room Layout Estimation from Panoramic Images Robust to Occlusion using Knowledge Distillation with 3D Shapes as Additional Inputs M. Tabata Kana Kurata Junichiro Tamamatsu 3DV 3DPC 27 4 0 25 Apr 2023
On Robustness in Multimodal Learning Brandon McKinzie Joseph Cheng Vaishaal Shankar Yinfei Yang Jonathon Shlens Alexander Toshev 45 2 0 10 Apr 2023
Class Based Thresholding in Early Exit Semantic Segmentation Networks Alperen Görmez Erdem Koyuncu 28 5 0 27 Oct 2022
Classification of Quasars, Galaxies, and Stars in the Mapping of the Universe Multi-modal Deep Learning Sabeesh Ethiraj B. Bolla 16 2 0 22 May 2022
SceneTrilogy: On Human Scene-Sketch and its Complementarity with Photo and Text Pinaki Nath Chowdhury A. Bhunia Aneeshan Sain Subhadeep Koley Tao Xiang Yi-Zhe Song 43 29 0 25 Apr 2022
Dynamic Multimodal Fusion Zihui Xue R. Marculescu 51 48 0 31 Mar 2022
Graph Neural Networks in IoT: A Survey Guimin Dong Mingyue Tang Zhiyuan Wang Jiechao Gao Sikun Guo Lihua Cai Robert Gutierrez Brad Campbell Laura E. Barnes M. Boukhechba GNN AI4CE 47 97 0 29 Mar 2022
Interpretable Prediction of Pulmonary Hypertension in Newborns using Echocardiograms H. Ragnarsdóttir Laura Manduchi H. Michel F. Laumer S. Wellmann Ece Ozkan Julia-Franziska Vogt 21 3 0 24 Mar 2022
Modality Competition: What Makes Joint Training of Multi-modal Network Fail in Deep Learning? (Provably) Yu Huang Junyang Lin Chang Zhou Hongxia Yang Longbo Huang 21 91 0 23 Mar 2022
FS-COCO: Towards Understanding of Freehand Sketches of Common Objects in Context Pinaki Nath Chowdhury Aneeshan Sain A. Bhunia Tao Xiang Yulia Gryaditskaya Yi-Zhe Song 3DV 48 52 0 04 Mar 2022
Characterizing and overcoming the greedy nature of learning in multi-modal deep neural networks Nan Wu Stanislaw Jastrzebski Kyunghyun Cho Krzysztof J. Geras 21 72 0 10 Feb 2022
MIXER: A Principled Framework for Multimodal, Multiway Data Association Parker C. Lusk Ronak Roy Kaveh Fathian Jonathan P. How 21 0 0 29 Nov 2021
A cross-modal fusion network based on self-attention and residual structure for multimodal emotion recognition Ziwang Fu Feng Liu Hanyang Wang Jiayin Qi Xiangling Fu Aimin Zhou Zhibin Li 33 30 0 03 Nov 2021
Event and Activity Recognition in Video Surveillance for Cyber-Physical Systems Swarnabja Bhaumik Prithwish Jana Partha Pratim Mohanta 9 4 0 03 Nov 2021
A Survey on Multi-modal Summarization Anubhav Jangra Sourajit Mukherjee Adam Jatowt S. Saha M. Hasanuzzaman 33 59 0 11 Sep 2021
Data Fusion for Deep Learning on Transport Mode Detection: A Case Study Hugues Moreau A. Vassilev Liming Chen 16 2 0 31 May 2021
A Review on Explainability in Multimodal Deep Neural Nets Gargi Joshi Rahee Walambe K. Kotecha 34 140 0 17 May 2021
What is the appropriate speed for an autonomous vehicle? Designing a Pedestrian Aware Contextual Speed Controller D. Jiang Stewart Worrall Mao Shan 13 2 0 13 Apr 2021
MSAF: Multimodal Split Attention Fusion Lang Su Chuqing Hu Guofa Li Dongpu Cao 32 37 0 13 Dec 2020
Trajformer: Trajectory Prediction with Local Self-Attentive Contexts for Autonomous Driving Manoj Bhat Jonathan M Francis Jean Oh 35 22 0 30 Nov 2020
Multi-modal Summarization for Video-containing Documents Xiyan Fu Jun Wang Zhenglu Yang 28 23 0 17 Sep 2020
Towards Robust Pattern Recognition: A Review Xu-Yao Zhang Cheng-Lin Liu C. Suen OOD HAI 26 103 0 12 Jun 2020
A Deep Learning-based Radar and Camera Sensor Fusion Architecture for Object Detection Felix Nobis Maximilian Geisslinger Markus Weber Johannes Betz Markus Lienkamp 22 257 0 15 May 2020
Multimodal Categorization of Crisis Events in Social Media Mahdi Abavisani Liwei Wu Shengli Hu Joel R. Tetreault A. Jaimes 34 87 0 10 Apr 2020
From Generalized zero-shot learning to long-tail with class descriptors Dvir Samuel Yuval Atzmon Gal Chechik VLM 32 1 0 05 Apr 2020
Deep Multimodal Feature Encoding for Video Ordering Vivek Sharma Makarand Tapaswi Rainer Stiefelhagen 36 10 0 05 Apr 2020
Multimodal Material Classification for Robots using Spectroscopy and High Resolution Texture Imaging Zackory M. Erickson Eliot Xing Bharat Srirangam Sonia Chernova Charles C. Kemp 26 38 0 02 Apr 2020
EmotiCon: Context-Aware Multimodal Emotion Recognition using Frege's Principle Trisha Mittal P. Guhan Uttaran Bhattacharya Rohan Chandra Aniket Bera Tianyi Zhou 54 132 0 14 Mar 2020
A multimodal deep learning approach for named entity recognition from social media M. Asgari-Chenaghlu M. Feizi-Derakhshi Leili Farzinvash M. Balafar C. Motamed 19 28 0 19 Jan 2020
Multimodal Generative Models for Compositional Representation Learning Mike Wu Noah D. Goodman GAN DRL 43 17 0 11 Dec 2019
Modal-aware Features for Multimodal Hashing Haien Zeng Hanjiang Lai Hanlu Chu Yong Tang Jian Yin 20 0 0 19 Nov 2019
M3ER: Multiplicative Multimodal Emotion Recognition Using Facial, Textual, and Speech Cues Trisha Mittal Uttaran Bhattacharya Rohan Chandra Aniket Bera Tianyi Zhou 22 236 0 09 Nov 2019
Watch, Listen and Tell: Multi-modal Weakly Supervised Dense Event Captioning Tanzila Rahman Bicheng Xu Leonid Sigal 30 78 0 22 Sep 2019
Cross-view Relation Networks for Mammogram Mass Detection Jiechao Ma Sen Liang Xiang Li Hongwei Bran Li Bjoern H. Menze Rongguo Zhang Weishi Zheng 16 30 0 01 Jul 2019
Collaborative Layer-wise Discriminative Learning in Deep Neural Networks Xiaojie Jin Yunpeng Chen Jian Dong Jiashi Feng Shuicheng Yan 28 21 0 19 Jul 2016
EmoNets: Multimodal deep learning approaches for emotion recognition in video Samira Ebrahimi Kahou Xavier Bouthillier Pascal Lamblin Çağlar Gülçehre Vincent Michalski ... Aaron Courville Pascal Vincent Roland Memisevic C. Pal Yoshua Bengio 140 401 0 05 Mar 2015
Convolutional Neural Networks for Sentence Classification Yoon Kim AILaw VLM 312 13,377 0 25 Aug 2014