Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems

2 July 2019

Nancy F. Chen

Papers citing "Multimodal Transformer Networks for End-to-End Video-Grounded Dialogue Systems"

23 / 23 papers shown

Title
OneLLM: One Framework to Align All Modalities with Language Jiaming Han Kaixiong Gong Yiyuan Zhang Jiaqi Wang Kaipeng Zhang D. Lin Yu Qiao Peng Gao Xiangyu Yue MLLM 104 109 0 10 Jan 2025
Multimodal Cross-Document Event Coreference Resolution Using Linear Semantic Transfer and Mixed-Modality Ensembles Abhijnan Nath Huma Jamil Shafiuddin Rehan Ahmed George Baker Rahul Ghosh James H. Martin Nathaniel Blanchard Nikhil Krishnaswamy 32 2 0 13 Apr 2024
OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialog Adnen Abdessaied Manuel von Hochmeister Andreas Bulling 35 2 0 20 Feb 2024
HEAR: Hearing Enhanced Audio Response for Video-grounded Dialogue Sunjae Yoon Dahyun Kim Eunseop Yoon Hee Suk Yoon Junyeong Kim C. Yoo 37 6 0 15 Dec 2023
Uncovering Hidden Connections: Iterative Search and Reasoning for Video-grounded Dialog Haoyu Zhang Meng Liu Yaowei Wang Da Cao Weili Guan Liqiang Nie 28 0 0 11 Oct 2023
A Unified Framework for Slot based Response Generation in a Multimodal Dialogue System Mauajama Firdaus Avinash Madasu Asif Ekbal 33 7 0 27 May 2023
SPRING: Situated Conversation Agent Pretrained with Multimodal Questions from Incremental Layout Graph Yuxing Long Binyuan Hui Fulong Ye Yanyang Li Zhuoxin Han Caixia Yuan Yongbin Li Xiaojie Wang LLMAG 20 7 0 05 Jan 2023
Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Review Hao Wang Bin Guo Y. Zeng Yasan Ding Chen Qiu Ying Zhang Li Yao Zhiwen Yu 27 2 0 02 Jul 2022
The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training Gi-Cheon Kang Sungdong Kim Jin-Hwa Kim Donghyun Kwak Byoung-Tak Zhang 16 10 0 25 May 2022
Learning to Retrieve Videos by Asking Questions Avinash Madasu Junier Oliva Gedas Bertasius VGen 30 16 0 11 May 2022
Serving and Optimizing Machine Learning Workflows on Heterogeneous Infrastructures Yongji Wu Matthew Lentz Danyang Zhuo Yao Lu 21 22 0 10 May 2022
Multimodal Interactions Using Pretrained Unimodal Models for SIMMC 2.0 Joosung Lee Kijong Han 31 6 0 10 Dec 2021
FLAVA: A Foundational Language And Vision Alignment Model Amanpreet Singh Ronghang Hu Vedanuj Goswami Guillaume Couairon Wojciech Galuba Marcus Rohrbach Douwe Kiela CLIP VLM 38 686 0 08 Dec 2021
Interventional Video Grounding with Dual Contrastive Learning Guoshun Nan Rui Qiao Yao Xiao Jun Liu Sicong Leng H. Zhang Wei Lu 16 144 0 21 Jun 2021
Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey Jinjie Ni Tom Young Vlad Pandelea Fuzhao Xue Erik Cambria 52 267 0 10 May 2021
Multiscale Vision Transformers Haoqi Fan Bo Xiong K. Mangalam Yanghao Li Zhicheng Yan Jitendra Malik Christoph Feichtenhofer ViT 19 1,219 0 22 Apr 2021
Structured Co-reference Graph Attention for Video-grounded Dialogue Junyeong Kim Sunjae Yoon Dahyun Kim Chang-Dong Yoo 18 26 0 24 Mar 2021
Learning Reasoning Paths over Semantic Graphs for Video-grounded Dialogues Hung Le Nancy F. Chen S. Hoi 31 14 0 01 Mar 2021
Is Space-Time Attention All You Need for Video Understanding? Gedas Bertasius Heng Wang Lorenzo Torresani ViT 280 1,981 0 09 Feb 2021
TMT: A Transformer-based Modal Translator for Improving Multimodal Sequence Representations in Audio Visual Scene-aware Dialog Wubo Li Dongwei Jiang Wei Zou Xiangang Li 18 6 0 21 Oct 2020
Dynamic Graph Representation Learning for Video Dialog via Multi-Modal Shuffled Transformers Shijie Geng Peng Gao Moitreya Chatterjee Chiori Hori Jonathan Le Roux Yongfeng Zhang Hongsheng Li A. Cherian 19 11 0 08 Jul 2020
Multiresolution and Multimodal Speech Recognition with Transformers Georgios Paraskevopoulos Srinivas Parthasarathy Aparna Khare Shiva Sundaram 18 29 0 29 Apr 2020
Span-based Localizing Network for Natural Language Video Localization Hao Zhang Aixin Sun Wei Jing Joey Tianyi Zhou 15 311 0 29 Apr 2020