v1v2 (latest)

Let's Face It: Probabilistic Multi-modal Interlocutor-aware Generation of Facial Gestures in Dyadic Settings

11 June 2020

Papers citing "Let's Face It: Probabilistic Multi-modal Interlocutor-aware Generation of Facial Gestures in Dyadic Settings"

50 / 61 papers shown

Title
DiTaiListener: Controllable High Fidelity Listener Video Generation with Diffusion Maksim Siniukov Di Chang Minh Tran Hongkun Gong Ashutosh Chaubey Mohammad Soleymani DiffM VGen 154 1 0 05 Apr 2025
MAG: Multi-Modal Aligned Autoregressive Co-Speech Gesture Generation without Vector Quantization Binjie Liu Lina Liu Sanyi Zhang Songen Gu Yihao Zhi Tianyi Zhu Lei Yang Long Ye SLR 142 0 0 18 Mar 2025
Allo-AVA: A Large-Scale Multimodal Conversational AI Dataset for Allocentric Avatar Gesture Animation Saif Punjwani Larry Heck SLR VGen 88 1 0 21 Oct 2024
Towards a GENEA Leaderboard -- an Extended, Living Benchmark for Evaluating and Advancing Conversational Motion Synthesis Rajmund Nagy Hendric Voss Youngwoo Yoon Taras Kucherenko Teodor Nikolov Thanh Hoang-Minh R. Mcdonnell Stefan Kopp Michael Neff G. Henter 97 2 0 08 Oct 2024
How Video Meetings Change Your Expression Sumit Sarin Utkarsh Mall Purva Tendulkar Carl Vondrick CVBM 131 0 0 03 Jun 2024
Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis Shivam Mehta Anna Deichler Jim O'Regan Birger Moëll Jonas Beskow G. Henter Simon Alexanderson 124 6 0 30 Apr 2024
Beyond Talking -- Generating Holistic 3D Human Dyadic Motion for Communication Mingze Sun Chao Xu Xinyu Jiang Yang Liu Baigui Sun Ruqi Huang 113 7 0 28 Mar 2024
Dyadic Interaction Modeling for Social Behavior Generation Minh Tran Di Chang Maksim Siniukov Mohammad Soleymani VGen 144 13 0 14 Mar 2024
ReNeLiB: Real-time Neural Listening Behavior Generation for Socially Interactive Agents Daksitha Senel Withanage Don Philipp Müller Fabrizio Nunnari Elisabeth André Patrick Gebhard 156 2 0 12 Feb 2024
Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance Qingcheng Zhao Pengyu Long Qixuan Zhang Dafei Qin Hanming Liang Longwen Zhang Yingliang Zhang Jingyi Yu Lan Xu DiffM 3DH 139 39 0 28 Jan 2024
Unified speech and gesture synthesis using flow matching Shivam Mehta Ruibo Tu Simon Alexanderson Jonas Beskow Éva Székely G. Henter 140 3 0 08 Oct 2023
Emotional Listener Portrait: Neural Listener Head Generation with Emotion Luchuan Song Guojun Yin Zhenchao Jin Xiaoyi Dong Chenliang Xu 130 16 0 29 Sep 2023
FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion Stefan Stan Kazi Injamamul Haque Zerrin Yumak DiffM 133 70 0 20 Sep 2023
Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent Alice Delbosc M. Ochs Nicolas Sabouret Brian Ravenet Stéphane Ayache 131 9 0 15 Sep 2023
Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation Anna Deichler Shivam Mehta Simon Alexanderson Jonas Beskow DiffM 102 26 0 11 Sep 2023
The GENEA Challenge 2023: A large scale evaluation of gesture generation models in monadic and dyadic settings Taras Kucherenko Rajmund Nagy Youngwoo Yoon Jieyeon Woo Teodor Nikolov Mihail Tsakov G. Henter VGen 101 48 0 24 Aug 2023
Towards an architectural framework for intelligent virtual agents using probabilistic programming A. Andreev G. Cattan 55 0 0 20 Jul 2023
The Ethical Implications of Generative Audio Models: A Systematic Literature Review J. Barnett 123 40 0 07 Jul 2023
MRecGen: Multimodal Appropriate Reaction Generator Jiaqi Xu Cheng Luo Weicheng Xie Linlin Shen Xiaofeng Liu Lu Liu Hatice Gunes Siyang Song VGen 63 3 0 05 Jul 2023
Diff-TTSG: Denoising probabilistic integrated speech and gesture synthesis Shivam Mehta Siyang Wang Simon Alexanderson Jonas Beskow Éva Székely G. Henter DiffM 143 14 0 15 Jun 2023
REACT2023: the first Multi-modal Multiple Appropriate Facial Reaction Generation Challenge Siyang Song Micol Spitale Cheng Luo German Barquero Cristina Palmero ... Michel Valstar Tobias Baur Fabien Ringeval Elisabeth Andre Hatice Gunes 118 11 0 11 Jun 2023
Reversible Graph Neural Network-based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation Tong Xu Micol Spitale Haozhan Tang Lu Liu Hatice Gunes Siyang Song CVBM 138 12 0 24 May 2023
ZS-MSTM: Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding Mireille Fares Catherine Pelachaud Nicolas Obin 79 0 0 22 May 2023
AMII: Adaptive Multimodal Inter-personal and Intra-personal Model for Adapted Behavior Synthesis Jieyeon Woo Mireille Fares Catherine Pelachaud Catherine Achard LLMAG 107 5 0 18 May 2023
Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022 Taras Kucherenko Pieter Wolfert Youngwoo Yoon Carla Viegas Teodor Nikolov Mihail Tsakov G. Henter 103 26 0 15 Mar 2023
FaceXHuBERT: Text-less Speech-driven E(X)pressive 3D Facial Animation Synthesis Using Self-Supervised Speech Representation Learning Kazi Injamamul Haque Zerrin Yumak 87 33 0 09 Mar 2023
Affective Faces for Goal-Driven Dyadic Communication Scott Geng Revant Teotia Purva Tendulkar Sachit Menon Carl Vondrick VGen 79 27 0 26 Jan 2023
A Comprehensive Review of Data-Driven Co-Speech Gesture Generation Simbarashe Nyatsanga Taras Kucherenko Chaitanya Ahuja G. Henter Michael Neff SLR 194 104 0 13 Jan 2023
Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models Simon Alexanderson Rajmund Nagy Jonas Beskow G. Henter DiffM VGen 145 194 0 17 Nov 2022
The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation Youngwoo Yoon Pieter Wolfert Taras Kucherenko Carla Viegas Teodor Nikolov Mihail Tsakov G. Henter VGen 107 86 0 22 Aug 2022
Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding Mireille Fares Michele Grimaldi Catherine Pelachaud Nicolas Obin 114 18 0 03 Aug 2022
A Probabilistic Model Of Interaction Dynamics for Dyadic Face-to-Face Settings Renke Wang Ifeoma Nwogu CVBM 80 0 0 10 Jul 2022
Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motion Evonne Ng Hanbyul Joo Liwen Hu Hao Li Trevor Darrell Angjoo Kanazawa Shiry Ginosar VGen 98 107 0 18 Apr 2022
Mechanical Chameleons: Evaluating the effects of a social robot's non-verbal behavior on social influence Patrik Jonell Anna Deichler Ilaria Torre Iolanda Leite Jonas Beskow 58 1 0 02 Sep 2021
A large, crowdsourced evaluation of gesture generation systems on common data: The GENEA Challenge 2020 Taras Kucherenko Patrik Jonell Youngwoo Yoon Pieter Wolfert G. Henter 95 79 0 23 Feb 2021
HEMVIP: Human Evaluation of Multiple Videos in Parallel Patrik Jonell Youngwoo Yoon Pieter Wolfert Taras Kucherenko G. Henter 151 21 0 28 Jan 2021
Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation Taras Kucherenko Dai Hasegawa Naoshi Kaneko G. Henter Hedvig Kjellström 151 43 0 16 Jul 2020
Normalizing Flows for Probabilistic Modeling and Inference George Papamakarios Eric T. Nalisnick Danilo Jimenez Rezende S. Mohamed Balaji Lakshminarayanan TPM AI4CE 395 1,823 0 05 Dec 2019
PyTorch: An Imperative Style, High-Performance Deep Learning Library Adam Paszke Sam Gross Francisco Massa Adam Lerer James Bradbury ... Sasank Chilamkurthy Benoit Steiner Lu Fang Junjie Bai Soumith Chintala ODL 772 44,568 0 03 Dec 2019
To React or not to React: End-to-End Visual Pose Forecasting for Personalized Avatar during Dyadic Conversations Chaitanya Ahuja Shugao Ma Louis-Philippe Morency Yaser Sheikh 92 59 0 05 Oct 2019
Optuna: A Next-generation Hyperparameter Optimization Framework Takuya Akiba Shotaro Sano Toshihiko Yanase Takeru Ohta Masanori Koyama 713 6,608 0 25 Jul 2019
Learning to Regress 3D Face Shape and Expression from an Image without 3D Supervision Soubhik Sanyal Timo Bolkart Haiwen Feng Michael J. Black 3DH CVBM 96 303 0 16 May 2019
MoGlow: Probabilistic and controllable motion synthesis using normalising flows G. Henter Simon Alexanderson Jonas Beskow 149 102 0 16 May 2019
Speaker-Independent Speech-Driven Visual Speech Synthesis using Domain-Adapted Acoustic Models Ahmed Hussen Abdelaziz B. Theobald Justin Binder Gabriele Fanelli Paul Dixon N. Apostoloff T. Weise Sachin Kajareker 82 11 0 15 May 2019
Capture, Learning, and Synthesis of 3D Speaking Styles Daniel Cudeiro Timo Bolkart Cassidy Laidlaw Anurag Ranjan Michael J. Black CVBM 3DH 159 364 0 08 May 2019
Analyzing Input and Output Representations for Speech-Driven Gesture Generation Taras Kucherenko Dai Hasegawa G. Henter Naoshi Kaneko Hedvig Kjellström 139 158 0 08 Mar 2019
A Face-to-Face Neural Conversation Model Hang Chu Daiqing Li Sanja Fidler 3DH CVBM 80 22 0 04 Dec 2018
Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots Youngwoo Yoon Woo-Ri Ko Minsu Jang Jaeyeon Lee Jaehong Kim Geehyuk Lee SLR 92 239 0 30 Oct 2018
Glow: Generative Flow with Invertible 1x1 Convolutions Diederik P. Kingma Prafulla Dhariwal BDL DRL 480 3,229 0 09 Jul 2018
Interactive Generative Adversarial Networks for Facial Expression Generation in Dyadic Interactions Behnaz Nojavanasghari Yuchi Huang Saad M. Khan CVBM GAN 96 27 0 27 Jan 2018