A Comprehensive Review of Data-Driven Co-Speech Gesture Generation

13 January 2023

Papers citing "A Comprehensive Review of Data-Driven Co-Speech Gesture Generation"

50 / 53 papers shown

Title
SARGes: Semantically Aligned Reliable Gesture Generation via Intent Chain Nan Gao Yihua Bao Dongdong Weng Jiayi Zhao Jia Li Yan Zhou Pengfei Wan Di Zhang SLR 96 0 0 26 Mar 2025
Large Language Models for Virtual Human Gesture Selection P. Torshizi Laura Birka Hensel Ari Shapiro Stacy Marsella SLR 66 0 0 18 Mar 2025
Cosh-DiT: Co-Speech Gesture Video Synthesis via Hybrid Audio-Visual Diffusion Transformers Yasheng Sun Zhiliang Xu Hang Zhou Jiazhi Guan Quanwei Yang ... Yingying Li Haocheng Feng J. Wang Ziwei Liu Koike Hideki VGen 54 0 0 13 Mar 2025
Streaming Generation of Co-Speech Gestures via Accelerated Rolling Diffusion Evgeniia Vu Andrei Boiarov Dmitry Vetrov VGen 48 0 0 13 Mar 2025
ExGes: Expressive Human Motion Retrieval and Modulation for Audio-Driven Gesture Synthesis Xukun Zhou Fengxin Li Ming Chen Yan Zhou Pengfei Wan Di Zhang Yeying Jin Zhaoxin Fan Hongyan Liu Jun He DiffM VGen 43 0 0 09 Mar 2025
HOP: Heterogeneous Topology-based Multimodal Entanglement for Co-Speech Gesture Generation Hongye Cheng Tianyu Wang Guangsi Shi Zexing Zhao Yanwei Fu SLR 38 0 0 03 Mar 2025
Human-like Nonverbal Behavior with MetaHumans in Real-World Interaction Studies: An Architecture Using Generative Methods and Motion Capture Oliver Chojnowski Alexander Eberhard Michael Schiffmann Ana Müller Anja Richert AI4CE 26 0 0 18 Jan 2025
A Review of Human Emotion Synthesis Based on Generative Technology Fei Ma Y. Li Yifan Xie Y. He Y. Zhang ... Z. Liu Wei Yao Fuji Ren Fei Richard Yu Shiguang Ni 76 0 0 10 Dec 2024
Large Body Language Models Saif Punjwani Larry Heck 18 0 0 21 Oct 2024
Mitigation of gender bias in automatic facial non-verbal behaviors generation Alice Delbosc M. Ochs Nicolas Sabouret Brian Ravenet Stéphane Ayache 21 0 0 09 Oct 2024
Towards a GENEA Leaderboard -- an Extended, Living Benchmark for Evaluating and Advancing Conversational Motion Synthesis Rajmund Nagy Hendric Voss Youngwoo Yoon Taras Kucherenko Teodor Nikolov Thanh Hoang-Minh R. Mcdonnell Stefan Kopp Michael Neff G. Henter 19 1 0 08 Oct 2024
LLM Gesticulator: Leveraging Large Language Models for Scalable and Controllable Co-Speech Gesture Synthesis Haozhou Pang Tianwei Ding Lanshan He Ming Tao Lu Zhang Qi Gan 18 1 0 06 Oct 2024
TANGO: Co-Speech Gesture Video Reenactment with Hierarchical Audio Motion Embedding and Diffusion Interpolation Haiyang Liu Xingchao Yang Tomoya Akiyama Yuantian Huang Qiaoge Li Shigeru Kuriyama Takafumi Taketomi VGen SLR 19 7 0 05 Oct 2024
2D or not 2D: How Does the Dimensionality of Gesture Representation Affect 3D Co-Speech Gesture Generation? Teo Guichoux Laure Soulier Nicolas Obin Catherine Pelachaud SLR 24 0 0 16 Sep 2024
Gesture Generation from Trimodal Context for Humanoid Robots Shiyi Tang Christian Dondrup 16 0 0 08 Sep 2024
Learning Co-Speech Gesture Representations in Dialogue through Contrastive Learning: An Intrinsic Evaluation E. Ghaleb Bulat Khaertdinov Wim Pouw Marlou Rasenberg Judith Holler Aslı Özyürek Raquel Fernández SSL 18 1 0 31 Aug 2024
MambaGesture: Enhancing Co-Speech Gesture Generation with Mamba and Disentangled Multi-Modality Fusion Chencan Fu Yabiao Wang Jiangning Zhang Zhengkai Jiang Xiaofeng Mao Jiafu Wu Weijian Cao Chengjie Wang Yanhao Ge Yong Liu Mamba 35 2 0 29 Jul 2024
The Effects of Embodiment and Personality Expression on Learning in LLM-based Educational Agents Sinan Sonlu Bennie Bendiksen Funda Durupinar U. Güdükbay 28 7 0 24 Jun 2024
SIGGesture: Generalized Co-Speech Gesture Synthesis via Semantic Injection with Large-Scale Pre-Training Diffusion Models Qingrong Cheng Xu Li Xinghui Fu DiffM 27 2 0 22 May 2024
LLAniMAtion: LLAMA Driven Gesture Animation John T. Windle Iain Matthews Sarah Taylor 36 0 0 13 May 2024
Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis Shivam Mehta Anna Deichler Jim O'Regan Birger Moëll Jonas Beskow G. Henter Simon Alexanderson 31 4 0 30 Apr 2024
Leveraging Speech for Gesture Detection in Multimodal Communication E. Ghaleb I. Burenko Marlou Rasenberg Wim Pouw Ivan Toni Peter Uhrig Anna Wilson Judith Holler Asli Ozyurek Raquel Fernández SLR 16 4 0 23 Apr 2024
A Unified Editing Method for Co-Speech Gesture Generation via Diffusion Inversion Zeyu Zhao Nan Gao Zhi Zeng Guixuan Zhang Jie Liu Shuwu Zhang DiffM 29 0 0 03 Apr 2024
ConvoFusion: Multi-Modal Conversational Diffusion for Co-Speech Gesture Synthesis Muhammad Hamza Mughal Rishabh Dabral I. Habibie Lucia Donatelli Marc Habermann Christian Theobalt SLR 28 14 0 26 Mar 2024
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models Zunnan Xu Yukang Lin Haonan Han Sicheng Yang Ronghui Li Yachao Zhang Xiu Li Mamba 46 24 0 14 Mar 2024
Freetalker: Controllable Speech and Text-Driven Gesture Generation Based on Diffusion Models for Enhanced Speaker Naturalness Sicheng Yang Zunnan Xu Haiwei Xue Yongkang Cheng Shaoli Huang Mingming Gong Zhiyong Wu DiffM VGen 22 11 0 07 Jan 2024
Chain of Generation: Multi-Modal Gesture Synthesis via Cascaded Conditional Control Zunnan Xu Yachao Zhang Sicheng Yang Ronghui Li Xiu Li SLR 16 6 0 26 Dec 2023
Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion Kiran Chhatre Radek Danvevcek Nikos Athanasiou Giorgio Becherini Christopher Peters Michael J. Black Timo Bolkart DiffM 25 14 0 07 Dec 2023
Unified speech and gesture synthesis using flow matching Shivam Mehta Ruibo Tu Simon Alexanderson Jonas Beskow Éva Székely G. Henter 17 3 0 08 Oct 2023
Large language models in textual analysis for gesture selection Laura Birka Hensel Nutchanon Yongsatianchot P. Torshizi E. Minucci Stacy Marsella SLR 19 3 0 04 Oct 2023
Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent Alice Delbosc M. Ochs Nicolas Sabouret Brian Ravenet Stéphane Ayache 19 7 0 15 Sep 2023
UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons Sicheng Yang Z. Wang Zhiyong Wu Minglei Li Zhensong Zhang ... Lei Hao Songcen Xu Xiaofei Wu Changpeng Yang Zonghong Dai DiffM 26 14 0 13 Sep 2023
Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation Anna Deichler Shivam Mehta Simon Alexanderson Jonas Beskow DiffM 8 23 0 11 Sep 2023
C2G2: Controllable Co-speech Gesture Generation with Latent Diffusion Model Longbin Ji Pengfei Wei Yi Ren Jinglin Liu Chen Zhang Xiang Yin DiffM 21 3 0 29 Aug 2023
The DiffuseStyleGesture+ entry to the GENEA Challenge 2023 Sicheng Yang Haiwei Xue Zhensong Zhang Minglei Li Zhiyong Wu Xiaofei Wu Songcen Xu Zonghong Dai DiffM 16 15 0 26 Aug 2023
The GENEA Challenge 2023: A large scale evaluation of gesture generation models in monadic and dyadic settings Taras Kucherenko Rajmund Nagy Youngwoo Yoon Jieyeon Woo Teodor Nikolov Mihail Tsakov G. Henter VGen 16 40 0 24 Aug 2023
A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation Li Liu Lufei Gao Wen-Ling Lei Fengji Ma Xiaotian Lin Jin-Tao Wang CVBM 16 1 0 17 Aug 2023
Audio is all in one: speech-driven gesture synthetics using WavLM pre-trained model Fan Zhang Naye Ji Fuxing Gao Siyuan Zhao Zhaohan Wang Shunman Li 19 0 0 11 Aug 2023
Augmented Co-Speech Gesture Generation: Including Form and Meaning Features to Guide Learning-Based Gesture Synthesis Hendric Voss S. Kopp SLR 45 4 0 13 Jul 2023
Diff-TTSG: Denoising probabilistic integrated speech and gesture synthesis Shivam Mehta Siyang Wang Simon Alexanderson Jonas Beskow Éva Székely G. Henter DiffM 11 14 0 15 Jun 2023
QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generation Sicheng Yang Zhiyong Wu Minglei Li Zhensong Zhang Lei Hao Weihong Bao Hao-Wen Zhuang SLR 11 40 0 18 May 2023
DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Models Sicheng Yang Zhiyong Wu Minglei Li Zhensong Zhang Lei Hao Weihong Bao Ming Cheng Long Xiao 11 64 0 08 May 2023
AQ-GT: a Temporally Aligned and Quantized GRU-Transformer for Co-Speech Gesture Synthesis Hendric Voss S. Kopp SLR 36 6 0 02 May 2023
GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents Tenglong Ao Zeyi Zhang Libin Liu DiffM VGen 65 143 0 26 Mar 2023
GesGPT: Speech Gesture Synthesis With Text Parsing from ChatGPT Nan Gao Zeyu Zhao Zhi Zeng Shuwu Zhang Dongdong Weng Yihua Bao 32 8 0 23 Mar 2023
Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022 Taras Kucherenko Pieter Wolfert Youngwoo Yoon Carla Viegas Teodor Nikolov Mihail Tsakov G. Henter 30 24 0 15 Mar 2023
Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models Simon Alexanderson Rajmund Nagy Jonas Beskow G. Henter DiffM VGen 22 164 0 17 Nov 2022
Human Motion Diffusion Model Guy Tevet Sigal Raab Brian Gordon Yonatan Shafir Daniel Cohen-Or Amit H. Bermano DiffM VGen 177 713 0 29 Sep 2022
Text/Speech-Driven Full-Body Animation Wenlin Zhuang Jinwei Qi Peng Zhang Bang Zhang Ping Tan 25 6 0 31 May 2022
Multimodal analysis of the predictability of hand-gesture properties Taras Kucherenko Rajmund Nagy Michael Neff Hedvig Kjellström G. Henter 20 22 0 12 Aug 2021