Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models

17 November 2022

Papers citing "Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models"

30 / 130 papers shown

Title
FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion Stefan Stan Kazi Injamamul Haque Zerrin Yumak DiffM 24 53 0 20 Sep 2023
UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletons Sicheng Yang Z. Wang Zhiyong Wu Minglei Li Zhensong Zhang ... Lei Hao Songcen Xu Xiaofei Wu Changpeng Yang Zonghong Dai DiffM 23 14 0 13 Sep 2023
Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation Anna Deichler Shivam Mehta Simon Alexanderson Jonas Beskow DiffM 8 23 0 11 Sep 2023
ImitationNet: Unsupervised Human-to-Robot Motion Retargeting via Shared Latent Space Yashuai Yan Esteve Valls Mascaro Dongheui Lee 17 13 0 11 Sep 2023
Matcha-TTS: A fast TTS architecture with conditional flow matching Shivam Mehta Ruibo Tu Jonas Beskow Éva Székely G. Henter 4 68 0 06 Sep 2023
MDSC: Towards Evaluating the Style Consistency Between Music and Dance Zixiang Zhou Weiyuan Li Baoyuan Wang 16 1 0 04 Sep 2023
The GENEA Challenge 2023: A large scale evaluation of gesture generation models in monadic and dyadic settings Taras Kucherenko Rajmund Nagy Youngwoo Yoon Jieyeon Woo Teodor Nikolov Mihail Tsakov G. Henter VGen 14 40 0 24 Aug 2023
Audio is all in one: speech-driven gesture synthetics using WavLM pre-trained model Fan Zhang Naye Ji Fuxing Gao Siyuan Zhao Zhaohan Wang Shunman Li 16 0 0 11 Aug 2023
Human Motion Generation: A Survey Wentao Zhu Xiaoxuan Ma Dongwoo Ro Hai Ci Jinlu Zhang Jiaxin Shi Feng Gao Qi Tian Yizhou Wang VGen 19 50 0 20 Jul 2023
Diff-TTSG: Denoising probabilistic integrated speech and gesture synthesis Shivam Mehta Siyang Wang Simon Alexanderson Jonas Beskow Éva Székely G. Henter DiffM 11 14 0 15 Jun 2023
NAP: Neural 3D Articulation Prior Jiahui Lei Congyue Deng Bokui Shen Leonidas J. Guibas Kostas Daniilidis DiffM 14 16 0 25 May 2023
Guided Motion Diffusion for Controllable Human Motion Synthesis Korrawe Karunratanakul Konpat Preechakul Supasorn Suwajanakorn Siyu Tang DiffM 21 118 0 21 May 2023
Towards Consistent Stochastic Human Motion Prediction via Motion Diffusion Jiarui Sun Girish Chowdhary DiffM 29 6 0 21 May 2023
Synthesizing Diverse Human Motions in 3D Indoor Scenes Kaifeng Zhao Yan Zhang Shaofei Wang Thabo Beeler Siyu Tang 25 64 0 21 May 2023
QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generation Sicheng Yang Zhiyong Wu Minglei Li Zhensong Zhang Lei Hao Weihong Bao Hao-Wen Zhuang SLR 11 40 0 18 May 2023
DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Models Sicheng Yang Zhiyong Wu Minglei Li Zhensong Zhang Lei Hao Weihong Bao Ming Cheng Long Xiao 11 64 0 08 May 2023
Probabilistic Human Mesh Recovery in 3D Scenes from Egocentric Views Siwei Zhang Qianli Ma Yan Zhang S. Aliakbarian Darren Cosker Siyu Tang EgoV 17 13 0 12 Apr 2023
Controllable Motion Synthesis and Reconstruction with Autoregressive Diffusion Models Wenjie Yin Ruibo Tu Hang Yin Danica Kragic Hedvig Kjellström Mårten Björkman DiffM 11 2 0 03 Apr 2023
GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents Tenglong Ao Zeyi Zhang Libin Liu DiffM VGen 65 88 0 26 Mar 2023
Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022 Taras Kucherenko Pieter Wolfert Youngwoo Yoon Carla Viegas Teodor Nikolov Mihail Tsakov G. Henter 27 24 0 15 Mar 2023
HumanMAC: Masked Motion Completion for Human Motion Prediction Ling-Hao Chen Jiawei Zhang Ye-rong Li Yiren Pang Xiaobo Xia Tongliang Liu DiffM VGen 25 54 0 07 Feb 2023
Bipartite Graph Diffusion Model for Human Interaction Generation Baptiste Chopin Hao Tang Mohamed Daoudi DiffM 11 8 0 24 Jan 2023
Audio2Gestures: Generating Diverse Gestures from Audio Jing Li Di Kang Wenjie Pei Xuefei Zhe Ying Zhang Linchao Bao Zhenyu He DiffM SLR 23 7 0 17 Jan 2023
Human Motion Diffusion Model Guy Tevet Sigal Raab Brian Gordon Yonatan Shafir Daniel Cohen-Or Amit H. Bermano DiffM VGen 177 713 0 29 Sep 2022
Denoising Diffusion Probabilistic Models for Styled Walking Synthesis Edmund J. C. Findlay Hao Zhang Ziyi Chang Hubert P. H. Shum DiffM 29 6 0 29 Sep 2022
TM2T: Stochastic and Tokenized Modeling for the Reciprocal Generation of 3D Human Motions and Texts Chuan Guo Xinxin Xuo Sen Wang Li Cheng VGen 60 225 0 04 Jul 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 234 690 0 27 Aug 2021
Multimodal analysis of the predictability of hand-gesture properties Taras Kucherenko Rajmund Nagy Michael Neff Hedvig Kjellström G. Henter 20 22 0 12 Aug 2021
Robust Classification using Hidden Markov Models and Mixtures of Normalizing Flows Anubhab Ghosh Antoine Honoré Dong Liu G. Henter S. Chatterjee BDL VLM 13 7 0 15 Feb 2021
AI Choreographer: Music Conditioned 3D Dance Generation with AIST++ Ruilong Li Sha Yang David A. Ross Angjoo Kanazawa ViT 198 467 0 21 Jan 2021