Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models

17 November 2022

Papers citing "Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models"

50 / 130 papers shown

Title
Investigating the impact of 2D gesture representation on co-speech gesture generation Teo Guichoux Laure Soulier Nicolas Obin Catherine Pelachaud SLR 19 0 0 21 Jun 2024
Complex Image-Generative Diffusion Transformer for Audio Denoising Junhui Li Pu Wang Jialu Li Youshan Zhang DiffM 16 1 0 13 Jun 2024
Monkey See, Monkey Do: Harnessing Self-attention in Motion Diffusion for Zero-shot Motion Transfer Sigal Raab Inbar Gat Nathan Sala Guy Tevet Rotem Shalev-Arkushin Ohad Fried Amit H. Bermano Daniel Cohen-Or 24 3 0 10 Jun 2024
Should you use a probabilistic duration model in TTS? Probably! Especially for spontaneous speech Shivam Mehta Harm Lameris Rajiv Punmiya Jonas Beskow Éva Székely G. Henter 18 1 0 08 Jun 2024
GaussianPrediction: Dynamic 3D Gaussian Prediction for Motion Extrapolation and Free View Synthesis Boming Zhao Yuan Li Ziyu Sun Lin Zeng Yujun Shen Rui-ya Ma Yinda Zhang Hujun Bao Zhaopeng Cui 3DGS 3DV 35 3 0 30 May 2024
Programmable Motion Generation for Open-Set Motion Control Tasks Hanchao Liu Xiaohang Zhan Shaoli Huang Tai-Jiang Mu Ying Shan 27 5 0 29 May 2024
DanceGen: Supporting Choreography Ideation and Prototyping with Generative AI Yimeng Liu Misha Sra 27 2 0 28 May 2024
SIGGesture: Generalized Co-Speech Gesture Synthesis via Semantic Injection with Large-Scale Pre-Training Diffusion Models Qingrong Cheng Xu Li Xinghui Fu DiffM 27 2 0 22 May 2024
InterAct: Capture and Modelling of Realistic, Expressive and Interactive Activities between Two Persons in Daily Scenarios Yinghao Huang Leo Ho Dafei Qin Mingyi Shi Taku Komura VGen 24 1 0 19 May 2024
Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis Zeyi Zhang Tenglong Ao Yuyao Zhang Qingzhe Gao Chuan Lin Baoquan Chen Libin Liu SLR 21 15 0 16 May 2024
LLAniMAtion: LLAMA Driven Gesture Animation John T. Windle Iain Matthews Sarah Taylor 33 0 0 13 May 2024
Interaction Design for Human-AI Choreography Co-creation Yimeng Liu 25 1 0 07 May 2024
LGTM: Local-to-Global Text-Driven Human Motion Diffusion Model Haowen Sun Ruikun Zheng Haibin Huang Chongyang Ma Hui Huang Ruizhen Hu DiffM 34 7 0 06 May 2024
Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesis Shivam Mehta Anna Deichler Jim O'Regan Birger Moëll Jonas Beskow G. Henter Simon Alexanderson 28 4 0 30 Apr 2024
Taming Diffusion Probabilistic Models for Character Control Rui Chen Mingyi Shi Shaoli Huang Ping Tan Taku Komura Xuelin Chen VGen DiffM AI4CE 22 21 0 23 Apr 2024
Learn2Talk: 3D Talking Face Learns from 2D Talking Face Yixiang Zhuang Baoping Cheng Yao Cheng Yuntao Jin Renshuai Liu Chengyang Li Xuan Cheng Jing Liao Juncong Lin CVBM 3DH 29 6 0 19 Apr 2024
A Unified Editing Method for Co-Speech Gesture Generation via Diffusion Inversion Zeyu Zhao Nan Gao Zhi Zeng Guixuan Zhang Jie Liu Shuwu Zhang DiffM 29 0 0 03 Apr 2024
THOR: Text to Human-Object Interaction Diffusion via Relation Intervention Qianyang Wu Ye-ling Shi 0001 Xiaoshui Huang Jingyi Yu Lan Xu Jingya Wang 44 24 0 17 Mar 2024
Speech-driven Personalized Gesture Synthetics: Harnessing Automatic Fuzzy Feature Inference Fan Zhang Zhaohan Wang Xin Lyu Siyuan Zhao Mengjian Li ... Naye Ji Hui Du Fuxing Gao Hao Wu Shunman Li VGen 25 3 0 16 Mar 2024
LM2D: Lyrics- and Music-Driven Dance Synthesis Wenjie Yin Xuejiao Zhao Yi Yu Hang Yin Danica Kragic Mårten Björkman 23 1 0 14 Mar 2024
Seamless Human Motion Composition with Blended Positional Encodings Germán Barquero Sergio Escalera Cristina Palmero DiffM 34 26 0 23 Feb 2024
Exploring AI-assisted Ideation and Prototyping for Choreography Yimeng Liu Misha Sra 18 7 0 20 Feb 2024
DiffSpeaker: Speech-Driven 3D Facial Animation with Diffusion Transformer Zhiyuan Ma Xiangyu Zhu Guojun Qi Chen Qian Zhaoxiang Zhang Zhen Lei 17 9 0 08 Feb 2024
Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance Qingcheng Zhao Pengyu Long Qixuan Zhang Dafei Qin Hanming Liang Longwen Zhang Yingliang Zhang Jingyi Yu Lan Xu DiffM 3DH 12 7 0 28 Jan 2024
Diffusion-based Graph Generative Methods Hongyang Chen Can Xu Lingyu Zheng Qiang Zhang Xuemin Lin DiffM MedIm 15 0 0 28 Jan 2024
Multi-Track Timeline Control for Text-Driven 3D Human Motion Generation Mathis Petrovich Or Litany Umar Iqbal Michael J. Black Gül Varol Xue Bin Peng Davis Rempe DiffM VGen 27 40 0 16 Jan 2024
DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation Junming Chen Yunfei Liu Jianan Wang Ailing Zeng Yu Li Qifeng Chen VGen 22 10 0 09 Jan 2024
Freetalker: Controllable Speech and Text-Driven Gesture Generation Based on Diffusion Models for Enhanced Speaker Naturalness Sicheng Yang Zunnan Xu Haiwei Xue Yongkang Cheng Shaoli Huang Mingming Gong Zhiyong Wu DiffM VGen 22 11 0 07 Jan 2024
From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations Evonne Ng Javier Romero Timur M. Bagautdinov Shaojie Bai Trevor Darrell Angjoo Kanazawa Alexander Richard VGen 17 36 0 03 Jan 2024
EnchantDance: Unveiling the Potential of Music-Driven Dance Movement Bo Han Yi Ren Hao Peng Teng Zhang Zeyu Ling Xiang Yin Feilin Han 11 1 0 26 Dec 2023
Optimizing Diffusion Noise Can Serve As Universal Motion Priors Korrawe Karunratanakul Konpat Preechakul Emre Aksan Thabo Beeler Supasorn Suwajanakorn Siyu Tang DiffM 11 37 0 19 Dec 2023
MotionScript: Natural Language Descriptions for Expressive 3D Human Motions Payam Jome Yazdian Eric Liu Li Cheng Angelica Lim Li Cheng Angelica Lim 33 7 0 19 Dec 2023
FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models Shivangi Aneja Justus Thies Angela Dai Matthias Nießner DiffM VGen 14 29 0 13 Dec 2023
Scalable Motion Style Transfer with Constrained Diffusion Generation Wenjie Yin Yi Yu Hang Yin Danica Kragic Mårten Björkman DiffM 9 3 0 12 Dec 2023
Disentangled Clothed Avatar Generation from Text Descriptions Jiong-Qi Wang Yuan Liu Zhiyang Dou Zhengming Yu Yongqing Liang Xin Li Wenping Wang Rong Xie Li-Na Song 17 21 0 08 Dec 2023
Emotional Speech-driven 3D Body Animation via Disentangled Latent Diffusion Kiran Chhatre Radek Danvevcek Nikos Athanasiou Giorgio Becherini Christopher Peters Michael J. Black Timo Bolkart DiffM 25 14 0 07 Dec 2023
EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation Wenyang Zhou Zhiyang Dou Zeyu Cao Zhouyingcheng Liao Jingbo Wang Wenjia Wang Yuan-Bin Liu Taku Komura Wenping Wang Lingjie Liu VGen 23 47 0 04 Dec 2023
AAMDM: Accelerated Auto-regressive Motion Diffusion Model Tianyu Li Calvin Qiao Guanqiao Ren KangKang Yin Sehoon Ha 8 4 0 02 Dec 2023
DanceMeld: Unraveling Dance Phrases with Hierarchical Latent Codes for Music-to-Dance Synthesis Xin Gao Liucheng Hu Peng Zhang Bang Zhang Liefeng Bo DiffM 12 4 0 30 Nov 2023
SpeechAct: Towards Generating Whole-body Motion from Speech Jinsong Zhang Minjie Zhu Yuxiang Zhang Yebin Liu Kun Li 18 0 0 29 Nov 2023
TLControl: Trajectory and Language Control for Human Motion Synthesis Weilin Wan Zhiyang Dou Taku Komura Wenping Wang Dinesh Jayaraman Lingjie Liu 20 43 0 28 Nov 2023
A Unified Framework for Multimodal, Multi-Part Human Motion Synthesis Zixiang Zhou Yu Wan Baoyuan Wang 17 9 0 28 Nov 2023
CG-HOI: Contact-Guided 3D Human-Object Interaction Generation Christian Diller Angela Dai 17 59 0 27 Nov 2023
Controllable Group Choreography using Contrastive Diffusion Nhat Le Tuong Khanh Long Do Khoa Do Hien Nguyen Erman Tjiputra Quang-Dieu Tran Anh Nguyen 32 10 0 29 Oct 2023
Orientation-Aware Leg Movement Learning for Action-Driven Human Motion Prediction Chunzhi Gu Chao Zhang Shigeru Kuriyama 25 4 0 23 Oct 2023
MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations Heyuan Yao Zhenhua Song Yuyang Zhou Tenglong Ao Baoquan Chen Libin Liu 11 38 0 16 Oct 2023
State of the Art on Diffusion Models for Visual Computing Ryan Po Wang Yifan Vladislav Golyanik Kfir Aberman Jonathan T. Barron ... Matthias Nießner Bjorn Ommer Christian Theobalt Peter Wonka Gordon Wetzstein 16 101 0 11 Oct 2023
Unified speech and gesture synthesis using flow matching Shivam Mehta Ruibo Tu Simon Alexanderson Jonas Beskow Éva Székely G. Henter 17 3 0 08 Oct 2023
Music- and Lyrics-driven Dance Synthesis Wenjie Yin Qingyuan Yao Yi Yu Hang Yin Danica Kragic Mårten Björkman DiffM 11 0 0 30 Sep 2023
DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models Zhiyao Sun Tian Lv Sheng Ye Matthieu Lin Jenny Sheng Yuhui Wen Minjing Yu Yong-jin Liu DiffM 21 44 0 30 Sep 2023