DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation

10 January 2023

Wenliang Zhao

Jie Zhou

Papers citing "DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation"

29 / 79 papers shown

Title
Superior and Pragmatic Talking Face Generation with Teacher-Student Framework Chao Liang Jianwen Jiang Tianyun Zhong Gaojie Lin Zhengkun Rong Jiaqi Yang Yongming Zhu 37 1 0 26 Mar 2024
DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactment Stella Bounareli Christos Tzelepis Vasileios Argyriou Ioannis Patras Georgios Tzimiropoulos DiffM 43 7 0 25 Mar 2024
DiffSal: Joint Audio and Video Learning for Diffusion Saliency Prediction Jun Xiong Peng Zhang Tao You Chuanyue Li Wei Huang Yufei Zha DiffM 27 5 0 02 Mar 2024
G4G:A Generic Framework for High Fidelity Talking Face Generation with Fine-grained Intra-modal Alignment Juan Zhang Jiahao Chen Cheng Wang Zhi-Yang Yu Tangquan Qi Di Wu CVBM 30 0 0 28 Feb 2024
Context-aware Talking Face Video Generation Meidai Xuanyuan Yuwang Wang Honglei Guo Qionghai Dai DiffM 27 0 0 28 Feb 2024
EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions Linrui Tian Qi Wang Bang Zhang Liefeng Bo DiffM 61 101 0 27 Feb 2024
EmoTalker: Emotionally Editable Talking Face Generation via Diffusion Model Bingyuan Zhang Xulong Zhang Ning Cheng Jun Yu Jing Xiao Jianzong Wang DiffM 23 5 0 16 Jan 2024
Dubbing for Everyone: Data-Efficient Visual Dubbing using Neural Rendering Priors Jack D. Saunders Vinay P. Namboodiri VGen DiffM 31 1 0 11 Jan 2024
A Generalist FaceX via Learning Unified Facial Representation Yue Han Jiangning Zhang Junwei Zhu Xiangtai Li Yanhao Ge Wei Li Chengjie Wang Yong Liu Xiaoming Liu Ying Tai DiffM 27 13 0 31 Dec 2023
PIA: Your Personalized Image Animator via Plug-and-Play Modules in Text-to-Image Models Yiming Zhang Zhening Xing Yanhong Zeng Youqing Fang Kai Chen VGen 31 27 0 21 Dec 2023
DREAM-Talk: Diffusion-based Realistic Emotional Audio-driven Method for Single Image Talking Face Generation Chenxu Zhang Chao Wang Jianfeng Zhang Hongyi Xu Guoxian Song You Xie Linjie Luo Yapeng Tian Xiaohu Guo Jiashi Feng 30 19 0 21 Dec 2023
AE-NeRF: Audio Enhanced Neural Radiance Field for Few Shot Talking Head Synthesis Dongze Li Kang Zhao Wei Wang Bo Peng Yingya Zhang Jing Dong Tien-Ping Tan DiffM VGen 27 12 0 18 Dec 2023
FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models Shivangi Aneja Justus Thies Angela Dai Matthias Nießner DiffM VGen 26 29 0 13 Dec 2023
GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits Yibo Xia Lizhen Wang Xiang Deng Xiaoyan Luo Yunhong Wang Yebin Liu VGen 33 1 0 12 Dec 2023
SingingHead: A Large-scale 4D Dataset for Singing Head Animation Sijing Wu Yunhao Li Weitian Zhang Jun Jia Yucheng Zhu Yichao Yan Guangtao Zhai Xiaokang Yang 41 2 0 07 Dec 2023
AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset Zhixi Cai Shreya Ghosh Aman Pankaj Adatia Munawar Hayat Abhinav Dhall Kalin Stefanov 19 27 0 26 Nov 2023
GAIA: Zero-shot Talking Avatar Generation Tianyu He Junliang Guo Runyi Yu Yuchi Wang Jialiang Zhu ... Chunyu Wang Han Hu HsiangTao Wu Sheng Zhao Jiang Bian 23 25 0 26 Nov 2023
DiffDub: Person-generic Visual Dubbing Using Inpainting Renderer with Diffusion Auto-encoder Tao Liu Chenpeng Du Shuai Fan Feilong Chen Kai Yu DiffM VGen 14 6 0 03 Nov 2023
Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization Soumik Mukhopadhyay Saksham Suri R. Gadde Abhinav Shrivastava DiffM 38 20 0 18 Aug 2023
A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation Li Liu Lufei Gao Wen-Ling Lei Fengji Ma Xiaotian Lin Jin-Tao Wang CVBM 25 5 0 17 Aug 2023
Learning and Evaluating Human Preferences for Conversational Head Generation Mohan Zhou Yalong Bai Wei Zhang Ting Yao Tiejun Zhao Tao Mei 27 2 0 20 Jul 2023
Audio-driven Talking Face Generation with Stabilized Synchronization Loss Dogucan Yaman Fevziye Irem Eyiokur Leonard Barmann H. K. Ekenel Alexander Waibel CVBM 27 3 0 18 Jul 2023
Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis Jiahe Li Jiawei Zhang Xiao Bai Jun Zhou L. Gu 3DH 26 62 0 18 Jul 2023
Text-driven Talking Face Synthesis by Reprogramming Audio-driven Models J. Choi Minsu Kim Se Jin Park Y. Ro CVBM 16 3 0 28 Jun 2023
Speech Driven Video Editing via an Audio-Conditioned Diffusion Model Dan Bigioi Shubhajit Basak Michał Stypułkowski Maciej Ziȩba H. Jordan R. Mcdonnell Peter Corcoran DiffM VGen 19 34 0 10 Jan 2023
Face Super-Resolution Using Stochastic Differential Equations Marcelo dos Santos Rayson Laroca Rafael O. Ribeiro João Neves Hugo Proencca David Menotti DiffM 24 11 0 24 Sep 2022
RePaint: Inpainting using Denoising Diffusion Probabilistic Models Andreas Lugmayr Martin Danelljan Andrés Romero F. I. F. Richard Yu Radu Timofte Luc Van Gool DiffM 213 1,354 0 24 Jan 2022
Multimodal Image Synthesis and Editing: The Generative AI Era Fangneng Zhan Yingchen Yu Rongliang Wu Jiahui Zhang Shijian Lu Lingjie Liu Adam Kortylewski Christian Theobalt Eric Xing EGVM 24 48 0 27 Dec 2021
Zero-Shot Text-to-Image Generation Aditya A. Ramesh Mikhail Pavlov Gabriel Goh Scott Gray Chelsea Voss Alec Radford Mark Chen Ilya Sutskever VLM 255 4,774 0 24 Feb 2021