LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalization

8 June 2021

Papers citing "LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces from Video using Pose and Lighting Normalization"

50 / 67 papers shown

Title
Exploiting Temporal Audio-Visual Correlation Embedding for Audio-Driven One-Shot Talking Head Animation Zhihua Xu Tianshui Chen Zhijing Yang Siyuan Peng Keze Wang Liang Lin 24 0 0 08 Apr 2025
SyncDiff: Diffusion-based Talking Head Synthesis with Bottlenecked Temporal Visual Prior for Improved Synchronization Xulin Fan Heting Gao Ziyi Chen Peng Chang Mei Han Mark Hasegawa-Johnson DiffM 57 0 0 17 Mar 2025
MuseTalk: Real-Time High-Fidelity Video Dubbing via Spatio-Temporal Sampling Yue Zhang Minhao Liu Zhaokang Chen Bin Wu Yubin Zeng Chao Zhan Yingjie He Junxin Huang Wenjiang Zhou Wenjiang Zhou 36 6 0 14 Oct 2024
StyleTalk++: A Unified Framework for Controlling the Speaking Styles of Talking Heads Suzhen Wang Yifeng Ma Yu Ding Zhipeng Hu Changjie Fan Tangjie Lv Zhidong Deng Xin Yu 38 9 0 14 Sep 2024
EmoFace: Emotion-Content Disentangled Speech-Driven 3D Talking Face Animation Yihong Lin Liang Peng Jianqiao Hu Xiandong Li Wenxiong Kang Songju Lei Xianjia Wu Huang Xu CVBM 3DH 22 1 0 21 Aug 2024
Content and Style Aware Audio-Driven Facial Animation Qingju Liu Hyeongwoo Kim Gaurav Bharaj DiffM 30 1 0 13 Aug 2024
Solutions to Deepfakes: Can Camera Hardware, Cryptography, and Deep Learning Verify Real Images? Alexander Vilesov Yuan Tian Nader Sehatbakhsh A. Kadambi 29 0 0 04 Jul 2024
Speech2UnifiedExpressions: Synchronous Synthesis of Co-Speech Affective Face and Body Expressions from Affordable Inputs Uttaran Bhattacharya Aniket Bera Dinesh Manocha CVBM 33 2 0 26 Jun 2024
NLDF: Neural Light Dynamic Fields for Efficient 3D Talking Head Generation Niu Guanchen 3DH 39 0 0 17 Jun 2024
Embedded Representation Learning Network for Animating Styled Video Portrait Tianyong Wang Xiangyu Liang Wangguandong Zheng Dan Niu Haifeng Xia Siyu Xia 3DH 21 0 0 29 Apr 2024
Towards Image Ambient Lighting Normalization Florin-Alexandru Vasluianu Tim Seizinger Zongwei Wu Rakesh Ranjan Radu Timofte DiffM 46 2 0 27 Mar 2024
Dyadic Interaction Modeling for Social Behavior Generation Minh Tran Di Chang Maksim Siniukov Mohammad Soleymani VGen 32 6 0 14 Mar 2024
An Implicit Physical Face Model Driven by Expression and Style Lingchen Yang Gaspard Zoss Prashanth Chandran Paulo F. U. Gotardo Markus Gross B. Solenthaler Eftychios Sifakis Derek Bradley CVBM 3DH AI4CE 27 2 0 27 Jan 2024
DreamTalk: When Expressive Talking Head Generation Meets Diffusion Probabilistic Models Yifeng Ma Shiwei Zhang Jiayu Wang Xiang Wang Yingya Zhang Zhidong Deng DiffM 39 23 0 15 Dec 2023
GSmoothFace: Generalized Smooth Talking Face Generation via Fine Grained 3D Face Guidance Haiming Zhang Zhihao Yuan Chaoda Zheng Xu Yan Baoyuan Wang Guanbin Li Song Wu Shuguang Cui Zhen Li CVBM 42 1 0 12 Dec 2023
SingingHead: A Large-scale 4D Dataset for Singing Head Animation Sijing Wu Yunhao Li Weitian Zhang Jun Jia Yucheng Zhu Yichao Yan Guangtao Zhai Xiaokang Yang 41 2 0 07 Dec 2023
GAIA: Zero-shot Talking Avatar Generation Tianyu He Junliang Guo Runyi Yu Yuchi Wang Jialiang Zhu ... Chunyu Wang Han Hu HsiangTao Wu Sheng Zhao Jiang Bian 23 25 0 26 Nov 2023
OSM-Net: One-to-Many One-shot Talking Head Generation with Spontaneous Head Motions Jin Liu Xi Wang Xiaomeng Fu Yesheng Chai Cai Yu Jiao Dai Jizhong Han 19 3 0 28 Sep 2023
RADIO: Reference-Agnostic Dubbing Video Synthesis Dongyeun Lee Chaewon Kim Sangjoon Yu Jaejun Yoo Gyeong-Moon Park VGen DiffM 34 1 0 05 Sep 2023
MFR-Net: Multi-faceted Responsive Listening Head Generation via Denoising Diffusion Model Jin Liu Xi Wang Xiaomeng Fu Yesheng Chai Cai Yu Jiao Dai Jizhong Han DiffM 23 10 0 31 Aug 2023
Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization Soumik Mukhopadhyay Saksham Suri R. Gadde Abhinav Shrivastava DiffM 38 20 0 18 Aug 2023
A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation Li Liu Lufei Gao Wen-Ling Lei Fengji Ma Xiaotian Lin Jin-Tao Wang CVBM 25 5 0 17 Aug 2023
Controlling Character Motions without Observable Driving Source Weiyuan Li Bin Dai Ziyi Zhou Qi Yao Baoyuan Wang VGen 6 1 0 11 Aug 2023
VAST: Vivify Your Talking Avatar via Zero-Shot Expressive Facial Style Transfer Liyang Chen Zhiyong Wu Runnan Li Weihong Bao Jun Ling Xuejiao Tan Sheng Zhao 21 5 0 09 Aug 2023
UniBriVL: Robust Universal Representation and Generation of Audio Driven Diffusion Models Sen Fang Bowen Gao Yangjian Wu T. Teoh DiffM 18 1 0 29 Jul 2023
MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentions Yunfei Liu Lijian Lin Fei Yu Changyin Zhou Yu Li DiffM VGen 42 23 0 19 Jul 2023
FACTS: Facial Animation Creation using the Transfer of Styles Jack D. Saunders Steven Caulkin Vinay P. Namboodiri 3DH CVBM 36 0 0 18 Jul 2023
SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces Ziqiao Peng Yihao Luo Yue Shi Hao-Xuan Xu Xiangyu Zhu Jun He Hongyan Liu Zhaoxin Fan 52 39 0 19 Jun 2023
ReactFace: Multiple Appropriate Facial Reaction Generation in Dyadic Interactions Cheng Luo Siyang Song Weicheng Xie Micol Spitale Linlin Shen Hatice Gunes CVBM DiffM 28 9 0 25 May 2023
CPNet: Exploiting CLIP-based Attention Condenser and Probability Map Guidance for High-fidelity Talking Face Generation Jingning Xu Benlai Tang Mingjie Wang Minghao Li Meirong Ma CVBM 13 0 0 23 May 2023
StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generator Jiazhi Guan Zhanwang Zhang Hang Zhou Tianshu Hu Kaisiyuan Wang ... Haocheng Feng Jingtuo Liu Errui Ding Ziwei Liu Jingdong Wang 37 57 0 09 May 2023
DAE-Talker: High Fidelity Speech-Driven Talking Face Generation with Diffusion Autoencoder Chenpeng Du Qi Chen Xie Chen K. Yu DiffM 27 50 0 30 Mar 2023
Style Transfer for 2D Talking Head Animation Trong-Thang Pham Nhat Le Tuong Khanh Long Do Hung Nguyen Erman Tjiputra Quang-Dieu Tran A. Nguyen 17 3 0 17 Mar 2023
MMFace4D: A Large-Scale Multi-Modal 4D Face Dataset for Audio-Driven 3D Face Animation Haozhe Wu Jia Jia Junliang Xing Hongwei Xu Xiangyuan Wang Jelo Wang CVBM 24 7 0 17 Mar 2023
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition Xize Cheng Lin Li Tao Jin Rongjie Huang Wang Lin Zehan Wang Huangdai Liu Yejin Wang Aoxiong Yin Zhou Zhao 21 24 0 09 Mar 2023
Exploring Efficient-Tuned Learning Audio Representation Method from BriVL Sen Fang Yang Wu Bowen Gao Jingwen Cai T. Teoh DiffM 16 1 0 08 Mar 2023
DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Video Zhimeng Zhang Zhipeng Hu W. Deng Changjie Fan Tangjie Lv Yu-qiong Ding 3DH CVBM 24 59 0 07 Mar 2023
READ Avatars: Realistic Emotion-controllable Audio Driven Avatars Jack D. Saunders Vinay P. Namboodiri VGen 21 11 0 01 Mar 2023
Pose-Controllable 3D Facial Animation Synthesis using Hierarchical Audio-Vertex Attention Bin Liu Xiaolin K. Wei Bo Li Junjie Cao Yunyu Lai CVBM 19 1 0 24 Feb 2023
Regeneration Learning: A Learning Paradigm for Data Generation Xu Tan Tao Qin Jiang Bian Tie-Yan Liu Yoshua Bengio GAN 38 15 0 21 Jan 2023
Speech Driven Video Editing via an Audio-Conditioned Diffusion Model Dan Bigioi Shubhajit Basak Michał Stypułkowski Maciej Ziȩba H. Jordan R. Mcdonnell Peter Corcoran DiffM VGen 19 34 0 10 Jan 2023
Diffused Heads: Diffusion Models Beat GANs on Talking-Face Generation Michal Stypulkowski Konstantinos Vougioukas Sen He Maciej Ziȩba Stavros Petridis M. Pantic DiffM 16 116 0 06 Jan 2023
CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior Jinbo Xing Menghan Xia Yuechen Zhang Xiaodong Cun Jue Wang T. Wong 24 140 0 06 Jan 2023
StyleTalk: One-shot Talking Head Generation with Controllable Speaking Styles Yifeng Ma Suzhe Wang Zhipeng Hu Changjie Fan Tangjie Lv Yu-qiong Ding Zhidong Deng Xin Yu 54 82 0 03 Jan 2023
Imitator: Personalized Speech-driven 3D Facial Animation Balamurugan Thambiraja I. Habibie S. Aliakbarian Darren Cosker Christian Theobalt Justus Thies CVBM 39 49 0 30 Dec 2022
Memories are One-to-Many Mapping Alleviators in Talking Face Generation Anni Tang Tianyu He Xuejiao Tan Jun Ling Liang Song CVBM 26 23 0 09 Dec 2022
Masked Lip-Sync Prediction by Audio-Visual Contextual Exploitation in Transformers Yasheng Sun Hang Zhou Kaisiyuan Wang Qianyi Wu Zhibin Hong Jingtuo Liu Errui Ding Jingdong Wang Ziwei Liu Koike Hideki 27 34 0 09 Dec 2022
VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild K. Cheng Xiaodong Cun Yong Zhang Menghan Xia Fei Yin Mingrui Zhu Xuanxia Wang Jue Wang Nan Wang CVBM 25 92 0 27 Nov 2022
Continuously Controllable Facial Expression Editing in Talking Face Videos Zhiyao Sun Yu-Hui Wen Tian Lv Y. Sun Ziyang Zhang Yaoyuan Wang Y. Liu CVBM 32 10 0 17 Sep 2022
Talking Head from Speech Audio using a Pre-trained Image Generator M. M. Alghamdi He-Nan Wang A. Bulpitt David C. Hogg 70 21 0 09 Sep 2022