SyncTalkFace: Talking Face Generation with Precise Lip-Syncing via Audio-Lip Memory

2 November 2022

Papers citing "SyncTalkFace: Talking Face Generation with Precise Lip-Syncing via Audio-Lip Memory"

50 / 56 papers shown

Title
KeySync: A Robust Approach for Leakage-free Lip Synchronization in High Resolution Antoni Bigata Rodrigo Mira Stella Bounareli Michał Stypułkowski Konstantinos Vougioukas Stavros Petridis Maja Pantic 49 0 0 01 May 2025
AlignDiT: Multimodal Aligned Diffusion Transformer for Synchronized Speech Generation J. Choi Ji-Hoon Kim Kim Sung-Bin Tae-Hyun Oh Joon Son Chung DiffM 48 0 0 29 Apr 2025
FluentLip: A Phonemes-Based Two-stage Approach for Audio-Driven Lip Synthesis with Optical Flow Consistency Shiyan Liu Rui Qu Yan Jin 26 0 0 06 Apr 2025
SyncDiff: Diffusion-based Talking Head Synthesis with Bottlenecked Temporal Visual Prior for Improved Synchronization Xulin Fan Heting Gao Ziyi Chen Peng Chang Mei Han Mark Hasegawa-Johnson DiffM 45 0 0 17 Mar 2025
MAVFlow: Preserving Paralinguistic Elements with Conditional Flow Matching for Zero-Shot AV2AV Multilingual Translation Sungwoo Cho J. Choi Sungnyun Kim Se-Young Yun 54 0 0 14 Mar 2025
Cosh-DiT: Co-Speech Gesture Video Synthesis via Hybrid Audio-Visual Diffusion Transformers Yasheng Sun Zhiliang Xu Hang Zhou Jiazhi Guan Quanwei Yang ... Yingying Li Haocheng Feng J. Wang Ziwei Liu Koike Hideki VGen 54 0 0 13 Mar 2025
Emotion Recognition and Generation: A Comprehensive Review of Face, Speech, and Text Modalities Rebecca Mobbs Dimitrios Makris Vasileios Argyriou 33 0 0 02 Feb 2025
SyncFlow: Toward Temporally Aligned Joint Audio-Video Generation from Text Haohe Liu Gaël Le Lan Xinhao Mei Zhaoheng Ni Anurag Kumar Varun K. Nagaraja Wenwu Wang Mark D. Plumbley Yangyang Shi Vikas Chandra VGen 61 1 0 03 Dec 2024
JoyVASA: Portrait and Animal Image Animation with Diffusion-Based Audio-Driven Facial Dynamics and Head Motion Generation Xuyang Cao Guoxin Wang Sheng Shi Jun Zhao Yang Yao Jintao Fei Minyu Gao VGen 32 1 0 14 Nov 2024
Separation of Neural Drives to Muscles from Transferred Polyfunctional Nerves using Implanted Micro-electrode Arrays Laura Ferrante Anna Boesendorfer D. Barsakcioglu Benedikt Baumgartner Yazan Al-Ajam Alex Woollard Norbert Venantius Kang Oskar Aszmann D. Farina 26 7 0 14 Oct 2024
MuseTalk: Real-Time High-Fidelity Video Dubbing via Spatio-Temporal Sampling Yue Zhang Minhao Liu Zhaokang Chen Bin Wu Yubin Zeng Chao Zhan Yingjie He Junxin Huang Wenjiang Zhou Wenjiang Zhou 34 6 0 14 Oct 2024
Interpretable Convolutional SyncNet Sungjoon Park Jaesub Yun Donggeon Lee Minsik Park 39 0 0 02 Sep 2024
ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performer Jiazhi Guan Zhiliang Xu Hang Zhou Kaisiyuan Wang Shengyi He ... Errui Ding Jingtuo Liu Jingdong Wang Youjian Zhao Ziwei Liu VGen 36 2 0 06 Aug 2024
GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer Yihong Lin Zhaoxin Fan Lingyu Xiong Liang Peng Xiandong Li Wenxiong Kang Xianjia Wu Songju Lei Huang Xu 27 3 0 03 Aug 2024
The Tug-of-War Between Deepfake Generation and Detection Hannah Lee Changyeon Lee Kevin Farhat Lin Qiu Steve Geluso Aerin Kim O. Etzioni 28 1 0 08 Jul 2024
RealTalk: Real-time and Realistic Audio-driven Face Generation with 3D Facial Prior-guided Identity Alignment Network Xiaozhong Ji Chuming Lin Zhonggan Ding Ying Tai Junwei Zhu Xiaobin Hu Donghao Luo Yanhao Ge Chengjie Wang CVBM 19 2 0 26 Jun 2024
Let's Go Real Talk: Spoken Dialogue Model for Face-to-Face Conversation Se Jin Park Chae Won Kim Hyeongseop Rha Minsu Kim Joanna Hong Jeong Hun Yeo Yong Man Ro CVBM AuLLM 37 6 0 12 Jun 2024
Faces that Speak: Jointly Synthesising Talking Face and Speech from Text Youngjoon Jang Ji-Hoon Kim Junseok Ahn Doyeop Kwak Hong-Sun Yang Yooncheol Ju Il-Hwan Kim Byeong-Yeol Kim Joon Son Chung CVBM 19 9 0 16 May 2024
Audio-Visual Speech Representation Expert for Enhanced Talking Face Video Generation and Evaluation Dogucan Yaman Fevziye Irem Eyiokur Leonard Barmann Seymanur Akti H. K. Ekenel Alexander H. Waibel EGVM 15 9 0 07 May 2024
Voice Attribute Editing with Text Prompt Zheng-Yan Sheng Yang Ai Li-Juan Liu Jia Pan Zhenhua Ling 21 4 0 13 Apr 2024
EDTalk: Efficient Disentanglement for Emotional Talking Head Synthesis Shuai Tan Bin Ji Mengxiao Bi Ye Pan 33 24 0 02 Apr 2024
Say Anything with Any Style Shuai Tan Bin Ji Yu Ding Ye Pan VGen DiffM 19 10 0 11 Mar 2024
Audio-Synchronized Visual Animation Lin Zhang Shentong Mo Yijing Zhang Pedro Morgado DiffM 30 18 0 08 Mar 2024
G4G:A Generic Framework for High Fidelity Talking Face Generation with Fine-grained Intra-modal Alignment Juan Zhang Jiahao Chen Cheng Wang Zhi-Yang Yu Tangquan Qi Di Wu CVBM 22 0 0 28 Feb 2024
GSmoothFace: Generalized Smooth Talking Face Generation via Fine Grained 3D Face Guidance Haiming Zhang Zhihao Yuan Chaoda Zheng Xu Yan Baoyuan Wang Guanbin Li Song Wu Shuguang Cui Zhen Li CVBM 34 1 0 12 Dec 2023
GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits Yibo Xia Lizhen Wang Xiang Deng Xiaoyan Luo Yunhong Wang Yebin Liu VGen 33 1 0 12 Dec 2023
AV2AV: Direct Audio-Visual Speech to Audio-Visual Speech Translation with Unified Audio-Visual Speech Representation J. Choi Se Jin Park Minsu Kim Y. Ro 9 12 0 05 Dec 2023
HyperLips: Hyper Control Lips with High Resolution Decoder for Talking Face Generation Yaosen Chen Yu Yao Zhiqiang Li Wei Wang Yanru Zhang Han Yang Xuming Wen 17 8 0 09 Oct 2023
Diverse and Aligned Audio-to-Video Generation via Text-to-Video Model Adaptation Guy Yariv Itai Gat Sagie Benaim Lior Wolf Idan Schwartz Yossi Adi DiffM VGen 29 36 0 28 Sep 2023
Face-Driven Zero-Shot Voice Conversion with Memory-based Face-Voice Alignment Zheng-Yan Sheng Yang Ai Yan-Nian Chen Zhenhua Ling CVBM 9 4 0 18 Sep 2023
RADIO: Reference-Agnostic Dubbing Video Synthesis Dongyeun Lee Chaewon Kim Sangjoon Yu Jaejun Yoo Gyeong-Moon Park VGen DiffM 13 1 0 05 Sep 2023
MFR-Net: Multi-faceted Responsive Listening Head Generation via Denoising Diffusion Model Jin Liu Xi Wang Xiaomeng Fu Yesheng Chai Cai Yu Jiao Dai Jizhong Han DiffM 16 10 0 31 Aug 2023
DF-3DFace: One-to-Many Speech Synchronized 3D Face Animation with Diffusion Se Jin Park Joanna Hong Minsu Kim Y. Ro 19 3 0 23 Aug 2023
Diff2Lip: Audio Conditioned Diffusion Models for Lip-Synchronization Soumik Mukhopadhyay Saksham Suri R. Gadde Abhinav Shrivastava DiffM 33 20 0 18 Aug 2023
A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation Li Liu Lufei Gao Wen-Ling Lei Fengji Ma Xiaotian Lin Jin-Tao Wang CVBM 16 1 0 17 Aug 2023
VAST: Vivify Your Talking Avatar via Zero-Shot Expressive Facial Style Transfer Liyang Chen Zhiyong Wu Runnan Li Weihong Bao Jun Ling Xuejiao Tan Sheng Zhao 10 5 0 09 Aug 2023
MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentions Yunfei Liu Lijian Lin Fei Yu Changyin Zhou Yu Li DiffM VGen 26 23 0 19 Jul 2023
Audio-driven Talking Face Generation with Stabilized Synchronization Loss Dogucan Yaman Fevziye Irem Eyiokur Leonard Barmann H. K. Ekenel Alexander Waibel CVBM 19 3 0 18 Jul 2023
A Comprehensive Multi-scale Approach for Speech and Dynamics Synchrony in Talking Head Generation Louis Airale Dominique Vaufreydaz Xavier Alameda-Pineda 11 1 0 04 Jul 2023
Text-driven Talking Face Synthesis by Reprogramming Audio-driven Models J. Choi Minsu Kim Se Jin Park Y. Ro CVBM 11 3 0 28 Jun 2023
SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces Ziqiao Peng Yihao Luo Yue Shi Hao-Xuan Xu Xiangyu Zhu Jun He Hongyan Liu Zhaoxin Fan 44 39 0 19 Jun 2023
Hearing Lips in Noise: Universal Viseme-Phoneme Mapping and Transfer for Robust Audio-Visual Speech Recognition Yuchen Hu Ruizhe Li Cheng Chen Chengwei Qin Qiu-shi Zhu E. Chng 18 5 0 18 Jun 2023
Exploring Phonetic Context-Aware Lip-Sync For Talking Face Generation Se Jin Park Minsu Kim J. Choi Y. Ro CVBM 6 4 0 31 May 2023
Incorporating Ultrasound Tongue Images for Audio-Visual Speech Enhancement through Knowledge Distillation Ruixin Zheng Yang Ai Zhenhua Ling 12 8 0 24 May 2023
Identity-Preserving Talking Face Generation with Landmark and Appearance Priors Wei‐Tao Zhong Chaowei Fang Yinqi Cai Pengxu Wei Gangming Zhao Liang Lin Guanbin Li 16 74 0 15 May 2023
StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generator Jiazhi Guan Zhanwang Zhang Hang Zhou Tianshu Hu Kaisiyuan Wang ... Haocheng Feng Jingtuo Liu Errui Ding Ziwei Liu Jingdong Wang 35 57 0 09 May 2023
StyleLipSync: Style-based Personalized Lip-sync Video Generation Taekyung Ki Dong Min 32 11 0 30 Apr 2023
That's What I Said: Fully-Controllable Talking Face Generation Youngjoon Jang Kyeongha Rho Jong-Bin Woo Hyeongkeun Lee Jihwan Park Youshin Lim Byeong-Yeol Kim Joon Son Chung CVBM 6 8 0 06 Apr 2023
Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert Jiadong Wang Xinyuan Qian Malu Zhang R. Tan Haizhou Li EGVM 14 92 0 29 Mar 2023
DINet: Deformation Inpainting Network for Realistic Face Visually Dubbing on High Resolution Video Zhimeng Zhang Zhipeng Hu W. Deng Changjie Fan Tangjie Lv Yu-qiong Ding 3DH CVBM 16 59 0 07 Mar 2023