A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild

23 August 2020

Prajwal K R

Rudrabha Mukhopadhyay

Papers citing "A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild"

50 / 115 papers shown

Title
CoGenAV: Versatile Audio-Visual Representation Learning via Contrastive-Generative Synchronization Detao Bai Zhiheng Ma Xihan Wei Liefeng Bo 102 0 0 06 May 2025
OT-Talk: Animating 3D Talking Head with Optimal Transportation Xinmu Wang Xiang Gao Xiyun Song Heather Yu Zongfang Lin Liang Peng Xianfeng Gu 22 0 0 03 May 2025
FlowDubber: Movie Dubbing with LLM-based Semantic-aware Learning and Flow Matching based Voice Enhancing Gaoxiang Cong Liang-Sheng Li Jiadong Pan Zhedong Zhang Amin Beheshti A. Hengel Yuankai Qi Qingming Huang 121 0 0 02 May 2025
KeySync: A Robust Approach for Leakage-free Lip Synchronization in High Resolution Antoni Bigata Rodrigo Mira Stella Bounareli Michał Stypułkowski Konstantinos Vougioukas Stavros Petridis Maja Pantic 52 0 0 01 May 2025
MagicPortrait: Temporally Consistent Face Reenactment with 3D Geometric Guidance Mengting Wei Yante Li Tuomas Varanka Yan Jiang Licai Sun DiffM VGen 62 0 0 30 Apr 2025
Audio-Driven Talking Face Video Generation with Joint Uncertainty Learning Yifan Xie Fei Ma Yi Bin Ying He Fei Richard Yu 57 0 0 26 Apr 2025
Disentangle Identity, Cooperate Emotion: Correlation-Aware Emotional Talking Portrait Generation Weipeng Tan Chuming Lin Chengming Xu F. Xu Xiaobin Hu Xiaozhong Ji Junwei Zhu Chengjie Wang Yanwei Fu 44 0 0 25 Apr 2025
Contrastive Decoupled Representation Learning and Regularization for Speech-Preserving Facial Expression Manipulation Tianshui Chen Jianman Lin Zhijing Yang Chumei Qing Yukai Shi Liang Lin 39 2 0 08 Apr 2025
Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation Fa-Ting Hong Zunnan Xu Zixiang Zhou Jun Zhou Xiu Li Qin Lin Qinglin Lu D. Xu DiffM VGen 57 2 0 03 Apr 2025
EmoHead: Emotional Talking Head via Manipulating Semantic Expression Parameters Xuli Shen Hua Cai Dingding Yu Weilin Shen Qing-Song Xu Xiangyang Xue 32 0 0 25 Mar 2025
PC-Talk: Precise Facial Animation Control for Audio-Driven Talking Face Generation Baiqin Wang Xiangyu Zhu Fan Shen Hao-Xuan Xu Zhen Lei 55 0 0 18 Mar 2025
FREAK: Frequency-modulated High-fidelity and Real-time Audio-driven Talking Portrait Synthesis Ziqi Ni Ao Fu Yi Zhou 61 0 0 06 Mar 2025
KeyFace: Expressive Audio-Driven Facial Animation for Long Sequences via KeyFrame Interpolation Antoni Bigata Michał Stypułkowski Rodrigo Mira Stella Bounareli Konstantinos Vougioukas Zoe Landgraf Nikita Drobyshev Maciej Ziȩba Stavros Petridis M. Pantic DiffM VGen 65 2 0 03 Mar 2025
Steganography Beyond Space-Time with Chain of Multimodal AI Ching-Chun Chang Isao Echizen 69 0 0 25 Feb 2025
A Survey on Bridging EEG Signals and Generative AI: From Image and Text to Beyond Shreya Shukla Jose Torres Abhijit Mishra Jacek Gwizdka Shounak Roychowdhury 43 0 0 20 Feb 2025
SayAnything: Audio-Driven Lip Synchronization with Conditional Video Diffusion Junxian Ma Shiwen Wang Jian Yang Junyi Hu Jian Liang Guosheng Lin Jingbo Chen Kai Li Yu Meng DiffM VGen 61 3 0 17 Feb 2025
Towards Dynamic Neural Communication and Speech Neuroprosthesis Based on Viseme Decoding Ji-Ha Park Seo-Hyun Lee Soowon Kim Seong-Whan Lee 41 0 0 28 Jan 2025
FADA: Fast Diffusion Avatar Synthesis with Mixed-Supervised Multi-CFG Distillation Tianyun Zhong Chao Liang Jianwen Jiang Gaojie Lin Jiaqi Yang Zhou Zhao DiffM 96 1 0 22 Dec 2024
EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbing Gaoxiang Cong Jiadong Pan Liang-Sheng Li Yuankai Qi Yuxin Peng A. Hengel Jian Yang Qingming Huang 90 6 0 12 Dec 2024
Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer Jiahao Cui Hui Li Yun Zhan Hanlin Shang K. Cheng Yuqi Ma Shan Mu Hang Zhou Jingdong Wang Siyu Zhu ViT VGen 102 6 0 01 Dec 2024
Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning Dragos-Alexandru Boldisor Stefan Smeu Dan Oneaţă Elisabeta Oneata 98 1 0 29 Nov 2024
Passive Deepfake Detection Across Multi-modalities: A Comprehensive Survey Hong-Hanh Nguyen-Le Van-Tuan Tran Dinh-Thuc Nguyen Nhien-An Le-Khac AAML 107 1 0 26 Nov 2024
DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video Generation Hanbo Cheng Limin Lin Chenyu Liu Pengcheng Xia Pengfei Hu Jiefeng Ma Jun Du Jia Pan DiffM VGen 116 0 0 17 Oct 2024
Beyond Fixed Topologies: Unregistered Training and Comprehensive Evaluation Metrics for 3D Talking Heads Federico Nocentini T. Besnier Claudio Ferrari Sylvain Arguillere Stefano Berretti Mohamed Daoudi 59 1 0 14 Oct 2024
Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animation Jiahao Cui Hui Li Yao Yao Hao Zhu Hanlin Shang Kaihui Cheng Hang Zhou Siyu Zhu Jingdong Wang DiffM VGen 38 22 0 10 Oct 2024
EmoGene: Audio-Driven Emotional 3D Talking-Head Generation Wenqing Wang Yun Fu VGen 76 0 0 07 Oct 2024
LaDTalk: Latent Denoising for Synthesizing Talking Head Videos with High Frequency Details Jian Yang Xukun Wang Wentao Wang Guoming Li Qihang Fang Ruihong Yuan Tianyang Wang Jason Zhaoxin Fan Yeying Jin Zhaoxin Fan VGen 47 1 0 01 Oct 2024
Learning Frame-Wise Emotion Intensity for Audio-Driven Talking-Head Generation Jingyi Xu Hieu Le Zhixin Shu Yang Wang Yi-Hsuan Tsai Dimitris Samaras 29 0 0 29 Sep 2024
A Comprehensive Survey with Critical Analysis for Deepfake Speech Detection Lam Pham Phat Lam Dat Tran Hieu Tang Tin Nguyen Alexander Schindler Canh Vu Alexander Polonsky Canh Vu 46 3 0 23 Sep 2024
Multi-modal Speech Transformer Decoders: When Do Multiple Modalities Improve Accuracy? Yiwen Guan V. Trinh Vivek Voleti Jacob Whitehill 34 1 0 13 Sep 2024
CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attention Gaojie Lin Jianwen Jiang Chao Liang Tianyun Zhong Jiaqi Yang Yanbo Zheng VGen DiffM 61 13 0 03 Sep 2024
High-fidelity and Lip-synced Talking Face Synthesis via Landmark-based Diffusion Model Weizhi Zhong Junfan Lin Peixin Chen Liang Lin Guanbin Li 34 1 0 10 Aug 2024
Landmark-guided Diffusion Model for High-fidelity and Temporally Coherent Talking Head Generation Jintao Tan Xize Cheng Lingyu Xiong Lei Zhu Xiandong Li Xianjia Wu Kai Gong Minglei Li Yi Cai DiffM 28 2 0 03 Aug 2024
Learning Online Scale Transformation for Talking Head Video Generation Fa-Ting Hong Dan Xu 58 1 0 13 Jul 2024
MobilePortrait: Real-Time One-Shot Neural Head Avatars on Mobile Devices Jianwen Jiang Gaojie Lin Zhengkun Rong Chao Liang Yongming Zhu Jiaqi Yang Tianyun Zhong 3DH 80 8 0 08 Jul 2024
Towards Attention-based Contrastive Learning for Audio Spoof Detection C. Goel Surya Koppisetti Ben Colman Ali Shahriyari Gaurav Bharaj 52 5 0 03 Jul 2024
NLDF: Neural Light Dynamic Fields for Efficient 3D Talking Head Generation Niu Guanchen 3DH 39 0 0 17 Jun 2024
Controllable Talking Face Generation by Implicit Facial Keypoints Editing Dong Zhao Jiaying Shi Wenjun Li Shudong Wang Shenghui Xu Zhaoming Pan CVBM 43 0 0 05 Jun 2024
Learn2Talk: 3D Talking Face Learns from 2D Talking Face Yixiang Zhuang Baoping Cheng Yao Cheng Yuntao Jin Renshuai Liu Chengyang Li Xuan Cheng Jing Liao Juncong Lin CVBM 3DH 29 6 0 19 Apr 2024
THQA: A Perceptual Quality Assessment Database for Talking Heads Yingjie Zhou Zicheng Zhang Wei Sun Xiaohong Liu Xiongkuo Min Zhihua Wang Xiao-Ping Zhang Guangtao Zhai EGVM 29 10 0 13 Apr 2024
Translation-based Video-to-Video Synthesis Pratim Saha Chengcui Zhang DiffM 23 1 0 03 Apr 2024
Talk3D: High-Fidelity Talking Portrait Synthesis via Personalized 3D Generative Prior Jaehoon Ko Kyusun Cho Joungbin Lee Heeji Yoon Sangmin Lee Sangjun Ahn Seungryong Kim 26 2 0 29 Mar 2024
DiffusionAct: Controllable Diffusion Autoencoder for One-shot Face Reenactment Stella Bounareli Christos Tzelepis Vasileios Argyriou Ioannis Patras Georgios Tzimiropoulos DiffM 43 7 0 25 Mar 2024
Context-aware Talking Face Video Generation Meidai Xuanyuan Yuwang Wang Honglei Guo Qionghai Dai DiffM 29 0 0 28 Feb 2024
Lips Are Lying: Spotting the Temporal Inconsistency between Audio and Visual in Lip-Syncing DeepFakes Weifeng Liu Tianyi She Jiawei Liu Run Wang Dongyu Yao Ziyou Liang 28 5 0 28 Jan 2024
AE-NeRF: Audio Enhanced Neural Radiance Field for Few Shot Talking Head Synthesis Dongze Li Kang Zhao Wei Wang Bo Peng Yingya Zhang Jing Dong Tien-Ping Tan DiffM VGen 27 12 0 18 Dec 2023
GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits Yibo Xia Lizhen Wang Xiang Deng Xiaoyan Luo Yunhong Wang Yebin Liu VGen 33 1 0 12 Dec 2023
Neural Text to Articulate Talk: Deep Text to Audiovisual Speech Synthesis achieving both Auditory and Photo-realism Georgios Milis P. Filntisis A. Roussos Petros Maragos CVBM 26 2 0 11 Dec 2023
Unsupervised Multimodal Deepfake Detection Using Intra- and Cross-Modal Inconsistencies Mulin Tian Mahyar Khayatkhoei Joe Mathai Wael AbdAlmageed 23 6 0 28 Nov 2023
AVTENet: Audio-Visual Transformer-based Ensemble Network Exploiting Multiple Experts for Video Deepfake Detection Ammarah Hashmi Sahibzada Adil Shahzad Chia-Wen Lin Yu Tsao Hsin-Min Wang ViT 40 6 0 19 Oct 2023