A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild

23 August 2020

Prajwal K R

Rudrabha Mukhopadhyay

Papers citing "A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild"

50 / 115 papers shown

Title
Audio-Driven Dubbing for User Generated Contents via Style-Aware Semi-Parametric Synthesis Linsen Song Wayne Wu Chaoyou Fu Chen Change Loy Ran He 23 10 0 31 Aug 2023
AI-Generated Content (AIGC) for Various Data Modalities: A Survey Lin Geng Foo Hossein Rahmani J. Liu 73 31 0 27 Aug 2023
Controlling Character Motions without Observable Driving Source Weiyuan Li Bin Dai Ziyi Zhou Qi Yao Baoyuan Wang VGen 6 1 0 11 Aug 2023
A Comprehensive Multi-scale Approach for Speech and Dynamics Synchrony in Talking Head Generation Louis Airale Dominique Vaufreydaz Xavier Alameda-Pineda 23 1 0 04 Jul 2023
Audio-Driven 3D Facial Animation from In-the-Wild Videos Liying Lu Tianke Zhang Yunfei Liu Xuangeng Chu Yu Li VGen 42 3 0 20 Jun 2023
Emotional Speech-Driven Animation with Content-Emotion Disentanglement Radek Danvevcek Kiran Chhatre Shashank Tripathi Yandong Wen Michael J. Black Timo Bolkart 16 65 0 15 Jun 2023
Ada-TTA: Towards Adaptive High-Quality Text-to-Talking Avatar Synthesis Zhe Ye Ziyue Jiang Yi Ren Jinglin Liu Chen Zhang Xiang Yin Zejun Ma Zhou Zhao 40 4 0 06 Jun 2023
A Unified Compression Framework for Efficient Speech-Driven Talking-Face Generation Bo-Kyeong Kim Jaemin Kang Daeun Seo Hancheol Park Shinkook Choi Hyoung-Kyu Song Hyungshin Kim Sungsu Lim 19 0 0 02 Apr 2023
Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert Jiadong Wang Xinyuan Qian Malu Zhang R. Tan Haizhou Li EGVM 22 93 0 29 Mar 2023
MusicFace: Music-driven Expressive Singing Face Synthesis Peng Liu W. Deng Hengda Li Jintai Wang Yinglin Zheng Yiwei Ding Xiaohu Guo Ming Zeng CVBM 30 10 0 24 Mar 2023
ModEFormer: Modality-Preserving Embedding for Audio-Video Synchronization using Transformers Akash Gupta Rohun Tripathi Won-Kap Jang 11 6 0 21 Mar 2023
Reference-Guided Large-Scale Face Inpainting with Identity and Texture Control Wuyang Luo Su Yang Weishan Zhang DiffM 40 10 0 13 Mar 2023
Progressive Open Space Expansion for Open-Set Model Attribution Tianyun Yang Danding Wang Fan Tang Xinying Zhao Juan Cao Sheng Tang 29 17 0 13 Mar 2023
READ Avatars: Realistic Emotion-controllable Audio Driven Avatars Jack D. Saunders Vinay P. Namboodiri VGen 21 11 0 01 Mar 2023
GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis Zhenhui Ye Ziyue Jiang Yi Ren Jinglin Liu Jinzheng He Zhou Zhao CVBM 20 122 0 31 Jan 2023
What Decreases Editing Capability? Domain-Specific Hybrid Refinement for Improved GAN Inversion Pu Cao Lu Yang Dongxu Liu Zhiwei Liu Shan Li Q. Song 27 6 0 28 Jan 2023
DPE: Disentanglement of Pose and Expression for General Video Portrait Editing Youxin Pang Yong Zhang Weize Quan Yanbo Fan Xiaodong Cun Ying Shan Dong-ming Yan VGen 24 33 0 16 Jan 2023
Memories are One-to-Many Mapping Alleviators in Talking Face Generation Anni Tang Tianyu He Xuejiao Tan Jun Ling Liang Song CVBM 26 23 0 09 Dec 2022
Learning to Dub Movies via Hierarchical Prosody Models Gaoxiang Cong Liang Li Yuankai Qi Zhengjun Zha Qi Wu Wen-yu Wang Bin Jiang Ming Yang Qin Huang 62 25 0 08 Dec 2022
Audio-Driven Co-Speech Gesture Video Generation Xian Liu Qianyi Wu Hang Zhou Yuanqi Du Wayne Wu Dahua Lin Ziwei Liu SLR VGen 23 47 0 05 Dec 2022
FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection Gil Knafo Ohad Fried 21 5 0 01 Dec 2022
VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wild K. Cheng Xiaodong Cun Yong Zhang Menghan Xia Fei Yin Mingrui Zhu Xuanxia Wang Jue Wang Nan Wang CVBM 25 92 0 27 Nov 2022
Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decomposition Jiaxiang Tang Kaisiyuan Wang Hang Zhou Xiaokang Chen Dongliang He Tianshu Hu Jingtuo Liu Gang Zeng Jingdong Wang 3DH 18 76 0 22 Nov 2022
SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation Wenxuan Zhang Xiaodong Cun Xuan Wang Yong Zhang Xiaodong Shen Yu-Xiao Guo Ying Shan Fei-Yue Wang VGen 29 232 0 22 Nov 2022
SPACE: Speech-driven Portrait Animation with Controllable Expression Siddharth Gururani Arun Mallya Ting-Chun Wang Rafael Valle Ming-Yu Liu VGen 26 45 0 17 Nov 2022
MARLIN: Masked Autoencoder for facial video Representation LearnINg Zhixi Cai Shreya Ghosh Kalin Stefanov Abhinav Dhall Jianfei Cai Hamid Rezatofighi Reza Haffari Munawar Hayat ViT CVBM 20 60 0 12 Nov 2022
SyncTalkFace: Talking Face Generation with Precise Lip-Syncing via Audio-Lip Memory Se Jin Park Minsu Kim Joanna Hong J. Choi Y. Ro CVBM 19 85 0 02 Nov 2022
Multimodal Transformer Distillation for Audio-Visual Synchronization Xuan-Bo Chen Haibin Wu Chung-Che Wang Hung-yi Lee J. Jang 24 3 0 27 Oct 2022
Pre-Avatar: An Automatic Presentation Generation Framework Leveraging Talking Avatar Aolan Sun Xulong Zhang Tiandong Ling Jianzong Wang Ning Cheng Jing Xiao 24 4 0 13 Oct 2022
Compressing Video Calls using Synthetic Talking Heads Madhav Agarwal Anchit Gupta Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar 17 10 0 07 Oct 2022
A Keypoint Based Enhancement Method for Audio Driven Free View Talking Head Synthesis Yichen Han Ya Li Yingming Gao Jinlong Xue Songpo Wang Lei Yang 11 2 0 07 Oct 2022
Audio-Visual Face Reenactment Madhav Agarwal Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar DiffM VGen 19 22 0 06 Oct 2022
Facial Expression Translation using Landmark Guided GANs Hao Tang N. Sebe CVBM 16 3 0 05 Sep 2022
StableFace: Analyzing and Improving Motion Stability for Talking Face Generation Jun Ling Xuejiao Tan Liyang Chen Runnan Li Yuchao Zhang Sheng Zhao Liang Song CVBM 34 13 0 29 Aug 2022
DualVoice: Speech Interaction that Discriminates between Normal and Whispered Voice Input Jun Rekimoto 22 6 0 22 Aug 2022
Extreme-scale Talking-Face Video Upsampling with Audio-Visual Priors Sindhu B. Hegde Rudrabha Mukhopadhyay Vinay P. Namboodiri C. V. Jawahar CVBM 16 1 0 17 Aug 2022
Video Manipulations Beyond Faces: A Dataset with Human-Machine Analysis Trisha Mittal Ritwik Sinha Viswanathan Swaminathan John Collomosse Dinesh Manocha 14 9 0 26 Jul 2022
Audio-driven Neural Gesture Reenactment with Video Motion Graphs Yang Zhou Jimei Yang Dingzeyu Li Jun Saito Deepali Aneja E. Kalogerakis DiffM SLR 34 20 0 23 Jul 2022
Object-Compositional Neural Implicit Surfaces Qianyi Wu Xian Liu Yuedong Chen Kejie Li Chuanxia Zheng Jianfei Cai Jianmin Zheng OCL AI4CE 22 73 0 20 Jul 2022
Cut Inner Layers: A Structured Pruning Strategy for Efficient U-Net GANs Bo-Kyeong Kim Shinkook Choi Hancheol Park 13 4 0 29 Jun 2022
EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model Xinya Ji Hang Zhou Kaisiyuan Wang Qianyi Wu Wayne Wu Feng Xu Xun Cao CVBM 54 157 0 30 May 2022
Deep Learning and Synthetic Media Raphaël Millière 18 18 0 11 May 2022
Audio-Visual Person-of-Interest DeepFake Detection D. Cozzolino Alessandro Pianese Matthias Nießner L. Verdoliva 28 59 0 06 Apr 2022
VocaLiST: An Audio-Visual Synchronisation Model for Lips and Voices V. S. Kadandale Juan F. Montesinos G. Haro 19 23 0 05 Apr 2022
On the role of Lip Articulation in Visual Speech Perception Zakaria Aldeneh Masha Fedzechkina Skyler Seto Katherine Metcalf Miguel Sarabia N. Apostoloff B. Theobald 19 1 0 18 Mar 2022
StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGAN Fei Yin Yong Zhang Xiaodong Cun Ming Cao Yanbo Fan Xuanxia Wang Qingyan Bai Baoyuan Wu Jue Wang Yujiu Yang CVBM 31 171 0 08 Mar 2022
Attention-Based Lip Audio-Visual Synthesis for Talking Face Generation in the Wild Gang Wang Peng Zhang Lei Xie Wei Huang Yufei Zha CVBM 19 14 0 08 Mar 2022
Freeform Body Motion Generation from Speech Jing-Fen Xu Wei Zhang Yalong Bai Qi-Biao Sun Tao Mei SLR 31 18 0 04 Mar 2022
Audio-Driven Talking Face Video Generation with Dynamic Convolution Kernels Zipeng Ye Mengfei Xia Ran Yi Juyong Zhang Yu-Kun Lai Xuanteng Huang Guoxin Zhang Yong-jin Liu CVBM 22 39 0 16 Jan 2022
DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering Shunyu Yao Ruizhe Zhong Yichao Yan Guangtao Zhai Xiaokang Yang CVBM 19 90 0 03 Jan 2022