FaceFormer: Speech-Driven 3D Facial Animation with Transformers

10 December 2021

Yingruo Fan

Zhaojiang Lin

Jun Saito

Wenping Wang

Taku Komura

CVBM

ArXiv PDF HTML

Papers citing "FaceFormer: Speech-Driven 3D Facial Animation with Transformers"

31 / 131 papers shown

Title
REACT2023: the first Multi-modal Multiple Appropriate Facial Reaction Generation Challenge Siyang Song Micol Spitale Cheng Luo Germán Barquero Cristina Palmero ... M. Valstar Tobias Baur F. Ringeval E. André Hatice Gunes 25 10 0 11 Jun 2023
Learning Landmarks Motion from Speech for Speaker-Agnostic 3D Talking Heads Generation Federico Nocentini Claudio Ferrari Stefano Berretti 14 7 0 02 Jun 2023
ReactFace: Multiple Appropriate Facial Reaction Generation in Dyadic Interactions Cheng Luo Siyang Song Weicheng Xie Micol Spitale Linlin Shen Hatice Gunes CVBM DiffM 28 9 0 25 May 2023
Reversible Graph Neural Network-based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation Tong Bill Xu Micol Spitale Haozhan Tang Lu Liu Hatice Gunes Siyang Song CVBM 19 10 0 24 May 2023
An Android Robot Head as Embodied Conversational Agent Marcel Heisler C. Becker-Asano LM&Ro LLMAG 11 0 0 18 May 2023
StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based Generator Jiazhi Guan Zhanwang Zhang Hang Zhou Tianshu Hu Kaisiyuan Wang ... Haocheng Feng Jingtuo Liu Errui Ding Ziwei Liu Jingdong Wang 35 57 0 09 May 2023
HACK: Learning a Parametric Head and Neck Model for High-fidelity Animation Longwen Zhang Zijun Zhao Xinzhou Cong Qixuan Zhang Shuqian Gu Yuchong Gao Ruichen Zheng Wei Yang Lan Xu Jingyi Yu 3DH 22 8 0 08 May 2023
AVFace: Towards Detailed Audio-Visual 4D Face Reconstruction Aggelina Chatziagapi Dimitris Samaras 3DH CVBM 8 3 0 25 Apr 2023
DAE-Talker: High Fidelity Speech-Driven Talking Face Generation with Diffusion Autoencoder Chenpeng Du Qi Chen Xie Chen K. Yu DiffM 11 50 0 30 Mar 2023
Robust Dancer: Long-term 3D Dance Synthesis Using Unpaired Data Bin Feng Tenglong Ao Zequn Liu Wei Ju Libin Liu Ming Zhang 22 3 0 29 Mar 2023
Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert Jiadong Wang Xinyuan Qian Malu Zhang R. Tan Haizhou Li EGVM 17 92 0 29 Mar 2023
EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face Animation Ziqiao Peng Hao Wu Zhenbo Song Hao-Xuan Xu Xiangyu Zhu Jun He Hongyan Liu Zhaoxin Fan CVBM 8 99 0 20 Mar 2023
MMFace4D: A Large-Scale Multi-Modal 4D Face Dataset for Audio-Driven 3D Face Animation Haozhe Wu Jia Jia Junliang Xing Hongwei Xu Xiangyuan Wang Jelo Wang CVBM 6 7 0 17 Mar 2023
FaceXHuBERT: Text-less Speech-driven E(X)pressive 3D Facial Animation Synthesis Using Self-Supervised Speech Representation Learning Kazi Injamamul Haque Zerrin Yumak 6 23 0 09 Mar 2023
Improving Few-Shot Learning for Talking Face System with TTS Data Augmentation Qi Chen Ziyang Ma Tao Liu Xuejiao Tan Qu Lu Xie Chen K. Yu CVBM 35 5 0 09 Mar 2023
Pose-Controllable 3D Facial Animation Synthesis using Hierarchical Audio-Vertex Attention Bin Liu Xiaolin K. Wei Bo Li Junjie Cao Yunyu Lai CVBM 6 1 0 24 Feb 2023
Learning Audio-Driven Viseme Dynamics for 3D Face Animation Linchao Bao Haoxian Zhang Yue Qian Tangli Xue Changan Chen Xuefei Zhe Di Kang 3DH 15 12 0 15 Jan 2023
CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior Jinbo Xing Menghan Xia Yuechen Zhang Xiaodong Cun Jue Wang T. Wong 24 134 0 06 Jan 2023
Expressive Speech-driven Facial Animation with controllable emotions Yutong Chen Junhong Zhao Weiqiang Zhang 17 8 0 05 Jan 2023
Imitator: Personalized Speech-driven 3D Facial Animation Balamurugan Thambiraja I. Habibie S. Aliakbarian Darren Cosker Christian Theobalt Justus Thies CVBM 23 49 0 30 Dec 2022
Generating Holistic 3D Human Motion from Speech Hongwei Yi Hualin Liang Yifei Liu Qiong Cao Yandong Wen Timo Bolkart Dacheng Tao Michael J. Black SLR 21 137 0 08 Dec 2022
SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animation Wenxuan Zhang Xiaodong Cun Xuan Wang Yong Zhang Xiaodong Shen Yu-Xiao Guo Ying Shan Fei-Yue Wang VGen 25 227 0 22 Nov 2022
SPACE: Speech-driven Portrait Animation with Controllable Expression Siddharth Gururani Arun Mallya Ting-Chun Wang Rafael Valle Ming-Yu Liu VGen 10 45 0 17 Nov 2022
Autoregressive GAN for Semantic Unconditional Head Motion Generation Louis Airale Xavier Alameda-Pineda Stéphane Lathuilière Dominique Vaufreydaz 17 3 0 02 Nov 2022
StableFace: Analyzing and Improving Motion Stability for Talking Face Generation Jun Ling Xuejiao Tan Liyang Chen Runnan Li Yuchao Zhang Sheng Zhao Liang Song CVBM 23 12 0 29 Aug 2022
Multimodal Learning with Transformers: A Survey P. Xu Xiatian Zhu David A. Clifton ViT 41 518 0 13 Jun 2022
Deep Learning for Visual Speech Analysis: A Survey Changchong Sheng Gangyao Kuang L. Bai Chen Hou Y. Guo Xin Xu M. Pietikäinen Li Liu VLM 13 33 0 22 May 2022
TEMOS: Generating diverse human motions from textual descriptions Mathis Petrovich Michael J. Black Gül Varol 40 365 0 25 Apr 2022
Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation Ofir Press Noah A. Smith M. Lewis 242 690 0 27 Aug 2021
3D-TalkEmo: Learning to Synthesize 3D Emotional Talking Head Qianyun Wang Zhenfeng Fan Shi-hong Xia 3DH 45 17 0 25 Apr 2021
Transformers in Vision: A Survey Salman Khan Muzammal Naseer Munawar Hayat Syed Waqas Zamir F. Khan M. Shah ViT 225 2,404 0 04 Jan 2021