FaceFormer: Speech-Driven 3D Facial Animation with Transformers

10 December 2021

Yingruo Fan

Zhaojiang Lin

Jun Saito

Wenping Wang

Taku Komura

CVBM

ArXiv PDF HTML

Papers citing "FaceFormer: Speech-Driven 3D Facial Animation with Transformers"

50 / 131 papers shown

Title
Towards Variable and Coordinated Holistic Co-Speech Motion Generation Yifei Liu Qiong Cao Yandong Wen Huaiguang Jiang Changxing Ding SLR 63 13 0 30 Mar 2024
Beyond Talking -- Generating Holistic 3D Human Dyadic Motion for Communication Mingze Sun Chao Xu Xinyu Jiang Yang Liu Baigui Sun Ruqi Huang 41 3 0 28 Mar 2024
AnimateMe: 4D Facial Expressions via Diffusion Models Dimitrios Gerogiannis Foivos Paraperas-Papantoniou Rolandos Alexandros Potamias Alexandros Lattas Stylianos Moschoglou Stylianos Ploumpis S. Zafeiriou 22 3 0 25 Mar 2024
EmoVOCA: Speech-Driven Emotional 3D Talking Heads Federico Nocentini Claudio Ferrari Stefano Berretti 48 4 0 19 Mar 2024
Driving Animatronic Robot Facial Expression From Speech Boren Li Hang Li Hangxin Liu 26 0 0 19 Mar 2024
ScanTalk: 3D Talking Heads from Unregistered Scans Federico Nocentini T. Besnier Claudio Ferrari Sylvain Arguillere Stefano Berretti Mohamed Daoudi 30 5 0 16 Mar 2024
MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Models Zunnan Xu Yukang Lin Haonan Han Sicheng Yang Ronghui Li Yachao Zhang Xiu Li Mamba 46 25 0 14 Mar 2024
Dyadic Interaction Modeling for Social Behavior Generation Minh Tran Di Chang Maksim Siniukov Mohammad Soleymani VGen 21 0 0 14 Mar 2024
VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis Enric Corona Andrei Zanfir Eduard Gabriel Bazavan Nikos Kolotouros Thiemo Alldieck C. Sminchisescu VGen DiffM 27 25 0 13 Mar 2024
EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions Linrui Tian Qi Wang Bang Zhang Liefeng Bo DiffM 58 101 0 27 Feb 2024
Learning Dynamic Tetrahedra for High-Quality Talking Head Synthesis Zicheng Zhang Ruobing Zheng Ziwen Liu Congying Han Tianqi Li Meng Wang Tiande Guo Jingdong Chen Bonan Li Ming Yang 3DH 24 5 0 27 Feb 2024
AVI-Talking: Learning Audio-Visual Instructions for Expressive 3D Talking Face Generation Yasheng Sun Wenqing Chu Hang Zhou Kaisiyuan Wang Hideki Koike 32 5 0 25 Feb 2024
DiffSpeaker: Speech-Driven 3D Facial Animation with Diffusion Transformer Zhiyuan Ma Xiangyu Zhu Guojun Qi Chen Qian Zhaoxiang Zhang Zhen Lei 19 9 0 08 Feb 2024
Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance Qingcheng Zhao Pengyu Long Qixuan Zhang Dafei Qin Hanming Liang Longwen Zhang Yingliang Zhang Jingyi Yu Lan Xu DiffM 3DH 25 25 0 28 Jan 2024
DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generation Junming Chen Yunfei Liu Jianan Wang Ailing Zeng Yu Li Qifeng Chen VGen 25 28 0 09 Jan 2024
EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling Haiyang Liu Zihao Zhu Giorgio Becherini Yichen Peng Mingyang Su You Zhou Xuefei Zhe Naoya Iwamoto Bo Zheng Michael J. Black SLR 23 29 0 31 Dec 2023
SAiD: Speech-driven Blendshape Facial Animation with Diffusion Inkyu Park Jaewoong Cho 26 4 0 25 Dec 2023
Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animation Hui Fu Zeqing Wang Ke Gong Keze Wang Tianshui Chen Haojie Li Haifeng Zeng Wenxiong Kang 25 10 0 18 Dec 2023
FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models Shivangi Aneja Justus Thies Angela Dai Matthias Nießner DiffM VGen 24 29 0 13 Dec 2023
GSmoothFace: Generalized Smooth Talking Face Generation via Fine Grained 3D Face Guidance Haiming Zhang Zhihao Yuan Chaoda Zheng Xu Yan Baoyuan Wang Guanbin Li Song Wu Shuguang Cui Zhen Li CVBM 36 1 0 12 Dec 2023
GMTalker: Gaussian Mixture-based Audio-Driven Emotional Talking Video Portraits Yibo Xia Lizhen Wang Xiang Deng Xiaoyan Luo Yunhong Wang Yebin Liu VGen 33 1 0 12 Dec 2023
Neural Text to Articulate Talk: Deep Text to Audiovisual Speech Synthesis achieving both Auditory and Photo-realism Georgios Milis P. Filntisis A. Roussos Petros Maragos CVBM 24 2 0 11 Dec 2023
SingingHead: A Large-scale 4D Dataset for Singing Head Animation Sijing Wu Yunhao Li Weitian Zhang Jun Jia Yucheng Zhu Yichao Yan Guangtao Zhai Xiaokang Yang 41 2 0 07 Dec 2023
PMMTalk: Speech-Driven 3D Facial Animation from Complementary Pseudo Multi-modal Features Tianshun Han Shengnan Gui Yiqing Huang Baihui Li Lijian Liu ... Quan Lu Ruicong Zhi Yanyan Liang Du Zhang Jun Wan VGen 16 1 0 05 Dec 2023
3DiFACE: Diffusion-based Speech-driven 3D Facial Animation and Editing Balamurugan Thambiraja S. Aliakbarian Darren Cosker Justus Thies DiffM VGen 38 10 0 01 Dec 2023
Probabilistic Speech-Driven 3D Facial Motion Synthesis: New Benchmarks, Methods, and Applications Karren D. Yang Anurag Ranjan Jen-Hao Rick Chang Raviteja Vemulapalli Oncel Tuzel 14 8 0 30 Nov 2023
SpeechAct: Towards Generating Whole-body Motion from Speech Jinsong Zhang Minjie Zhu Yuxiang Zhang Yebin Liu Kun Li 21 0 0 29 Nov 2023
DiffusionTalker: Personalization and Acceleration for Speech-Driven 3D Face Diffuser Peng Chen Xiaobao Wei Ming Lu Yitong Zhu Nai-Ming Yao Xingyu Xiao Hui Chen 18 11 0 28 Nov 2023
DualTalker: A Cross-Modal Dual Learning Approach for Speech-Driven 3D Facial Animation Guinan Su Yanwu Yang Zhifeng Li VGen 20 1 0 08 Nov 2023
DiffDub: Person-generic Visual Dubbing Using Inpainting Renderer with Diffusion Auto-encoder Tao Liu Chenpeng Du Shuai Fan Feilong Chen Kai Yu DiffM VGen 9 6 0 03 Nov 2023
LaughTalk: Expressive 3D Talking Head Generation with Laughter Kim Sung-Bin Lee Hyun Da Hye Hong Suekyeong Nam Janghoon Ju Tae-Hyun Oh 20 20 0 02 Nov 2023
Breathing Life into Faces: Speech-driven 3D Facial Animation with Natural Head Pose and Detailed Shape Wei Zhao Yijun Wang Tianyu He Li-Ping Yin Jianxin Lin Xin Jin 3DH 6 2 0 31 Oct 2023
Orientation-Aware Leg Movement Learning for Action-Driven Human Motion Prediction Chunzhi Gu Chao Zhang Shigeru Kuriyama 32 4 0 23 Oct 2023
CorrTalk: Correlation Between Hierarchical Speech and Facial Activity Variances for 3D Animation Zhaojie Chu K. Guo Xiaofen Xing Yilin Lan Bolun Cai Xiangmin Xu 30 5 0 17 Oct 2023
AdaMesh: Personalized Facial Expressions and Head Poses for Adaptive Speech-Driven 3D Facial Animation Liyang Chen Weihong Bao Shunwei Lei Boshi Tang Zhiyong Wu Shiyin Kang Haozhi Huang Helen M. Meng 35 1 0 11 Oct 2023
DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models Zhiyao Sun Tian Lv Sheng Ye Matthieu Lin Jenny Sheng Yuhui Wen Minjing Yu Yong-jin Liu DiffM 26 44 0 30 Sep 2023
FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion Stefan Stan Kazi Injamamul Haque Zerrin Yumak DiffM 29 53 0 20 Sep 2023
LivelySpeaker: Towards Semantic-Aware Co-Speech Gesture Generation Yihao Zhi Xiaodong Cun Xuelin Chen Xi Shen Wen Guo Shaoli Huang Shenghua Gao 22 26 0 17 Sep 2023
ReliTalk: Relightable Talking Portrait Generation from a Single Video Haonan Qiu Zhaoxi Chen Yuming Jiang Hang Zhou Xiangyu Fan Lei Yang Wayne Wu Ziwei Liu DiffM VGen 24 10 0 05 Sep 2023
ExpCLIP: Bridging Text and Facial Expressions via Semantic Alignment Yicheng Zhong Huawei Wei Pei-Yin Yang Zhisheng Wang CLIP 19 6 0 28 Aug 2023
DF-3DFace: One-to-Many Speech Synchronized 3D Face Animation with Diffusion Se Jin Park Joanna Hong Minsu Kim Y. Ro 22 3 0 23 Aug 2023
Speech-Driven 3D Face Animation with Composite and Regional Facial Movements Haozhe Wu Songtao Zhou Jia Jia Junliang Xing Qi Wen Xiang Wen CVBM 17 14 0 10 Aug 2023
MODA: Mapping-Once Audio-driven Portrait Animation with Dual Attentions Yunfei Liu Lijian Lin Fei Yu Changyin Zhou Yu Li DiffM VGen 26 23 0 19 Jul 2023
FACTS: Facial Animation Creation using the Transfer of Styles Jack D. Saunders Steven Caulkin Vinay P. Namboodiri 3DH CVBM 18 0 0 18 Jul 2023
A Comprehensive Multi-scale Approach for Speech and Dynamics Synchrony in Talking Head Generation Louis Airale Dominique Vaufreydaz Xavier Alameda-Pineda 13 1 0 04 Jul 2023
Audio-Driven 3D Facial Animation from In-the-Wild Videos Liying Lu Tianke Zhang Yunfei Liu Xuangeng Chu Yu Li VGen 40 3 0 20 Jun 2023
EMoG: Synthesizing Emotive Co-speech 3D Gesture with Diffusion Model Li-Ping Yin Yijun Wang Tianyu He Jinming Liu Wei Zhao Bohan Li Xin Jin Jianxin Lin DiffM 11 14 0 20 Jun 2023
SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking Faces Ziqiao Peng Yihao Luo Yue Shi Hao-Xuan Xu Xiangyu Zhu Jun He Hongyan Liu Zhaoxin Fan 47 39 0 19 Jun 2023
Emotional Speech-Driven Animation with Content-Emotion Disentanglement Radek Danvevcek Kiran Chhatre Shashank Tripathi Yandong Wen Michael J. Black Timo Bolkart 14 65 0 15 Jun 2023
Parametric Implicit Face Representation for Audio-Driven Facial Reenactment Ricong Huang Puxiang Lai Yipeng Qin Guanbin Li CVBM DiffM 17 13 0 13 Jun 2023