FaceFormer: Speech-Driven 3D Facial Animation with Transformers

10 December 2021

Yingruo Fan

Zhaojiang Lin

Jun Saito

Wenping Wang

Taku Komura

CVBM

ArXiv PDF HTML

Papers citing "FaceFormer: Speech-Driven 3D Facial Animation with Transformers"

50 / 131 papers shown

Title
OT-Talk: Animating 3D Talking Head with Optimal Transportation Xinmu Wang Xiang Gao Xiyun Song Heather Yu Zongfang Lin Liang Peng Xianfeng Gu 10 0 0 03 May 2025
Model See Model Do: Speech-Driven Facial Animation with Style Control Yifang Pan Karan Singh Luiz Gustavo Hafemann DiffM 48 0 0 02 May 2025
Efficient Listener: Dyadic Facial Motion Synthesis via Action Diffusion Z. Wang Alexandre Bruckert P. Le Callet Guangtao Zhai VGen 32 0 0 29 Apr 2025
Supervising 3D Talking Head Avatars with Analysis-by-Audio-Synthesis Radek Daněček Carolin Schmitt Senya Polikovsky Michael J. Black 32 0 0 18 Apr 2025
EchoMask: Speech-Queried Attention-based Mask Modeling for Holistic Co-Speech Motion Generation Xiangyue Zhang Jianfang Li Jiaxu Zhang Jianqiang Ren Liefeng Bo Zhigang Tu 20 0 0 12 Apr 2025
EasyGenNet: An Efficient Framework for Audio-Driven Gesture Video Generation Based on Diffusion Model Renda Li Xiaohua Qi Q. Ling Jun Yu Ziyi Chen Peng Chang Mei HanJing Xiao DiffM VGen 45 0 0 11 Apr 2025
Perceptually Accurate 3D Talking Head Generation: New Definitions, Speech-Mesh Representation, and Evaluation Metrics Lee Chae-Yeon Oh Hyun-Bin Han EunGi Kim Sung-Bin Suekyeong Nam Tae-Hyun Oh EGVM 3DH 80 0 1 26 Mar 2025
DiffusionTalker: Efficient and Compact Speech-Driven 3D Talking Head via Personalizer-Guided Distillation Peng Chen Xiaobao Wei Ming Lu Hui Chen Feng Tian 39 1 0 23 Mar 2025
PC-Talk: Precise Facial Animation Control for Audio-Driven Talking Face Generation Baiqin Wang Xiangyu Zhu Fan Shen Hao-Xuan Xu Zhen Lei 53 0 0 18 Mar 2025
EmoDiffusion: Enhancing Emotional 3D Facial Animation with Latent Diffusion Models Yixuan Zhang Qing Chang Yuxi Wang Guang Chen Z. Zhang Junran Peng 36 0 0 14 Mar 2025
Cafe-Talk: Generating 3D Talking Face Animation with Multimodal Coarse- and Fine-grained Control Hejia Chen Haoxian Zhang Shoulong Zhang Xiaoqiang Liu Sisi Zhuang Yuan Zhang Pengfei Wan Di Zhang Shuai Li 54 1 0 14 Mar 2025
Towards High-fidelity 3D Talking Avatar with Personalized Dynamic Texture X. Li Jianyu Wang Yuhao Cheng Yikun Zeng X. Ren W. Zhu Weiming Zhao Yichao Yan 31 0 0 01 Mar 2025
Skinned Motion Retargeting with Dense Geometric Interaction Perception Zijie Ye Jia-Wei Liu Jia Jia Shikun Sun Mike Zheng Shou 39 2 0 17 Jan 2025
Joint Co-Speech Gesture and Expressive Talking Face Generation using Diffusion with Adapters S. Hogue Chenxu Zhang Yapeng Tian Xiaohu Guo DiffM 64 0 0 18 Dec 2024
D $^2$ -World: An Efficient World Model through Decoupled Dynamic Flow Haiming Zhang Xu Yan Ying Xue Zixuan Guo Shuguang Cui Z. Li Bingbing Liu 59 0 0 26 Nov 2024
Takin-ADA: Emotion Controllable Audio-Driven Animation with Canonical and Landmark Loss Optimization Bin Lin Yanzhen Yu Jianhao Ye Ruitao Lv Y. Yang Ruoye Xie Pan Yu Hongbin Zhou VGen 30 1 0 18 Oct 2024
Beyond Fixed Topologies: Unregistered Training and Comprehensive Evaluation Metrics for 3D Talking Heads Federico Nocentini T. Besnier Claudio Ferrari Sylvain Arguillere Stefano Berretti Mohamed Daoudi 56 1 0 14 Oct 2024
MMHead: Towards Fine-grained Multi-modal 3D Facial Animation Sijing Wu Yunhao Li Yichao Yan Huiyu Duan Ziwei Liu Guangtao Zhai 3DH VGen 32 4 0 10 Oct 2024
Diverse Code Query Learning for Speech-Driven Facial Animation Chunzhi Gu Shigeru Kuriyama Katsuya Hotta DiffM 21 0 0 27 Sep 2024
FastTalker: Jointly Generating Speech and Conversational Gestures from Text Zixin Guo Jian Zhang 24 1 0 24 Sep 2024
MIMAFace: Face Animation via Motion-Identity Modulated Appearance Feature Learning Yue Han Junwei Zhu Yuxiang Feng Xiaozhong Ji Keke He Xiangtai Li Zhucun Xue Yong Liu 21 0 0 23 Sep 2024
PainDiffusion: Learning to Express Pain Quang Tien Dam Tri Tung Nguyen Nguyen D. Tran Joo-Ho Lee Joo-Ho Lee VGen 20 0 0 18 Sep 2024
3DFacePolicy: Speech-Driven 3D Facial Animation with Diffusion Policy Xuanmeng Sha Liyun Zhang Tomohiro Mashita Yuki Uranishi VGen 18 0 0 17 Sep 2024
ProbTalk3D: Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAE Sichun Wu Kazi Injamamul Haque Zerrin Yumak VGen 28 2 0 12 Sep 2024
TalkLoRA: Low-Rank Adaptation for Speech-Driven Animation Jack D. Saunders Vinay P. Namboodiri 43 1 0 25 Aug 2024
T3M: Text Guided 3D Human Motion Synthesis from Speech Wenshuo Peng Kaipeng Zhang Sai Qian Zhang 20 0 0 23 Aug 2024
DEGAS: Detailed Expressions on Full-Body Gaussian Avatars Zhijing Shao D. B. Wang Qing-Yao Tian Yao-Dong Yang Hengyu Meng Zeyu Cai Bo Dong Yu Zhang Kang Zhang Z. Wang 3DGS 34 3 0 20 Aug 2024
Combo: Co-speech holistic 3D human motion generation and efficient customizable adaptation in harmony Chao Xu Mingze Sun Zhi-Qi Cheng Fei-Yue Wang Yang Liu Baigui Sun Ruqi Huang Alexander G. Hauptmann VGen 32 2 0 18 Aug 2024
FD2Talk: Towards Generalized Talking Head Generation with Facial Decoupled Diffusion Model Ziyu Yao Xuxin Cheng Zhiqi Huang DiffM 19 3 0 18 Aug 2024
Content and Style Aware Audio-Driven Facial Animation Qingju Liu Hyeongwoo Kim Gaurav Bharaj DiffM 16 1 0 13 Aug 2024
ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performer Jiazhi Guan Zhiliang Xu Hang Zhou Kaisiyuan Wang Shengyi He ... Errui Ding Jingtuo Liu Jingdong Wang Youjian Zhao Ziwei Liu VGen 36 2 0 06 Aug 2024
GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer Yihong Lin Zhaoxin Fan Lingyu Xiong Liang Peng Xiandong Li Wenxiong Kang Xianjia Wu Songju Lei Huang Xu 27 3 0 03 Aug 2024
JambaTalk: Speech-Driven 3D Talking Head Generation Based on Hybrid Transformer-Mamba Language Model Farzaneh Jafari Stefano Berretti Anup Basu Mamba 29 1 0 03 Aug 2024
UniTalker: Scaling up Audio-Driven 3D Facial Animation through A Unified Model Xiangyu Fan Jiaqi Li Zhiqian Lin Weiye Xiao Lei Yang CVBM VGen 26 3 0 01 Aug 2024
The impact of differences in facial features between real speakers and 3D face models on synthesized lip motions Rabab Algadhy Yoshihiko Gotoh Steve Maddock CVBM 13 0 0 24 Jul 2024
EmoFace: Audio-driven Emotional 3D Face Animation Chang Liu Qunfen Lin Zijiao Zeng Ye Pan CVBM 19 4 0 17 Jul 2024
LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control Jianzhu Guo Dingyun Zhang Xiaoqiang Liu Zhizhou Zhong Yuan Zhang Pengfei Wan Di Zhang VGen 56 52 0 03 Jul 2024
Enhancing Speech-Driven 3D Facial Animation with Audio-Visual Guidance from Lip Reading Expert Han EunGi Oh Hyun-Bin Kim Sung-Bin Corentin Nivelet Etcheberry Suekyeong Nam Janghoon Joo Tae-Hyun Oh 18 5 0 01 Jul 2024
MultiTalk: Enhancing 3D Talking Head Generation Across Languages with Multilingual Video Dataset Kim Sung-Bin Lee Chae-Yeon Gihun Son Oh Hyun-Bin Janghoon Ju Suekyeong Nam Tae-Hyun Oh 19 11 0 20 Jun 2024
A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing Ming Meng Yufei Zhao Bo Zhang Yonggui Zhu Weimin Shi Maxwell Wen Zhaoxin Fan VGen 29 1 0 15 Jun 2024
PianoMotion10M: Dataset and Benchmark for Hand Motion Generation in Piano Performance Qijun Gan Song Wang Shengtao Wu Jianke Zhu 50 1 0 13 Jun 2024
Emotional Conversation: Empowering Talking Faces with Cohesive Expression, Gaze and Pose Generation Jiadong Liang Feng Lu CVBM 21 0 0 12 Jun 2024
DNPM: A Neural Parametric Model for the Synthesis of Facial Geometric Details Haitao Cao Baoping Cheng Qiran Pu Haocheng Zhang Bin Luo Yixiang Zhuang Juncong Lin Liyan Chen Xuan Cheng 3DH 33 0 0 30 May 2024
InterAct: Capture and Modelling of Realistic, Expressive and Interactive Activities between Two Persons in Daily Scenarios Yinghao Huang Leo Ho Dafei Qin Mingyi Shi Taku Komura VGen 27 1 0 19 May 2024
AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encoding Tao Liu Feilong Chen Shuai Fan Chenpeng Du Qi Chen Xie Chen Kai Yu DiffM PINN 34 25 0 06 May 2024
Embedded Representation Learning Network for Animating Styled Video Portrait Tianyong Wang Xiangyu Liang Wangguandong Zheng Dan Niu Haifeng Xia Siyu Xia 3DH 16 0 0 29 Apr 2024
CSTalk: Correlation Supervised Speech-driven 3D Emotional Facial Animation Generation Xiangyu Liang Wenlin Zhuang Tianyong Wang Guangxing Geng Guangyue Geng Haifeng Xia Siyu Xia CVBM 25 1 0 29 Apr 2024
Learn2Talk: 3D Talking Face Learns from 2D Talking Face Yixiang Zhuang Baoping Cheng Yao Cheng Yuntao Jin Renshuai Liu Chengyang Li Xuan Cheng Jing Liao Juncong Lin CVBM 3DH 29 6 0 19 Apr 2024
VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time Sicheng Xu Guojun Chen Yu-Xiao Guo Jiaolong Yang Chong Li Zhenyu Zang Yizhong Zhang Xin Tong Baining Guo 32 85 0 16 Apr 2024
3D Face Tracking from 2D Video through Iterative Dense UV to Image Flow Felix Taubner Prashant Raina Mathieu Tuli Eu Wern Teh Chul Lee Jinmiao Huang 3DH CVBM 30 4 0 15 Apr 2024