v1v2 (latest)

MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement

IEEE International Conference on Computer Vision (ICCV), 2021

16 April 2021

ArXiv (abs)PDF HTML Github (384★)

Papers citing "MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement"

45 / 145 papers shown

Title
Unpaired Multi-domain Attribute Translation of 3D Facial Shapes with a Square and Symmetric Geometric MapIEEE International Conference on Computer Vision (ICCV), 2023 Zhenfeng Fan Zhiheng Zhang Shuang Yang Chongyang Zhong Min Cao Shi-hong Xia 3DH 207 2 0 25 Aug 2023
DF-3DFace: One-to-Many Speech Synchronized 3D Face Animation with Diffusion Se Jin Park Joanna Hong Minsu Kim Y. Ro 178 4 0 23 Aug 2023
Speech-Driven 3D Face Animation with Composite and Regional Facial MovementsACM Multimedia (ACM MM), 2023 Haozhe Wu Songtao Zhou Jia Jia Junliang Xing Qi Wen Xiang Wen CVBM 216 21 0 10 Aug 2023
UniBriVL: Robust Universal Representation and Generation of Audio Driven Diffusion Models Sen Fang Bowen Gao Yangjian Wu T. Teoh DiffM 153 1 0 29 Jul 2023
Interactive Conversational Head GenerationIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023 Mohan Zhou Yalong Bai Wei Zhang Tingjun Yao Tiejun Zhao 101 9 0 05 Jul 2023
Audio-Driven 3D Facial Animation from In-the-Wild Videos Liying Lu Tianke Zhang Yunfei Liu Xuangeng Chu Yu Li VGen 128 6 0 20 Jun 2023
SelfTalk: A Self-Supervised Commutative Training Diagram to Comprehend 3D Talking FacesACM Multimedia (ACM MM), 2023 Ziqiao Peng Yihao Luo Yue Shi Hao-Xuan Xu Xiangyu Zhu Jun He Hongyan Liu Zhaoxin Fan 209 62 0 19 Jun 2023
Emotional Speech-Driven Animation with Content-Emotion DisentanglementACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia), 2023 Radek Danvevcek Kiran Chhatre Shashank Tripathi Yandong Wen Michael J. Black Timo Bolkart 236 99 0 15 Jun 2023
REACT2023: the first Multi-modal Multiple Appropriate Facial Reaction Generation Challenge Siyang Song Micol Spitale Cheng Luo German Barquero Cristina Palmero ... Michel Valstar Tobias Baur Fabien Ringeval Elisabeth Andre Hatice Gunes 228 12 0 11 Jun 2023
Learning Landmarks Motion from Speech for Speaker-Agnostic 3D Talking Heads GenerationInternational Conference on Image Analysis and Processing (ICIAP), 2023 Federico Nocentini Claudio Ferrari Stefano Berretti 130 8 0 02 Jun 2023
Reversible Graph Neural Network-based Reaction Distribution Learning for Multiple Appropriate Facial Reactions Generation Tong Xu Micol Spitale Haozhan Tang Lu Liu Hatice Gunes Siyang Song CVBM 265 15 0 24 May 2023
RenderMe-360: A Large Digital Asset Library and Benchmarks Towards High-fidelity Head AvatarsNeural Information Processing Systems (NeurIPS), 2023 Dongwei Pan Long Zhuo Jingtan Piao Huiwen Luo Wei Cheng ... Chen Change Loy Chao Qian Wayne Wu Dahua Lin Kwan-Yee Lin 199 37 0 22 May 2023
An Android Robot Head as Embodied Conversational Agent Marcel Heisler C. Becker-Asano LM&Ro LLMAG 73 2 0 18 May 2023
StyleSync: High-Fidelity Generalized and Personalized Lip Sync in Style-based GeneratorComputer Vision and Pattern Recognition (CVPR), 2023 Jiazhi Guan Zhanwang Zhang Hang Zhou Tianshu Hu Kaisiyuan Wang ... Haocheng Feng Jingtuo Liu Errui Ding Ziwei Liu Jingdong Wang 249 91 0 09 May 2023
Multimodal-driven Talking Face Generation via a Unified Diffusion-based Generator Chao Xu Shaoting Zhu Junwei Zhu Alexander I. Rudnicky Jiangning Zhang Ying Tai Yong Liu DiffM 197 15 0 04 May 2023
AVFace: Towards Detailed Audio-Visual 4D Face ReconstructionComputer Vision and Pattern Recognition (CVPR), 2023 Aggelina Chatziagapi Dimitris Samaras 3DH CVBM 139 5 0 25 Apr 2023
A Complete Survey on Generative AI (AIGC): Is ChatGPT from GPT-4 to GPT-5 All You Need? Chaoning Zhang Chenshuang Zhang Sheng Zheng Yu Qiao Chenghao Li ... Lik-Hang Lee Yang Yang Heng Tao Shen In So Kweon Choong Seon Hong 267 193 0 21 Mar 2023
EmoTalk: Speech-Driven Emotional Disentanglement for 3D Face AnimationIEEE International Conference on Computer Vision (ICCV), 2023 Ziqiao Peng Hao Wu Zhenbo Song Hao-Xuan Xu Xiangyu Zhu Jun He Hongyan Liu Zhaoxin Fan CVBM 325 158 0 20 Mar 2023
Style Transfer for 2D Talking Head Animation Trong-Thang Pham Nhat Le Tuong Khanh Long Do Hung Nguyen Erman Tjiputra Quang-Dieu Tran A. Nguyen 252 3 0 17 Mar 2023
MMFace4D: A Large-Scale Multi-Modal 4D Face Dataset for Audio-Driven 3D Face Animation Haozhe Wu Jia Jia Junliang Xing Hongwei Xu Xiangyuan Wang Jelo Wang CVBM 137 9 0 17 Mar 2023
FaceXHuBERT: Text-less Speech-driven E(X)pressive 3D Facial Animation Synthesis Using Self-Supervised Speech Representation LearningInternational Conference on Multimodal Interaction (ICMI), 2023 Kazi Injamamul Haque Zerrin Yumak 163 39 0 09 Mar 2023
Exploring Efficient-Tuned Learning Audio Representation Method from BriVLInternational Conference on Neural Information Processing (ICONIP), 2023 Sen Fang Yang Wu Bowen Gao Jingwen Cai T. Teoh DiffM 114 1 0 08 Mar 2023
Pose-Controllable 3D Facial Animation Synthesis using Hierarchical Audio-Vertex Attention Yinan Han Xiaolin K. Wei Bo Li Junjie Cao Yunyu Lai CVBM 120 2 0 24 Feb 2023
Learning Audio-Driven Viseme Dynamics for 3D Face Animation Linchao Bao Haoxian Zhang Yue Qian Tangli Xue Changan Chen Xuefei Zhe Di Kang 3DH 156 15 0 15 Jan 2023
Speech Driven Video Editing via an Audio-Conditioned Diffusion ModelImage and Vision Computing (IVC), 2023 Dan Bigioi Shubhajit Basak Michał Stypułkowski Maciej Ziȩba H. Jordan R. Mcdonnell Peter Corcoran DiffM VGen 235 39 0 10 Jan 2023
CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion PriorComputer Vision and Pattern Recognition (CVPR), 2023 Jinbo Xing Menghan Xia Yuechen Zhang Xiaodong Cun Jue Wang T. Wong 252 191 0 06 Jan 2023
Expressive Speech-driven Facial Animation with controllable emotions Yutong Chen Junhong Zhao Weiqiang Zhang 185 13 0 05 Jan 2023
Imitator: Personalized Speech-driven 3D Facial AnimationIEEE International Conference on Computer Vision (ICCV), 2022 Balamurugan Thambiraja I. Habibie S. Aliakbarian Darren Cosker Christian Theobalt Justus Thies CVBM 208 86 0 30 Dec 2022
Generating Holistic 3D Human Motion from SpeechComputer Vision and Pattern Recognition (CVPR), 2022 Hongwei Yi Hualin Liang Yifei Liu Qiong Cao Yandong Wen Timo Bolkart Dacheng Tao Michael J. Black SLR 225 191 0 08 Dec 2022
Audio-Driven Co-Speech Gesture Video GenerationNeural Information Processing Systems (NeurIPS), 2022 Xian Liu Qianyi Wu Hang Zhou Yuanqi Du Wayne Wu Dahua Lin Ziwei Liu SLR VGen 241 67 0 05 Dec 2022
Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural EmbeddingsACM Transactions on Graphics (TOG), 2022 Tenglong Ao Qingzhe Gao Yuke Lou Baoquan Chen Libin Liu SLR 261 76 0 04 Oct 2022
Synthesizing Photorealistic Virtual Humans Through Cross-modal DisentanglementComputer Vision and Pattern Recognition (CVPR), 2022 S. Ravichandran Ondrej Texler Dimitar Dinev Hyun Jae Kang 137 4 0 03 Sep 2022
Multiface: A Dataset for Neural Face Rendering Cheng-hsin Wuu N. Zheng Scott Ardisson Rohan Bali Danielle Belko ... Xinshuo Weng David Whitewolf Chenglei Wu Shoou-I Yu Yaser Sheikh 3DH CVBM 250 96 0 22 Jul 2022
u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled ModalityNeural Information Processing Systems (NeurIPS), 2022 Wei-Ning Hsu Bowen Shi SSL VLM 286 51 0 14 Jul 2022
Deep Learning for Visual Speech Analysis: A SurveyIEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2022 Changchong Sheng Gangyao Kuang L. Bai Chen Hou Yike Guo Xin Xu M. Pietikäinen Tianpeng Liu VLM 263 52 0 22 May 2022
TEMOS: Generating diverse human motions from textual descriptionsEuropean Conference on Computer Vision (ECCV), 2022 Mathis Petrovich Michael J. Black Gül Varol 389 500 0 25 Apr 2022
Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-SynthesisComputer Vision and Pattern Recognition (CVPR), 2022 Karren D. Yang Dejan Marković Steven Krenn Vasu Agrawal Alexander Richard VGen 151 44 0 31 Mar 2022
DialogueNeRF: Towards Realistic Avatar Face-to-Face Conversation Video Generation Manwen Liao Zanwei Zhou Zi Wang Chen-Ning Yang Xiaokang Yang CVBM 155 34 0 15 Mar 2022
FLAG: Flow-based 3D Avatar Generation from Sparse ObservationsComputer Vision and Pattern Recognition (CVPR), 2022 S. Aliakbarian Pashmina Cameron Federica Bogo Andrew Fitzgibbon T. Cashman 3DH 157 65 0 11 Mar 2022
BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures SynthesisEuropean Conference on Computer Vision (ECCV), 2022 Haiyang Liu Zihao Zhu Naoya Iwamoto Yichen Peng Zhengqing Li You Zhou E. Bozkurt Bo Zheng SLR CVBM 451 189 0 10 Mar 2022
Responsive Listening Head Generation: A Benchmark Dataset and BaselineEuropean Conference on Computer Vision (ECCV), 2021 Mohan Zhou Yalong Bai Wei Zhang Ting Yao Tiejun Zhao Tao Mei EGVM 164 66 0 27 Dec 2021
FaceFormer: Speech-Driven 3D Facial Animation with Transformers Yingruo Fan Mohammad Kachuee Jun Saito Wenping Wang Taku Komura CVBM 581 256 0 10 Dec 2021
Joint Audio-Text Model for Expressive Speech-Driven 3D Facial AnimationProceedings of the ACM on Computer Graphics and Interactive Techniques (PACMCGIT), 2021 Yingruo Fan Mohammad Kachuee Jun Saito Wenping Wang Taku Komura 162 27 0 04 Dec 2021
Neural Dubber: Dubbing for Videos According to Scripts Chenxu Hu Qiao Tian Tingle Li Yuping Wang Yuxuan Wang Hang Zhao DiffM VGen 218 50 0 15 Oct 2021
Learning an Animatable Detailed 3D Face Model from In-The-Wild Images Yao Feng Haiwen Feng Michael J. Black Timo Bolkart CVBM 3DH 489 699 0 07 Dec 2020