Audio to Body Dynamics

19 December 2017

Eli Shlizerman

Lucio Dery

Hayden Schoen

Ira Kemelmacher-Shlizerman

VGen

ArXiv PDF HTML

Papers citing "Audio to Body Dynamics"

50 / 79 papers shown

Title
ELGAR: Expressive Cello Performance Motion Generation for Audio Rendition Zhiping Qiu Yitong Jin Y. Wang Yi Shi C. Wang Chao Tan Xiaobing Li Feng Yu Tao Yu Qionghai Dai 19 0 0 07 May 2025
A Survey on Cross-Modal Interaction Between Music and Multimodal Data Sifei Li Mining Tan Feier Shen Minyan Luo Zijiao Yin Fan Tang W. Dong Changsheng Xu 55 0 0 17 Apr 2025
BGM2Pose: Active 3D Human Pose Estimation with Non-Stationary Sounds Yuto Shibata Yusuke Oumi Go Irie Akisato Kimura Yoshimitsu Aoki Mariko Isogawa 24 0 0 01 Mar 2025
Synchronize Dual Hands for Physics-Based Dexterous Guitar Playing Pei Xu Ruocheng Wang 43 2 0 20 Feb 2025
SyncViolinist: Music-Oriented Violin Motion Generation Based on Bowing and Fingering Hiroki Nishizawa Keitaro Tanaka Asuka Hirata Shugo Yamaguchi Qi Feng Masatoshi Hamanaka Shigeo Morishima 62 0 0 11 Dec 2024
Acoustic-based 3D Human Pose Estimation Robust to Human Position Yusuke Oumi Yuto Shibata Go Irie Akisato Kimura Yoshimitsu Aoki Mariko Isogawa 18 1 0 08 Nov 2024
STNet: Deep Audio-Visual Fusion Network for Robust Speaker Tracking Yidi Li Hong Liu Bing Yang 25 4 0 08 Oct 2024
FürElise: Capturing and Physically Synthesizing Hand Motions of Piano Performance Ruocheng Wang Pei Xu Haochen Shi Elizabeth Schumann C. Karen Liu 17 3 0 08 Oct 2024
VMAS: Video-to-Music Generation via Semantic Alignment in Web Music Videos Yan-Bo Lin Yu Tian L. Yang Gedas Bertasius Heng Wang VGen 26 7 0 11 Sep 2024
CLHOP: Combined Audio-Video Learning for Horse 3D Pose and Shape Estimation Ci Li Elin Hernlund Hedvig Kjellström Silvia Zuffi 3DH 23 2 0 01 Jul 2024
MOSA: Music Motion with Semantic Annotation Dataset for Cross-Modal Music Processing Yu-Fen Huang Nikki Moran Simon Coleman Jon Kelly Shun-Hwa Wei ... Chih-Hsuan Li Da-Yu Huang Hsuan-Kai Kao Ting-Wei Lin Li Su 21 1 0 10 Jun 2024
Dance Any Beat: Blending Beats with Visuals in Dance Video Generation Xuanchen Wang Heng Wang Dongnan Liu Weidong Cai 30 3 0 15 May 2024
Cross-modal Generative Model for Visual-Guided Binaural Stereo Generation Zhaojian Li Bin Zhao Yuan Yuan 12 1 0 13 Nov 2023
Tackling Data Bias in MUSIC-AVQA: Crafting a Balanced Dataset for Unbiased Question-Answering Xiulong Liu Zhikang Dong Peng Zhang 12 21 0 10 Oct 2023
BodyFormer: Semantics-guided 3D Body Gesture Synthesis with Transformer Kunkun Pang Dafei Qin Yingruo Fan Julian Habekost Takaaki Shiratori Junichi Yamagishi Taku Komura SLR ViT 14 19 0 07 Sep 2023
DiffDance: Cascaded Human Motion Diffusion Model for Dance Generation Qiaosong Qi Le Zhuo Aixi Zhang Yue Liao Fei Fang Si Liu Shuicheng Yan 11 22 0 05 Aug 2023
ZS-MSTM: Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding Mireille Fares Catherine Pelachaud Nicolas Obin 9 0 0 22 May 2023
Spatial-temporal Transformer-guided Diffusion based Data Augmentation for Efficient Skeleton-based Action Recognition Yifan Jiang Han Chen Hanseok Ko DiffM 26 3 0 26 Feb 2023
Audio2Gestures: Generating Diverse Gestures from Audio Jing Li Di Kang Wenjie Pei Xuefei Zhe Ying Zhang Linchao Bao Zhenyu He DiffM SLR 23 7 0 17 Jan 2023
FLAG3D: A 3D Fitness Activity Dataset with Language Instruction Yansong Tang Jinpeng Liu Aoyang Liu B. Yang Wen-Dao Dai Yongming Rao Jiwen Lu Jie Zhou Xiu Li 27 22 0 09 Dec 2022
MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis Rishabh Dabral Muhammad Hamza Mughal Vladislav Golyanik Christian Theobalt DiffM VGen 14 111 0 08 Dec 2022
PaCMO: Partner Dependent Human Motion Generation in Dyadic Human Activity using Neural Operators Md Ashiqur Rahman Jasorsi Ghosh Hrishikesh Viswanath Kamyar Azizzadenesheli Aniket Bera 27 8 0 25 Nov 2022
Learning in Audio-visual Context: A Review, Analysis, and New Perspective Yake Wei Di Hu Yapeng Tian Xuelong Li 31 54 0 20 Aug 2022
Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding Mireille Fares Michele Grimaldi Catherine Pelachaud Nicolas Obin 14 11 0 03 Aug 2022
Finding Fallen Objects Via Asynchronous Audio-Visual Integration Chuang Gan Yi Gu Siyuan Zhou Jeremy Schwartz S. Alter James Traer Dan Gutfreund J. Tenenbaum Josh H. McDermott Antonio Torralba 32 19 0 07 Jul 2022
Programmatic Concept Learning for Human Motion Description and Synthesis Sumith Kulal Jiayuan Mao A. Aiken Jiajun Wu 17 6 0 27 Jun 2022
Weakly-supervised Action Transition Learning for Stochastic Human Motion Prediction Wei Mao Miaomiao Liu Mathieu Salzmann 31 28 0 31 May 2022
Text/Speech-Driven Full-Body Animation Wenlin Zhuang Jinwei Qi Peng Zhang Bang Zhang Ping Tan 25 6 0 31 May 2022
Learning Visual Styles from Audio-Visual Associations Tingle Li Yichen Liu Andrew Owens Hang Zhao DiffM 15 20 0 10 May 2022
Quantized GAN for Complex Music Generation from Dance Videos Ye Zhu Kyle Olszewski Yuehua Wu Panos Achlioptas Menglei Chai Yan Yan Sergey Tulyakov MGen 17 44 0 01 Apr 2022
AIMusicGuru: Music Assisted Human Pose Correction Snehesh Shrestha Cornelia Fermuller Tianyu Huang Pyone Thant Win Adam Zukerman Chethan Parameshwara Yiannis Aloimonos 3DH 11 7 0 24 Mar 2022
Freeform Body Motion Generation from Speech Jing-Fen Xu Wei Zhang Yalong Bai Qi-Biao Sun Tao Mei SLR 17 18 0 04 Mar 2022
Rhythm is a Dancer: Music-Driven Motion Synthesis with Global Structure A. Aristidou Anastasios Yiannakidis Kfir Aberman Daniel Cohen-Or Ariel Shamir Y. Chrysanthou 30 72 0 23 Nov 2021
Action2video: Generating Videos of Human 3D Actions Chuan Guo X. Zuo Sen Wang Xinshuang Liu Shihao Zou Minglun Gong Li Cheng 3DH 63 22 0 12 Nov 2021
TriBERT: Full-body Human-centric Audio-visual Representation Learning for Visual Sound Separation Tanzila Rahman Mengyu Yang Leonid Sigal ViT 13 8 0 26 Oct 2021
Multi-Modulation Network for Audio-Visual Event Localization Hao Wang Zhengjun Zha Liang Li Xuejin Chen Jiebo Luo 20 2 0 26 Aug 2021
Speech Drives Templates: Co-Speech Gesture Synthesis with Learned Templates Shenhan Qian Zhi Tu Yihao Zhi Wen Liu Shenghua Gao SLR 6 70 0 18 Aug 2021
Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders Jing Li Di Kang Wenjie Pei Xuefei Zhe Ying Zhang Zhenyu He Linchao Bao SLR 25 99 0 15 Aug 2021
Egocentric Videoconferencing Mohamed A. Elgharib Mohit Mendiratta Justus Thies Matthias Nießner Hans-Peter Seidel A. Tewari Vladislav Golyanik Christian Theobalt EgoV 20 17 0 07 Jul 2021
Dance Generation with Style Embedding: Learning and Transferring Latent Representations of Dance Styles Xinjian Zhang Yi Xu Su Yang Longwen Gao Huyang Sun 13 10 0 30 Apr 2021
FixMyPose: Pose Correctional Captioning and Retrieval Hyounghun Kim Abhaysinh Zala Graham Burri Mohit Bansal 14 13 0 04 Apr 2021
Learning Speech-driven 3D Conversational Gestures from Video I. Habibie Weipeng Xu Dushyant Mehta Lingjie Liu Hans-Peter Seidel Gerard Pons-Moll Mohamed A. Elgharib Christian Theobalt SLR CVBM 3DH 31 104 0 13 Feb 2021
AI Choreographer: Music Conditioned 3D Dance Generation with AIST++ Ruilong Li Sha Yang David A. Ross Angjoo Kanazawa ViT 198 467 0 21 Jan 2021
AudioViewer: Learning to Visualize Sounds Chunjin Song Yuchi Zhang Willis Peng Parmis Mohaghegh Bastian Wandt Helge Rhodin 17 1 0 22 Dec 2020
Multi-Instrumentalist Net: Unsupervised Generation of Music from Body Movements Kun Su Xiulong Liu Eli Shlizerman 17 18 0 07 Dec 2020
Lets Play Music: Audio-driven Performance Video Generation Hao Zhu Yi Li Feixia Zhu A. Zheng R. He 17 6 0 05 Nov 2020
DanceIt: Music-inspired Dancing Video Synthesis Xin Guo Yifan Zhao Jia Li 8 10 0 17 Sep 2020
Temporally Guided Music-to-Body-Movement Generation Hsuan-Kai Kao Li Su 31 34 0 17 Sep 2020
A Human-Computer Duet System for Music Performance Yuen-Jen Lin Hsuan-Kai Kao Yih-Chih Tseng Ming Tsai Li Su 8 7 0 16 Sep 2020
Learning to Generate Diverse Dance Motions with Transformer Jiaman Li Yihang Yin Hang Chu Yi Zhou Tingwu Wang Sanja Fidler Hao Li 6 122 0 18 Aug 2020