emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation

23 December 2023

Xie Chen

Papers citing "emotion2vec: Self-Supervised Pre-Training for Speech Emotion Representation"

44 / 44 papers shown

Title
Affect Models Have Weak Generalizability to Atypical Speech Jaya Narain Amrit Romana Vikramjit Mitra Colin S. Lea Shirley Ren 12 0 0 22 Apr 2025
VoiceCraft-Dub: Automated Video Dubbing with Neural Codec Language Models Kim Sung-Bin Jeongsoo Choi Puyuan Peng Joon Son Chung Tae-Hyun Oh David F. Harwath VGen 45 1 0 03 Apr 2025
EmoHead: Emotional Talking Head via Manipulating Semantic Expression Parameters Xuli Shen Hua Cai Dingding Yu Weilin Shen Qing-Song Xu Xiangyang Xue 32 0 0 25 Mar 2025
Large Language Models Meet Contrastive Learning: Zero-Shot Emotion Recognition Across Languages Heqing Zou Fengmao Lv Desheng Zheng E. Chng D. Rajan 24 0 0 25 Mar 2025
Shushing! Let's Imagine an Authentic Speech from the Silent Video Jiaxin Ye Hongming Shan DiffM VGen 57 1 0 19 Mar 2025
MAVFlow: Preserving Paralinguistic Elements with Conditional Flow Matching for Zero-Shot AV2AV Multilingual Translation Sungwoo Cho J. Choi Sungnyun Kim Se-Young Yun 54 0 0 14 Mar 2025
Scaling Rich Style-Prompted Text-to-Speech Datasets Anuj Diwan Zhisheng Zheng David F. Harwath Eunsol Choi CLIP VLM 72 0 0 06 Mar 2025
Qieemo: Speech Is All You Need in the Emotion Recognition in Conversations Jinming Chen Jingyi Fang Yuanzhong Zheng Yaoxuan Wang Haojun Fei 41 0 0 05 Mar 2025
Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens X. Wang Mingqi Jiang Z. Ma Ziyu Zhang S. Liu ... Zhifei Li Xie Chen Lei Xie Y. Guo Wei Xue 67 9 0 03 Mar 2025
Residual Speech Embeddings for Tone Classification: Removing Linguistic Content to Enhance Paralinguistic Analysis Hamdan Al Ahbabi Gautier Marti Saeed AlMarri Ibrahim Elfadel 49 0 0 26 Feb 2025
voc2vec: A Foundation Model for Non-Verbal Vocalization Alkis Koudounas Moreno La Quatra Marco Sabato Siniscalchi Elena Baralis 38 0 0 22 Feb 2025
Gender Bias in Instruction-Guided Speech Synthesis Models Chun-Yi Kuan Hung-yi Lee 56 0 0 08 Feb 2025
Enhancing Expressive Voice Conversion with Discrete Pitch-Conditioned Flow Matching Model Jialong Zuo Shengpeng Ji Minghui Fang Ziyue Jiang Xize Cheng ... Wenrui Liu Guangyan Zhang Zehai Tu Yiwen Guo Zhou Zhao 49 0 0 08 Feb 2025
Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation Haorui He Zengqiang Shang Chaoren Wang Xuyuan Li Yicheng Gu ... Peiyang Shi Y. Wang Kai Chen Pengyuan Zhang Z. Wu AuLLM 47 3 0 28 Jan 2025
DeSTA2: Developing Instruction-Following Speech Language Model Without Speech Instruction-Tuning Data Ke-Han Lu Zhehuai Chen Szu-Wei Fu Chao-Han Huck Yang Jagadeesh Balam Boris Ginsburg Yu-Te Wang Hung-yi Lee AuLLM SyDa 95 5 0 28 Jan 2025
OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academia Xuelong Geng Kun Wei Qijie Shao Shuiyun Liu Zhennan Lin ... Yuhang Dai Xinfa Zhu Yue Li Li Zhang Lei Xie 62 3 0 23 Jan 2025
SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words Junyi Ao Yuancheng Wang Xiaohai Tian Dekun Chen J. Zhang Lu Lu Y. Wang Haizhou Li Z. Wu AuLLM 71 16 0 17 Jan 2025
ZSVC: Zero-shot Style Voice Conversion with Disentangled Latent Diffusion Models and Adversarial Training Xinfa Zhu Lei He Yujia Xiao Xi Wang Xu Tan Sheng Zhao Lei Xie DiffM 33 0 0 08 Jan 2025
Improving Speech Emotion Recognition in Under-Resourced Languages via Speech-to-Speech Translation with Bootstrapping Data Selection Hsi-Che Lin Yi-Cheng Lin Huang-Cheng Chou Hung-yi Lee 21 0 0 08 Jan 2025
OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios Xize Cheng Dongjie Fu Xiaoda Yang Minghui Fang Ruofan Hu ... Rongjie Huang Linjun Li Yu Chen Tao Jin Zhou Zhao 41 1 0 03 Jan 2025
EmoReg: Directional Latent Vector Modeling for Emotional Intensity Regularization in Diffusion-based Voice Conversion Ashishkumar Gudmalwar Ishan D. Biyani Nirmesh J. Shah Pankaj Wasnik R. Shah DiffM 21 0 0 31 Dec 2024
EmoSphere++: Emotion-Controllable Zero-Shot Text-to-Speech via Emotion-Adaptive Spherical Vector Deok-Hyeon Cho Hyung-Seok Oh Seung-Bin Kim Seong-Whan Lee 39 3 0 04 Nov 2024
Enhancing Multimodal Sentiment Analysis for Missing Modality through Self-Distillation and Unified Modality Cross-Attention Yuzhe Weng Haotian Wang Tian Gao Kewei Li Shutong Niu Jun Du 28 0 0 19 Oct 2024
EmoPro: A Prompt Selection Strategy for Emotional Expression in LM-based Speech Synthesis Haoyu Wang Chunyu Qiang Tianrui Wang Cheng Gong Qiuyu Liu Yu Jiang Xiaobao Wang Chenyang Wang Chen Zhang 28 0 0 27 Sep 2024
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions Kai Chen Yunhao Gou Runhui Huang Zhili Liu Daxin Tan ... Qun Liu Jun Yao Lu Hou Hang Xu Hang Xu AuLLM MLLM VLM 58 21 0 26 Sep 2024
Emotional Dimension Control in Language Model-Based Text-to-Speech: Spanning a Broad Spectrum of Human Emotions Kun Zhou You Zhang Shengkui Zhao Hao Wang Zexu Pan ... Chongjia Ni Yukun Ma Trung Hieu Nguyen J. Yip Bin Ma 42 4 0 25 Sep 2024
Codec-SUPERB @ SLT 2024: A lightweight benchmark for neural audio codec models Haibin Wu Xuanjun Chen Yi-Cheng Lin Kaiwei Chang Jiawei Du ... Yi-Chiao Wu Xu Tan James Glass Shinji Watanabe Hung-yi Lee 16 0 0 21 Sep 2024
Emo-DPO: Controllable Emotional Speech Synthesis through Direct Preference Optimization Xiaoxue Gao Chen Zhang Yiming Chen Huayun Zhang Nancy F. Chen 24 6 0 16 Sep 2024
MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer Yuancheng Wang Haoyue Zhan Liwei Liu Ruihong Zeng Haotian Guo Jiachen Zheng Qiang Zhang Shunsi Zhang Shunsi Zhang Zhizheng Wu 23 37 0 01 Sep 2024
WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling Shengpeng Ji Ziyue Jiang Xize Cheng Yifu Chen Minghui Fang ... Rongjie Huang Yidi Jiang Qian Chen Zhou Zhao Zhou Zhao VLM 45 32 0 29 Aug 2024
DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation Jisoo Kim Jungbin Cho Joonho Park Soonmin Hwang Da Eun Kim Geon Kim Youngjae Yu 34 0 0 12 Aug 2024
Adapting General Disentanglement-Based Speaker Anonymization for Enhanced Emotion Preservation Xiaoxiao Miao Yuxiang Zhang Xin Wang N. Tomashenko D. Soh Ian Mcloughlin 28 1 0 12 Aug 2024
Laugh Now Cry Later: Controlling Time-Varying Emotional States of Flow-Matching-Based Zero-Shot Text-to-Speech Haibin Wu Xiaofei Wang Sefik Emre Eskimez Manthan Thakker Daniel Tompkins ... Canrun Li Zhen Xiao Sheng Zhao Jinyu Li Naoyuki Kanda 15 6 0 17 Jul 2024
Speech-Copilot: Leveraging Large Language Models for Speech Processing via Task Decomposition, Modularization, and Program Generation Chun-Yi Kuan Chih-Kai Yang Wei-Ping Huang Ke-Han Lu Hung-yi Lee 39 5 0 13 Jul 2024
Emotion Talk: Emotional Support via Audio Messages for Psychological Assistance Fabrycio Leite Nakano Almada Kauan Divino Pouso Mariano Maykon Adriell Dutra Victor Emanuel da Silva Monteiro 18 0 0 12 Jul 2024
FunAudioLLM: Voice Understanding and Generation Foundation Models for Natural Interaction Between Humans and LLMs Keyu An Qian Chen Chong Deng Zhihao Du Changfeng Gao ... Bin Zhang Qinglin Zhang Shiliang Zhang Nan Zhao Siqi Zheng AuLLM 24 42 0 04 Jul 2024
Towards Probing Speech-Specific Risks in Large Multimodal Models: A Taxonomy, Benchmark, and Insights Hao Yang Lizhen Qu Ehsan Shareghi Gholamreza Haffari 23 0 0 25 Jun 2024
EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark Ziyang Ma Mingjie Chen Hezhao Zhang Zhisheng Zheng Wenxi Chen Xiquan Li Jiaxin Ye Xie Chen Thomas Hain 25 12 0 11 Jun 2024
ControlSpeech: Towards Simultaneous Zero-shot Speaker Cloning and Zero-shot Language Style Control With Decoupled Codec Shengpeng Ji Jia-li Zuo Minghui Fang Siqi Zheng Qian Chen ... Ziyue Jiang Hai Huang Xize Cheng Rongjie Huang Zhou Zhao 45 7 0 03 Jun 2024
1st Place Solution to Odyssey Emotion Recognition Challenge Task1: Tackling Class Imbalance Problem Mingjie Chen Hezhao Zhang Yuanchao Li Jiachen Luo Wen Wu ... Lin Wang P. Woodland Xie Chen Huy P Phan Thomas Hain 18 0 0 30 May 2024
MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition Zheng Lian Haiyang Sun Licai Sun Zhuofan Wen Siyuan Zhang ... Bin Liu Erik Cambria Guoying Zhao Björn W. Schuller Jianhua Tao VLM 31 11 0 26 Apr 2024
Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations Guan-Ting Lin Cheng-Han Chiang Hung-yi Lee 28 22 0 20 Feb 2024
DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations without Text Alignment Hyoung-Seok Oh Sang-Hoon Lee Deok-Hyun Cho Seong-Whan Lee 26 1 0 16 Jan 2024
Exploration of A Self-Supervised Speech Model: A Study on Emotional Corpora Yuanchao Li Yumnah Mohamied P. Bell Catherine Lai SSL 30 45 0 05 Oct 2022