Title
Recent Advances in Speech Language Models: A Survey Wenqian Cui Dianzhi Yu Xiaoqi Jiao Ziqiao Meng Guangyan Zhang Qichao Wang Yiwen Guo Irwin King AuLLM 59 14 0 01 Oct 2024
MIO: A Foundation Model on Multimodal Tokens Zekun Wang King Zhu Chunpu Xu Wangchunshu Zhou Jiaheng Liu ... Yuanxing Zhang Ge Zhang Ke Xu Jie Fu Wenhao Huang MLLM AuLLM 58 11 0 26 Sep 2024
EMOVA: Empowering Language Models to See, Hear and Speak with Vivid Emotions Kai Chen Yunhao Gou Runhui Huang Zhili Liu Daxin Tan ... Qun Liu Jun Yao Lu Hou Hang Xu Hang Xu AuLLM MLLM VLM 67 21 0 26 Sep 2024
Enhancing Polyglot Voices by Leveraging Cross-Lingual Fine-Tuning in Any-to-One Voice Conversion Giuseppe Ruggiero Matteo Testa Jurgen Van de Walle Luigi Di Caro 21 0 0 25 Sep 2024
VoiceGuider: Enhancing Out-of-Domain Performance in Parameter-Efficient Speaker-Adaptive Text-to-Speech via Autoguidance Jiheum Yeom Heeseung Kim Jooyoung Choi Che Hyun Lee Nohil Park Sungroh Yoon 24 1 0 24 Sep 2024
FeruzaSpeech: A 60 Hour Uzbek Read Speech Corpus with Punctuation, Casing, and Context Anna Povey Katherine Povey 25 0 0 23 Sep 2024
Semi-supervised Learning For Robust Speech Evaluation Huayun Zhang Jeremy H. M. Wong Geyu Lin Nancy F. Chen 26 0 0 23 Sep 2024
LM-assisted keyword biasing with Aho-Corasick algorithm for Transducer-based ASR Iuliia Thorbecke Juan Zuluaga-Gomez Esaú Villatoro-Tello Andres Carofilis Shashi Kumar P. Motlícek Karthik Pandia A. Ganapathiraju 32 0 0 20 Sep 2024
Fast Streaming Transducer ASR Prototyping via Knowledge Distillation with Whisper Iuliia Thorbecke Juan Zuluaga-Gomez Esaú Villatoro-Tello Shashi Kumar Pradeep Rangappa Sergio Burdisso P. Motlícek Karthik Pandia A. Ganapathiraju 31 0 0 20 Sep 2024
Preference Alignment Improves Language Model-Based TTS Jinchuan Tian Chunlei Zhang Jiatong Shi Hao Zhang Jianwei Yu Shinji Watanabe Dong Yu 30 7 0 19 Sep 2024
SpoofCeleb: Speech Deepfake Detection and SASV In The Wild Jee-weon Jung Yihan Wu Xin Wang Ji-Hoon Kim Soumi Maiti ... Joon Son Chung Wangyou Zhang Seyun Um Shinnosuke Takamichi Shinji Watanabe 62 1 0 18 Sep 2024
Enhancing Low-Resource Language and Instruction Following Capabilities of Audio Language Models Potsawee Manakul Guangzhi Sun Warit Sirichotedumrong Kasima Tharnpipitchai Kunat Pipatanakul AuLLM 36 4 0 17 Sep 2024
Optimizing Dysarthria Wake-Up Word Spotting: An End-to-End Approach for SLT 2024 LRDWWS Challenge Shuiyun Liu Yuxiang Kong Pengcheng Guo Weiji Zhuang Peng Gao Yujun Wang Lei Xie 39 0 0 16 Sep 2024
Exploring SSL Discrete Speech Features for Zipformer-based Contextual ASR Mingyu Cui Yifan Yang Jiajun Deng Jiawen Kang Shujie Hu Tianzi Wang Zhaoqing Li Shiliang Zhang Xie Chen Xunying Liu 23 1 0 13 Sep 2024
Evaluation of real-time transcriptions using end-to-end ASR models Carlos Arriaga Alejandro Pozo Javier Conde Alvaro Alonso 16 3 0 09 Sep 2024
Longer is (Not Necessarily) Stronger: Punctuated Long-Sequence Training for Enhanced Speech Recognition and Translation Nithin Rao Koluguri Travis M. Bartley Hainan Xu Oleksii Hrinchuk Jagadeesh Balam Boris Ginsburg Georg Kucsko 32 2 0 09 Sep 2024
WHISMA: A Speech-LLM to Perform Zero-shot Spoken Language Understanding Mohan Li Cong-Thanh Do Simon Keizer Youmna Farag Svetlana Stoyanchev R. Doddipatla 30 2 0 29 Aug 2024
Enabling Beam Search for Language Model-Based Text-to-Speech Synthesis Zehai Tu Guangyan Zhang Yiting Lu Adaeze Adigwe Simon King Yiwen Guo 35 0 0 29 Aug 2024
VoxInstruct: Expressive Human Instruction-to-Speech Generation with Unified Multilingual Codec Language Modelling Yixuan Zhou Xiaoyu Qin Zeyu Jin Shuoyi Zhou Shun Lei Songtao Zhou Zhiyong Wu Jia Jia AuLLM 28 5 0 28 Aug 2024
SpeechCraft: A Fine-grained Expressive Speech Dataset with Natural Language Description Zeyu Jin Jia Jia Qixin Wang Kehan Li Shuoyi Zhou Songtao Zhou Xiaoyu Qin Zhiyong Wu 27 10 0 24 Aug 2024
CMU's IWSLT 2024 Simultaneous Speech Translation System Xi Xu Siqi Ouyang Brian Yan Patrick Fernandes William Chen Lei Li Graham Neubig Shinji Watanabe 31 1 0 14 Aug 2024
VITA: Towards Open-Source Interactive Omni Multimodal LLM Chaoyou Fu Haojia Lin Zuwei Long Yunhang Shen Meng Zhao ... Ran He Rongrong Ji Yunsheng Wu Caifeng Shan Xing Sun MLLM 39 80 0 09 Aug 2024
Body of Her: A Preliminary Study on End-to-End Humanoid Agent Tenglong Ao LM&Ro 26 1 0 06 Aug 2024
Decoding Linguistic Representations of Human Brain Yu Wang Heyang Liu Yuhao Wang Chuan Xuan Yixuan Hou Sheng Feng Hongcheng Liu Yusheng Liao Yanfeng Wang AI4CE 31 1 0 30 Jul 2024
ELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Tasks Nakamasa Inoue Shinta Otake Takumi Hirose Masanari Ohi Rei Kawakami 34 1 0 28 Jul 2024
Speech Editing -- a Summary Tobias Kässmann Yining Liu Danni Liu 29 0 0 24 Jul 2024
J-CHAT: Japanese Large-scale Spoken Dialogue Corpus for Spoken Dialogue Language Modeling Wataru Nakata Kentaro Seki Hitomi Yanaka Yuki Saito Shinnosuke Takamichi Hiroshi Saruwatari AuLLM 43 0 0 22 Jul 2024
Overview of Speaker Modeling and Its Applications: From the Lens of Deep Speaker Representation Learning Shuai Wang Zheng-Shou Chen Kong Aik Lee Yan-min Qian Haizhou Li 32 4 0 21 Jul 2024
Consent in Crisis: The Rapid Decline of the AI Data Commons Shayne Longpre Robert Mahari Ariel N. Lee Campbell Lund Hamidah Oderinwale ... Hanlin Li Daphne Ippolito Sara Hooker Jad Kabbara Sandy Pentland 69 35 0 20 Jul 2024
Seal: Advancing Speech Language Models to be Few-Shot Learners Shuyu Lei Lingen Liu Jiaolong Yang Yasen Jiao Yuxiang Yang Yushu Yang Xiang Guo VLM 30 0 0 20 Jul 2024
Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generation Haorui He Zengqiang Shang Chaoren Wang Xuyuan Li Yicheng Gu ... Peiyang Shi Yuancheng Wang Kai Chen Pengyuan Zhang Zhizheng Wu 33 34 0 07 Jul 2024
On the Effectiveness of Acoustic BPE in Decoder-Only TTS Bohan Li Feiyu Shen Yiwei Guo Shuai Wang Xie Chen Kai Yu 37 2 0 04 Jul 2024
Finetuning End-to-End Models for Estonian Conversational Spoken Language Translation Tiia Sildam Andra Velve Tanel Alumäe 33 0 0 04 Jul 2024
Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization Yuchen Hu Chen Chen Siyin Wang Eng Siong Chng C. Zhang 43 3 0 02 Jul 2024
Investigating the Effects of Large-Scale Pseudo-Stereo Data and Different Speech Foundation Model on Dialogue Generative Spoken Language Model Yu-Kuan Fu Cheng-Kuang Lee Hsiu-Hsuan Wang Hung-yi Lee 22 0 0 02 Jul 2024
Towards Robust Speech Representation Learning for Thousands of Languages William Chen Wangyou Zhang Yifan Peng Xinjian Li Jinchuan Tian Jiatong Shi Xuankai Chang Soumi Maiti Karen Livescu Shinji Watanabe ELM 40 6 0 30 Jun 2024
Less Forgetting for Better Generalization: Exploring Continual-learning Fine-tuning Methods for Speech Self-supervised Representations Salah Zaiem Titouan Parcollet S. Essid CLL 28 3 0 30 Jun 2024
BESTOW: Efficient and Streamable Speech Language Model with the Best of Two Worlds in GPT and T5 Zhehuai Chen He Huang Oleksii Hrinchuk Krishna C. Puvvada Nithin Rao Koluguri Piotr Żelasko Jagadeesh Balam Boris Ginsburg AuLLM RALM 34 10 0 28 Jun 2024
WavRx: a Disease-Agnostic, Generalizable, and Privacy-Preserving Speech Health Diagnostic Model Yi Zhu Tiago H. Falk MedIm 41 0 0 26 Jun 2024
MSR-86K: An Evolving, Multilingual Corpus with 86,300 Hours of Transcribed Audio for Speech Recognition Research Song Li Yongbin You Xuezhi Wang Zhengkun Tian Ke Ding Guanglu Wan 21 1 0 26 Jun 2024
A Comprehensive Solution to Connect Speech Encoder and Large Language Model for ASR Van Tung Pham Yist Y. Lin Tao Han Wei Li Jun Zhang Lu Lu Yuxuan Wang AuLLM 32 1 0 25 Jun 2024
Exploring the Capability of Mamba in Speech Applications Koichi Miyazaki Yoshiki Masuyama Masato Murata Mamba 30 12 0 24 Jun 2024
AudioBench: A Universal Benchmark for Audio Large Language Models Bin Wang Xunlong Zou Geyu Lin S. Zhuohan Liu Wenyu Zhang Zhengyuan Liu AiTi Aw Nancy F. Chen AuLLM ELM LM&MA 87 20 0 23 Jun 2024
Speech Analysis of Language Varieties in Italy Moreno La Quatra Alkis Koudounas Elena Baralis Sabato Marco Siniscalchi 27 3 0 22 Jun 2024
The Greek podcast corpus: Competitive speech models for low-resourced languages with weakly supervised data Georgios Paraskevopoulos Chara Tsoukala Athanasios Katsamanis V. Katsouros OffRL 23 0 0 21 Jun 2024
GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinement Yifan Yang Zheshu Song Jianheng Zhuo Mingyu Cui Jinpeng Li ... Shuai Fan Kai Yu Wei-Qiang Zhang Guoguo Chen Xie Chen 14 8 0 17 Jun 2024
Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models Dominik Wagner Ilja Baumann K. Riedhammer Tobias Bocklet MQ 30 1 0 16 Jun 2024
On the Evaluation of Speech Foundation Models for Spoken Language Understanding Siddhant Arora Ankita Pasad Chung-Ming Chien Jionghao Han Roshan S. Sharma ... William Chen Suwon Shon Hung-yi Lee Karen Livescu Shinji Watanabe ELM 43 4 0 14 Jun 2024
Vec-Tok-VC+: Residual-enhanced Robust Zero-shot Voice Conversion with Progressive Constraints in a Dual-mode Training Strategy Linhan Ma Xinfa Zhu Yuanjun Lv Zhichao Wang Ziqian Wang Wendi He Hongbin Zhou Lei Xie 37 2 0 14 Jun 2024
On the Effects of Heterogeneous Data Sources on Speech-to-Text Foundation Models Jinchuan Tian Yifan Peng William Chen Kwanghee Choi Karen Livescu Shinji Watanabe 24 5 0 13 Jun 2024