v1v2 (latest)

SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models

International Conference on Learning Representations (ICLR), 2023

31 August 2023

Xipeng Qiu

ArXiv (abs)PDF HTML HuggingFace (1 upvotes)Github (560★)

Papers citing "SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models"

23 / 73 papers shown

Title
SSDM: Scalable Speech Dysfluency ModelingNeural Information Processing Systems (NeurIPS), 2024 Jiachen Lian Xuanru Zhou Z. Ezzes Jet M J Vonk Brittany Morin D. Baquirin Zachary Mille M. G. Tempini Gopala Anumanchipalli AuLLM 239 18 0 29 Aug 2024
WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language ModelingInternational Conference on Learning Representations (ICLR), 2024 Shengpeng Ji Ziyue Jiang Xize Cheng Yifu Chen Minghui Fang ... Rongjie Huang Yidi Jiang Qian Chen Zhou Zhao Zhou Zhao VLM 320 110 0 29 Aug 2024
dMel: Speech Tokenization made Simple Richard He Bai Tatiana Likhomanenko Ruixiang Zhang Zijin Gu Zakaria Aldeneh Navdeep Jaitly 279 10 0 22 Jul 2024
Overview of Speaker Modeling and Its Applications: From the Lens of Deep Speaker Representation Learning Shuai Wang Zheng-Shou Chen Kong Aik Lee Yan-min Qian Haizhou Li 297 23 0 21 Jul 2024
Codec-ASR: Training Performant Automatic Speech Recognition Systems with Discrete Speech Representations Kunal Dhawan Nithin Rao Koluguri Ante Jukić Ryan Langman Jagadeesh Balam Boris Ginsburg 185 13 0 03 Jul 2024
Robust Zero-Shot Text-to-Speech Synthesis with Reverse Inference Optimization Yuchen Hu Chen Chen Siyin Wang Eng Siong Chng C. Zhang 186 12 0 02 Jul 2024
DASB -- Discrete Audio and Speech Benchmark Pooneh Mousavi Luca Della Libera J. Duret Artem Ploujnikov Cem Subakan Mirco Ravanelli 290 35 0 20 Jun 2024
Articulatory Encodec: Coding Speech through Vocal Tract KinematicsIEEE Journal on Selected Topics in Signal Processing (JSTSP), 2024 Cheol Jun Cho Peter Wu Tejas S. Prabhune Dhruv Agarwal Gopala K. Anumanchipalli 256 21 0 18 Jun 2024
UniAudio 1.5: Large Language Model-driven Audio Codec is A Few-shot Audio Task LearnerNeural Information Processing Systems (NeurIPS), 2024 Dongchao Yang Haohan Guo Yuanyuan Wang Rongjie Huang Xiang Li Xu Tan Xixin Wu Helen Meng AuLLM 163 30 0 14 Jun 2024
Autoregressive Diffusion Transformer for Text-to-Speech Synthesis Zhijun Liu Shuai Wang Sho Inoue Qibing Bai Haizhou Li DiffM 152 30 0 08 Jun 2024
VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizers Sanyuan Chen Shujie Liu Long Zhou Yanqing Liu Xu Tan Jinyu Li Sheng Zhao Yao Qian Furu Wei VLM 240 143 0 08 Jun 2024
Neural Codec-based Adversarial Sample Detection for Speaker VerificationInterspeech (Interspeech), 2024 Xuanjun Chen Jiawei Du Haibin Wu Jyh-Shing Roger Jang Hung-yi Lee 333 8 0 07 Jun 2024
Addressing Index Collapse of Large-Codebook Speech Tokenizer with Dual-Decoding Product-Quantized Variational Auto-Encoder Haohan Guo Fenglong Xie Dongchao Yang Hui Lu Xixin Wu Helen Meng 163 8 0 05 Jun 2024
ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Control Shengpeng Ji Jia-li Zuo Wen Wang Jialong Zuo Minghui Fang ... Ziyue Jiang Hai Huang Xize Cheng Siqi Zheng Zhou Zhao 413 8 0 03 Jun 2024
Enhancing Zero-shot Text-to-Speech Synthesis with Human Feedback Chen Chen Yuchen Hu Wen Wu Helin Wang Chng Eng Siong Chao Zhang 165 25 0 02 Jun 2024
SpeechAlign: Aligning Speech Generation to Human Preferences Dong Zhang Zhaowei Li Shimin Li Xin Zhang Pengyu Wang Yaqian Zhou Xipeng Qiu ALM AuLLM 183 35 0 08 Apr 2024
NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models Zeqian Ju Yuancheng Wang Kai Shen Xu Tan Detai Xin ... Shikun Zhang Jiang Bian Lei He Jinyu Li Sheng Zhao DiffM 370 285 0 05 Mar 2024
Towards audio language modeling -- an overview Haibin Wu Xuanjun Chen Yi-Cheng Lin Kai-Wei Chang Ho-Lam Chung Alexander H. Liu Hung-yi Lee AuLLM 215 58 0 20 Feb 2024
AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling Jun Zhan Junqi Dai Jiasheng Ye Yunhua Zhou Dong Zhang ... Jie Fu Tao Gui Tianxiang Sun Yugang Jiang Xinyu Zhou MLLM 433 196 0 19 Feb 2024
APCodec: A Neural Audio Codec with Parallel Amplitude and Phase Spectrum Encoding and Decoding Yang Ai Xiao-Hang Jiang Ye-Xin Lu Hui-Peng Du Zhenhua Ling 162 40 0 16 Feb 2024
BASE TTS: Lessons from building a billion-parameter Text-to-Speech model on 100K hours of data Mateusz Lajszczak Guillermo Cámbara Yang Li Fatih Beyhan Arent van Korlaar ... Bartosz Putrycz Soledad López Gambino Kayeon Yoo Elena Sokolova Thomas Drugman LM&MA 318 110 0 12 Feb 2024
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models Yunfei Chu Jin Xu Xiaohuan Zhou Qian Yang Shiliang Zhang Zhijie Yan Chang Zhou Jingren Zhou AuLLM 266 578 0 14 Nov 2023
Sparks of Large Audio Models: A Survey and Outlook S. Latif Moazzam Shoukat Fahad Shamshad Muhammad Usama Yi Ren ... Wenwu Wang Xulong Zhang Roberto Togneri Xiaoshi Zhong Björn W. Schuller LM&MA AuLLM 573 50 0 24 Aug 2023