SoundStream: An End-to-End Neural Audio Codec

7 July 2021

Papers citing "SoundStream: An End-to-End Neural Audio Codec"

50 / 102 papers shown

Title
LiveSpeech: Low-Latency Zero-shot Text-to-Speech via Autoregressive Modeling of Audio Discrete Codes Trung D. Q. Dang David Aponte Dung Tran K. Koishida 36 3 0 05 Jun 2024
Discrete Multimodal Transformers with a Pretrained Large Language Model for Mixed-Supervision Speech Processing V. Trinh Rosy Southwell Yiwen Guan Xinlu He Zhiyong Wang Jacob Whitehill OffRL 36 2 0 04 Jun 2024
MAD Speech: Measures of Acoustic Diversity of Speech Matthieu Futeral A. Agostinelli Marco Tagliasacchi Neil Zeghidour Eugene Kharitonov 48 1 0 16 Apr 2024
HyperVQ: MLR-based Vector Quantization in Hyperbolic Space Nabarun Goswami Yusuke Mukuta Tatsuya Harada 40 3 0 18 Mar 2024
Advancing Large Language Models to Capture Varied Speaking Styles and Respond Properly in Spoken Conversations Guan-Ting Lin Cheng-Han Chiang Hung-yi Lee 34 22 0 20 Feb 2024
StreamVoice: Streamable Context-Aware Language Modeling for Real-time Zero-Shot Voice Conversion Zhichao Wang Yuan-Jui Chen Xinsheng Wang Lei Xie Yuping Wang 22 6 0 19 Jan 2024
Learning to Act without Actions Dominik Schmidt Minqi Jiang OffRL 26 30 0 17 Dec 2023
4M: Massively Multimodal Masked Modeling David Mizrahi Roman Bachmann Ouguzhan Fatih Kar Teresa Yeo Mingfei Gao Afshin Dehghan Amir Zamir MLLM 41 62 0 11 Dec 2023
SVQ: Sparse Vector Quantization for Spatiotemporal Forecasting Chao Chen Tian Zhou Yanjun Zhao Hui Liu Liang Sun Rong Jin 37 0 0 06 Dec 2023
MoMask: Generative Masked Modeling of 3D Human Motions Chuan Guo Yuxuan Mu Muhammad Gohar Javed Sen Wang Li Cheng VGen 19 117 0 29 Nov 2023
Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models Yunfei Chu Jin Xu Xiaohuan Zhou Qian Yang Shiliang Zhang Zhijie Yan Chang Zhou Jingren Zhou AuLLM 28 267 0 14 Nov 2023
InstrumentGen: Generating Sample-Based Musical Instruments From Text S. Nercessian Johannes Imort 27 2 0 07 Nov 2023
Content-based Controls For Music Large Language Modeling Liwei Lin Gus Xia Junyan Jiang Yixiao Zhang 18 14 0 26 Oct 2023
AdaMesh: Personalized Facial Expressions and Head Poses for Adaptive Speech-Driven 3D Facial Animation Liyang Chen Weihong Bao Shunwei Lei Boshi Tang Zhiyong Wu Shiyin Kang Haozhi Huang Helen M. Meng 35 1 0 11 Oct 2023
Discrete Audio Representation as an Alternative to Mel-Spectrograms for Speaker and Speech Recognition Krishna C. Puvvada Nithin Rao Koluguri Kunal Dhawan Jagadeesh Balam Boris Ginsburg 24 12 0 19 Sep 2023
FunCodec: A Fundamental, Reproducible and Integrable Open-source Toolkit for Neural Speech Codec Zhihao Du Shiliang Zhang Kai Hu Siqi Zheng 24 54 0 14 Sep 2023
Self-Supervised Disentanglement of Harmonic and Rhythmic Features in Music Audio Signals Yiming Wu CoGe DRL 24 0 0 06 Sep 2023
AudioFormer: Audio Transformer learns audio feature representations from discrete acoustic codes Zhaohui Li Haitao Wang Xinghua Jiang 40 1 0 14 Aug 2023
AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining Haohe Liu Yiitan Yuan Xubo Liu Xinhao Mei Qiuqiang Kong Qiao Tian Yuping Wang Wenwu Wang Yuxuan Wang Mark D. Plumbley DiffM 25 221 0 10 Aug 2023
JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models Peike Li Bo-Yu Chen Yao Yao Yikai Wang Allen Wang Alex Jinpeng Wang MGen VLM DiffM 60 37 0 09 Aug 2023
Minimally-Supervised Speech Synthesis with Conditional Diffusion Model and Language Model: A Comparative Study of Semantic Coding Chunyu Qiang Hao Li Hao Ni He Qu Ruibo Fu Tao Wang Longbiao Wang J. Dang DiffM 30 8 0 28 Jul 2023
Incrementally-Computable Neural Networks: Efficient Inference for Dynamic Inputs Or Sharir Anima Anandkumar 24 0 0 27 Jul 2023
SoundStorm: Efficient Parallel Audio Generation Zalan Borsos Matthew Sharifi Damien Vincent Eugene Kharitonov Neil Zeghidour Marco Tagliasacchi 23 97 0 16 May 2023
Leveraging Neural Representations for Audio Manipulation Scott H. Hawley C. Steinmetz 25 2 0 10 Apr 2023
Native Multi-Band Audio Coding within Hyper-Autoencoded Reconstruction Propagation Networks Darius Petermann Inseon Jang Minje Kim 11 1 0 14 Mar 2023
FoundationTTS: Text-to-Speech for ASR Customization with Generative Language Model Rui Xue Yanqing Liu Lei He Xuejiao Tan Linquan Liu Ed Lin Sheng Zhao 26 7 0 06 Mar 2023
Hypernetworks build Implicit Neural Representations of Sounds Filip Szatkowski Karol J. Piczak Przemtslaw Spurek Jacek Tabor Tomasz Trzciñski 22 11 0 09 Feb 2023
SingSong: Generating musical accompaniments from singing Chris Donahue Antoine Caillon Adam Roberts Ethan Manilow P. Esling ... Mauro Verzetti Ian Simon Olivier Pietquin Neil Zeghidour Jesse Engel 32 52 0 30 Jan 2023
Learning from Mistakes: Self-Regularizing Hierarchical Representations in Point Cloud Semantic Segmentation Elena Camuffo Umberto Michieli Simone Milani 3DPC 22 4 0 26 Jan 2023
Mesostructures: Beyond Spectrogram Loss in Differentiable Time-Frequency Analysis Cyrus Vahidi Han Han Changhong Wang Mathieu Lagrange Gyorgy Fazekas Vincent Lostanlen 14 8 0 24 Jan 2023
Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers Chengyi Wang Sanyuan Chen Yu-Huan Wu Zi-Hua Zhang Long Zhou ... Huaming Wang Jinyu Li Lei He Sheng Zhao Furu Wei 43 639 0 05 Jan 2023
High Quality Audio Coding with MDCTNet G. Davidson M. Vinton P. Ekstrand Cong Zhou Lars Villemoes Lie Lu MedIm 15 8 0 08 Dec 2022
Autovocoder: Fast Waveform Generation from a Learned Speech Representation using Differentiable Digital Signal Processing J. Webber Cassia Valentini-Botinhao Evelyn Williams G. Henter Simon King 11 9 0 13 Nov 2022
Neural Feature Predictor and Discriminative Residual Coding for Low-Bitrate Speech Coding Haici Yang Wootaek Lim Minje Kim 19 9 0 04 Nov 2022
HyperSound: Generating Implicit Neural Representations of Audio Signals with Hypernetworks Filip Szatkowski Karol J. Piczak P. Spurek Jacek Tabor Tomasz Trzciñski 23 12 0 03 Nov 2022
Audio Language Modeling using Perceptually-Guided Discrete Representations Felix Kreuk Yaniv Taigman Adam Polyak Jade Copet Gabriel Synnaeve Alexandre Défossez Yossi Adi 27 4 0 02 Nov 2022
WaveFit: An Iterative and Non-autoregressive Neural Vocoder based on Fixed-Point Iteration Yuma Koizumi Kohei Yatabe Heiga Zen M. Bacchiani DiffM 42 29 0 03 Oct 2022
AudioGen: Textually Guided Audio Generation Felix Kreuk Gabriel Synnaeve Adam Polyak Uriel Singer Alexandre Défossez Jade Copet Devi Parikh Yaniv Taigman Yossi Adi DiffM 17 289 0 30 Sep 2022
AudioLM: a Language Modeling Approach to Audio Generation Zalan Borsos Raphaël Marinier Damien Vincent Eugene Kharitonov Olivier Pietquin ... Dominik Roblek O. Teboul David Grangier Marco Tagliasacchi Neil Zeghidour AuLLM 28 566 0 07 Sep 2022
Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications Deniz Gunduz Zhijin Qin Iñaki Estella Aguerri Harpreet S. Dhillon Zhaohui Yang Aylin Yener Kai‐Kit Wong C. Chae 16 431 0 19 Jul 2022
Guaranteed Discovery of Control-Endogenous Latent States with Multi-Step Inverse Models Alex Lamb Riashat Islam Yonathan Efroni Aniket Didolkar Dipendra Kumar Misra Dylan J. Foster Lekan Molu Rajan Chari A. Krishnamurthy John Langford 41 24 0 17 Jul 2022
NESC: Robust Neural End-2-End Speech Coding with GANs N. Pia Kishan Gupta Srikanth Korse M. Multrus Guillaume Fuchs 28 15 0 07 Jul 2022
Cross-Scale Vector Quantization for Scalable Neural Speech Coding Xue Jiang Xiulian Peng Huaying Xue Yuan Zhang Yan Lu MQ 31 9 0 07 Jul 2022
Lossy Compression with Gaussian Diffusion Lucas Theis Tim Salimans Matthew D. Hoffman Fabian Mentzer DiffM 28 77 0 17 Jun 2022
Audio-Visual Speech Codecs: Rethinking Audio-Visual Speech Enhancement by Re-Synthesis Karren D. Yang Dejan Marković Steven Krenn Vasu Agrawal Alexander Richard VGen 16 32 0 31 Mar 2022
Real time spectrogram inversion on mobile phone Oleg Rybakov Marco Tagliasacchi Yunpeng Li Liyang Jiang Xia Zhang Fadi Biadsy 13 4 0 01 Mar 2022
End-to-end LPCNet: A Neural Vocoder With Fully-Differentiable LPC Estimation Krishna Subramani J. Valin Umut Isik Paris Smaragdis A. Krishnaswamy 21 11 0 23 Feb 2022
General-purpose, long-context autoregressive modeling with Perceiver AR Curtis Hawthorne Andrew Jaegle Cătălina Cangea Sebastian Borgeaud C. Nash ... Hannah R. Sheahan Neil Zeghidour Jean-Baptiste Alayrac João Carreira Jesse Engel 35 65 0 15 Feb 2022
End-to-End Neural Speech Coding for Real-Time Communications Xue Jiang Xiulian Peng Chengyu Zheng Huaying Xue Yuan Zhang Yan Lu 21 27 0 24 Jan 2022
Taming Visually Guided Sound Generation Vladimir E. Iashin Esa Rahtu VLM 28 120 0 17 Oct 2021