Textless Speech-to-Speech Translation on Real Data

15 December 2021

Ann Lee

Hongyu Gong

Paul-Ambroise Duquenne

Papers citing "Textless Speech-to-Speech Translation on Real Data"

50 / 110 papers shown

Title
UniAudio: An Audio Foundation Model Toward Universal Audio Generation Dongchao Yang Jinchuan Tian Xuejiao Tan Rongjie Huang Songxiang Liu ... Jiang Bian Xixin Wu Zhou Zhao Shinji Watanabe Helen M. Meng CVBM AuLLM 22 114 0 01 Oct 2023
Joint Prediction and Denoising for Large-scale Multilingual Self-supervised Learning William Chen Jiatong Shi Brian Yan Dan Berrebbi Wangyou Zhang Yifan Peng Xuankai Chang Soumi Maiti Shinji Watanabe 24 8 0 26 Sep 2023
Towards Practical and Efficient Image-to-Speech Captioning with Vision-Language Pre-training and Multi-modal Tokens Minsu Kim J. Choi Soumi Maiti Jeong Hun Yeo Shinji Watanabe Y. Ro VLM 21 6 0 15 Sep 2023
Speech-to-Speech Translation with Discrete-Unit-Based Style Transfer Yongqiang Wang Jionghao Bai Rongjie Huang Ruiqi Li Zhiqing Hong Zhou Zhao 17 3 0 14 Sep 2023
Direct Text to Speech Translation System using Acoustic Units Victoria Mingote Pablo Gimeno Luis Vicente Sameer Khurana Antoine Laurent J. Duret 23 3 0 14 Sep 2023
RepCodec: A Speech Representation Codec for Speech Tokenization Zhichao Huang Chutong Meng Tom Ko 14 22 0 31 Aug 2023
Many-to-Many Spoken Language Translation via Unified Speech and Text Representation Learning with Unit-to-Unit Translation Minsu Kim J. Choi Dahun Kim Y. Ro 35 10 0 03 Aug 2023
The Effect of Spoken Language on Speech Enhancement using Self-Supervised Speech Representation Loss Functions George Close Thomas Hain Stefan Goetze 24 8 0 27 Jul 2023
Multilingual Speech-to-Speech Translation into Multiple Target Languages Hongyu Gong Ning Dong Sravya Popuri Vedanuj Goswami Ann Lee J. Pino 55 4 0 17 Jul 2023
Towards cross-language prosody transfer for dialog Jonathan Avila Nigel G. Ward 28 6 0 09 Jul 2023
Learning Multilingual Expressive Speech Representation for Prosody Prediction without Parallel Data J. Duret Titouan Parcollet Yannick Esteve 27 4 0 29 Jun 2023
AudioPaLM: A Large Language Model That Can Speak and Listen Paul Kishan Rubenstein Chulayuth Asawaroengchai D. Nguyen Ankur Bapna Zalan Borsos ... Neil Zeghidour Yu Zhang Zhishuai Zhang Lukás Zilka Christian Frank LM&MA AuLLM VLM 35 257 0 22 Jun 2023
PolyVoice: Language Models for Speech to Speech Translation Qianqian Dong Zhiying Huang Qiao Tian Chen Xu Tom Ko ... Lu Lu Zejun Ma Yuping Wang Mingxuan Wang Yuxuan Wang 20 23 0 05 Jun 2023
Learning When to Speak: Latency and Quality Trade-offs for Simultaneous Speech-to-Speech Translation with Offline Models Liam Dugan Anshul Wadhawan Kyle Spence Chris Callison-Burch Morgan McGuire Victor Zordan OffRL 30 2 0 01 Jun 2023
Make-A-Voice: Unified Voice Synthesis With Discrete Representation Rongjie Huang Chunlei Zhang Yongqiang Wang Dongchao Yang Lu Liu Zhenhui Ye Ziyue Jiang Chao Weng Zhou Zhao Dong Yu DiffM 29 26 0 30 May 2023
Translatotron 3: Speech to Speech Translation with Monolingual Data Eliya Nachmani Alon Levkovitch Yi-Yang Ding Chulayutsh Asawaroengchai Heiga Zen Michelle Tadmor Ramanovich 21 14 0 27 May 2023
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation Rongjie Huang Huadai Liu Xize Cheng Yi Ren Lin Li ... Jinzheng He Lichao Zhang Jinglin Liu Xiaoyue Yin Zhou Zhao 67 8 0 24 May 2023
Spoken Question Answering and Speech Continuation Using Spectrogram-Powered LLM Eliya Nachmani Alon Levkovitch Roy Hirsch Julián Salazar Chulayutsh Asawaroengchai Soroosh Mariooryad Ehud Rivlin RJ Skerry-Ryan Michelle Tadmor Ramanovich AuLLM 19 30 0 24 May 2023
Textually Pretrained Speech Language Models Michael Hassid Tal Remez Tu Nguyen Itai Gat Alexis Conneau ... Alexandre Défossez Gabriel Synnaeve Emmanuel Dupoux Roy Schwartz Yossi Adi VLM SyDa 29 53 0 22 May 2023
MParrotTTS: Multilingual Multi-speaker Text to Speech Synthesis in Low Resource Setting Neil Shah Vishal Tambrahalli Saiteja Kosgi N. Pedanekar Vineet Gandhi 33 0 0 19 May 2023
DUB: Discrete Unit Back-translation for Speech Translation Dong Zhang Rong Ye Tom Ko Mingxuan Wang Yaqian Zhou 13 23 0 19 May 2023
ML-SUPERB: Multilingual Speech Universal PERformance Benchmark Jiatong Shi Dan Berrebbi William Chen Ho-Lam Chung En-Pei Hu ... Xuankai Chang Shang-Wen Li Abdel-rahman Mohamed Hung-yi Lee Shinji Watanabe ELM 55 58 0 18 May 2023
Back Translation for Speech-to-text Translation Without Transcripts Qingkai Fang Yang Feng 30 13 0 15 May 2023
Joint Multi-scale Cross-lingual Speaking Style Transfer with Bidirectional Attention Mechanism for Automatic Dubbing Jingbei Li Sipan Li Ping Chen Lu Zhang Yi Meng Zhiyong Wu H. Meng Qiao Tian Yuping Wang Yuxuan Wang 29 3 0 09 May 2023
Looking Similar, Sounding Different: Leveraging Counterfactual Cross-Modal Pairs for Audiovisual Representation Learning Nikhil Singh Chih-Wei Wu Iroro Orife Mahdi M. Kalayeh 23 2 0 12 Apr 2023
Enhancing Speech-to-Speech Translation with Multiple TTS Targets Jiatong Shi Yun Tang Ann Lee H. Inaguma Changhan Wang J. Pino Shinji Watanabe 38 9 0 10 Apr 2023
ESPnet-ST-v2: Multipurpose Spoken Language Translation Toolkit Brian Yan Jiatong Shi Yun Tang H. Inaguma Yifan Peng ... Zhaoheng Ni Moto Hira Soumi Maiti J. Pino Shinji Watanabe 19 20 0 10 Apr 2023
Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling Zi-Hua Zhang Long Zhou Chengyi Wang Sanyuan Chen Yu Wu ... Huaming Wang Jinyu Li Lei He Sheng Zhao Furu Wei VLM 28 170 0 07 Mar 2023
ParrotTTS: Text-to-Speech synthesis by exploiting self-supervised representations N. Shah Saiteja Kosgi Vishal Tambrahalli Neha Sahipjohn Anil Nelakanti Vineet Gandhi 17 8 0 01 Mar 2023
A Holistic Cascade System, benchmark, and Human Evaluation Protocol for Expressive Speech-to-Speech Translation Wen-Chin Huang Benjamin Peloquin Justine T. Kao Changhan Wang Hongyu Gong Elizabeth Salesky Yossi Adi Ann Lee Peng-Jen Chen 25 16 0 25 Jan 2023
Speaking Style Conversion in the Waveform Domain Using Discrete Self-Supervised Units Gallil Maimon Yossi Adi 21 13 0 19 Dec 2022
BLASER: A Text-Free Speech-to-Speech Translation Evaluation Metric Mingda Chen Paul-Ambroise Duquenne Pierre Yves Andrews Justine T. Kao Alexandre Mourachko Holger Schwenk Marta R. Costa-jussá 14 17 0 16 Dec 2022
UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units H. Inaguma Sravya Popuri Ilia Kulikov Peng-Jen Chen Changhan Wang Yu-An Chung Yun Tang Ann Lee Shinji Watanabe J. Pino 43 51 0 15 Dec 2022
Direct Speech-to-speech Translation without Textual Annotation using Bottleneck Features Junhui Zhang Junjie Pan Xiang Yin Zejun Ma 19 0 0 12 Dec 2022
Speech-to-Speech Translation For A Real-world Unwritten Language Peng-Jen Chen Ke M. Tran Yilin Yang Jingfei Du Justine T. Kao ... Sravya Popuri Changhan Wang J. Pino Wei-Ning Hsu Ann Lee 29 26 0 11 Nov 2022
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations Paul-Ambroise Duquenne Hongyu Gong Ning Dong Jingfei Du Ann Lee Vedanuj Goswani Changhan Wang J. Pino Benoît Sagot Holger Schwenk 37 34 0 08 Nov 2022
Audio Language Modeling using Perceptually-Guided Discrete Representations Felix Kreuk Yaniv Taigman Adam Polyak Jade Copet Gabriel Synnaeve Alexandre Défossez Yossi Adi 27 4 0 02 Nov 2022
Textless Direct Speech-to-Speech Translation with Discrete Speech Representation Xinjian Li Ye Jia Chung-Cheng Chiu 23 23 0 31 Oct 2022
Joint Pre-Training with Speech and Bilingual Text for Direct Speech to Speech Translation Kun Wei Long Zhou Zi-Hua Zhang Liping Chen Shujie Liu Lei He Jinyu Li Furu Wei 14 13 0 31 Oct 2022
Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challenge Ewan Dunbar Nicolas Hamilakis Emmanuel Dupoux SSL 26 30 0 27 Oct 2022
Improving Speech-to-Speech Translation Through Unlabeled Text Xuan-Phi Nguyen Sravya Popuri Changhan Wang Yun Tang Ilia Kulikov Hongyu Gong 12 9 0 26 Oct 2022
High Fidelity Neural Audio Compression Alexandre Défossez Jade Copet Gabriel Synnaeve Yossi Adi 11 595 0 24 Oct 2022
Bootstrapping meaning through listening: Unsupervised learning of spoken sentence embeddings Jian Zhu Zuoyu Tian Yadong Liu Cong Zhang Chia-wen Lo SSL 30 2 0 23 Oct 2022
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training Zi-Hua Zhang Long Zhou Junyi Ao Shujie Liu Lirong Dai Jinyu Li Furu Wei 61 57 0 07 Oct 2022
Augmentation Invariant Discrete Representation for Generative Spoken Language Modeling Itai Gat Felix Kreuk Tu Nguyen Ann Lee Jade Copet Gabriel Synnaeve Emmanuel Dupoux Yossi Adi 20 11 0 30 Sep 2022
AudioGen: Textually Guided Audio Generation Felix Kreuk Gabriel Synnaeve Adam Polyak Uriel Singer Alexandre Défossez Jade Copet Devi Parikh Yaniv Taigman Yossi Adi DiffM 17 289 0 30 Sep 2022
T-Modules: Translation Modules for Zero-Shot Cross-Modal Machine Translation Paul-Ambroise Duquenne Hongyu Gong Benoît Sagot Holger Schwenk 22 18 0 24 May 2022
Self-Supervised Speech Representation Learning: A Review Abdel-rahman Mohamed Hung-yi Lee Lasse Borgholt Jakob Drachmann Havtorn Joakim Edin ... Shang-Wen Li Karen Livescu Lars Maaløe Tara N. Sainath Shinji Watanabe SSL AI4TS 128 349 0 21 May 2022
Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation Qianqian Dong Fengpeng Yue Tom Ko Mingxuan Wang Qibing Bai Yu Zhang 32 16 0 18 May 2022
Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation Sravya Popuri Peng-Jen Chen Changhan Wang J. Pino Yossi Adi Jiatao Gu Wei-Ning Hsu Ann Lee 20 56 0 06 Apr 2022