A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks

21 October 2020

Papers citing "A General Multi-Task Learning Framework to Leverage Text Data for Speech to Text Tasks"

49 / 49 papers shown

Title
LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Models Xi Chen Songyang Zhang Qibing Bai Kai-xiang Chen Satoshi Nakamura AuLLM 35 6 0 22 Jul 2024
An Adapter-Based Unified Model for Multiple Spoken Language Processing Tasks Varsha Suresh Salah Ait-Mokhtar Caroline Brun Ioan Calapodescu 21 0 0 20 Jun 2024
CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving Bhavani Shankar P. Jyothi Pushpak Bhattacharyya 31 1 0 16 Jun 2024
StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task Learning Shaolei Zhang Qingkai Fang Shoutao Guo Zhengrui Ma Min Zhang Yang Feng 29 4 0 05 Jun 2024
Automatic Speech Recognition using Advanced Deep Learning Approaches: A survey Hamza Kheddar Mustapha Hemis Yassine Himeur OffRL 32 58 0 02 Mar 2024
Pushing the Limits of Zero-shot End-to-End Speech Translation Ioannis Tsiamas Gerard I. Gállego José A. R. Fonollosa Marta R. Costa-jussá 35 7 0 16 Feb 2024
Rethinking and Improving Multi-task Learning for End-to-end Speech Translation Yuhao Zhang Chen Xu Bei Li Hao Chen Tong Xiao Chunliang Zhang Jingbo Zhu 18 5 0 07 Nov 2023
Audio-AdapterFusion: A Task-ID-free Approach for Efficient and Non-Destructive Multi-task Speech Recognition Hillary Ngai Rohan Agrawal Neeraj Gaur Ronny Huang Parisa Haghani P. M. Mengibar MoMe 21 0 0 17 Oct 2023
An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation Pengzhi Gao Ruiqing Zhang Zhongjun He Hua-Hong Wu Haifeng Wang 14 4 0 28 Aug 2023
Improving Joint Speech-Text Representations Without Alignment Cal Peyser Zhong Meng Ke Hu Rohit Prabhavalkar Andrew Rosenberg Tara N. Sainath M. Picheny Kyunghyun Cho VLM 19 4 0 11 Aug 2023
Improving End-to-End Speech Translation by Imitation-Based Knowledge Distillation with Synthetic Transcripts Rebekka Hubert Artem Sokolov Stefan Riezler 11 1 0 17 Jul 2023
Performance Comparison of Pre-trained Models for Speech-to-Text in Turkish: Whisper-Small and Wav2Vec2-XLS-R-300M Ö. B. Mercan Sercan Cepni D. E. Tasar ¸Sükrü Ozan VLM 8 1 0 06 Jul 2023
Recent Advances in Direct Speech-to-text Translation Chen Xu Rong Ye Qianqian Dong Chengqi Zhao Tom Ko Mingxuan Wang Tong Xiao Jingbo Zhu 12 18 0 20 Jun 2023
End-to-End Simultaneous Speech Translation with Differentiable Segmentation Shaolei Zhang Yang Feng 13 17 0 25 May 2023
CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation Yan Zhou Qingkai Fang Yang Feng OT 23 25 0 24 May 2023
Improving speech translation by fusing speech and text Wenbiao Yin Zhicheng Liu Chengqi Zhao Tao Wang Jian-Fei Tong Rong Ye 13 4 0 23 May 2023
Back Translation for Speech-to-text Translation Without Transcripts Qingkai Fang Yang Feng 19 13 0 15 May 2023
Understanding and Bridging the Modality Gap for Speech Translation Qingkai Fang Yang Feng 15 25 0 15 May 2023
Hybrid Transducer and Attention based Encoder-Decoder Modeling for Speech-to-Text Tasks Yun Tang Anna Y. Sun H. Inaguma Xinyue Chen Ning Dong Xutai Ma Paden Tomasello J. Pino 27 19 0 04 May 2023
Deep Transfer Learning for Automatic Speech Recognition: Towards Better Generalization Hamza Kheddar Yassine Himeur S. Al-Maadeed Abbes Amira F. Bensaali 35 76 0 27 Apr 2023
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition Xize Cheng Lin Li Tao Jin Rongjie Huang Wang Lin Zehan Wang Huangdai Liu Yejin Wang Aoxiong Yin Zhou Zhao 18 24 0 09 Mar 2023
Multi-task Highly Adaptive Lasso Ivana Malenica Rachael V. Phillips D. Lazzareschi Jeremy Coyle Romain Pirracchio Mark van der Laan 13 0 0 27 Jan 2023
Pre-training for Speech Translation: CTC Meets Optimal Transport Hang Le Hongyu Gong Changhan Wang J. Pino Benjamin Lecouteux D. Schwab OT 13 20 0 27 Jan 2023
$Mu$^{2}$SLAM: Multitask, Multilingual Speech and Language Models$ Mu $^{2}$ SLAM: Multitask, Multilingual Speech and Language Models Yong Cheng Yu Zhang Melvin Johnson Wolfgang Macherey Ankur Bapna 20 8 0 19 Dec 2022
WACO: Word-Aligned Contrastive Learning for Speech Translation Siqi Ouyang Rong Ye Lei Li 13 25 0 19 Dec 2022
AdaTranS: Adapting with Boundary-based Shrinking for End-to-End Speech Translation Xingshan Zeng Liangyou Li Qun Liu 24 5 0 17 Dec 2022
Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Data Yuhao Zhang Chen Xu Bojie Hu Chunliang Zhang Tong Xiao Jingbo Zhu 16 15 0 04 Dec 2022
MMSpeech: Multi-modal Multi-task Encoder-Decoder Pre-training for Speech Recognition Xiaohuan Zhou Jiaming Wang Zeyu Cui Shiliang Zhang Zhijie Yan Jingren Zhou Chang Zhou 30 12 0 29 Nov 2022
T5lephone: Bridging Speech and Text Self-supervised Models for Spoken Language Understanding via Phoneme level T5 Chan-Jan Hsu Ho-Lam Chung Hung-yi Lee Yu Tsao 12 6 0 01 Nov 2022
Speech-text based multi-modal training with bidirectional attention for improved speech recognition Yuhang Yang Haihua Xu Hao-Ming Huang E. Chng Sheng Li 34 7 0 01 Nov 2022
Don't Discard Fixed-Window Audio Segmentation in Speech-to-Text Translation Chantal Amrhein Barry Haddow 17 8 0 24 Oct 2022
Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translation Chen Wang Yuchen Liu Boxing Chen Jiajun Zhang Wei Luo Zhongqiang Huang Chengqing Zong 23 10 0 18 Oct 2022
Efficient acoustic feature transformation in mismatched environments using a Guided-GAN Walter Heymans Marelie Hattingh Davel C. van Heerden 13 1 0 03 Oct 2022
Improving Deliberation by Text-Only and Semi-Supervised Training Ke Hu Tara N. Sainath Yanzhang He Rohit Prabhavalkar Trevor Strohman S. Mavandadi Weiran Wang 17 12 0 29 Jun 2022
Cross-modal Contrastive Learning for Speech Translation Rong Ye Mingxuan Wang Lei Li SSL 13 84 0 05 May 2022
Hear No Evil: Towards Adversarial Robustness of Automatic Speech Recognition via Multi-Task Learning Nilaksh Das Duen Horng Chau AAML 21 0 0 05 Apr 2022
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation Qingkai Fang Rong Ye Lei Li Yang Feng Mingxuan Wang 22 95 0 20 Mar 2022
Tackling data scarcity in speech translation using zero-shot multilingual machine translation techniques Tu Anh Dinh Danni Liu J. Niehues 6 6 0 26 Jan 2022
Optimizing Alignment of Speech and Language Latent Spaces for End-to-End Speech Recognition and Understanding Wei Wang Shuo Ren Yao Qian Shujie Liu Yu Shi Y. Qian Michael Zeng 18 16 0 23 Oct 2021
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing Junyi Ao Rui Wang Long Zhou Chengyi Wang Shuo Ren ... Yu Zhang Zhihua Wei Yao Qian Jinyu Li Furu Wei 110 192 0 14 Oct 2021
ASR Rescoring and Confidence Estimation with ELECTRA Hayato Futami H. Inaguma Masato Mimura S. Sakai Tatsuya Kawahara KELM 54 20 0 05 Oct 2021
FST: the FAIR Speech Translation System for the IWSLT21 Multilingual Shared Task Yun Tang Hongyu Gong Xian Li Changhan Wang J. Pino Holger Schwenk Naman Goyal 23 10 0 14 Jul 2021
Zero-shot Speech Translation Tu Anh Dinh 14 6 0 13 Jul 2021
Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task Yun Tang J. Pino Xian Li Changhan Wang Dmitriy Genzel 103 81 0 12 Jul 2021
The Volctrans Neural Speech Translation System for IWSLT 2021 Chengqi Zhao Zhicheng Liu Jian-Fei Tong Tao Wang Mingxuan Wang Rong Ye Qianqian Dong Jun Cao Lei Li 8 8 0 16 May 2021
Learning Shared Semantic Space for Speech-to-Text Translation Chi Han Mingxuan Wang Heng Ji Lei Li 16 76 0 07 May 2021
End-to-end Speech Translation via Cross-modal Progressive Training Rong Ye Mingxuan Wang Lei Li 19 71 0 21 Apr 2021
Large-Scale Self- and Semi-Supervised Learning for Speech Translation Changhan Wang Anne Wu J. Pino Alexei Baevski Michael Auli Alexis Conneau SSL 31 44 0 14 Apr 2021
Tied Multitask Learning for Neural Speech Translation Antonios Anastasopoulos David Chiang 91 171 0 19 Feb 2018