End-to-end Speech Translation via Cross-modal Progressive Training

21 April 2021

Lei Li

Papers citing "End-to-end Speech Translation via Cross-modal Progressive Training"

44 / 44 papers shown

Title
DoCIA: An Online Document-Level Context Incorporation Agent for Speech Translation Xinglin Lyu Wei Tang Y. Li X. Zhao Ming Zhu ... Y. Lu Min Zhang Daimeng Wei Hao Yang Min Zhang 70 0 0 07 Apr 2025
Speech Translation Refinement using Large Language Models Huaixia Dou Xinyu Tian Xinglin Lyu Jie Zhu Junhui Li Lifan Guo 86 0 0 28 Jan 2025
LLaST: Improved End-to-end Speech Translation System Leveraged by Large Language Models Xi Chen Songyang Zhang Qibing Bai Kai-xiang Chen Satoshi Nakamura AuLLM 35 6 0 22 Jul 2024
CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving Bhavani Shankar P. Jyothi Pushpak Bhattacharyya 34 1 0 16 Jun 2024
Pushing the Limits of Zero-shot End-to-End Speech Translation Ioannis Tsiamas Gerard I. Gállego José A. R. Fonollosa Marta R. Costa-jussá 38 7 0 16 Feb 2024
Speech Translation with Large Language Models: An Industrial Practice Zhichao Huang Rong Ye Tom Ko Qianqian Dong Shanbo Cheng Mingxuan Wang Hang Li 62 15 0 21 Dec 2023
Soft Alignment of Modality Space for End-to-end Speech Translation Yuhao Zhang Kaiqi Kou Bei Li Chen Xu Chunliang Zhang Tong Xiao Jingbo Zhu 21 0 0 18 Dec 2023
End-to-End Speech-to-Text Translation: A Survey Nivedita Sethiya Chandresh Kumar Maurya 13 7 0 02 Dec 2023
Rethinking and Improving Multi-task Learning for End-to-end Speech Translation Yuhao Zhang Chen Xu Bei Li Hao Chen Tong Xiao Chunliang Zhang Jingbo Zhu 18 5 0 07 Nov 2023
Towards a Deep Understanding of Multilingual End-to-End Speech Translation Haoran Sun Xiaohu Zhao Yikun Lei Shaolin Zhu Deyi Xiong 28 8 0 31 Oct 2023
Towards Real-World Streaming Speech Translation for Code-Switched Speech Belen Alastruey Matthias Sperber Christian Gollan Dominic Telaar Tim Ng Aashish Agarwal 14 2 0 19 Oct 2023
Bridging the Gaps of Both Modality and Language: Synchronous Bilingual CTC for Speech Translation and Speech Recognition Chen Xu Xiaoqian Liu Erfeng He Yuhao Zhang Qianqian Dong Tong Xiao Jingbo Zhu Dapeng Man Wu Yang 21 0 0 21 Sep 2023
An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation Pengzhi Gao Ruiqing Zhang Zhongjun He Hua-Hong Wu Haifeng Wang 17 4 0 28 Aug 2023
Improving End-to-End Speech Translation by Imitation-Based Knowledge Distillation with Synthetic Transcripts Rebekka Hubert Artem Sokolov Stefan Riezler 13 1 0 17 Jul 2023
Recent Advances in Direct Speech-to-text Translation Chen Xu Rong Ye Qianqian Dong Chengqi Zhao Tom Ko Mingxuan Wang Tong Xiao Jingbo Zhu 12 18 0 20 Jun 2023
Modality Adaption or Regularization? A Case Study on End-to-End Speech Translation Yucheng Han Chen Xu Tong Xiao Jingbo Zhu 20 3 0 13 Jun 2023
CTC-based Non-autoregressive Speech Translation Chen Xu Xiaoqian Liu Xiaowen Liu Qingxuan Sun Yuhao Zhang ... Tom Ko Mingxuan Wang Tong Xiao Anxiang Ma Jingbo Zhu 17 11 0 27 May 2023
ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation Chenyang Le Yao Qian Long Zhou Shujie Liu Yanmin Qian Michael Zeng Xuedong Huang 17 12 0 24 May 2023
CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation Yan Zhou Qingkai Fang Yang Feng OT 23 25 0 24 May 2023
Improving speech translation by fusing speech and text Wenbiao Yin Zhicheng Liu Chengqi Zhao Tao Wang Jian-Fei Tong Rong Ye 13 4 0 23 May 2023
DUB: Discrete Unit Back-translation for Speech Translation Dong Zhang Rong Ye Tom Ko Mingxuan Wang Yaqian Zhou 11 23 0 19 May 2023
AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translation Sara Papi Marco Turchi Matteo Negri 21 19 0 19 May 2023
Back Translation for Speech-to-text Translation Without Transcripts Qingkai Fang Yang Feng 22 13 0 15 May 2023
Understanding and Bridging the Modality Gap for Speech Translation Qingkai Fang Yang Feng 18 25 0 15 May 2023
Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learning Hao Zhang Nianwen Si Yaqi Chen Wenlin Zhang Xukui Yang Dan Qu Weiqiang Zhang 25 9 0 20 Apr 2023
Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inference Biao Fu Minpeng Liao Kai Fan Zhongqiang Huang Boxing Chen Yidong Chen Xiaodon Shi 41 8 0 14 Mar 2023
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition Xize Cheng Lin Li Tao Jin Rongjie Huang Wang Lin Zehan Wang Huangdai Liu Yejin Wang Aoxiong Yin Zhou Zhao 18 24 0 09 Mar 2023
Pre-training for Speech Translation: CTC Meets Optimal Transport Hang Le Hongyu Gong Changhan Wang J. Pino Benjamin Lecouteux D. Schwab OT 13 20 0 27 Jan 2023
WACO: Word-Aligned Contrastive Learning for Speech Translation Siqi Ouyang Rong Ye Lei Li 13 25 0 19 Dec 2022
AdaTranS: Adapting with Boundary-based Shrinking for End-to-End Speech Translation Xingshan Zeng Liangyou Li Qun Liu 24 5 0 17 Dec 2022
M3ST: Mix at Three Levels for Speech Translation Xuxin Cheng Qianqian Dong Fengpeng Yue Tom Ko Mingxuan Wang Yuexian Zou 13 40 0 07 Dec 2022
Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Data Yuhao Zhang Chen Xu Bojie Hu Chunliang Zhang Tong Xiao Jingbo Zhu 16 15 0 04 Dec 2022
Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translation Chen Wang Yuchen Liu Boxing Chen Jiajun Zhang Wei Luo Zhongqiang Huang Chengqing Zong 26 10 0 18 Oct 2022
$RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise$ RedApt: An Adaptor for wav2vec 2 Encoding \\ Faster and Smaller Speech Translation without Quality Compromise Jinming Zhao Haomiao Yang Gholamreza Haffari Ehsan Shareghi VLM 11 2 0 16 Oct 2022
Generating Synthetic Speech from SpokenVocab for Speech Translation Jinming Zhao Gholamreza Haffar Ehsan Shareghi 11 5 0 15 Oct 2022
M-Adapter: Modality Adaptation for End-to-End Speech-to-Text Translation Jinming Zhao Haomiao Yang Ehsan Shareghi Gholamreza Haffari 37 19 0 03 Jul 2022
On the Impact of Noises in Crowd-Sourced Data for Speech Translation Siqi Ouyang Rong Ye Lei Li 12 8 0 28 Jun 2022
Cross-modal Contrastive Learning for Speech Translation Rong Ye Mingxuan Wang Lei Li SSL 16 84 0 05 May 2022
An Analysis of Semantically-Aligned Speech-Text Embeddings M. Huzaifah Ivan Kukanov 17 7 0 04 Apr 2022
STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation Qingkai Fang Rong Ye Lei Li Yang Feng Mingxuan Wang 22 95 0 20 Mar 2022
SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing Junyi Ao Rui Wang Long Zhou Chengyi Wang Shuo Ren ... Yu Zhang Zhihua Wei Yao Qian Jinyu Li Furu Wei 110 192 0 14 Oct 2021
Learning When to Translate for Streaming Speech Qianqian Dong Yaoming Zhu Mingxuan Wang Lei Li 47 29 0 15 Sep 2021
The Volctrans Neural Speech Translation System for IWSLT 2021 Chengqi Zhao Zhicheng Liu Jian-Fei Tong Tao Wang Mingxuan Wang Rong Ye Qianqian Dong Jun Cao Lei Li 8 8 0 16 May 2021
End-to-End Automatic Speech Translation of Audiobooks Alexandre Berard Laurent Besacier A. Kocabiyikoglu Olivier Pietquin 71 189 0 12 Feb 2018