STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation

20 March 2022

Lei Li

Papers citing "STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation"

50 / 54 papers shown

Title
DoCIA: An Online Document-Level Context Incorporation Agent for Speech Translation Xinglin Lyu Wei Tang Y. Li X. Zhao Ming Zhu ... Y. Lu Min Zhang Daimeng Wei Hao Yang Min Zhang 70 0 0 07 Apr 2025
Detecting underdetermination in parameterized quantum circuits Marie Kempkes Jakob Spiegelberg Evert van Nieuwenburg Vedran Dunjko 34 0 0 04 Apr 2025
Adaptive Inner Speech-Text Alignment for LLM-based Speech Translation Henglyu Liu Andong Chen Kehai Chen X. Bai M. Zhong Yuan Qiu Min Zhang 40 0 0 13 Mar 2025
Speech Translation Refinement using Large Language Models Huaixia Dou Xinyu Tian Xinglin Lyu Jie Zhu Junhui Li Lifan Guo 83 0 0 28 Jan 2025
Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison Tsz Kin Lam Marco Gaido Sara Papi L. Bentivogli Barry Haddow 29 0 0 04 Jan 2025
CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving Bhavani Shankar P. Jyothi Pushpak Bhattacharyya 31 1 0 16 Jun 2024
CTC-based Non-autoregressive Textless Speech-to-Speech Translation Qingkai Fang Zhengrui Ma Yan Zhou Min Zhang Yang Feng 50 0 0 11 Jun 2024
Can We Achieve High-quality Direct Speech-to-Speech Translation without Parallel Speech Data? Qingkai Fang Shaolei Zhang Zhengrui Ma Min Zhang Yang Feng VLM 35 1 0 11 Jun 2024
Pushing the Limits of Zero-shot End-to-End Speech Translation Ioannis Tsiamas Gerard I. Gállego José A. R. Fonollosa Marta R. Costa-jussá 28 7 0 16 Feb 2024
R-BI: Regularized Batched Inputs enhance Incremental Decoding Framework for Low-Latency Simultaneous Speech Translation Jiaxin Guo Zhanglin Wu Zongyao Li Hengchao Shang Daimeng Wei Xiaoyu Chen Zhiqiang Rao Shaojun Li Hao-Yu Yang 25 1 0 11 Jan 2024
Soft Alignment of Modality Space for End-to-end Speech Translation Yuhao Zhang Kaiqi Kou Bei Li Chen Xu Chunliang Zhang Tong Xiao Jingbo Zhu 15 0 0 18 Dec 2023
End-to-End Speech-to-Text Translation: A Survey Nivedita Sethiya Chandresh Kumar Maurya 11 7 0 02 Dec 2023
Rethinking and Improving Multi-task Learning for End-to-end Speech Translation Yuhao Zhang Chen Xu Bei Li Hao Chen Tong Xiao Chunliang Zhang Jingbo Zhu 18 5 0 07 Nov 2023
Towards a Deep Understanding of Multilingual End-to-End Speech Translation Haoran Sun Xiaohu Zhao Yikun Lei Shaolin Zhu Deyi Xiong 23 8 0 31 Oct 2023
Unified Segment-to-Segment Framework for Simultaneous Sequence Generation Shaolei Zhang Yang Feng 15 7 0 27 Oct 2023
Bridging the Gap between Synthetic and Authentic Images for Multimodal Machine Translation Wenyu Guo Qingkai Fang Dong Yu Yang Feng 13 6 0 20 Oct 2023
DASpeech: Directed Acyclic Transformer for Fast and High-quality Speech-to-Speech Translation Qingkai Fang Yan Zhou Yangzhou Feng 27 6 0 11 Oct 2023
Tuning Large language model for End-to-end Speech Translation Hao Zhang Nianwen Si Yaqi Chen Wenlin Zhang Xu Yang Dan Qu Xiaolin Jiao 15 8 0 03 Oct 2023
Cross-Modal Multi-Tasking for Speech-to-Text Translation via Hard Parameter Sharing B. Grimstad Xuankai Chang Antonios Anastasopoulos Yuya Fujita Shinji Watanabe 13 2 0 27 Sep 2023
Bridging the Gaps of Both Modality and Language: Synchronous Bilingual CTC for Speech Translation and Speech Recognition Chen Xu Xiaoqian Liu Erfeng He Yuhao Zhang Qianqian Dong Tong Xiao Jingbo Zhu Dapeng Man Wu Yang 19 0 0 21 Sep 2023
An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation Pengzhi Gao Ruiqing Zhang Zhongjun He Hua-Hong Wu Haifeng Wang 12 4 0 28 Aug 2023
Recent Advances in Direct Speech-to-text Translation Chen Xu Rong Ye Qianqian Dong Chengqi Zhao Tom Ko Mingxuan Wang Tong Xiao Jingbo Zhu 12 18 0 20 Jun 2023
Modality Adaption or Regularization? A Case Study on End-to-End Speech Translation Yucheng Han Chen Xu Tong Xiao Jingbo Zhu 20 3 0 13 Jun 2023
CTC-based Non-autoregressive Speech Translation Chen Xu Xiaoqian Liu Xiaowen Liu Qingxuan Sun Yuhao Zhang ... Tom Ko Mingxuan Wang Tong Xiao Anxiang Ma Jingbo Zhu 14 11 0 27 May 2023
ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation Chenyang Le Yao Qian Long Zhou Shujie Liu Yanmin Qian Michael Zeng Xuedong Huang 15 12 0 24 May 2023
CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation Yan Zhou Qingkai Fang Yang Feng OT 20 25 0 24 May 2023
Improving speech translation by fusing speech and text Wenbiao Yin Zhicheng Liu Chengqi Zhao Tao Wang Jian-Fei Tong Rong Ye 13 4 0 23 May 2023
DUB: Discrete Unit Back-translation for Speech Translation Dong Zhang Rong Ye Tom Ko Mingxuan Wang Yaqian Zhou 11 23 0 19 May 2023
AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translation Sara Papi Marco Turchi Matteo Negri 16 19 0 19 May 2023
Cross-modality Data Augmentation for End-to-End Sign Language Translation Jinhui Ye Wenxiang Jiao Xing Wang Zhaopeng Tu Hui Xiong SLR 11 21 0 18 May 2023
Back Translation for Speech-to-text Translation Without Transcripts Qingkai Fang Yang Feng 17 13 0 15 May 2023
Understanding and Bridging the Modality Gap for Speech Translation Qingkai Fang Yang Feng 13 25 0 15 May 2023
Improving Speech Translation by Cross-Modal Multi-Grained Contrastive Learning Hao Zhang Nianwen Si Yaqi Chen Wenlin Zhang Xukui Yang Dan Qu Weiqiang Zhang 25 9 0 20 Apr 2023
Decouple Non-parametric Knowledge Distillation For End-to-end Speech Translation Hao Zhang Nianwen Si Yaqi Chen Wenlin Zhang Xukui Yang Dan Qu Zhen Li 11 3 0 20 Apr 2023
Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inference Biao Fu Minpeng Liao Kai Fan Zhongqiang Huang Boxing Chen Yidong Chen Xiaodon Shi 41 8 0 14 Mar 2023
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition Xize Cheng Lin Li Tao Jin Rongjie Huang Wang Lin Zehan Wang Huangdai Liu Yejin Wang Aoxiong Yin Zhou Zhao 15 24 0 09 Mar 2023
Pre-training for Speech Translation: CTC Meets Optimal Transport Hang Le Hongyu Gong Changhan Wang J. Pino Benjamin Lecouteux D. Schwab OT 13 20 0 27 Jan 2023
A Survey of Mix-based Data Augmentation: Taxonomy, Methods, Applications, and Explainability Chengtai Cao Fan Zhou Yurou Dai Jianping Wang Kunpeng Zhang AAML 16 27 0 21 Dec 2022
SegAugment: Maximizing the Utility of Speech Translation Data with Segmentation-based Augmentations Ioannis Tsiamas José A. R. Fonollosa Marta R. Costa-jussá 28 6 0 19 Dec 2022
WACO: Word-Aligned Contrastive Learning for Speech Translation Siqi Ouyang Rong Ye Lei Li 13 25 0 19 Dec 2022
AdaTranS: Adapting with Boundary-based Shrinking for End-to-End Speech Translation Xingshan Zeng Liangyou Li Qun Liu 24 5 0 17 Dec 2022
M3ST: Mix at Three Levels for Speech Translation Xuxin Cheng Qianqian Dong Fengpeng Yue Tom Ko Mingxuan Wang Yuexian Zou 11 40 0 07 Dec 2022
Beyond Instance Discrimination: Relation-aware Contrastive Self-supervised Learning Yifei Zhang Chang-rui Liu Yu Zhou Weiping Wang QiXiang Ye Xiangyang Ji SSL ISeg BDL 19 6 0 02 Nov 2022
ULN: Towards Underspecified Vision-and-Language Navigation Weixi Feng Tsu-jui Fu Yujie Lu William Yang Wang 31 4 0 18 Oct 2022
Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translation Chen Wang Yuchen Liu Boxing Chen Jiajun Zhang Wei Luo Zhongqiang Huang Chengqing Zong 21 10 0 18 Oct 2022
Low-resource Neural Machine Translation with Cross-modal Alignment Zhe Yang Qingkai Fang Yang Feng VLM 13 9 0 13 Oct 2022
SpeechUT: Bridging Speech and Text with Hidden-Unit for Encoder-Decoder Based Speech-Text Pre-training Zi-Hua Zhang Long Zhou Junyi Ao Shujie Liu Lirong Dai Jinyu Li Furu Wei 61 57 0 07 Oct 2022
Cross-modal Contrastive Learning for Speech Translation Rong Ye Mingxuan Wang Lei Li SSL 11 84 0 05 May 2022
Neural Machine Translation with Phrase-Level Universal Visual Representations Qingkai Fang Yang Feng 24 40 0 19 Mar 2022
Geodesic Multi-Modal Mixup for Robust Fine-Tuning Changdae Oh Junhyuk So Hoyoon Byun Yongtaek Lim Minchul Shin Jong-June Jeon Kyungwoo Song 21 26 0 08 Mar 2022