Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation

15 December 2022

Papers citing "Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation"

6 / 6 papers shown

Title
SiRA: Sparse Mixture of Low Rank Adaptation Yun Zhu Nevan Wichers Chu-Cheng Lin Xinyi Wang Tianlong Chen ... Han Lu Canoee Liu Liangchen Luo Jindong Chen Lei Meng MoE 19 27 0 15 Nov 2023
Tutel: Adaptive Mixture-of-Experts at Scale Changho Hwang Wei Cui Yifan Xiong Ziyue Yang Ze Liu ... Joe Chau Peng Cheng Fan Yang Mao Yang Y. Xiong MoE 92 108 0 07 Jun 2022
Multilingual Machine Translation with Hyper-Adapters Christos Baziotis Mikel Artetxe James Cross Shruti Bhosale 63 21 0 22 May 2022
Scalable and Efficient MoE Training for Multitask Multilingual Models Young Jin Kim A. A. Awan Alexandre Muzio Andres Felipe Cruz Salinas Liyang Lu Amr Hendy Samyam Rajbhandari Yuxiong He Hany Awadalla MoE 94 83 0 22 Sep 2021
Scaling Laws for Neural Language Models Jared Kaplan Sam McCandlish T. Henighan Tom B. Brown B. Chess R. Child Scott Gray Alec Radford Jeff Wu Dario Amodei 226 4,424 0 23 Jan 2020
Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation Rui Wang Masao Utiyama Eiichiro Sumita 13 27 0 01 May 2018