Boosting Neural Machine Translation

Training efficiency is one of the main problems for Neural Machine Translation (NMT). Deep networks, very large data and many training iterations are necessary to achieve state-of-the-art performance for NMT. This results in very high computation cost and slow down research and industrialization. In this paper, we first investigate the instability by randomizations for NMT training, and further propose an efficient training method based on data boosting and bootstrapping with no modifications to the neural network. Experiments show that this method can converge much faster compared with a baseline system and achieve stable improvement up to 2.36 BLEU points with 80% training cost.