model in most cases while they are trained identically.
One particular outlier in this case is German-toEnglish, where the many-to-one model is 2 BLEU
points below the many-to-many model. We examine the BLEU score of this language pair on its
dedicated German-English development set during training in the many-to-one model and ﬁnd
that it highly ﬂuctuates. We then measure the
performance on the test set for this language pair
by choosing the best checkpoint on the dedicated
German-English development set (instead of on
the mixed multilingual development set) and ﬁnd
it to be 38.07, which is actually higher in 1 BLEU
than the best result of the many-to-many model.
This shows that while training many languages together, there is no “silver bullet”: some languages
may suffer from severe interference during training (i.e. a reduction of 3 BLEU in this case, from38.07 to 35.05) while other languages continue to
improve with more updates.
Table 6 describes the results when translating
out-of-English. Again, both of the massively multilingual models perform better than the baselines when averaged across the 10 evaluated language pairs, while handling up to 102 languages
Multilingual neural machine translation (NMT) enables training a single model
that supports translation from multiple source languages into multiple target
languages. In this paper, we push the limits of multilingual NMT in terms of
number of languages being used. We perform extensive experiments in training
massively multilingual NMT models, translating up to 102 languages to and from
English within a single model. We explore different setups for training such
models and analyze the trade-offs between translation quality and various
modeling decisions. We report results on the publicly available TED talks
multilingual corpus where we show that massively multilingual many-to-many
models are effective in low resource settings, outperforming the previous
state-of-the-art while supporting up to 59 languages. Our experiments on a
large-scale dataset with 102 languages to and from English and up to one
million examples per direction also show promising results, surpassing strong
bilingual baselines and encouraging future work on massively multilingual NMT.
Roee Aharoni Melvin Johnson Orhan Firat