out-of-English. Again, both of the massively multilingual models perform better than the baselines when averaged across the 10 evaluated language pairs, while handling up to 102 languages
to-and-from English and 204 translation tasks simultaneously. In this case the results are similar to those we observed on the TED talks corpus, where the one-to-many model performs better
than the many-to-many model. Again, this advantage may be due to the one-to-many model handling a smaller number of tasks while not being
biased towards English in the target side like the
many-to-many model.
4 Analysis
The above results show that massively multilingual NMT is indeed possible in large scale settings
and can improve performance over strong bilingual baselines. However, it was shown in a somewhat extreme case with more than 100 languages
trained jointly, where we saw that in some cases
the joint training may harm the performance for
some language pairs (i.e. German-English above).
In the following analysis we would like to better understand the trade-off between the number
of languages involved and the translation accuracy while keeping the model capacity and training conﬁguration ﬁxed.
4.1 Multilinguality & Supervised
Performance
We ﬁrst study the effect of varying the number of languages on the translation accuracy in
Multilingual neural machine translation (NMT) enables training a single model
that supports translation from multiple source languages into multiple target
languages. In this paper, we push the limits of multilingual NMT in terms of
number of languages being used. We perform extensive experiments in training
massively multilingual NMT models, translating up to 102 languages to and from
English within a single model. We explore different setups for training such
models and analyze the trade-offs between translation quality and various
modeling decisions. We report results on the publicly available TED talks
multilingual corpus where we show that massively multilingual many-to-many
models are effective in low resource settings, outperforming the previous
state-of-the-art while supporting up to 59 languages. Our experiments on a
large-scale dataset with 102 languages to and from English and up to one
million examples per direction also show promising results, surpassing strong
bilingual baselines and encouraging future work on massively multilingual NMT.
Roee Aharoni Melvin Johnson Orhan Firat