Ar-Fr Fr-Ar Ru-Uk Uk-Ru Avg.
5-to-5 1.66 4.49 3.7 3.02 3.21
25-to-25 1.83 5.52 16.67 4.31 7.08
50-to-50 4.34 4.72 15.14 20.23 11.1
75-to-75 1.85 4.26 11.2 15.88 8.3
103-to-103 2.87 3.05 12.3 18.49 9.17
Table 8: Zero-Shot performance while varying the
number of languages involved
to-many models. We create four subsets of the
in-house dataset by sub-sampling it to a different number of languages in each subset. In
this way we create four additional English-centric
datasets, containing 5, 25, 50 and 75 languages
each to-and-from English. We make sure that
each subset contains all the languages from the
next smaller subsets – i.e. the 25 language subset contains the 5 language subset, the 50 language subset contains the 25 language subset and
so on. We train a similar-capacity large Transformer model (with 473.7M parameters) on each
of these subsets and measure the performance for
each model on the 8 supervised language pairs
Multilingual neural machine translation (NMT) enables training a single model
that supports translation from multiple source languages into multiple target
languages. In this paper, we push the limits of multilingual NMT in terms of
number of languages being used. We perform extensive experiments in training
massively multilingual NMT models, translating up to 102 languages to and from
English within a single model. We explore different setups for training such
models and analyze the trade-offs between translation quality and various
modeling decisions. We report results on the publicly available TED talks
multilingual corpus where we show that massively multilingual many-to-many
models are effective in low resource settings, outperforming the previous
state-of-the-art while supporting up to 59 languages. Our experiments on a
large-scale dataset with 102 languages to and from English and up to one
million examples per direction also show promising results, surpassing strong
bilingual baselines and encouraging future work on massively multilingual NMT.
Roee Aharoni Melvin Johnson Orhan Firat