95667 0.2852652967
97386 0.286067158
99069 0.285282582
101647 0.2873030901
103358 0.2812287509
105932 0.2841886282
107656 0.2752591372
109350 0.2852451503
111069 0.2802457809
113631 0.2839408815
115325 0.2698247731
117038 0.2862476707
119599 0.2838998437
121302 0.2805997431
123015 0.2826767564
Figure 1: Development BLEU on
fIt,Ro,Nl,De,Ar g!En vs. training BLEU for the
many-to-one and many-to-many models. Best viewed
in color.
the results of our experiments when evaluating
on the same language pairs as they did. The results under “Neubig & Hu 18” are their bilingual baselines and their best many-to-one models.
Their many-to-one models use similar-languageregularization, i.e. ﬁne-tuning a pre-trained manyto-one model with data from the language pair of
interest together with data from a language pair
Multilingual neural machine translation (NMT) enables training a single model
that supports translation from multiple source languages into multiple target
languages. In this paper, we push the limits of multilingual NMT in terms of
number of languages being used. We perform extensive experiments in training
massively multilingual NMT models, translating up to 102 languages to and from
English within a single model. We explore different setups for training such
models and analyze the trade-offs between translation quality and various
modeling decisions. We report results on the publicly available TED talks
multilingual corpus where we show that massively multilingual many-to-many
models are effective in low resource settings, outperforming the previous
state-of-the-art while supporting up to 59 languages. Our experiments on a
large-scale dataset with 102 languages to and from English and up to one
million examples per direction also show promising results, surpassing strong
bilingual baselines and encouraging future work on massively multilingual NMT.
Roee Aharoni Melvin Johnson Orhan Firat