0 0 0 0 0 0
25000 3.360397369 7.345648855 7.028211653 6.687645614 6.629930437
50000 2.003555186 10.08476391 11.66040972 11.46485135 11.34905592
100000 2.383616194 9.54657495 15.44517726 14.72926438 13.17522973
150000 2.588021383 7.121089101 14.5408541 15.54533243 15.89359492
200000 2.589718811 10.99432111 15.82096368 17.38970876 16.26121253
250000 2.854427323 14.00393397 16.53215885 15.97282737 15.3215304
300000 2.823847346 6.934611499 16.78672731 16.52613729 16.16107672
350000 2.950227261 4.779103771 16.73545986 16.12752229 17.03165472
400000 4.301280901 6.631205231 16.16190225 17.38892049 17.55904406
Multilingual neural machine translation (NMT) enables training a single model
that supports translation from multiple source languages into multiple target
languages. In this paper, we push the limits of multilingual NMT in terms of
number of languages being used. We perform extensive experiments in training
massively multilingual NMT models, translating up to 102 languages to and from
English within a single model. We explore different setups for training such
models and analyze the trade-offs between translation quality and various
modeling decisions. We report results on the publicly available TED talks
multilingual corpus where we show that massively multilingual many-to-many
models are effective in low resource settings, outperforming the previous
state-of-the-art while supporting up to 59 languages. Our experiments on a
large-scale dataset with 102 languages to and from English and up to one
million examples per direction also show promising results, surpassing strong
bilingual baselines and encouraging future work on massively multilingual NMT.
Roee Aharoni Melvin Johnson Orhan Firat