produced translations. Similarly, Malaviya et al.
(2017) trained a many-to-English system including 1017 languages from bible translations, and
used it to infer typological features for the different languages (without evaluating the translation quality). In another relevant work, Artetxe
and Schwenk (2018) trained an NMT model on
93 languages and used the learned representations
to perform cross-lingual transfer learning. Again,
they did not report the performance of the translation model learned in that massively multilingual
setting.
6 Conclusions and Future Work
We showed that NMT models can successfully
scale to 102 languages to-and-from English with
204 trained directions and up to one million examples per direction. Such models improve the
translation quality over similar single-pair baselines when evaluated to and from English by more
than 2 BLEU when averaged over 10 diverse language pairs in each case. We show a similar result on the low-resource TED Talks corpus with 59
languages and 116 trained directions. We analyze
the trade-offs between translation quality and the
number of languages involved, pointing on capacity bottlenecks even with very large models and
showing that massively multilingual models can
generalize better to zero-shot settings.
We hope this work will encourage future research on massively multilingual NMT, enabling
Multilingual neural machine translation (NMT) enables training a single model
that supports translation from multiple source languages into multiple target
languages. In this paper, we push the limits of multilingual NMT in terms of
number of languages being used. We perform extensive experiments in training
massively multilingual NMT models, translating up to 102 languages to and from
English within a single model. We explore different setups for training such
models and analyze the trade-offs between translation quality and various
modeling decisions. We report results on the publicly available TED talks
multilingual corpus where we show that massively multilingual many-to-many
models are effective in low resource settings, outperforming the previous
state-of-the-art while supporting up to 59 languages. Our experiments on a
large-scale dataset with 102 languages to and from English and up to one
million examples per direction also show promising results, surpassing strong
bilingual baselines and encouraging future work on massively multilingual NMT.
Roee Aharoni Melvin Johnson Orhan Firat