showing that massively multilingual models can
generalize better to zero-shot settings.
We hope this work will encourage future research on massively multilingual NMT, enabling
easier support for systems that can serve more people around the globe. There are many possible avenues for future work, including semi-supervised
learning in such settings, exploring ways to reduce the performance degradation when increasing the number of languages, or using such models
for multilingual transfer learning (McCann et al.,
2017; Eriguchi et al., 2018; Artetxe and Schwenk,
2018). Understanding and improving zero-shot
performance in such scenarios is also a promising
direction for future work.
Acknowledgments
We would like to thank the Google Brain and
Google Translate teams for their useful inputs and
discussions. We would also like to thank the entire
Lingvo development team for their foundational
contributions to this project.
References
Mikel Artetxe and Holger Schwenk. 2018. Massively multilingual sentence embeddings for zeroshot cross-lingual transfer and beyond. arXiv
preprint arXiv:1812.10464 .
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly
learning to align and translate. arXiv preprint
Multilingual neural machine translation (NMT) enables training a single model
that supports translation from multiple source languages into multiple target
languages. In this paper, we push the limits of multilingual NMT in terms of
number of languages being used. We perform extensive experiments in training
massively multilingual NMT models, translating up to 102 languages to and from
English within a single model. We explore different setups for training such
models and analyze the trade-offs between translation quality and various
modeling decisions. We report results on the publicly available TED talks
multilingual corpus where we show that massively multilingual many-to-many
models are effective in low resource settings, outperforming the previous
state-of-the-art while supporting up to 59 languages. Our experiments on a
large-scale dataset with 102 languages to and from English and up to one
million examples per direction also show promising results, surpassing strong
bilingual baselines and encouraging future work on massively multilingual NMT.
Roee Aharoni Melvin Johnson Orhan Firat