InProceedings of the Third Conference on Machine
Translation: Research Papers , Belgium, Brussels.
Chaitanya Malaviya, Graham Neubig, and Patrick Littell. 2017. Learning language representations for typology prediction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language
Processing .
Bryan McCann, James Bradbury, Caiming Xiong, and
Richard Socher. 2017. Learned in translation: Contextualized word vectors. In Advances in Neural Information Processing Systems .
Graham Neubig and Junjie Hu. 2018. Rapid adaptation
of neural machine translation to new languages. In
Proceedings of the 2018 Conference on Empirical
Methods in Natural Language Processing .
Toan Q. Nguyen and David Chiang. 2017. Transfer
learning across low-resource, related languages for
neural machine translation. In Proc. IJCNLP .
Myle Ott, Sergey Edunov, David Grangier, and
Michael Auli. 2018. Scaling neural machine translation. In Proceedings of the Third Conference on
Machine Translation: Research Papers .
Kishore Papineni, Salim Roukos, Todd Ward, and WeiJing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of
40th Annual Meeting of the Association for Computational Linguistics , Philadelphia, Pennsylvania,
USA.
Multilingual neural machine translation (NMT) enables training a single model
that supports translation from multiple source languages into multiple target
languages. In this paper, we push the limits of multilingual NMT in terms of
number of languages being used. We perform extensive experiments in training
massively multilingual NMT models, translating up to 102 languages to and from
English within a single model. We explore different setups for training such
models and analyze the trade-offs between translation quality and various
modeling decisions. We report results on the publicly available TED talks
multilingual corpus where we show that massively multilingual many-to-many
models are effective in low resource settings, outperforming the previous
state-of-the-art while supporting up to 59 languages. Our experiments on a
large-scale dataset with 102 languages to and from English and up to one
million examples per direction also show promising results, surpassing strong
bilingual baselines and encouraging future work on massively multilingual NMT.
Roee Aharoni Melvin Johnson Orhan Firat